CN113723548A - Method and device for preprocessing decision tree rule set - Google Patents
Method and device for preprocessing decision tree rule set Download PDFInfo
- Publication number
- CN113723548A CN113723548A CN202111038482.6A CN202111038482A CN113723548A CN 113723548 A CN113723548 A CN 113723548A CN 202111038482 A CN202111038482 A CN 202111038482A CN 113723548 A CN113723548 A CN 113723548A
- Authority
- CN
- China
- Prior art keywords
- node
- rule
- num
- tree
- rules
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/24323—Tree-organised classifiers
Abstract
The invention relates to a method and a device for preprocessing a decision tree rule set, and belongs to the technical field of communication. The invention identifies a small number of large rules in the rule set by one-time decision tree pre-creation, and the user rule set is divided into two parts after being preprocessed by the rule set introduced in the patent: and a small number of large rules and a large number of small rules are used for establishing the decision tree, the decision tree information is stored in hardware, and the small number of large rules are searched in parallel by means of the hardware. And when the data packet is inquired in hardware and is matched with the rule, the data packet is respectively searched through the decision tree searching module and the parallel searching module. The invention obviously reduces the time for creating the decision tree, obviously reduces the RAM resources in the FPGA chip occupied by the decision tree nodes, lowers the tree height, reduces the corresponding logic resources and reduces the processing delay.
Description
Technical Field
The invention belongs to the field of technology, and particularly relates to a method and a device for preprocessing a decision tree rule set.
Background
The packet classification algorithm is one of the algorithms commonly used in networks. The method aims to divide the message into different types according to the header information carried by the message, generally 5 dimensions of a source IP, a destination IP, a source port, a destination port and a protocol number, so that the message can be processed differently. Common applications are: access control lists, firewalls, flow-based billing statistics. There are different applications in both core network devices and edge network devices. Decision trees are a common one of many packet classification algorithms.
The key point in packet classification by decision tree is the creation of the decision tree. The decision tree is created by dividing the rules of the parent node into a plurality of child nodes as evenly as possible according to the characteristics of the rule set. The root node has all the rules in the rule set, and the leaf nodes only contain the rules less than the specified number after the multi-level tree nodes are divided. Therefore, the purpose of dividing a large rule set into a plurality of small rule sets and searching the data packet in the specified small rule set is achieved. When searching the data packet, searching the lower level node from the root node according to the information when the decision tree is established until the leaf node, and then searching in a small rule set contained in the leaf node.
FIG. 1 is an example of a rule, with the fields shown in sequence: a source ip address value/source ip mask, a destination ip address value/destination ip mask, a source port number range, a destination port number range, and a protocol number value/protocol number mask value. Each rule represents a range, and a rule with a small mask represents a large range.
In a decision tree implementation, since a rule represents a range, the same rule may be divided into multiple children nodes when dividing the rule from a parent node to a child node, which is referred to as rule replication in this patent. Rule replication can lead to problems of lengthy decision tree creation time and excessive consumption of storage resources. After the decision tree is created, the number of times of copying each rule in the rule set is different. In order to ensure the creation efficiency of the decision tree, the rules with a large number of copying times can be removed from the rule set and then the decision tree can be created again. Rules with a high number of copies are referred to as large rules in this patent.
The large rule is expressed as: the first type: representing a wide range of rules. I.e. small ip mask, large range of port numbers and small protocol number mask. The range represented by rule 5 in fig. 2 is relatively large, and the number of copies is relatively large. The second type: rules of mutual coverage. Although the scope of a single regular expression is not large, the scope of multiple similar regular expressions cover each other. In fig. 3, the ranges represented by rules 1 to 5 are not large, but the ranges represented by the 5 rules are mutually overlapped and cannot be equally divided into different child nodes as much as possible, which also increases the number of times of rule replication. Therefore, it is necessary to determine a rule having a relatively large number of copies as a large rule based on the number of copies of these 5 rules.
The first large rule can be conveniently identified by the size of the ip mask and the size of the port number range. But it is not easy to identify several sets of rules that overlap each other among tens or even hundreds of thousands of rules, and to identify which rule in each overlapping set of rules is the large rule. The embodiment of the invention provides a method for preprocessing a rule set, which can accurately identify two types of large rules in the rule set. The rule set is preprocessed and then the decision tree is created, so that rule copying can be reduced, and the problems of too long creation time of the decision tree and too much consumption of storage resources are solved.
Disclosure of Invention
Technical problem to be solved
The technical problem to be solved by the invention is how to provide a method and a device for preprocessing a decision tree rule set so as to solve the problems of too long creation time of a decision tree and too much consumption of storage resources.
(II) technical scheme
In order to solve the above technical problem, the present invention provides a method for preprocessing a rule set of a decision tree, comprising the following steps:
step1, judging whether the number of the current child node rules is larger than the upper limit threshold value pre _ leaf _ rules _ max of the number of the leaf node rules, if so, executing step7 and the subsequent steps; otherwise step2 is executed;
step2, judging whether the child node still has effective division dimension; if the current child node does not have a valid division dimension, step7 and the following steps are executed; otherwise step3 is executed.
step3, judging whether the designated dimension still has the dividing bit; if there is no partition bit for the specified dimension, step7 and the following steps are executed; otherwise step4 is executed;
step4, judging whether the height of the current child node exceeds the tree height upper limit threshold value pre _ tree _ height _ max; if the tree height of the current child node is larger than or equal to the threshold value, executing step7 and the subsequent steps; otherwise, step5 and step6 are executed;
step 5: the number of copies of each rule of the child node is increased by 1, step 6: the child node is used as a new father node to continue to divide the rule step by step; after step6 is executed, the child nodes are used as new father nodes, step1 to step4 are executed again until all the nodes are divided by rules, and each child node can execute step7 and the subsequent steps;
step 7: the child node is set as a leaf node, step 8: the copy number of each rule of the child node is increased by 1, and the success of tree pre-building is marked.
Further, a plurality of thresholds are preset before the decision tree is created: tree height upper limit tree _ height _ max and leaf node rule number upper limit leaf _ rules _ max, and in the preprocessing stage, a new threshold value is set for the pre-built tree: the tree upper limit pre _ tree _ height _ max and the leaf node rule number upper limit pre _ leaf _ rules _ max, the tree upper limit pre _ tree _ height _ max is larger than the tree _ height _ max, and the leaf node rule number upper limit pre _ leaf _ rules _ max is larger than the leaf _ rules _ max.
Further, a count variable is maintained for each rule, all rules in a rule set in an initial state belong to a root node of a decision tree, all count variables are initialized to be zero, the rules are divided into lower-layer nodes layer by layer from the root node in the process of creating the tree until the number of the rules contained in the lower-layer nodes is smaller than the upper limit of the number of the rules of leaf nodes pre _ leaf _ rules _ max, and the nodes are marked as leaf nodes; after a rule is divided from a father node to a child node each time, the counting variable corresponding to the rule is automatically increased by one, namely the copying times of the rule are automatically increased by one.
Further, each node in the decision tree includes the following key information:
1) current node tree height tree _ height: representing the number of levels from the root node to the current node tree;
2) the partition dimension cut _ dim of the current node: the number of bits is 5, five dimensions of a protocol number, a destination port number, a source port number, a destination ip address and a source ip address are sequentially represented from bit 0 to bit 4, and when a certain bit of cut _ dim is 1, the corresponding dimension is used as a dividing dimension;
3) the current node uses several bits as partition bits cut _ num [5] in the specified partition dimension: expressed by an array containing 5 elements, the number of dividing bits of five dimensions of a protocol number, a destination port number, a source port number, a destination ip address and a source ip address is expressed from cut _ num [0] to cut _ num [4] in sequence;
4) on a path from a root node to a father node of a current node, the sum of the dividing bit numbers of each node is called the accumulated dividing bit number of the current node and is represented by an accu _ cut _ num [5], and the accumulated dividing bit numbers of five dimensions of a protocol number, a destination port number, a source port number, a destination ip address and a source ip address are sequentially represented from the accu _ cut _ num [0] to the accu _ cut _ num [4 ];
5) pointer children _ ptr to child node memory: a plurality of child nodes of a father node are stored in continuous memory blocks, and children _ ptr points to the head address of the memory block;
6) child node number child _ num: calculating the sum of cut _ num _ sum of the dividing bit numbers corresponding to the dividing dimensions, wherein child node number child _ num is as follows: cut _ num _ sum to the power of 2.
Further, the method for regularly dividing the parent node into the child nodes comprises the following steps: when the father node only uses one dimension i as a dividing dimension, dividing the bit number into cut _ num [ i ], and accumulating the dividing bit number into accumulatively dividing the bit number into accumu _ cut _ num [ i ]; when a certain rule r of a parent node needs to be divided into child nodes, a value of the rule from the bit of the first accu _ cut _ num [ i ] of the dimension i to the bit of the second accu _ cut _ num [ i ] + cut _ num [ i ] -1 is obtained, the value is a child node index value idx, and represents that the rule r is divided into the idx child nodes of the parent node, and when the mask value mask of the dimension i is smaller than the bit of the second accu _ cut _ num [ i ] + cut _ num [ i ], bits from the bit of the second mask to the bit of the second accu _ cut _ num [ i ] + cut _ num [ i ] of the dimension i are wildly matched, so the rule r is divided into the child nodes.
Further, the method for regularly dividing the parent node into the child nodes comprises the following steps: when two dimensions i and j are used as the dividing dimensions, acquiring a value IDxi of the dividing dimension i from the bit of the accum _ cut _ num [ i ] to the bit of the accum _ cut _ num [ i ] + the accumulation _ num [ i ] -1; obtaining a value IDXj of a division dimension j from a bit of accum _ num [ j ] to a bit of accum _ num [ j ] + accum _ num [ j ] -1; after the IDXj is shifted to the left by cut _ num [ i ] bits, carrying out bit OR operation on the IDXj and the IDxi to obtain an index value idx, wherein the index value idx is an index of a child node and represents that a rule r is divided into the idx child nodes of a father node; also due to the mask wildcard, rule r may be divided into multiple child nodes.
Further, the step2 is determined by the following method: the maximum bit number occupied by the data of each dimension is fixed, and when the cumulative dividing bit number accu _ cut _ num of each dimension of the child node is equal to the corresponding maximum bit number, the child node does not have an effective dividing dimension;
the step3 judging method comprises the following steps: if all regular masks are less than or equal to the cumulative dividing bit number accu _ cut _ num of the corresponding dimension in all the designated dividing dimensions, judging that no dividing bit exists in the designated dimension;
the step4 judging method comprises the following steps: the node information contains the tree height of the current node, and whether the tree height in the child node is larger than or equal to the tree height threshold pre _ tree _ height _ max is directly compared.
Further, step8 is followed by the following steps: after the tree is successfully pre-built, traversing each leaf node of the pre-built decision tree, if the rule number rule _ num contained in the leaf node is greater than the leaf node rule number threshold value leaf _ rule _ max when the decision tree is built, sorting the rules in the leaf nodes from small to large according to the rule copying times, and marking the rules from the leaf _ rule _ max +1 to the rule _ num-1 as large rules.
Further, after being preprocessed, the user rule set is divided into two parts: and a small number of large rules and a large number of small rules are used, the large number of small rules are used for establishing the decision tree, the decision tree information is stored in hardware and searched by a decision tree searching module, and the small number of large rules are searched in parallel by means of the hardware.
The invention also provides a device for preprocessing the rule set of the decision tree, which comprises a processor used for executing the method.
(III) advantageous effects
The invention provides a method and a device for preprocessing a decision tree rule set, which are found by comparing the time for creating a decision tree and the data occupied by a memory before and after preprocessing the rule set, and have the following obvious advantages:
1) the decision tree creation time is significantly reduced.
2) And the RAM resources in the FPGA chip occupied by the decision tree nodes are obviously reduced.
3) The tree height is reduced, corresponding logic resources are reduced, and processing delay is reduced.
Drawings
FIG. 1 is an example of a rule;
FIG. 2 is big rule 1: a wide range of rule examples;
FIG. 3 is big rule 2: examples of rules that overlap each other;
FIG. 4 is a memory layout of a parent node and a child node;
FIG. 5 is a rule set preprocessing flow;
fig. 6 is an overall process flow.
Detailed Description
In order to make the objects, contents and advantages of the present invention clearer, the following detailed description of the embodiments of the present invention will be made in conjunction with the accompanying drawings and examples.
The invention identifies a small number of large rules in the rule set through one-time decision tree pre-creation. Typically, multiple thresholds are preset before the decision tree is created: tree height upper limit tree _ height _ max, leaf node rule number upper limit leaf _ rules _ max, and the like. In the pre-tree building stage, in order to ensure the success of pre-tree building and reduce the pre-tree building time and storage resource occupation, a new threshold value is set for the pre-tree building: a tree upper limit pre _ tree _ height _ max (greater than tree _ height _ max) and a leaf node rule number upper limit pre _ leaf _ rules _ max (greater than leaf _ rules _ max).
Before the tree is pre-built, all the rules in the rule set are distributed to the root node. The process of decision tree pre-creation is similar to that of decision tree creation, and since the focus of this patent is the pre-processing of the rule set, the decision tree creation process is not described too much. The pre-creation of the decision tree is recursive, and the patent describes the process of single recursive processing in detail, so as to illustrate the principle of pre-creating the accurate marking large rule through the decision tree. The single recursion processing is a process of dividing the rule of the father node into the child nodes and then respectively processing each child node according to different thresholds.
First, a statistical method of the number of times of rule copying is explained: one count variable is maintained for each of all rules. All rules in the rule set in the initial state belong to the decision tree root node, and all count variables are initialized to zero. In the process of creating the decision tree, the rules are divided into lower-layer nodes layer by layer from the root node until the number of the rules contained in the lower-layer nodes is smaller than the upper limit of the leaf node rule number pre _ leaf _ rules _ max, and the nodes are marked as leaf nodes. After a rule is divided from a father node to a child node each time, the counting variable corresponding to the rule is automatically increased by one, namely the copying times of the rule are automatically increased by one.
Each node in the decision tree includes the following key information:
1) current node tree height tree _ height: representing the number of levels from the root node to the current node tree. The tree height of the root node is 0, the tree heights of the child nodes of the root node are 1, the tree heights of the grandchild nodes are 2, and so on.
2) The partition dimension cut _ dim of the current node: the total number of the bits is 5, and five dimensions of a protocol number, a destination port number, a source port number, a destination ip address and a source ip address are represented from bit 0 to bit 4 in sequence. When a certain bit of cut _ dim is 1, it indicates that the corresponding dimension is used as the division dimension.
The method for acquiring the division dimension comprises the following steps: different numbers of rules contained in the current node in each dimensionality are respectively stored in a plurality of groups diff _ num [5], the average value avg _ num of five members of the diff _ num [5] is calculated, the dimensionality with diff _ num [ i ] larger than avg _ num is taken as a dividing dimensionality, and at most three dimensionalities are selected.
3) The current node uses several bits as partition bits cut _ num [5] in the specified partition dimension: represented by an array containing 5 elements. And dividing bit numbers of five dimensions of a protocol number, a destination port number, a source port number, a destination ip address and a source ip address are sequentially expressed from cut _ num [0] to cut _ num [4 ].
The method for acquiring the division bit number comprises the following steps: trying different division bit numbers, dividing the father node rule into a plurality of child nodes, and calculating the standard deviation of the rule number of the child nodes as a basis for reflecting the average degree of the rule division. And taking the division bit with the highest average degree of the regular division as the final division bit number.
4) On the path from the root node to the parent node of the current node, the sum of the number of division bits of each node, called the cumulative number of division bits of the current node, is represented by accumu _ cut _ num [5 ]. And the accumulated dividing bit number of five dimensions of a protocol number, a destination port number, a source port number, a destination ip address and a source ip address is sequentially represented from the accu _ cut _ num [0] to the accu _ cut _ num [4 ].
5) Pointer children _ ptr to child node memory: child nodes of the parent node are stored in contiguous blocks of memory, and children _ ptr points to the head address of the block of memory.
6) Child node number child _ num: after the dividing dimension cut _ dim and the dividing bit number cut _ num [5] are determined, the number of child nodes is also determined.
The child node number calculation method comprises the following steps: calculating the sum of cut _ num _ sum of the dividing bit numbers corresponding to the dividing dimensions, wherein child node number child _ num is as follows: cut _ num _ sum to the power of 2. The memory layout of the parent node and the child node is shown in fig. 4. The indices of the child nodes are numbered from 0, and the index value of child node n in the figure is child _ num-1.
The method for regularly dividing the father node into the child nodes comprises the following steps:
1) when the father node only uses one dimension i as a dividing dimension, the dividing bit number is cut _ num [ i ], and the accumulated dividing bit number is cut _ num [ i ]. When a certain rule r of a parent node needs to be divided into child nodes, a value of the rule from the bit of the second accu _ cut _ num [ i ] to the bit of the second accu _ cut _ num [ i ] + cut _ num [ i ] -1 (note that the highest bit is marked as the 0 th bit, the second highest bit is marked as the 1 st bit, and the like) in the dimension i is obtained, the value is a child node index value idx, and the rule r is shown to be divided into the idx child node of the parent node. When the mask value mask of dimension i is smaller than accum _ num [ i ] + accum _ num [ i ], that is, the bits from the mask of dimension i to the accum _ num [ i ] + accum _ num [ i ] are wildcarded, the rule r is divided into a plurality of sub-nodes.
2) When the parent node uses a plurality of dimensions as the division dimension, the processing method is similar to when only one dimension is used. For convenience of description, it is assumed herein that two dimensions i and j are used as the division dimension. Obtaining a value IDxi from the bit of a dividing dimension i from the bit of an accu _ cut _ num [ i ] to the bit of an accu _ cut _ num [ i ] + cut _ num [ i ] -1; and obtaining a value IDXj of the division dimension j from the bit of the accum _ cut _ num [ j ] to the bit of the accum _ cut _ num [ j ] + cut _ num [ j ] -1. And after the IDXj is shifted to the left by cut _ num [ i ] bits, carrying out bit OR operation on the IDXj and the IDxi to obtain an index value idx, wherein the index value idx is the index of the child node and represents that the rule r is divided into the idx child nodes of the father node. Also due to the mask wildcard, rule r may be divided into multiple child nodes. The case of using 3 dimensions as the division dimensions is similar to the case of using 2 dimensions, and is not described again.
3) And for other rules in the father node, adopting the same processing method as the rule r and sequentially dividing the rules into the child nodes.
The rule set preprocessing flow is shown in fig. 5. The figure details the processing flow after dividing the parent node rule into the child nodes.
1) step1, judging whether the rule number of the current child node is larger than a threshold value pre _ leaf _ rules _ max, if so, executing step7 and the subsequent steps; otherwise step2 is executed.
2) step2, determine if the child node still has a valid partition dimension.
The judging method comprises the following steps: the maximum number of bits occupied by data for each dimension is fixed. Such as: the source ip and the destination ip occupy 32 bits respectively, the source port and the destination port occupy 16 bits respectively, and the protocol number occupies 8 bits. When the cumulative number of partitioning bits accu _ cut _ num of each dimension of a child node is equal to the corresponding maximum number of bits, there is no effective partitioning dimension for the child node.
The reason why the effective division dimension is used up and the rules are not divided is that the rules overlapped with each other are too many, and at this time, the current child node needs to be marked as a leaf node in advance.
If the current child node does not have a valid division dimension, step7 and the following steps are executed; otherwise step3 is executed.
3) step3, determine if the partition bit still exists for the specified dimension.
The judging method comprises the following steps: and if all regular masks are less than or equal to the cumulative dividing bit number accu _ cut _ num of the corresponding dimension in all the designated dividing dimensions, judging that no dividing bit exists in the designated dimension.
And no effective division bit exists, which indicates that the current child node contains more large rules, and at this time, the current child node needs to be marked as a leaf node in advance.
If there is no partition bit for the specified dimension, step7 and the following steps are executed; otherwise step4 is executed.
4) step4, determine if the height of the current child node exceeds the tree height threshold pre _ tree _ height _ max.
The judging method comprises the following steps: the node information includes a tree height of the current node. And directly comparing whether the tree _ height in the child node is more than or equal to a tree high threshold pre _ tree _ height _ max.
To ensure the search efficiency, the height of the decision tree needs to be limited. When the tree height of the current child node is greater than or equal to the tree height threshold pre _ tree _ height _ max, the current child node needs to be marked as a leaf node in advance.
If the tree height of the current child node is larger than or equal to the threshold value, executing step7 and the subsequent steps; otherwise step5 and step6 are executed.
5) step 5: the number of copies of each rule of the child node is increased by 1, step 6: the child node is used as a new father node to continue to divide the rule step by step; after step6 is executed, the child node becomes the new parent node. Step 1) to step 4) are performed again. Until all nodes are divided by rules, each child node can execute step7 and the following steps, step 7: this node becomes a leaf node, step 8: the number of copies per rule of the node is increased by 1. Marking the success of tree pre-building.
6) After the tree is successfully pre-built, traversing each leaf node of the pre-built decision tree, if the rule number rule _ num contained in the leaf node is greater than the leaf node rule number threshold value leaf _ rule _ max when the decision tree is built, sorting the rules in the leaf nodes from small to large according to the rule copying times, and marking the rules from the leaf _ rule _ max +1 to the rule _ num-1 as large rules.
7) In the rule set with practical significance, the number of large rules is small, the large rules can be accurately identified through the embodiment, and the small large rules are searched in parallel through hardware.
8) The large rules have been marked when the rule set was preprocessed. The large number of rules that are unlabeled act as a rule set at the time of decision tree creation.
The rule set preprocessing is only needed to be performed once, and the preprocessed rule set can be used in subsequent possible multiple decision tree creations.
A small number of large rules separated from the original rule set are searched in parallel through hardware; the remaining large number of small rules are looked up through the decision tree. The whole processing flow is as shown in FIG. 6:
after the rule set introduced by the patent is preprocessed, the user rule set is divided into two parts: a small number of large rules and a large number of small rules. And (3) using a large number of small rules to create the decision tree, and saving the decision tree information to hardware (such as FPGA). A small number of large rules require parallel lookups by hardware. And when the data packet is inquired in hardware and is matched with the rule, the data packet is respectively searched through the decision tree searching module and the parallel searching module. If the two modules have a plurality of rules matched with the data packet, the rule with the highest priority is output as the final matching rule through the priority arbitration module.
By comparing the time before and after the rule set preprocessing, the time for creating the decision tree and the memory occupation data are found, the scheme has the following obvious advantages:
1) the decision tree creation time is significantly reduced.
2) And the RAM resources in the FPGA chip occupied by the decision tree nodes are obviously reduced.
3) The tree height is reduced, corresponding logic resources are reduced, and processing delay is reduced.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A method for preprocessing a rule set of a decision tree, the method comprising the steps of:
step1, judging whether the number of the current child node rules is larger than the upper limit threshold value pre _ leaf _ rules _ max of the number of the leaf node rules, if so, executing step7 and the subsequent steps; otherwise step2 is executed;
step2, judging whether the child node still has effective division dimension; if the current child node does not have a valid division dimension, step7 and the following steps are executed; otherwise step3 is executed.
step3, judging whether the designated dimension still has the dividing bit; if there is no partition bit for the specified dimension, step7 and the following steps are executed; otherwise step4 is executed;
step4, judging whether the height of the current child node exceeds the tree height upper limit threshold value pre _ tree _ height _ max; if the tree height of the current child node is larger than or equal to the threshold value, executing step7 and the subsequent steps; otherwise, step5 and step6 are executed;
step 5: the number of copies of each rule of the child node is increased by 1, step 6: the child node is used as a new father node to continue to divide the rule step by step; after step6 is executed, the child nodes are used as new father nodes, step1 to step4 are executed again until all the nodes are divided by rules, and each child node can execute step7 and the subsequent steps;
step 7: the child node is set as a leaf node, step 8: the copy number of each rule of the child node is increased by 1, and the success of tree pre-building is marked.
2. The method of decision tree rule set preprocessing of claim 1 wherein the decision tree is pre-set with a plurality of thresholds before creation: tree height upper limit tree _ height _ max and leaf node rule number upper limit leaf _ rules _ max, and in the preprocessing stage, a new threshold value is set for the pre-built tree: the tree upper limit pre _ tree _ height _ max and the leaf node rule number upper limit pre _ leaf _ rules _ max, the tree upper limit pre _ tree _ height _ max is larger than the tree _ height _ max, and the leaf node rule number upper limit pre _ leaf _ rules _ max is larger than the leaf _ rules _ max.
3. The method of claim 1, wherein a count variable is maintained for each rule, all rules in the rule set in an initial state belong to a root node of the decision tree, and all count variables are initialized to zero, and during the process of creating the tree, the rules are divided layer by layer from the root node to a lower node until the number of rules contained in the lower node is less than an upper limit of the number of rules of leaf nodes pre _ leaf _ rules _ max, and the node is marked as a leaf node; after a rule is divided from a father node to a child node each time, the counting variable corresponding to the rule is automatically increased by one, namely the copying times of the rule are automatically increased by one.
4. The method of decision tree rule set preprocessing of claim 1 wherein each node in the decision tree includes the following key information:
1) current node tree height tree _ height: representing the number of levels from the root node to the current node tree;
2) the partition dimension cut _ dim of the current node: the number of bits is 5, five dimensions of a protocol number, a destination port number, a source port number, a destination ip address and a source ip address are sequentially represented from bit 0 to bit 4, and when a certain bit of cut _ dim is 1, the corresponding dimension is used as a dividing dimension;
3) the current node uses several bits as partition bits cut _ num [5] in the specified partition dimension: expressed by an array containing 5 elements, the number of dividing bits of five dimensions of a protocol number, a destination port number, a source port number, a destination ip address and a source ip address is expressed from cut _ num [0] to cut _ num [4] in sequence;
4) on a path from a root node to a father node of a current node, the sum of the dividing bit numbers of each node is called the accumulated dividing bit number of the current node and is represented by an accu _ cut _ num [5], and the accumulated dividing bit numbers of five dimensions of a protocol number, a destination port number, a source port number, a destination ip address and a source ip address are sequentially represented from the accu _ cut _ num [0] to the accu _ cut _ num [4 ];
5) pointer children _ ptr to child node memory: a plurality of child nodes of a father node are stored in continuous memory blocks, and children _ ptr points to the head address of the memory block;
6) child node number child _ num: calculating the sum of cut _ num _ sum of the dividing bit numbers corresponding to the dividing dimensions, wherein child node number child _ num is as follows: cut _ num _ sum to the power of 2.
5. The method of decision tree rule set preprocessing of claim 4 wherein the method of parent node rule partitioning to child nodes comprises: when the father node only uses one dimension i as a dividing dimension, dividing the bit number into cut _ num [ i ], and accumulating the dividing bit number into accumulatively dividing the bit number into accumu _ cut _ num [ i ]; when a certain rule r of a parent node needs to be divided into child nodes, a value of the rule from the bit of the first accu _ cut _ num [ i ] of the dimension i to the bit of the second accu _ cut _ num [ i ] + cut _ num [ i ] -1 is obtained, the value is a child node index value idx, and represents that the rule r is divided into the idx child nodes of the parent node, and when the mask value mask of the dimension i is smaller than the bit of the second accu _ cut _ num [ i ] + cut _ num [ i ], bits from the bit of the second mask to the bit of the second accu _ cut _ num [ i ] + cut _ num [ i ] of the dimension i are wildly matched, so the rule r is divided into the child nodes.
6. The method of decision tree rule set preprocessing of claim 4 wherein the method of parent node rule partitioning to child nodes comprises: when two dimensions i and j are used as the dividing dimensions, acquiring a value IDxi of the dividing dimension i from the bit of the accum _ cut _ num [ i ] to the bit of the accum _ cut _ num [ i ] + the accumulation _ num [ i ] -1; obtaining a value IDXj of a division dimension j from a bit of accum _ num [ j ] to a bit of accum _ num [ j ] + accum _ num [ j ] -1; after the IDXj is shifted to the left by cut _ num [ i ] bits, carrying out bit OR operation on the IDXj and the IDxi to obtain an index value idx, wherein the index value idx is an index of a child node and represents that a rule r is divided into the idx child nodes of a father node; also due to the mask wildcard, rule r may be divided into multiple child nodes.
7. The method for decision tree rule set preprocessing of any one of claims 1-6,
the step2 judging method comprises the following steps: the maximum bit number occupied by the data of each dimension is fixed, and when the cumulative dividing bit number accu _ cut _ num of each dimension of the child node is equal to the corresponding maximum bit number, the child node does not have an effective dividing dimension;
the step3 judging method comprises the following steps: if all regular masks are less than or equal to the cumulative dividing bit number accu _ cut _ num of the corresponding dimension in all the designated dividing dimensions, judging that no dividing bit exists in the designated dimension;
the step4 judging method comprises the following steps: the node information contains the tree height of the current node, and whether the tree height in the child node is larger than or equal to the tree height threshold pre _ tree _ height _ max is directly compared.
8. The method for decision tree rule set preprocessing of claim 7 wherein step8 is followed by the further steps of: after the tree is successfully pre-built, traversing each leaf node of the pre-built decision tree, if the rule number rule _ num contained in the leaf node is greater than the leaf node rule number threshold value leaf _ rule _ max when the decision tree is built, sorting the rules in the leaf nodes from small to large according to the rule copying times, and marking the rules from the leaf _ rule _ max +1 to the rule _ num-1 as large rules.
9. The method of decision tree rule set preprocessing as set forth in claim 8, wherein the user rule set is divided into two parts after preprocessing: and a small number of large rules and a large number of small rules are used, the large number of small rules are used for establishing the decision tree, the decision tree information is stored in hardware and searched by a decision tree searching module, and the small number of large rules are searched in parallel by means of the hardware.
10. An apparatus for decision tree rule set preprocessing, the apparatus comprising a processor configured to perform the method of any of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111038482.6A CN113723548A (en) | 2021-09-06 | 2021-09-06 | Method and device for preprocessing decision tree rule set |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111038482.6A CN113723548A (en) | 2021-09-06 | 2021-09-06 | Method and device for preprocessing decision tree rule set |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113723548A true CN113723548A (en) | 2021-11-30 |
Family
ID=78681952
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111038482.6A Pending CN113723548A (en) | 2021-09-06 | 2021-09-06 | Method and device for preprocessing decision tree rule set |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113723548A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114638309A (en) * | 2022-03-21 | 2022-06-17 | 北京左江科技股份有限公司 | Hypercuts decision tree strategy set preprocessing method based on information entropy |
| CN114745453A (en) * | 2022-03-21 | 2022-07-12 | 北京左江科技股份有限公司 | Method for merging entropy binary tree into multi-branch tree |
-
2021
- 2021-09-06 CN CN202111038482.6A patent/CN113723548A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114638309A (en) * | 2022-03-21 | 2022-06-17 | 北京左江科技股份有限公司 | Hypercuts decision tree strategy set preprocessing method based on information entropy |
| CN114745453A (en) * | 2022-03-21 | 2022-07-12 | 北京左江科技股份有限公司 | Method for merging entropy binary tree into multi-branch tree |
| CN114638309B (en) * | 2022-03-21 | 2024-04-09 | 北京左江科技股份有限公司 | Information entropy-based hypercust decision tree strategy set preprocessing method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8937954B2 (en) | Decision tree level merging | |
| US9208438B2 (en) | Duplication in decision trees | |
| US9432284B2 (en) | Method and apparatus for compiling search trees for processing request keys based on a key size supported by underlying processing elements | |
| US20160071016A1 (en) | Scope In Decision Trees | |
| US8375433B2 (en) | Method for multi-core processor based packet classification on multiple fields | |
| US8732110B2 (en) | Method and device for classifying a packet | |
| US7684400B2 (en) | Logarithmic time range-based multifield-correlation packet classification | |
| US9521082B2 (en) | Methods and devices for creating, compressing and searching binary tree | |
| US9595003B1 (en) | Compiler with mask nodes | |
| US7386525B2 (en) | Data packet filtering | |
| CN100472480C (en) | Reverse path forwarding protection | |
| CN113723548A (en) | Method and device for preprocessing decision tree rule set | |
| US9268855B2 (en) | Processing request keys based on a key size supported by underlying processing elements | |
| CN104579941A (en) | Message classification method in OpenFlow switch | |
| US6970971B1 (en) | Method and apparatus for mapping prefixes and values of a hierarchical space to other representations | |
| CN114900474A (en) | Data packet classification method, system and related equipment for programmable switch | |
| US7546281B2 (en) | Reduction of ternary rules with common priority and actions | |
| WO2019183962A1 (en) | Method for classifying network packet on basis of equal length and equal density segmentation | |
| CN113810311A (en) | Data packet classification method based on multiple decision trees | |
| Donnelly et al. | IP route lookups as string matching | |
| CN114638309B (en) | Information entropy-based hypercust decision tree strategy set preprocessing method | |
| CN113723549A (en) | Method for realizing decision tree based on bitmap | |
| Abdulhassan et al. | Many-field packet classification using CR-tree | |
| Ashok et al. | Fast and efficient lookups via data-driven FIB designs | |
| KR101583439B1 (en) | Packet classification method and packet classification apparatus using an area-based quad trie with leaf-pushing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |