CN112699097B - Method, device and storage medium for realizing multi-element policy mirror image - Google Patents
Method, device and storage medium for realizing multi-element policy mirror image Download PDFInfo
- Publication number
- CN112699097B CN112699097B CN202011619775.9A CN202011619775A CN112699097B CN 112699097 B CN112699097 B CN 112699097B CN 202011619775 A CN202011619775 A CN 202011619775A CN 112699097 B CN112699097 B CN 112699097B
- Authority
- CN
- China
- Prior art keywords
- matching
- strategy
- policy
- field
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 91
- 238000010276 construction Methods 0.000 claims abstract description 7
- 238000012545 processing Methods 0.000 claims description 19
- 238000010606 normalization Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 description 23
- 230000014509 gene expression Effects 0.000 description 10
- 230000006870 function Effects 0.000 description 7
- 230000009471 action Effects 0.000 description 5
- 238000004590 computer program Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 239000002699 waste material Substances 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 239000002994 raw material Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a method, a device and a storage medium for realizing a multi-element strategy mirror image, wherein the method for realizing the multi-element strategy mirror image comprises the following steps: establishing a strategy database, wherein the strategy database comprises db data, strategy id and strategy field id; receiving flow data and a matching database; matching the first part of content in the streaming data by using a strategy database and a matching database for the first time; matching for the second time, namely matching the second part of content in the streaming data by using a strategy database and a matching database; matching for the third time, namely matching a third part of content in the streaming data by using a strategy database and a matching database; and receiving results of the first matching, the second matching and the third matching, and generating a construction mirror image message. The invention can realize the mirror image function aiming at the flow in the network, and match each field through the multivariate strategy to obtain a matching result.
Description
Technical field:
the present invention relates to the field of computer technologies, and in particular, to a method and apparatus for implementing a multi-policy image, and a storage medium.
The background technology is as follows:
the current mirror image implementation method for the multi-element strategy in the technical field of computers generally comprises the following steps: the method for mirroring policy matching based on efficient string matching algorithms such as KMP and BM algorithm, the method for mirroring based on finite automaton algorithm, the method for mirroring based on Naive String Matching Algorithm violent matching and the method for mirroring based on hyperscan hardware instruction of intel open source are generally adopted, but the flow mirroring function achieved by the algorithm usually adopts a similar architecture, namely, a policy set is firstly constructed, then various algorithms are used for carrying out related matching on relevant fields of the message, and finally, a mirroring message is constructed according to the matching result to achieve the flow mirroring function. The structure is complex in matching unit strategies, aiming at matching of multiple strategies, the result can be obtained only by completing the matching of the strategies in each dimension in all strategies, whether the flow needs to be mirrored or not is judged according to the matching result, and then the multiple strategies are classified according to the unit strategies in the multiple strategies or the multiple strategies are matched after the unit strategies are classified and quantized again, so that the realization efficiency is low, and the redundancy in the realization process is large.
Therefore, there is a need in the art for a method, apparatus, and storage medium for implementing multiple policy mirroring.
In view of this, the present invention has been proposed.
The invention comprises the following steps:
the invention aims to provide a method, a device and a storage medium which can be suitable for realizing multi-element policy mirror images, so as to solve at least one technical problem in the prior art.
In a first aspect of the present invention, a method for implementing a multivariate policy image is provided.
Specifically, the method for implementing the multi-element policy mirror image comprises the following steps:
the method for establishing the strategy database comprises a matching method, an offset, a matching length matchlen, a strategy id, a strategy field id, a strategy bitmap and various strategy field bitmaps, wherein the specific steps comprise,
a policy configuration is received and the policy configuration is received,
according to the policy configuration, parse the policy field,
according to the type of the policy field, respectively processing and generating a matching method, an offset, a matching length match, a policy id, a policy field id, a policy bitmap and various types of policy fields bitmap to construct a policy database,
when the type of the strategy field is a host strategy field and a url strategy field, analyzing the strategy field to obtain a matching method, an offset and a matching length match, obtaining a key matching word, setting a bitmap of the strategy to which the strategy field belongs, setting the host or url type strategy field bitmap of the strategy, using hyperscan logic combination to record strategy id, compiling and generating deserialized db data,
when the policy field type is a payload policy field, resolving the policy field to obtain a matching method, an offset and a matching length match, obtaining a key matching word, setting a bitmap of a policy to which the policy field belongs, setting a policy field bitmap of the payload type of the policy,
judging whether the types of the load offset and the packet tail forward offset in the strategy field are the same offset type, if yes, compiling and generating deserialized db data by using hyperscan logic combination, recording strategy ids, if not, respectively generating different types of deserialized db data by using hyperscan, recording strategy ids, and respectively recording strategy field ids of different offset types;
receiving flow data and a matching database;
the method comprises the steps of performing first matching, selecting data in host format by using a strategy database and a matching database, and matching a first part of content in flow data according to db data to obtain a first matching result, wherein the first part of content is host content;
a policy database and a matching database are used for second matching, data in a payload format is selected, and second partial content in the flow data is matched according to db data to obtain a second matching result, wherein the second partial content is payload content;
thirdly matching, namely selecting url format data by using a strategy database and a matching database, and matching a third part of content in the flow data according to db data to obtain a third matching result, wherein the third part of content is url content; and receiving results of the first matching, the second matching and the third matching, and generating a construction mirror image message.
By adopting the scheme, the mirror image function can be realized aiming at the flow in the network, and the matching of each field in the message is realized through the multi-element strategy in the mirror image process to obtain the matching result, so that a plurality of complex conditions in the current network environment can be effectively solved, the corresponding mirror image action is carried out on each field in the message, the matching efficiency of the multi-element strategy under the multi-dimension is improved, the mirror image efficiency is improved, and the processing is accelerated to improve the performance.
Preferably, the method for implementing the multi-element policy mirror further comprises the steps of: and establishing a matching database, wherein the matching database comprises db data formed by a matching method, offset and a key matching word.
Preferably, the policy configuration includes a matching method, an offset, a matching length, a policy id, a policy field type, and a key matching word.
By adopting the scheme, the policy configuration in the invention refers to a process which is guided according to the matching mode expected by the user and comprises the expected matching method, offset, matching length, policy id, policy field type, key matching words and other contents to form a raw material structure, the implementation mode of the process can be issued through a command line or other modes, and a policy database and a matching database used in the actual matching process are both generated by the raw material structure.
Preferably, the policy database creating step includes:
receiving a policy configuration;
analyzing a strategy field according to strategy configuration;
and respectively processing and generating a matching method, an offset, a matching length match, a policy id, a policy field id, a policy bitmap and various types of policy fields bitmap according to the type of the policy field, and constructing a policy database.
Further, the policy field includes a host policy field, a payload policy field, and a url policy field.
Further, the step of generating db data, the policy id and the policy field id according to the policy field type includes:
judging the type of the strategy field, classifying the host strategy field and the url strategy field into a first category, and classifying the payload strategy field into a second category.
Further, in the step of generating db data, the policy id and the policy field id according to the policy field type, the method further includes:
when the type of the strategy field is a host strategy field and a url strategy field, analyzing the strategy field to obtain a matching method, an offset and a matching length match, obtaining a key matching word, setting a bitmap of the strategy to which the strategy field belongs, setting the host or url type strategy field bitmap of the strategy, recording strategy id by using hyperscan logic combination, compiling and generating deserialized db data.
Further, in the step of generating db data, the policy id and the policy field id according to the policy field type, the method further includes:
when the type of the policy field is a payload policy field, analyzing the policy field to obtain a matching method, an offset and a matching length match, obtaining a key matching word, setting a bitmap of a policy to which the policy field belongs, and setting the bitmap of the policy field of the payload type of the policy;
judging whether the types of the load offset and the packet tail forward offset in the strategy field are the same offset type, if yes, compiling and generating deserialized db data by using hyperscan logic combination, recording strategy ids, if not, respectively generating different types of deserialized db data by using hyperscan, recording strategy ids, and respectively recording strategy field ids of different offset types.
By adopting the scheme, the load heads such as tcp, udp, ipv and ipv6 in the load fields can be effectively distinguished, so that the problems of head-tail offset, various matching methods, matching length and the like are improved, the processing efficiency of the multi-element strategy under the formed multi-dimension is improved, and the situation that the combination of single strategies is too complex is prevented.
Preferably, in the step of receiving the traffic data and the matching database, the matching database is Hyperscan, and the Hyperscan is a multiple regular expression matching library based on an intel hardware instruction set, which follows the regular expression grammar of the libpcre library, can use a hybrid automaton technology, allows a large number of regular expressions to be matched at the same time, and matches the regular expressions across data streams.
Further, the db data is reverse-serialized database data generated using hyperscan for use in matching.
Further, in the step of receiving the traffic data and matching the database, the method further includes: carrying out structural normalization on the flow data to obtain a normalization result, wherein the normalization result comprises the following steps: host format, payload format, url format.
By adopting the scheme, the mirror image realization efficiency can be effectively improved.
Preferably, the first matching step includes:
and selecting data in a host format, and matching the host content according to db data to obtain a first matching result.
Further, the first matching result comprises matching success and matching failure.
Further, in the first matching step, the method further includes:
when the first matching result is that the matching fails, ending the first matching step; when the first matching result is successful matching, recording the strategy id of successful matching.
Preferably, the second matching step includes:
and selecting data in the payload format, and matching the payload content according to db data to obtain a second matching result.
Further, the second matching result comprises matching success and matching failure.
Further, in the second matching step, the method further includes:
when the second matching result is that the matching fails, ending the second matching step; and when the second matching result is successful, performing payload secondary matching.
Further, the step of matching the payload twice includes:
recording the number natcnum of the fields successfully matched;
circularly confirming a specific matching strategy field id and strategy id in the natcnum matching fields;
collecting match to and a match length value of a successful field, judging whether the field is of a data packet tail forward offset type, if so, adding an offset match Len into the match length value, and if not, adding no offset match Len into the match length value;
judging whether the matching success field is smaller than or equal to the matching length value, if so, successfully matching the key matching words of the strategy field, setting the bitmap of the type of strategy field after the matching is successful, if not, failing to match the strategy field, and carrying out the next step;
and taking the bitmap in the policy database as an original object of policy arbitration, and performing policy arbitration on the bitmap successfully obtained by matching the key matching words of the policy fields, namely bit and operation, so as to obtain a payload secondary matching result.
Further, the payload secondary matching result comprises matching success and matching failure.
Further, in the step of payload secondary matching, the method further includes:
when the secondary matching result of the payload is successful, recording the strategy id of successful matching;
when the secondary matching result of the payload is a matching failure, the steps are re-entered: and (3) circularly confirming a specific matching strategy field id and strategy id in the natcnum matching fields.
Preferably, the third matching step includes:
and selecting url format data, and matching url contents according to db data to obtain a third matching result.
Further, the third matching result comprises matching success and matching failure.
Further, in the third matching step, the method further includes:
ending the third matching step when the third matching result is a matching failure; and when the third matching result is that the matching is successful, recording the strategy id of the successful matching.
Preferably, the step of receiving the results of the first matching, the second matching and the third matching and generating the mirror image message includes: and receiving the strategy id of successful matching obtained by the first matching, the second matching and the third matching, and generating a construction mirror image message.
By adopting the scheme, the interference of the combination condition of each field in the strategy can be reduced, each field in the strategy is divided into multiple dimensions, the fields in each dimension are subjected to type as granularity, the matching result is confirmed by taking the granularity as the lowest matching requirement according to the matching requirement of a plurality of fields under the granularity, and then whether the mirror image flow is needed is confirmed, the matching period is obviously reduced, the matching efficiency of the multi-element strategy under the multiple dimensions is improved, the mirror image efficiency is improved, the processing speed is accelerated, and the waste of computing resources is reduced.
In a second aspect of the present invention, a device for implementing multiple policy mirroring of network traffic is provided.
The file transmission device written by the multiple data sources comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the multi-element strategy mirror image realization method when executing the program.
In a third aspect of the present invention, a storage medium is provided.
Specifically, the storage medium has a computer program stored thereon, which when executed by a processor implements the above-described multi-policy mirroring implementation method.
In summary, the invention has the following beneficial effects:
1. the scheme provided by the invention can realize the mirror image function aiming at the flow in the network, and the matching of each field in the message is realized through the multi-element strategy in the mirror image process to obtain the matching result, so that a plurality of complex situations in the current network environment can be effectively solved, the corresponding mirror image action is carried out on each field in the message, the matching efficiency of the multi-element strategy under the multi-dimension is improved, the mirror image efficiency is improved, and the processing is accelerated to improve the performance;
2. the scheme provided by the invention can effectively distinguish the load heads such as tcp, udp, ipv, ipv6 and the like in the load fields respectively, so that the problems of head-tail offset, various matching methods, matching length and the like are improved, the processing efficiency of the multi-element strategy under the formed multi-dimension is improved, and the situation that the combination of single strategies is too complex is prevented;
3. the scheme provided by the invention can reduce the interference of the combination condition of each field in the strategy, divide each field in the strategy in a multi-dimensional way, take the type as granularity for each field in each dimension, confirm the matching result by taking the granularity as the lowest matching requirement according to the matching requirement of a plurality of fields under the granularity, further confirm whether mirror image flow is needed, obviously reduce the matching period, improve the matching efficiency of the multi-element strategy under the multi-dimension way, thereby improving the mirror image efficiency, accelerating the processing speed and reducing the waste of computing resources.
Description of the drawings:
in order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of one embodiment of a method for implementing a multiple policy image of the present invention;
FIG. 2 is a flowchart illustrating the steps for creating a policy database according to the present invention;
FIG. 3 is a flow chart of a second matching step in the present invention.
The specific embodiment is as follows:
reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. The implementations described in the following exemplary examples do not represent all implementations consistent with the invention. Rather, they are merely examples of apparatus and methods consistent with aspects of the invention as detailed in the accompanying claims.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.
The present invention will be described in detail by examples.
The invention is as follows:
compiling: compiling policy configuration of a command line or other interfaces to generate db data;
policy bitmap: the method is characterized in that a bitmap recorded in each strategy field of a single strategy is used for strategy arbitration after strategy configuration of a command line or other interfaces is analyzed in compiling;
policy field bitmap: the policy fields with the same dimension may have multiple policy fields, for example, the payload policy field may configure m policy fields with packet tails offset forward, and n policy fields with tcp payload heads offset backward, so as to form multiple policy fields with the same dimension, and the configuration may be recorded for policy field arbitration;
match length match len: refers to the matched length of the expected policy field type in the message;
load head matching type: the front offset of tcp/icmp/udp/ipv4/ipv6 and packet tail is included, which refers to the beginning of the matched content in the packet;
key matching words: the method is to refer to an object to be matched of a field to be matched in a data message;
matchto: and when the hyperscan executes the matching action, the returned matching end position after the original data is successfully matched is used for comparing the match length, namely the set expected matching length, in the strategy database, so as to judge whether the key matching words of the type strategy field are actually successfully matched.
The invention provides a method for realizing multi-element policy mirror image.
As shown in fig. 1, the method for implementing the multi-element policy mirror image includes the following steps:
s100, establishing a strategy database, wherein the strategy database comprises a matching method, an offset, a matching length match, a strategy id, a strategy field id, a strategy bitmap and various strategy fields bitmaps;
in the specific implementation process, the step S100 of establishing the policy database includes:
s110, receiving strategy configuration;
s120, analyzing a strategy field;
s130, respectively processing and generating db data, a strategy id and a strategy field id according to the strategy field type;
s140, receiving db data, a strategy id and a strategy field id, and generating a strategy database.
In a specific implementation process, the policy field includes a host policy field, a payload policy field, and a url policy field, and the step of processing and generating db data, a policy id, and a policy field id according to a policy field type includes: judging the type of the strategy field, classifying the host strategy field and the url strategy field into a first category, and classifying the payload strategy field into a second category.
In some preferred embodiments of the present invention, the step of generating db data, policy id, and policy field id according to the policy field type, respectively, further includes:
s131, when the type of the strategy field is a host strategy field and a url strategy field, analyzing the strategy field to obtain a matching method, an offset and a matching length match len, obtaining a key matching word, setting a bitmap of the strategy to which the strategy field belongs, setting the bitmap of the host or url type strategy field of the strategy, recording strategy ids by using hyperscan logic combination, compiling and generating deserialized db data.
As shown in fig. 2, in some preferred embodiments of the present invention, the step of generating db data, a policy id, and a policy field id according to the policy field type is further performed by the steps of:
s132, when the type of the policy field is a payload policy field, analyzing the policy field to obtain a matching method, an offset and a matching length match length, obtaining a key matching word, setting a bitmap of a policy to which the policy field belongs, and setting a host or url type policy field bitmap of the policy;
s133, judging whether the types of the load offset and the packet tail forward offset in the strategy field are the same offset type, if so, recording strategy ids by using hyperscan logic combination, compiling and generating deserialized db data, and if not, respectively generating different types of deserialized db data by using hyperscan, recording strategy ids and respectively recording strategy field ids of different offset types.
By adopting the scheme, the load heads such as tcp, udp, ipv and ipv6 in the load fields can be effectively distinguished, so that the problems of head-tail offset, various matching methods, matching length and the like are improved, the processing efficiency of the multi-element strategy under the formed multi-dimension is improved, and the situation that the combination of single strategies is too complex is prevented.
S199, establishing a matching database;
in the implementation process, the matching database comprises db data composed of a matching method, an offset and a key matching word.
S200, receiving flow data and a matching database;
in the implementation process, in the step of receiving flow data and matching a database, the matching database is Hyperscan, which is a multiple regular expression matching library based on an intel hardware instruction set, follows the regular expression grammar of a libpcre library, can use a hybrid automaton technology, allows a large number of regular expressions to be matched simultaneously, and matches the regular expressions across data streams, and the db data is inverse-sequenced database data generated by using Hyperscan, which is used for matching.
In some preferred embodiments of the present invention, the step s200 of receiving traffic data and matching a database further includes: carrying out structural normalization on the flow data to obtain a normalization result, wherein the normalization result comprises the following steps: host format, payload (data) format, url format. By adopting the scheme, the mirror image realization efficiency can be effectively improved.
S300, matching is carried out for the first time, and a strategy database and a matching database are used for matching the first part of content in the streaming data;
in a specific implementation process, the step s300 of first matching includes:
and selecting data in a host format, and matching the host content according to db data to obtain a first matching result, wherein the first matching result comprises matching success and matching failure.
Specifically, in the step s300, the first matching step further includes:
when the first matching result is that the matching fails, ending the first matching step; when the first matching result is successful matching, recording the strategy id of successful matching.
S400, matching the second part of content in the streaming data by using a strategy database and a matching database for the second time;
as shown in fig. 3, in the implementation process, the step s400 of the second matching includes:
s410, selecting data in a payload format, and matching payload contents according to db data to obtain a second matching result, wherein the second matching result comprises matching success and matching failure.
Specifically, in the step s400, the step of second matching further includes:
when the second matching result is that the matching fails, ending the second matching step; and when the second matching result is successful, performing payload secondary matching.
Further, in the step of s420.Payload secondary matching, the method includes:
s421, recording the number natcnum of the fields successfully matched;
s422, circularly confirming a specific matching strategy field id and strategy id in the natcnum matching fields;
s423, collecting matching end position matchto and matching length values of a successful field, judging whether the field is of a data packet tail forward offset type, if so, adding an offset matchLen into the matching length value, and if not, adding no offset matchLen into the matching length value;
s424, judging whether the matching success field is smaller than or equal to the matching length value, if so, considering that the key matching words of the strategy field are successfully matched, setting the bitmap of the type of strategy field after the matching is successful, if not, considering that the strategy field is actually matched to fail, and carrying out the next step;
s425, taking the bitmap in the policy database as an original object of policy arbitration, and performing policy arbitration, namely bit and operation, on the bitmap successfully obtained by matching the key matching words of the policy fields to obtain a payload secondary matching result, wherein the payload secondary matching result comprises successful matching and failed matching.
In some preferred embodiments of the present invention, the payload secondary matching step further includes:
when the secondary matching result of the payload is successful, recording the strategy id of successful matching;
when the secondary matching result of the payload is a matching failure, the steps are re-entered: and (3) circularly confirming a specific matching strategy field id and strategy id in the natcnum matching fields.
S500, matching is carried out for the third time, and a policy database and a matching database are used for matching the third part of content in the streaming data.
In a specific implementation process, the step s500 of third matching includes:
and selecting url format data, and matching url contents according to db data to obtain a third matching result, wherein the third matching result comprises matching success and matching failure.
Specifically, in the step s500, the third matching step further includes:
ending the third matching step when the third matching result is a matching failure; and when the third matching result is that the matching is successful, recording the strategy id of the successful matching.
S600, receiving results of the first matching, the second matching and the third matching, and generating a construction mirror image message.
In the implementation process, the steps of receiving the results of the first matching, the second matching and the third matching and generating the construction mirror image message include: and receiving the strategy id of successful matching obtained by the first matching, the second matching and the third matching, and generating a construction mirror image message.
By adopting the scheme, firstly, the mirror image function can be realized aiming at the flow in the network, and the matching of each field in the message is realized through the multi-element strategy in the mirror image process to obtain the matching result, so that a plurality of complex conditions in the current network environment can be effectively solved, the corresponding mirror image action is carried out on each field in the message, the matching efficiency of the multi-element strategy under the multi-dimension is improved, the mirror image efficiency is improved, and the processing is accelerated to improve the performance; secondly, interference of combination conditions of all fields in the policy can be reduced, the fields in the policy are divided into multiple dimensions, the fields in each dimension are in type granularity, according to matching requirements of a plurality of fields under granularity, matching results are confirmed by taking the granularity as the lowest matching requirement, whether mirror image traffic is needed or not is further confirmed, matching period is obviously reduced, matching efficiency of the multi-element policy under the multiple dimensions is improved, mirror image efficiency is improved, processing speed is accelerated, and waste of computing resources is reduced.
In a second aspect of the present invention, a device for implementing multiple policy mirroring of network traffic is provided.
The file transmission device written by the multiple data sources comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the multi-element strategy mirror image realization method when executing the program.
In a third aspect of the present invention, a storage medium is provided.
Specifically, the storage medium has a computer program stored thereon, which when executed by a processor implements the above-described multi-policy mirroring implementation method.
In some preferred embodiments of the present invention, the Bitmap is a Bitmap concept, and is set in the process of constructing the policy database, and is set by a bit or an algorithm, that is, one original data 0 is set by a bit or an algorithm, and after the analysis of the policy field, some positions of the original data 0 are set by a bit or an algorithm to be 1. The purpose of the Bitmap setting is to judge whether the strategy field matching is successful or not for strategy arbitration, and the Bitmap setting is an original object of arbitration of the whole strategy hit. The bitmaps of the policies and the bitmaps of the various types of policy fields are respectively bitmaps in the process of constructing a policy database, namely an original object in the process of policy arbitration, and bitmaps constructed in the process of matching, namely an arbitrated object in the process of policy arbitration.
In some preferred embodiments of the present invention, in the actual matching process, the message is parsed to obtain normalized key matching words for unified management, and the normalized key matching words are matched with the rule database of each dimension policy field generated in the compiling period one by one, so that the matching result is obtained by using policy arbitration. In the multi-element strategy matching process aiming at the multi-dimension, strategy field arbitration is used for obtaining the matching result of the strategy field. For the forward offset type of the data packet, a strategy with a matching length of not 0 and an offset of not 0 is configured, and after hyperscan matching is successful, the relation between the difference value between the end of matching and the offset and the matching length needs to be paid additional attention to confirm whether the strategy field is successfully matched.
In some preferred embodiments of the present invention, according to the mirroring policy configured by the command line or other interfaces, the policy field is decomposed into multiple dimensions, i.e. multiple types of policy fields, and hyperscan deserialized db data is respectively constructed, and db data is compiled and generated for different types of policy fields in a single dimension by using a logical matching manner of the policy fields bitmap+hyperscan.
In some preferred embodiments of the present invention, the supported matching method includes: string contains, string ignores case contains, equals, ignores case equals, regular matches, ignores case regular matches, non-contains, non-regular. For configured offsets, an array of expressions ending with null characters is constructed that contains the offset.
In summary, the solution provided by the present invention can realize the mirror function for the traffic in the network, and in the mirror process, the matching of each field in the message is realized through the multiple policies to obtain the matching result, so that many complex situations in the current network environment can be effectively solved, the corresponding mirror action is performed on each field in the message, the matching efficiency of the multiple policies in multiple dimensions is improved, the mirror efficiency is improved, and the processing is accelerated to improve the performance; the scheme provided by the invention can effectively distinguish the load heads such as tcp, udp, ipv, ipv6 and the like in the load fields respectively, so that the problems of head-tail offset, various matching methods, matching length and the like are improved, the processing efficiency of the multi-element strategy under the formed multi-dimension is improved, and the situation that the combination of single strategies is too complex is prevented; the scheme provided by the invention can reduce the interference of the combination condition of each field in the strategy, divide each field in the strategy in a multi-dimensional way, take the type as granularity for each field in each dimension, confirm the matching result by taking the granularity as the lowest matching requirement according to the matching requirement of a plurality of fields under the granularity, further confirm whether mirror image flow is needed, obviously reduce the matching period, improve the matching efficiency of the multi-element strategy under the multi-dimension way, thereby improving the mirror image efficiency, accelerating the processing speed and reducing the waste of computing resources.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
It should be understood that in the embodiments of the present application, the claims, the various embodiments, and the features may be combined with each other, so as to solve the foregoing technical problems.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. The method is applicable to multi-element strategy matching under other dimensions, and only needs to increase rule expression compiling at compiling period and strategy field arbitration at matching period. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (4)
1. A method for realizing multi-element policy mirror image is characterized in that: the method for realizing the multi-element policy mirror image comprises the following steps:
the method for establishing the strategy database comprises a matching method, an offset, a matching length matchlen, a strategy id, a strategy field id, a strategy bitmap and various strategy field bitmaps, wherein the specific steps comprise,
a policy configuration is received and the policy configuration is received,
according to the policy configuration, parse the policy field,
according to the type of the policy field, respectively processing and generating a matching method, an offset, a matching length match, a policy id, a policy field id, a policy bitmap and various types of policy fields bitmap to construct a policy database,
when the type of the strategy field is a host strategy field and a url strategy field, analyzing the strategy field to obtain a matching method, an offset and a matching length match, obtaining a key matching word, setting a bitmap of the strategy to which the strategy field belongs, setting the host or url type strategy field bitmap of the strategy, using hyperscan logic combination to record strategy id, compiling and generating deserialized db data,
when the policy field type is a payload policy field, resolving the policy field to obtain a matching method, an offset and a matching length match, obtaining a key matching word, setting a bitmap of a policy to which the policy field belongs, setting a policy field bitmap of the payload type of the policy,
judging whether the types of the load offset and the packet tail forward offset in the strategy field are the same offset type, if yes, compiling and generating deserialized db data by using hyperscan logic combination, recording strategy ids, if not, respectively generating different types of deserialized db data by using hyperscan, recording strategy ids, and respectively recording strategy field ids of different offset types;
receiving flow data and a matching database;
the method comprises the steps of performing first matching, selecting data in host format by using a strategy database and a matching database, and matching a first part of content in flow data according to db data to obtain a first matching result, wherein the first part of content is host content;
a policy database and a matching database are used for second matching, data in a payload format is selected, and second partial content in the flow data is matched according to db data to obtain a second matching result, wherein the second partial content is payload content;
thirdly matching, namely selecting url format data by using a strategy database and a matching database, and matching a third part of content in the flow data according to db data to obtain a third matching result, wherein the third part of content is url content;
and receiving results of the first matching, the second matching and the third matching, and generating a construction mirror image message.
2. The multi-element policy mirroring implementation method of claim 1, wherein: the policy fields include a host policy field, a payload policy field, and a url policy field.
3. The multi-element policy mirroring implementation method of claim 2, wherein: the step of generating db data, strategy id and strategy field id by processing respectively according to the strategy field type comprises the following steps:
judging the type of the strategy field, classifying the host strategy field and the url strategy field into a first category, and classifying the payload strategy field into a second category.
4. A method for implementing a multivariate policy image according to any one of claims 2-3, wherein: the step of receiving the flow data and matching the database further comprises the following steps: carrying out structural normalization on the flow data to obtain a normalization result, wherein the normalization result comprises the following steps: host format, payload format, url format.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011619775.9A CN112699097B (en) | 2020-12-31 | 2020-12-31 | Method, device and storage medium for realizing multi-element policy mirror image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011619775.9A CN112699097B (en) | 2020-12-31 | 2020-12-31 | Method, device and storage medium for realizing multi-element policy mirror image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112699097A CN112699097A (en) | 2021-04-23 |
| CN112699097B true CN112699097B (en) | 2024-03-08 |
Family
ID=75512885
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011619775.9A Active CN112699097B (en) | 2020-12-31 | 2020-12-31 | Method, device and storage medium for realizing multi-element policy mirror image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112699097B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101127691A (en) * | 2006-08-17 | 2008-02-20 | 王玉鹏 | A method for implementing stream-based policy routing on network processor |
| CN102045347A (en) * | 2010-11-30 | 2011-05-04 | 华为技术有限公司 | Method and device for identifying protocol |
| CN103248609A (en) * | 2012-02-06 | 2013-08-14 | 同方股份有限公司 | System, device and method for detecting data from end to end |
| CN103415842A (en) * | 2010-11-16 | 2013-11-27 | 阿克蒂菲奥股份有限公司 | Systems and methods for data management virtualization |
| CN107222496A (en) * | 2017-06-29 | 2017-09-29 | 北京东土军悦科技有限公司 | The security strategy matching process and field layer equipment of message based on field layer equipment |
| CN107465567A (en) * | 2017-06-29 | 2017-12-12 | 西安交大捷普网络科技有限公司 | A kind of data forwarding method of database fire wall |
| CN110866281A (en) * | 2019-11-20 | 2020-03-06 | 满江(上海)软件科技有限公司 | Safety compliance processing system and method for sensitive data |
| CN111277612A (en) * | 2020-05-08 | 2020-06-12 | 常州楠菲微电子有限公司 | Network message processing strategy generation method, system and medium |
| CN111711712A (en) * | 2020-06-03 | 2020-09-25 | 网根(南京)网络中心有限公司 | DNS (Domain name System) strategy configuration method, system and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7500158B1 (en) * | 2006-07-06 | 2009-03-03 | Referentia Systems, Inc. | System and method for network device configuration |
| EP2558988A4 (en) * | 2010-04-14 | 2016-12-21 | The Dun And Bradstreet Corp | Ascribing actionable attributes to data that describes a personal identity |
-
2020
- 2020-12-31 CN CN202011619775.9A patent/CN112699097B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101127691A (en) * | 2006-08-17 | 2008-02-20 | 王玉鹏 | A method for implementing stream-based policy routing on network processor |
| CN103415842A (en) * | 2010-11-16 | 2013-11-27 | 阿克蒂菲奥股份有限公司 | Systems and methods for data management virtualization |
| CN102045347A (en) * | 2010-11-30 | 2011-05-04 | 华为技术有限公司 | Method and device for identifying protocol |
| CN103248609A (en) * | 2012-02-06 | 2013-08-14 | 同方股份有限公司 | System, device and method for detecting data from end to end |
| CN107222496A (en) * | 2017-06-29 | 2017-09-29 | 北京东土军悦科技有限公司 | The security strategy matching process and field layer equipment of message based on field layer equipment |
| CN107465567A (en) * | 2017-06-29 | 2017-12-12 | 西安交大捷普网络科技有限公司 | A kind of data forwarding method of database fire wall |
| CN110866281A (en) * | 2019-11-20 | 2020-03-06 | 满江(上海)软件科技有限公司 | Safety compliance processing system and method for sensitive data |
| CN111277612A (en) * | 2020-05-08 | 2020-06-12 | 常州楠菲微电子有限公司 | Network message processing strategy generation method, system and medium |
| CN111711712A (en) * | 2020-06-03 | 2020-09-25 | 网根(南京)网络中心有限公司 | DNS (Domain name System) strategy configuration method, system and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| Flow Fields:Dense Correspondence Fields for Highly Accurate Large Displacement Optical Flow Estimation;Christian Bailer et al.;《Proceedings of the IEEE International Conference on Computer Vision》;4015-4023 * |
| 基于多层模式匹配技术的高速以太网NIDS实现方案;余扬 等;《微电子学与计算机》(第09期);32-36 * |
| 基于网络协议解析的入侵检测系统的研究;张华;;《网络安全技术与应用》(第06期);30-32 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112699097A (en) | 2021-04-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7162512B1 (en) | Guaranteed exactly once delivery of messages | |
| CN100424646C (en) | Delta object replication system and method for clustered system | |
| US7254632B2 (en) | Apparatus and method for pattern matching in text based protocol | |
| US8924389B2 (en) | Computer-implemented systems and methods for comparing and associating objects | |
| US6134664A (en) | Method and system for reducing the volume of audit data and normalizing the audit data received from heterogeneous sources | |
| US7437359B2 (en) | Merging multiple log entries in accordance with merge properties and mapping properties | |
| CN101095310A (en) | Programmable packet parsing processor | |
| US11151660B1 (en) | Intelligent routing control | |
| US9569285B2 (en) | Method and system for message handling | |
| US20210099538A1 (en) | Systems and methods for data exchange among network devices | |
| US7516475B1 (en) | Method and apparatus for managing security policies on a network | |
| Dai et al. | Identifying and estimating persistent items in data streams | |
| CN112699097B (en) | Method, device and storage medium for realizing multi-element policy mirror image | |
| CN114140127B (en) | Payment processing method and system based on block chain | |
| CN106843934A (en) | The treating method and apparatus of upgrade file | |
| Umbarkar et al. | Analysis of heuristic based feature reduction method in intrusion detection system | |
| CN111367479A (en) | Cloud printing method, printing method of cloud printing service platform and storage medium | |
| Hua et al. | Deep fidelity in DNN watermarking: A study of backdoor watermarking for classification models | |
| Gupta et al. | Preprocessor algorithm for network management codebook | |
| US6775834B2 (en) | System and method for facilitating the communication of data on a distributed medical scanner/workstation platform | |
| EP1839206A2 (en) | System and method for processing event predicates | |
| US7475095B2 (en) | Unread mark replication bounce-back prevention | |
| US7613778B2 (en) | Progressive de-featuring of electronic messages | |
| JP2790171B2 (en) | Data transfer method in HIPPI network | |
| US7676541B2 (en) | Peer communication channel partitioning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |