CN111404798A - System and method for multi-user rule matching and flow replication - Google Patents

System and method for multi-user rule matching and flow replication Download PDF

Info

Publication number
CN111404798A
CN111404798A CN202010157979.9A CN202010157979A CN111404798A CN 111404798 A CN111404798 A CN 111404798A CN 202010157979 A CN202010157979 A CN 202010157979A CN 111404798 A CN111404798 A CN 111404798A
Authority
CN
China
Prior art keywords
message
rule
hit
user
unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010157979.9A
Other languages
Chinese (zh)
Inventor
陈志华
曾祥刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hubei Weiyuan Zhuoyue Technology Co ltd
Original Assignee
Hubei Weiyuan Zhuoyue Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hubei Weiyuan Zhuoyue Technology Co ltd filed Critical Hubei Weiyuan Zhuoyue Technology Co ltd
Priority to CN202010157979.9A priority Critical patent/CN111404798A/en
Publication of CN111404798A publication Critical patent/CN111404798A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00Packet switching elements
    • H04L49/10Packet switching elements characterised by the switching fabric construction
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/28Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/46Interconnection of networks
    • H04L12/4641Virtual LANs, VLANs, e.g. virtual private networks [VPN]
    • H04L12/4645Details on frame tagging
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/10Flow control; Congestion control
    • H04L47/24Traffic characterised by specific attributes, e.g. priority or QoS
    • H04L47/2441Traffic characterised by specific attributes, e.g. priority or QoS relying on flow classification, e.g. using integrated services [IntServ]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00Packet switching elements
    • H04L49/35Switches specially adapted for specific applications
    • H04L49/354Switches specially adapted for specific applications for supporting virtual local area networks [VLAN]

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention discloses a system and a method for multi-user rule matching and flow duplication, wherein the system comprises a front-end exchange board, a signaling analysis subsystem and a user rule matching subsystem, the front-end exchange board receives AN original network message, divides the original network message into a signaling message and a data message according to message characteristics, obtains associated information metadata from the signaling message through the signaling analysis subsystem, marks the data message through AN FPGA message marking module, performs hierarchical structure analysis, caching, application identification and multi-user rule matching on the data message marked by the FPGA message matching subsystem, marks a hit rule tag on the data message to be duplicated through the FPGA message duplication module, and distributes the duplicated message to a corresponding user service system according to a V L AN number of a hit user.

Description

System and method for multi-user rule matching and flow replication
Technical Field
The invention belongs to the technical field of network communication, and particularly relates to a multi-user rule matching and flow copying system and method.
Background
With the continuous upgrading of communication technology, the internet technology is rapidly developed, various applications based on the internet are continuously developed, most of the existing distribution systems can only distribute data based on keywords of quintuple and specific positions, the application types of the data cannot be identified, more refined distribution rules cannot be used, a large amount of low-value flow flows to a back-end service analysis system, and therefore the system construction cost is greatly increased. Meanwhile, most of the existing shunting systems only support single-user issuing of rules, and if the back end has service analysis systems of multiple manufacturers, the number of front-end shunting devices needs to be increased, so that the construction cost of the system can be increased. If a multi-user rule is adopted for issuing, if a plurality of users are interested in the same traffic, the traffic needs to be copied into a plurality of copies, and the copying of the traffic can greatly degrade the overall processing performance of the system, which is also a difficult problem in the industry.
In addition, current reposition of redundant personnel system mostly integrates, and reposition of redundant personnel rule matches and the flow distribution plate is integrated to be accomplished on a board promptly, and the inconvenient later stage dilatation of this kind of structure.
Disclosure of Invention
In order to solve the problems in the prior art, the invention provides a system and a method for multi-user rule matching and flow replication.
In a first aspect of the present invention, a system for multi-user rule matching and traffic replication is provided, where the system includes a front-end switch board, a signaling analysis subsystem and a user rule matching subsystem, which are respectively connected to the front-end switch board; an FPGA message marking module and an FPGA message copying module are additionally arranged on the front-end exchange board;
the front-end exchange board is used for receiving AN original network message and dividing the original network message into a signaling message and a data message according to message characteristics, sending the signaling message to a signaling analysis subsystem to obtain associated information metadata output by the signaling analysis subsystem, marking the data message through AN FPGA message marking module according to the associated information metadata to obtain the data message carrying AN extended function label, sending the data message carrying the extended function label to a corresponding user rule matching subsystem to obtain the data message carrying the extended function label and a rule hit label output by the user rule matching subsystem, copying the data message to be copied through the FPGA message copying module, and distributing the copied message to a corresponding user service system according to a V L AN number of a hit user;
the signaling analysis subsystem is used for extracting the internet account information or the mobile phone number information in the signaling message and associating the internet account information or the mobile phone number information with the quintuple information of the data message to form associated information metadata; sending the associated information metadata to an FPGA message marking module of a front-end exchange board;
the user rule matching subsystem is used for carrying out hierarchical structure analysis, caching, application identification, multi-user rule matching and marking on the data message carrying the extended function tag to obtain the data message carrying the extended function tag and the rule hit tag.
Preferably, in the front-end switch board, the FPGA message marking module is configured to modify a corresponding data message header according to the associated information metadata, where the modifying includes modifying DMAC, SMAC, adding a new V L AN tag, and meanwhile, marking AN internet account or a mobile phone number information tag on the tail of the data message to obtain a data message carrying AN extended function tag.
Preferably, in the user rule matching subsystem, for a flow in which consecutive N messages are not hit, discarding all messages of the flow, and for a hit flow, marking corresponding rule hit labels at the tail of all messages of the flow, where the rule hit labels include AN identified application number, a rule ID of the highest priority of the hit user, a V L AN number of the hit user, and a message stripping type required by the user.
Preferably, the user rule matching subsystem specifically includes the following functional modules:
a data sorting module: the method is used for IP fragment recombination, caching and analyzing the data message, establishing a flow table, recombining the TCP out-of-order message, and carrying out exception processing on the error message or the huge message; the system specifically comprises an abnormal message processing unit, an IP fragment recombining unit, a multi-layer message structure analyzing unit, a message caching unit, a flow table unit and a TCP flow recombining unit;
basic rule matching module: the system is used for basic rule matching and comprises an internet account rule matching unit, a quintuple rule matching unit, a feature code rule matching unit and a model rule matching unit;
an application identification module: the device is used for identifying the application type and comprises an application characteristic library loading unit, an application characteristic library updating unit and an application identification unit;
applying a class rule matching module: the device is used for respectively matching different application rules according to the identified application types, and comprises an application rule matching unit, a DNS rule matching unit, an HTTP rule matching unit, an HTTPS rule matching unit and an audio and video rule matching unit;
a check class rule matching module: the attribute rule is used for correspondingly checking the attached attribute rule of each hit rule;
a combination rule processing module: the system is used for storing the hit result and judging whether the condition of the combination rule is met or not; the conditions of the combination rule are as follows: for the condition that one rule comprises a plurality of sub-rules, if each sub-rule meets the condition, the condition is determined as hit; if all the sub-rules are not hit in one message, recording the sub-rules which are hit by the flow until all the sub-rules are met or the matching is overtime;
a multi-user rule processing module: the system is used for carrying out user classification and priority sequencing on hit rules, only keeping hit rule numbers with the highest priority for multiple rules hit by the same user, and respectively storing the rule ID with the highest priority hit by each user; rules hit by different users are not influenced mutually and are stored respectively;
the message marking and sending module comprises: and the method is used for marking the hit message and all the messages on the flow to which the message belongs with the corresponding rule hit label and sending out the hit message and all the messages.
Preferably, in the front-end switch board, the FPGA packet copying module is configured to read tag information in a packet hit by a multi-user rule, copy a data packet according to a predetermined rule, modify a corresponding V L AN number, perform corresponding packet stripping on a packet having a stripping requirement, and deliver the processed packet to the front-end switch board.
In a second aspect of the present invention, a method for multi-user rule matching and traffic replication using the system of the first aspect of the present invention is provided, where the method includes:
s1, the front-end exchange board receives the original network message and divides the original network message into a signaling message and a data message according to the message characteristics;
s2, sending the signaling message to a signaling analysis subsystem, wherein the signaling analysis subsystem extracts internet account information or mobile phone number information in the signaling message and associates the internet account information or mobile phone number information with quintuple information of the data message to form associated information metadata;
s3, the FPGA message marking module marks the data message according to the associated information metadata to obtain the data message carrying the extended function label;
s4, the front-end switch board sends the data to the corresponding user rule matching subsystem according to the V L AN identification of the data message header carrying the extended function label;
s5, the user rule matching subsystem carries out hierarchical structure analysis, cache, application identification, multi-user rule matching and marking on the data message carrying the extended function label, obtains a data message carrying the extended function label and the rule hit label and sends the data message to the front-end exchange board;
s6, copying the message after the multi-user rule hits the mark through AN FPGA message copying module, and modifying a corresponding V L AN label;
and S7, the front-end exchange board respectively sends the messages to the corresponding user service systems according to the V L AN labels in the data messages.
7. The method according to claim 6, wherein in step S3, the marking of the data packet by the FPGA packet marking module according to the associated information metadata is specifically that the corresponding data packet header is modified, including modifying DMAC, SMAC, adding new V L AN, and simultaneously, the tail of the data packet is marked with AN Internet access account or a mobile phone number information tag.
Preferably, the step S5 specifically includes:
s51, inputting the data message with the extended function label into the IP fragment recombination unit, if the data message is an abnormal message, handing the abnormal message to the abnormal message processing unit for processing; for the IP fragment message, IP fragment recombination is carried out, the recombined message is sent to a message cache unit for processing, if the message is not the IP fragment message, the message is directly sent to the message cache unit for processing;
s52, the message buffer unit buffers the first N packets of a flow or buffers the message of the first N seconds of a flow according to the buffer strategy, and the message is delivered to the multi-layer message structure analysis unit for processing after buffer.
And S53, the multi-layer message structure analysis unit analyzes various package structures of the message, extracts message structure information and delivers the message structure information to the flow table unit.
S54, the flow table unit builds flow according to the message information of the innermost layer and maintains the state of the flow table; transmitting the message to an internet account rule matching unit and sharing the message to an application identification unit;
and S55, after being processed by the internet access account rule matching unit, the messages are sequentially transmitted to the quintuple rule matching unit, the feature code rule matching unit and the model rule matching unit for processing.
S56, the application identification unit identifies the application of the received message, outputs the identified application ID and matches the corresponding application rule; respectively delivering the application types to corresponding application rule matching units for processing;
s57, after all the rules are matched, entering an interface group rule checking unit, checking the hit rules in the interface group, and delivering the checked rules to a combined rule processing module;
s58, the combination rule processing module stores the hit result, judges whether the hit result meets the conditions of the combination rule, and if the hit result meets the conditions, the hit result is handed to the multi-user rule processing module;
s59, the multi-user rule processing module carries out user classification and priority sequencing on the hit rules, only the hit rule number with the highest priority is reserved for multiple rules hit by the same user, and the hit rule number with the highest priority hit by each user is respectively stored; rules hit by different users are not influenced mutually and are stored respectively;
and S510, according to the result of the multi-user rule processing, marking a corresponding rule hit label on the hit message and all messages on the flow to which the message belongs, and sending the messages to the front-end switch board.
Preferably, the step S510 specifically includes:
modifying internally defined communication protocol labels for the hit message and all messages on the flow to which the message belongs later, wherein the modified internally defined communication protocol labels carry message offset of a T L V initial position, message input interface group information, message flow direction and the number of hit users;
modifying the first V L AN label of the header of the message into a special V L AN number as a message copying identifier, and when the FPGA reads the V L AN label, performing message copying action;
and marking corresponding rule hit labels at the tail parts of the hit message and all messages on the flow to which the message belongs later, wherein the rule hit labels comprise the identified application number, the rule ID of the highest priority of the hit user, the V L AN number of the hit user and the message stripping type required by the user, and the rule ID of the highest priority of the hit user, the V L AN number of the hit user and the message stripping type required by the user are multiple groups and are determined according to the number of the hit users.
Preferably, the step S6 specifically includes:
reading the marked message with the multi-user rule hit by the FPGA message copying module, extracting the content in AN internally defined communication protocol label, reading the initial offset of T L V and the number of hit users, reading the rule ID of the highest priority of the hit users of each group of users at the tail of the message, the V L AN number of the hit users and a message stripping type label required by the user, copying the message, and replacing the special V L AN number in the copied message according to the V L AN number of the hit users in each group of labels;
and stripping the copied message according to the message stripping type of the message stripping type label required by the user, and adding a rule ID label which is corresponding to the user and hits the highest priority of the user into the copied message.
Due to the adoption of the technical scheme, the invention has the remarkable technical effects that:
1) the expansion is convenient, when the service flow is increased, the expansion can be realized only by increasing the number of the user rule matching subsystems.
2) And refining separation is performed, and processing pressure and cost of a back-end service system are reduced based on application classification.
3) And the system supports multi-user rules and can simultaneously support a plurality of back-end service systems.
4) And the message is copied by using the FPGA, so that the processing capacity of the whole machine is improved.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings needed to be used in the technical description of the present invention will be briefly introduced below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive labor.
FIG. 1 is a schematic diagram of a system architecture for multi-user rule matching and traffic replication disclosed in the present invention;
FIG. 2 is a functional block diagram of a user rule matching subsystem 300;
FIG. 3 is a flow chart of a method for multi-user rule matching and traffic replication according to the present invention;
fig. 4 is a specific format example of the marking of the message by the FPGA message marking module;
FIG. 5 is a specific format example of an internally defined communication protocol tag 1 (INFO-1-L ABE L);
FIG. 6 is a schematic flow chart of user rule matching subsystem data;
FIG. 7 is an example of a rule hit marking format for a user rule matching subsystem;
FIG. 8 is a specific format example of an internally defined communication protocol tag 2 (INFO-2-L ABE L);
fig. 9 is a message format example of the FPGA message copying module copying the message marked after the user rule matching subsystem rule hits.
Detailed Description
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the embodiments described below are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, the present invention provides a system for multi-user rule matching and traffic replication, which includes a front-end switch board 100, a signaling analysis subsystem 200 and a user rule matching subsystem 300, which are respectively connected to the front-end switch board; an FPGA message marking module 110 and an FPGA message copying module 120 are additionally arranged on the front-end exchange board;
the front-end exchange board 100 receives an original network packet, and divides the original network packet into a signaling packet and a data packet according to packet characteristics; sending the signaling message to a signaling analysis subsystem 200, wherein the signaling analysis subsystem 200 extracts internet account information or mobile phone number information in the signaling message, associates the internet account information or mobile phone number information with quintuple information of the data message to form associated information metadata, and sends the associated information metadata to an FPGA message marking module 110 of the front-end switch board 100;
marking the data message by the FPGA message marking module 110 according to the associated information metadata to obtain the data message carrying the extended function tag;
further, the FPGA message marking module 110 modifies the corresponding data message header according to the associated information metadata, including modifying DMAC, SMAC, adding a new V L AN tag, and meanwhile, marking AN internet account or mobile phone number information tag on the tail of the data message to obtain the data message carrying the extended function tag.
The front-end switch board 100 sends the data message carrying the extended function tag to the corresponding user rule matching subsystem 300 according to the V L AN tag, where the user rule matching subsystem is configured to perform hierarchical structure analysis, caching, application identification, multi-user rule matching, and marking on the data message carrying the extended function tag, so as to obtain the data message carrying the extended function tag and the rule hit tag.
Further, in the user rule matching subsystem 300, for a flow in which N consecutive messages are not hit, all messages of the flow are discarded, and for a hit flow, corresponding rule hit tags are marked on the tails of all messages of the flow, where the rule hit tags include AN identified application number, a rule ID of the highest priority of the hit user, a V L AN number of the hit user, and a message stripping type required by the user.
The data message to be copied is copied to the message marked by multi-user rule hit through the FPGA message copying module 120, and the front-end exchange board distributes the copied message to a corresponding user service system according to the V L AN number of the hit user;
further, the FPGA packet copying module is configured to read tag information in a packet hit by a multi-user rule, copy a data packet according to a predetermined rule, modify a corresponding V L AN number, perform corresponding packet stripping on a packet having a stripping requirement, and deliver the processed packet to the front-end switch board 100.
Referring to fig. 2, the user rule matching subsystem 300 specifically includes the following functional modules:
the data sorting module 310 is used for performing IP fragment reassembly, caching and analysis on data messages, establishing a flow table, reassembling TCP out-of-order messages and performing exception processing on error messages or huge messages, and specifically comprises AN exception message processing unit, AN IP fragment reassembly unit, a multilayer message structure analysis unit, a message caching unit, a flow table unit and a TCP flow reassembly unit, wherein the multilayer message structure analysis unit performs structure analysis on packages of various layers such as IP-In-IP, a tunnel, V L AN and the like, the message caching unit caches N messages for each flow, the flow table unit is used for establishing a flow table and maintaining a flow table state, and the TCP flow reassembly unit reassembles the TCP out-of-order messages.
Base class rule matching module 320: the system is used for basic rule matching and comprises an internet account rule matching unit, a quintuple rule matching unit, a feature code rule matching unit and a model rule matching unit; specifically, the internet access account rule matching unit comprises mobile phone number rule matching, the quintuple rule matching unit comprises accurate quintuple rule matching and mask quintuple rule matching, the feature code rule matching unit comprises fixed position feature code matching and floating position feature code matching, and the model rule matching unit comprises mobile phone number model rule matching and longitude and latitude model rule matching.
The application recognition module 330: the device is used for identifying the application type and comprises an application characteristic library loading unit, an application characteristic library updating unit and an application identification unit; when the characteristics of the application are changed or some application characteristics are customized by a user, an application characteristic library updating interface can be called for updating.
Application class rule matching module 340: the device is used for respectively matching different application rules according to the identified application types, and comprises an application rule matching unit, a DNS rule matching unit, an HTTP rule matching unit, an HTTPS rule matching unit and an audio and video rule matching unit; specifically, the DNS rule matching unit extracts domain name information, the HTTP rule matching unit extracts keyword information such as Host, and the HTTPS rule matching unit extracts keyword information such as an Https certificate. The module can also expand rule matching units of other applications according to actual application needs.
The check class rule matching module 350: the attribute rule is used for correspondingly checking the attached attribute rule of each hit rule; the most common is the rule check of the incoming interface group, and other types of rule check can be added according to actual situations.
Combination rule processing module 360: for the case that one rule comprises a plurality of sub-rules, if each sub-rule meets the condition, the rule is determined to be a hit; if all the sub-rules are not hit in one message, recording the sub-rules which are hit by the flow until all the sub-rules are met or the matching is overtime;
multi-user rule processing module 370: the system is used for carrying out user classification and priority sequencing on hit rules, only keeping hit rule numbers with the highest priority for multiple rules hit by the same user, and respectively storing the rule ID with the highest priority hit by each user; rules hit by different users are not influenced mutually and are stored respectively;
the message marking and sending module 380: and the method is used for marking the hit message and all the messages on the flow to which the message belongs with the corresponding rule hit label and sending out the hit message and all the messages.
Referring to fig. 3, the present invention provides a method for multi-user rule matching and traffic replication based on the system for multi-user rule matching and traffic replication, and the method specifically includes the following steps:
s1, the front-end exchange board receives the original network message and divides the original network message into a signaling message and a data message according to the message characteristics;
s2, sending the signaling message to a signaling analysis subsystem, wherein the signaling analysis subsystem extracts internet account information or mobile phone number information in the signaling message and associates the internet account information or mobile phone number information with quintuple information of the data message to form associated information metadata;
s3, the FPGA message marking module marks the data message according to the associated information metadata to obtain the data message carrying the extended function label;
further, in step S3, the front-end switch board determines whether to mark the data packet according to the attribute of the data packet, and the marking module of the FPGA packet marks the data packet according to the associated information metadata, specifically, the marking module modifies the head of the corresponding data packet, including modifying DMAC and SMAC, adding new V L AN, and simultaneously marks the tail of the data packet with AN internet access account or a mobile phone number information tag.
Referring to fig. 4, the specific format of the marking of the message by the FPGA message marking module in step S3 is illustrated. In fig. 4, the upper part is an original network message, the lower part is a message marked by the FPGA message marking module, and the data portions of the message marked by the FPGA are described as follows:
DMAC: DMAC address of the original message.
INFO-1-L ABE L changes the original SMAC part into an internally defined communication protocol label 1, wherein the label carries information such as message offset of a T L V starting position, message input interface group information, message flow direction and the like.
V L AN, a newly added V L AN label, and the front-end switch board judges which user rule matching subsystem the data are forwarded to according to the V L AN label.
PAY L OAD2 content between the MAC address of the original packet header and the CRC check of the trailer PAY L OAD2 is typically the packet Ethernet frame type and the entire IP section (including the IP header).
T L V-ACC is the information such as the internet account number or the mobile phone number associated with the data message.
Wherein INFO-1-L ABE L is an internally defined communication protocol tag 1, and the detailed format (network endianness) is shown in fig. 5, which is one of the implementation ways and can be adjusted according to the use condition, the data part of INFO-1-L ABE L illustrates:
d: occupies the bit of number [0], 0 represents the uplink, and 1 represents the downlink.
OFFSET occupies the bits 1-15 and represents the OFFSET of the T L V-ACC part relative to the initial part of the message.
INPUT-IFPORT-NUM: and the number [16-31] bit is occupied, and the number of the message incoming interface group is represented.
Reserved: occupying the bits 32-47, and reserving the later expansion use.
And S4, the front-end switch board sends the data to the corresponding user rule matching subsystem according to the V L AN identification of the data message header carrying the extended function label.
S5, the user rule matching subsystem carries out hierarchical structure analysis, cache, application identification, multi-user rule matching and marking on the data message carrying the extended function label to obtain the data message carrying the extended function label and the rule hit label, and the data message carrying the extended function label and the rule hit label is sent to the front-end switch board;
further, referring to fig. 6, the step S5 specifically includes:
s51, inputting the data message carrying the extended function label into the IP fragment recombination unit, and if the data message is an abnormal message such as a wrong packet, a huge packet and the like, handing the abnormal message to the abnormal message processing unit for processing; for the IP fragment message, IP fragment recombination is carried out, the recombined message is sent to a message cache unit for processing, if the message is not the IP fragment message, the message is directly sent to the message cache unit for processing;
s52, the message buffer unit buffers the first N packets of a flow or buffers the message of the first N seconds of a flow according to the buffer strategy, and the message is delivered to the multi-layer message structure analysis unit for processing after buffer.
S53, the multi-layer message structure analysis unit analyzes various encapsulation structures of the message, such as various tunnels or encapsulations of IP-IN-IP, IPSEC encapsulation, V L AN, MP L S, PPTP, L2 TP, GTP, GRE, etc., extracts the message structure information, and delivers the message structure information to the flow table unit.
S54, the flow table unit builds flow according to the message information of the innermost layer and maintains the state of the flow table; transmitting the message to an internet account rule matching unit and sharing the message to an application identification unit; specifically, since the matching process is only a read operation on the packet, the internal memory address of the block can be shared by the two processing units for processing respectively.
And S55, after being processed by the internet access account rule matching unit, the messages are sequentially transmitted to the quintuple rule matching unit, the feature code rule matching unit and the model rule matching unit for processing.
S56, the application identification unit identifies the application of the received message, outputs the identified application ID and matches the corresponding application rule; respectively delivering the application types to corresponding application rule matching units for processing; specifically, for example: the system comprises an HTTP rule matching unit, an HTTPS rule matching unit, a DNS rule matching unit and an audio and video rule matching unit.
S57, after all the rules are matched, entering an interface group rule checking unit, checking the hit rules in the interface group, and delivering the checked rules to a combined rule processing module;
s58, the combination rule processing module stores the hit result, judges whether the hit result meets the conditions of the combination rule, and if the hit result meets the conditions, the hit result is handed to the multi-user rule processing module;
s59, the multi-user rule processing module carries out user classification and priority sequencing on the hit rules, only the hit rule number with the highest priority is reserved for multiple rules hit by the same user, and the hit rule number with the highest priority hit by each user is respectively stored; rules hit by different users are not influenced mutually and are stored respectively;
and S510, according to the result of the multi-user rule processing, marking a corresponding rule hit label on the hit message and all messages on the flow to which the message belongs, and sending the messages to the front-end switch board.
Further, the specific process of step S510 is:
modifying internally defined communication protocol labels for the hit message and all messages on the flow to which the message belongs later, wherein the modified internally defined communication protocol labels carry message offset of a T L V initial position, message input interface group information, message flow direction and the number of hit users;
modifying the first V L AN label of the header of the message into a special V L AN number as a message copying identifier, and when the FPGA reads the V L AN label, performing message copying action;
and marking corresponding rule hit labels at the tail parts of the hit message and all messages on the flow to which the message belongs later, wherein the rule hit labels comprise the identified application number, the rule ID of the highest priority of the hit user, the V L AN number of the hit user and the message stripping type required by the user, and the rule ID of the highest priority of the hit user, the V L AN number of the hit user and the message stripping type required by the user are multiple groups and are determined according to the number of the hit users.
Referring to fig. 7, a rule hit marking format of the user rule matching subsystem is illustrated, which is an implementation manner of marking, and may be adjusted according to a use condition, in fig. 7, an upper part is a message marked by the FPGA, a lower part is a message marked by the user rule matching subsystem after the rule hit, and data portions of the message marked by the user rule matching subsystem after the rule hit are described as follows:
DMAC: and the user rule matching subsystem receives the DMAC address of the message.
And the INFO-2-L AB L E changes the INFO-1-L AB L E part of the message received by the user rule matching subsystem into an internally defined communication protocol label 2, wherein the label carries the message offset of the T L V initial position, the message input interface group information, the message flow direction and the number of users hit by the rule.
The V L AN is a special V L AN number, which is different from a commonly used V L AN number, such as 4095, and when the FPGA reads the V L AN identification, the action of message copying is performed.
PAY L OAD2 user rule matching subsystem PAY L OAD2 part of message received.
T L V-ACC is the Internet account number or mobile phone number associated with the data message.
T L V-APP, identified application number.
T L V-RU L E-ID hit the rule ID of the highest priority of the user.
T L V-USER-V L AN hit the USER's V L AN number.
T L V-STRIP-TYPE, message stripping TYPE required by user.
Wherein, T L V-RU L E-ID, T L V-USER-V L AN and T L V-STRIP-TYPE have a plurality of groups, which are determined according to the number of hit USERs.
INFO-2-L ABE L is an internally defined communication protocol tag 2, the specific format (network endianness) is shown in fig. 8, which is one implementation of the format and can be adjusted according to the use condition.
D: occupies the bit of number [0], 0 represents the uplink, and 1 represents the downlink.
OFFSET occupies the bits 1-15 and represents the OFFSET of the T L V-ACC part relative to the initial part of the message.
INPUT-IFPORT-NUM: and the number [16-31] bit is occupied, and the number of the message incoming interface is represented.
N: occupying bits 32-35, indicating the number of hits.
Reserved: occupying the No. 36-47 bits, and reserving the later expansion use.
S6, copying the message after the multi-user rule hits the mark through AN FPGA message copying module, and modifying a corresponding V L AN label;
further, the step S6 specifically includes:
the front-end exchange board checks the V L AN number of the message, judges whether the message needs to be copied according to the V L AN number, if the message is a special copy identifier V L AN number, judges that the message needs to be copied, and sends the message to AN FPGA message copying module for processing, the FPGA message copying module reads the marking information IN the multi-USER rule marking message, extracts the content IN AN INFO-2-L BA L E label, reads the initial offset of T L V and the hit multi-USER number, then sequentially reads the T L4V-RU L E-ID, the T356V-USER-V L AN and T L V-GREETTYPE labels of each group of USERs behind the message, copies the message, replaces the copy message INFO-2-L ABE L V L AN number of the back V L AN according to the T378V-GREEP-TYPE of the common label, STRIPs the IP-based on the T L V-USER-V L AN TYPE of the common label, STRIPs the copy message, STRIPs the IP tunnel ID, STRIPs the IP-P label after the IP label is added to the IP-P copy message, and the IP-SP-L.
Referring to fig. 9, the message format of each user after the message marked after the user rule matching subsystem rule hit is processed by the FPGA copying module is an example, the format is one implementation manner, and can be adjusted according to the use condition, and the data parts in the message of each user after the message marked after the user rule matching subsystem rule hit is copied are described as follows:
DMAC: and marking the DMAC address of the message according to the multi-user rule.
INFO-2-L ABE L. the INFO-2-L ABE L label of the multi-user rule marking message is not changed.
USER-N-V L AN, the V L AN number of the hit USER is obtained from T L V-USER-VAN in the multi-USER rule marking message.
PAY L OAD-AFTER-STRIPPED, multi-user rule marking the STRIPPED content of the message.
T L V-ACC is the Internet account number or mobile phone number associated with the data message.
T L V-APP, identified application number.
T L V-RU L E-ID-N, wherein the rule ID of the highest priority of the hit user is obtained from the T L V-RU L E-ID of the corresponding user in the multi-user rule marking message.
And S7, the front-end exchange board respectively sends the messages to the corresponding user service systems according to the V L AN labels in the data messages.
The front-end exchange board sends the messages to the corresponding user rule matching subsystems according to the difference of the V L AN number in the data messages processed by the FPGA marking module, and the front-end exchange board sends the messages to the corresponding user service systems according to the difference of the V L AN number of the user in the messages of each user after the FPGA copying module processes.
The above system embodiments and method embodiments are in one-to-one correspondence, and reference may be made to the method embodiments for a brief point of the system embodiments.
It can be clearly understood by those skilled in the art that for convenience and brevity of description, in the foregoing embodiments, descriptions of various embodiments have respective emphasis, and details or description which are not described in detail in a certain embodiment may be referred to in the related descriptions of other embodiments, and are not repeated herein.
Although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (10)

1. A multi-user rule matching and flow copying system is characterized by comprising a front-end exchange board, a signaling analysis subsystem and a user rule matching subsystem, wherein the signaling analysis subsystem and the user rule matching subsystem are respectively connected with the front-end exchange board; an FPGA message marking module and an FPGA message copying module are additionally arranged on the front-end exchange board;
the front-end exchange board is used for receiving AN original network message and dividing the original network message into a signaling message and a data message according to message characteristics, sending the signaling message to a signaling analysis subsystem to obtain associated information metadata output by the signaling analysis subsystem, marking the data message through AN FPGA message marking module according to the associated information metadata to obtain the data message carrying AN extended function label, sending the data message carrying the extended function label to a corresponding user rule matching subsystem to obtain the data message carrying the extended function label and a rule hit label output by the user rule matching subsystem, copying the data message to be copied through the FPGA message copying module, and distributing the copied message to a corresponding user service system according to a V L AN number of a hit user;
the signaling analysis subsystem is used for extracting the internet account information or the mobile phone number information in the signaling message and associating the internet account information or the mobile phone number information with the quintuple information of the data message to form associated information metadata; sending the associated information metadata to an FPGA message marking module of a front-end exchange board;
the user rule matching subsystem is used for carrying out hierarchical structure analysis, caching, application identification and multi-user rule matching and marking on the data message carrying the extended function tag to obtain the data message carrying the extended function tag and the rule hit tag.
2. The system according to claim 1, wherein in the front-end switch board, the FPGA packet marking module is configured to modify a header of a corresponding data packet according to associated information metadata, and includes modifying DMAC, SMAC, adding a new V L AN tag, and marking AN internet account number or a mobile phone number information tag on a tail of the data packet to obtain a data packet carrying AN extended function tag.
3. The system according to claim 1, wherein the user rule matching subsystem discards all the messages of the flow for flows where consecutive N messages are not hit, and for hit flows, marks corresponding rule hit labels on the tails of all the messages of the flow, wherein the rule hit labels include the identified application number, the rule ID of the highest priority of the hit user, the V L AN number of the hit user, and the type of message stripping required by the user.
4. The system for multi-user rule matching and traffic replication according to claim 1, wherein the user rule matching subsystem specifically comprises the following functional modules:
a data sorting module: the method is used for IP fragment recombination, caching and analyzing the data message, establishing a flow table, recombining the TCP out-of-order message, and carrying out exception processing on the error message or the huge message; the system specifically comprises an abnormal message processing unit, an IP fragment recombining unit, a multi-layer message structure analyzing unit, a message caching unit, a flow table unit and a TCP flow recombining unit;
basic rule matching module: the system is used for basic rule matching and comprises an internet account rule matching unit, a quintuple rule matching unit, a feature code rule matching unit and a model rule matching unit;
an application identification module: the device is used for identifying the application type and comprises an application characteristic library loading unit, an application characteristic library updating unit and an application identification unit;
applying a class rule matching module: the device is used for respectively matching different application rules according to the identified application types, and comprises an application rule matching unit, a DNS rule matching unit, an HTTP rule matching unit, an HTTPS rule matching unit and an audio and video rule matching unit;
a check class rule matching module: the attribute rule is used for correspondingly checking the attached attribute rule of each hit rule;
a combination rule processing module: for the case that one rule comprises a plurality of sub-rules, if each sub-rule meets the condition, the rule is determined to be a hit; if all the sub-rules are not hit in one message, recording the sub-rules which are hit by the flow until all the sub-rules are met or the matching is overtime;
a multi-user rule processing module: the system is used for carrying out user classification and priority sequencing on hit rules, only keeping hit rule numbers with the highest priority for multiple rules hit by the same user, and respectively storing the rule ID with the highest priority hit by each user; rules hit by different users are not influenced mutually and are stored respectively;
the message marking and sending module comprises: and the method is used for marking the hit message and all the messages on the flow to which the message belongs with the corresponding rule hit label and sending out the hit message and all the messages.
5. The system according to claim 1, wherein in the front-end switch board, the FPGA packet replication module is configured to read tag information in a packet hit by the multi-user rule, replicate a data packet according to a predetermined rule, modify a corresponding V L AN number, perform corresponding packet stripping on a packet having a stripping requirement, and deliver the processed packet to the front-end switch board.
6. A method for multi-user rule matching and traffic replication using the system of any of claims 1-5, the method comprising:
s1, the front-end exchange board receives the original network message and divides the original network message into a signaling message and a data message according to the message characteristics;
s2, sending the signaling message to a signaling analysis subsystem, wherein the signaling analysis subsystem extracts internet account information or mobile phone number information in the signaling message and associates the internet account information or mobile phone number information with quintuple information of the data message to form associated information metadata;
s3, the FPGA message marking module marks the data message according to the associated information metadata to obtain the data message carrying the extended function label;
and S4, the front-end exchange board sends the data message carrying the extended function label to a corresponding user rule matching subsystem according to the V L AN identifier of the data message header carrying the extended function label.
S5, the user rule matching subsystem carries out hierarchical structure analysis, cache, application identification and multi-user rule matching and marking on the data message carrying the extended function label, obtains a data message carrying the extended function label and the rule hit label and sends the data message to the front-end exchange board;
s6, for the messages marked by multi-user rule hit, the front-end switch board copies the messages through the FPGA message copying module and modifies the corresponding V L AN label;
and S7, the front-end exchange board respectively sends the messages to the corresponding user service systems according to the V L AN labels in the data messages.
7. The method according to claim 6, wherein in step S3, the marking of the data packet by the FPGA packet marking module according to the associated information metadata is specifically that the corresponding data packet header is modified, including modifying DMAC, SMAC, adding new V L AN, and simultaneously, the tail of the data packet is marked with AN Internet access account or a mobile phone number information tag.
8. The method for multi-user rule matching and traffic replication according to claim 6, wherein the step S5 specifically includes:
s51, inputting the data message with the extended function label into the IP fragment recombination unit, if the data message is an abnormal message, handing the abnormal message to the abnormal message processing unit for processing; for the IP fragment message, IP fragment recombination is carried out, the recombined message is sent to a message cache unit for processing, if the message is not the IP fragment message, the message is directly sent to the message cache unit for processing;
s52, the message buffer unit buffers the first N packets of a flow or buffers the message of the first N seconds of a flow according to the buffer strategy, and the message is delivered to the multi-layer message structure analysis unit for processing after buffer.
And S53, the multi-layer message structure analysis unit analyzes various package structures of the message, extracts message structure information and delivers the message structure information to the flow table unit.
S54, the flow table unit builds flow according to the message information of the innermost layer and maintains the state of the flow table; transmitting the message to an internet account rule matching unit and sharing the message to an application identification unit;
and S55, after being processed by the internet access account rule matching unit, the messages are sequentially transmitted to the quintuple rule matching unit, the feature code rule matching unit and the model rule matching unit for processing.
S56, the application identification unit identifies the application of the received message, outputs the identified application ID and matches the corresponding application rule; respectively delivering the application types to corresponding application rule matching units for processing;
s57, after all the rules are matched, entering an interface group rule checking unit, checking the hit rules in the interface group, and delivering the checked rules to a combined rule processing module;
s58, the combination rule processing module stores the hit result, judges whether the hit result meets the conditions of the combination rule, and if the hit result meets the conditions, the hit result is handed to the multi-user rule processing module;
s59, the multi-user rule processing module carries out user classification and priority sequencing on the hit rules, only the hit rule number with the highest priority is reserved for multiple rules hit by the same user, and the hit rule number with the highest priority hit by each user is respectively stored; rules hit by different users are not influenced mutually and are stored respectively;
and S510, according to the result of the multi-user rule processing, marking corresponding rule hit labels on the hit message and all messages on the flow to which the message belongs later, and sending out the messages.
9. The method for multi-user rule matching and traffic replication according to claim 8, wherein the step S510 specifically includes:
modifying internally defined communication protocol labels for the hit message and all messages on the flow to which the message belongs later, wherein the modified internally defined communication protocol labels carry message offset of a T L V initial position, message input interface group information, message flow direction and the number of hit users;
modifying the first V L AN label of the header of the message into a special V L AN number as a message copying identifier, and when the FPGA reads the V L AN label, performing message copying action;
and marking corresponding rule hit labels at the tail parts of the hit message and all messages on the flow to which the message belongs later, wherein the rule hit labels comprise the identified application number, the rule ID of the highest priority of the hit user, the V L AN number of the hit user and the message stripping type required by the user, and the rule ID of the highest priority of the hit user, the V L AN number of the hit user and the message stripping type required by the user are multiple groups and are determined according to the number of the hit users.
10. The method for multi-user rule matching and traffic replication according to claim 9, wherein the step S6 specifically comprises:
reading the marked message with the multi-user rule hit by the FPGA message copying module, extracting the content in AN internally defined communication protocol label, reading the initial offset of T L V and the number of hit users, reading the rule ID of the highest priority of the hit users of each group of users at the tail of the message, the V L AN number of the hit users and a message stripping type label required by the user, copying the message, and replacing the special V L AN number in the copied message according to the V L AN number of the hit users in each group of labels;
and stripping the copied message according to the message stripping type of the message stripping type label required by the user, and adding a rule ID label which is corresponding to the user and hits the highest priority of the user into the copied message.
CN202010157979.9A 2020-03-09 2020-03-09 System and method for multi-user rule matching and flow replication Pending CN111404798A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010157979.9A CN111404798A (en) 2020-03-09 2020-03-09 System and method for multi-user rule matching and flow replication

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010157979.9A CN111404798A (en) 2020-03-09 2020-03-09 System and method for multi-user rule matching and flow replication

Publications (1)

Publication Number Publication Date
CN111404798A true CN111404798A (en) 2020-07-10

Family

ID=71436094

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010157979.9A Pending CN111404798A (en) 2020-03-09 2020-03-09 System and method for multi-user rule matching and flow replication

Country Status (1)

Country Link
CN (1) CN111404798A (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111404834A (en) * 2020-03-23 2020-07-10 湖北微源卓越科技有限公司 System and method for multi-user rule matching and flow replication
CN112968841A (en) * 2021-03-04 2021-06-15 杭州迪普信息技术有限公司 Message convergence and distribution method and device and electronic equipment
CN113179229A (en) * 2021-03-10 2021-07-27 长沙星融元数据技术有限公司 Verification method, verification device, storage medium and electronic equipment
CN113507421A (en) * 2021-06-08 2021-10-15 南京中新赛克科技有限责任公司 High-performance refined shunting method based on application
CN115277582A (en) * 2022-07-13 2022-11-01 清华大学 Software data packet classification acceleration method, device, equipment and storage medium
CN115473819A (en) * 2022-08-30 2022-12-13 电信科学技术第十研究所有限公司 System and method for processing mass internet traffic based on dynamic rule driving

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103647708A (en) * 2013-11-29 2014-03-19 曙光信息产业(北京)有限公司 ATCA-based data message processing board
CN105450473A (en) * 2015-12-07 2016-03-30 湖南戎腾网络科技有限公司 User traceability association method for LTE network and front-end collector
US20170214625A1 (en) * 2016-01-22 2017-07-27 Citrix Systems, Inc. System and method of providing increased data optimization based on traffic priority on connection
CN107342926A (en) * 2017-06-13 2017-11-10 国家计算机网络与信息安全管理中心 A kind of method of multi-service Rapid matching distribution
CN108683610A (en) * 2018-04-12 2018-10-19 国家计算机网络与信息安全管理中心 A kind of system and method realized multi-service rule match and flow and replicated
CN108881033A (en) * 2018-06-20 2018-11-23 湖南戎腾网络科技有限公司 Highspeed user's source tracing method towards LTE network based on FPGA+NPU
CN110851672A (en) * 2019-11-13 2020-02-28 天津光电通信技术有限公司 Method for realizing multi-hit based on TCAM

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103647708A (en) * 2013-11-29 2014-03-19 曙光信息产业(北京)有限公司 ATCA-based data message processing board
CN105450473A (en) * 2015-12-07 2016-03-30 湖南戎腾网络科技有限公司 User traceability association method for LTE network and front-end collector
US20170214625A1 (en) * 2016-01-22 2017-07-27 Citrix Systems, Inc. System and method of providing increased data optimization based on traffic priority on connection
CN107342926A (en) * 2017-06-13 2017-11-10 国家计算机网络与信息安全管理中心 A kind of method of multi-service Rapid matching distribution
CN108683610A (en) * 2018-04-12 2018-10-19 国家计算机网络与信息安全管理中心 A kind of system and method realized multi-service rule match and flow and replicated
CN108881033A (en) * 2018-06-20 2018-11-23 湖南戎腾网络科技有限公司 Highspeed user's source tracing method towards LTE network based on FPGA+NPU
CN110851672A (en) * 2019-11-13 2020-02-28 天津光电通信技术有限公司 Method for realizing multi-hit based on TCAM

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111404834A (en) * 2020-03-23 2020-07-10 湖北微源卓越科技有限公司 System and method for multi-user rule matching and flow replication
CN112968841A (en) * 2021-03-04 2021-06-15 杭州迪普信息技术有限公司 Message convergence and distribution method and device and electronic equipment
CN113179229A (en) * 2021-03-10 2021-07-27 长沙星融元数据技术有限公司 Verification method, verification device, storage medium and electronic equipment
CN113507421A (en) * 2021-06-08 2021-10-15 南京中新赛克科技有限责任公司 High-performance refined shunting method based on application
CN115277582A (en) * 2022-07-13 2022-11-01 清华大学 Software data packet classification acceleration method, device, equipment and storage medium
CN115277582B (en) * 2022-07-13 2024-03-26 清华大学 Software data packet classification acceleration method, device, equipment and storage medium
CN115473819A (en) * 2022-08-30 2022-12-13 电信科学技术第十研究所有限公司 System and method for processing mass internet traffic based on dynamic rule driving
CN115473819B (en) * 2022-08-30 2024-05-17 电信科学技术第十研究所有限公司 Mass internet flow processing system and method based on dynamic rule driving

Similar Documents

Publication Publication Date Title
CN111404798A (en) System and method for multi-user rule matching and flow replication
US7373429B2 (en) Integrated IP network
CN1937541B (en) Network performance test method
EP2100406B1 (en) Method and apparatus for implementing multicast routing
CN100553199C (en) Method of realizing group broadcasting, system and equipment based on the PCIE switching network
US20050175022A1 (en) Bridge apparatus and logical queue control method
CN101632273A (en) Methods, systems, and computer program products for source-aware IP routing at a media gateway
CN105591974A (en) Message processing method, device and system
CN111404834A (en) System and method for multi-user rule matching and flow replication
CN106686553B (en) Method and device for carrying out multicast on core network EPC
CN109788247B (en) Method and device for identifying monitoring instruction
CN109120492B (en) Storage unit, source switch, message forwarding method and mirror image system
CN113132257B (en) Message processing method and device
CN109246081A (en) Aerial Electronic Equipment AFDX bus and A429, RS422 bus data conversion method
CN102771087A (en) Fast LSP alert mechanism
CN110809026A (en) File processing method and device, electronic equipment and storage medium
CN110417707B (en) Data transmission protection method, device, system and computer readable storage medium
CN112702254B (en) Message processing method and device and electronic equipment
US20050028043A1 (en) Method and apparatus for providing tandem connection, performance monitoring, and protection architectures over ethernet protocols
CN110691012B (en) Message processing method and tester
CN110677314B (en) Network interface testing method, system, electronic device and storage medium
CN106454882A (en) Method and device used for obtaining user call ticket xDR
CN104219160A (en) Method and device for generating input parameter
CN100527706C (en) Communication system, method and apparatus for providing mirroring service in the communication system
CN107124316A (en) Hardware based quick switching action implementation method in a kind of data communications equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
AD01 Patent right deemed abandoned

Effective date of abandoning: 20220208

AD01 Patent right deemed abandoned