CN117240947B - Message processing method, device and medium - Google Patents

Message processing method, device and medium Download PDF

Info

Publication number
CN117240947B
CN117240947B CN202311518307.6A CN202311518307A CN117240947B CN 117240947 B CN117240947 B CN 117240947B CN 202311518307 A CN202311518307 A CN 202311518307A CN 117240947 B CN117240947 B CN 117240947B
Authority
CN
China
Prior art keywords
message
matching
processing
strategy
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311518307.6A
Other languages
Chinese (zh)
Other versions
CN117240947A (en
Inventor
朱敏
曲贺
李桧
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuxi Muchuang Integrated Circuit Design Co ltd
Original Assignee
Wuxi Muchuang Integrated Circuit Design Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuxi Muchuang Integrated Circuit Design Co ltd filed Critical Wuxi Muchuang Integrated Circuit Design Co ltd
Priority to CN202311518307.6A priority Critical patent/CN117240947B/en
Publication of CN117240947A publication Critical patent/CN117240947A/en
Application granted granted Critical
Publication of CN117240947B publication Critical patent/CN117240947B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention discloses a message processing method, a message processing device and a message processing medium, which are suitable for the technical field of communication. And obtaining matching subscript information of the message to be processed, identifying the type of the message, and matching through a hardware parallel circuit to improve the matching processing efficiency. Determining a processing strategy according to the relation between the message type and the preset requirement, extracting fixed information relative to the existing message protocol, realizing the recombination of the data packets through the preset requirement, and finding out a corresponding processing strategy if the preset requirement is met or the preset requirement is not met, wherein the processing strategy is a data packet removing non-parsing data head strategy, a data packet skipping specified offset address strategy, a data packet removing and a data skipping specified strategy, and the corresponding information can be extracted according to different strategies without extracting from a fixed place, so that the flexibility of processing the message protocol is improved; meanwhile, for a new protocol message, a target message is extracted based on different processing strategies, so that redesign of a hardware circuit is avoided, the extraction period is shortened, and the research and development cost is saved.

Description

Message processing method, device and medium
Technical Field
The present invention relates to the field of communications technologies, and in particular, to a method, an apparatus, and a medium for processing a message.
Background
The data message parsing module of a conventional network processor is generally implemented by an integrated circuit (Application Specific Integrated Circuit, ASIC) circuit module for special applications, and the ASIC hardware circuit can only process the known message protocol, so that the fixed information can be extracted.
Because of certain limitation on processing of known message protocols, if a new protocol message is added currently, the original ASIC hardware circuit cannot be changed, hardware parameters need to be modified, and different hardware circuits need to be redesigned, so that design period and research and development cost are increased, and processing flexibility is poor.
Therefore, there is a need for a solution to the art to increase the flexibility of processing message protocols while reducing development costs.
Disclosure of Invention
The invention aims to provide a message processing method, a device and a medium, which are used for solving the technical problems of the limitation of the existing hardware circuit in extracting fixed information, the increase of design period and research and development cost caused by the need of redesigning a corresponding hardware circuit when a new message is added, and poor flexibility.
In order to solve the above technical problems, the present invention provides a method for processing a message, including:
Obtaining matching subscript information of a message to be processed, wherein the matching subscript information is obtained through matching of a hardware parallel circuit;
identifying the message type of the message to be processed according to the matching subscript information;
determining a corresponding processing strategy according to the relation between the message type and a preset requirement, wherein the preset requirement is a requirement for reorganizing a data packet corresponding to the message to be processed, and the processing strategy at least comprises one strategy of removing the data packet without analyzing a data header strategy, jumping to designate an offset address strategy, removing the data packet and jumping to designate the strategy;
processing the message to be processed according to the message type and the corresponding processing strategy to determine a final target message;
the determining a corresponding processing strategy according to the relation between the message type and a preset requirement comprises the following steps:
judging whether the message identifier of the message type meets the preset requirement or not;
if yes, determining that the processing strategy is the data head removing strategy or the data packet removing and jump specifying strategy;
if not, determining that the processing strategy is the jump specified offset address strategy.
Preferably, the determining process for removing the data packet without parsing the data header policy includes:
presetting a mapping relation between message types corresponding to each matched subscript information and unresolved message header offset information, wherein the number of the message types is the same as the parallel path number of the hardware parallel circuit;
obtaining a target message type;
determining a corresponding target offset address according to the target message type and the mapping relation;
and removing the message corresponding to the target offset address from the message to be processed to obtain a reorganized data packet so as to establish a strategy for removing the data packet without analyzing the data header.
Preferably, the determining process for removing the data packet and skipping the designated policy includes:
presetting a mapping relation between message types corresponding to each matched subscript information and unresolved message header offset information, wherein the number of the message types is the same as the parallel path number of the hardware parallel circuit;
obtaining a target message type;
determining a corresponding target offset address according to the target message type and the mapping relation;
removing the message corresponding to the target offset address from the message to be processed to obtain a reorganized data packet;
Presetting a mapping relation between a message type corresponding to each matched subscript information and specified analysis head offset information, wherein a first offset address of the specified analysis head offset information is a next offset address of an unresolved message head offset address and is relative offset information under a data message type;
presetting a mapping relation between each piece of matched subscript information and the data message type, wherein the message type at least comprises one data message type;
and pre-analyzing the recombined data packet according to the target message type to obtain a corresponding target data message type so as to establish the data packet removal and jump specifying strategy.
Preferably, the determining of the jump specified offset address policy includes:
presetting a mapping relation between a message type corresponding to each matched subscript information and specified analysis head offset information, wherein the specified analysis head offset information is relative offset information under a data message type;
presetting a mapping relation between each piece of matched subscript information and the data message type, wherein the message type at least comprises one data message type;
and pre-analyzing the message to be processed according to the target message type and the specified analysis head offset information to obtain the corresponding target data message type.
Preferably, the processing the message to be processed according to the message type and the corresponding processing policy to determine a final target message includes:
acquiring an extraction keyword strategy, wherein the extraction keyword strategy is based on a data packet extraction strategy corresponding to a data structure under the message type;
and extracting the current message to be processed according to the message type, the corresponding processing strategy and/or the extraction keyword strategy to obtain the target message.
Preferably, the determining process of the keyword extraction strategy includes:
the method comprises the steps of pre-establishing a mapping relation of data message types corresponding to each matched subscript information and extracting a specific offset address, wherein when a processing strategy corresponding to the message type is the data header removing strategy, the corresponding data message type is the pre-established data structure type of the message type and the reorganized data packet, and the specific offset address is the offset address of a message protocol custom field of the message to be processed;
obtaining a target message type of the message to be processed, wherein when a processing strategy corresponding to the message type is the data head removing strategy, the message to be processed is the reorganized data packet;
Analyzing the message to be processed according to the target message type to obtain a corresponding target message type;
determining a corresponding feature extraction position according to the target message type and the mapping relation of extracting the specific offset address;
and extracting according to the message data corresponding to the characteristic extraction position to establish the keyword extraction strategy.
Preferably, the determining process of the matching subscript information of the message to be processed includes:
acquiring a current data packet;
and carrying out matching processing on the current data packet through the hardware parallel circuit to obtain the matching subscript information.
Preferably, when the number of packet types of the current data packet is smaller than the parallel path number of the hardware parallel circuit, the matching processing of the current data packet by the hardware parallel circuit to obtain the matching subscript information includes:
determining a target matching circuit according to the number of message types of the current data packet and the hardware parallel circuit;
adding corresponding registers in each parallel circuit of the hardware parallel circuit;
determining a corresponding register mark value according to the target matching circuit, parallel circuits except the target matching circuit and a corresponding register;
And carrying out matching processing on the current data packet based on the target matching circuit with the register mark value as a matching mark to obtain the matching index information.
Preferably, when the number of packet types of the current data packet is smaller than the parallel path number of the hardware parallel circuit, the matching processing of the current data packet by the hardware parallel circuit to obtain the matching subscript information includes:
determining a target matching circuit according to the number of message types of the current data packet and the hardware parallel circuit;
matching the current data packet according to the hardware parallel circuit to obtain first matching subscript information;
determining successful matching index information except the target matching circuit in the first matching index information, and taking the successful matching index information as second matching index information;
modifying the second matching index information to determine failure matching index information;
and updating the first matching index information with the failure matching index information obtained after modification to obtain the final matching index information.
Preferably, the matching processing of the current data packet by the hardware parallel circuit to obtain the matching subscript information includes:
Determining each message keyword corresponding to the current data packet and having the same parallel path number as the hardware parallel circuit;
carrying out hash calculation on each message keyword to obtain a corresponding actual hash value;
and comparing each actual hash value with a corresponding preset hash value to determine the matching index information.
Preferably, the determining each message key word corresponding to the current data packet and having the same number of parallel paths of the hardware parallel circuit includes:
acquiring a starting byte address and a preset byte number in each parallel circuit in the hardware parallel circuit, wherein the preset byte number is a continuous byte number or a discontinuous byte number starting from the starting byte address;
and extracting the current data packet according to the starting byte address and the preset byte number to obtain a corresponding message keyword.
Preferably, the determining each message key word corresponding to the current data packet and having the same number of parallel paths of the hardware parallel circuit includes:
acquiring a starting byte address and a preset byte number in each parallel circuit in the hardware parallel circuit, wherein the preset byte number is a continuous byte number or a discontinuous byte number starting from the starting byte address;
Processing the current data packet according to the initial byte address and the preset byte number to determine a target byte, wherein corresponding bit operation is stored in each byte in advance, and the bit operation comprises mask and shift number;
and carrying out operation processing on the current data packet according to the mask and the shift number corresponding to the target byte so as to determine a corresponding message keyword.
Preferably, the matching processing of the current data packet by the hardware parallel circuit to obtain the matching subscript information includes:
determining each message keyword corresponding to the current data packet and having the same parallel path number as the hardware parallel circuit;
splitting the message keywords according to the number of register bits corresponding to the register types to obtain split message keywords, wherein the register types comprise a selected keyword index register, a starting boundary register, an ending boundary register and an operation register, and index values set by the selected keyword index register correspond to registers under each register type;
determining a corresponding target extracted message field according to the split message keywords and the index value;
Determining an extraction interval range according to the values corresponding to the initial boundary register and the end boundary register;
determining that the target extraction message segment is matched in the extraction interval range according to the value of the operation register so as to determine a corresponding preset value;
carrying out matching calculation on each message keyword to obtain a corresponding actual value;
and comparing the corresponding actual value with the corresponding preset value to determine the matching index information.
Preferably, after the hash calculation is performed on each message keyword to obtain a corresponding actual hash value, before comparing each actual hash value with a corresponding preset hash value to determine the matching subscript information, the method further includes:
obtaining mask information corresponding to each parallel circuit;
performing mask operation processing on each actual hash value and the corresponding mask information to obtain a masked hash value;
and comparing the masked hash value with the corresponding preset hash value to determine the matched index information.
Preferably, the comparing each of the actual hash values with a corresponding preset hash value to determine the matching index information includes:
Acquiring a target actual hash value of the message to be processed;
when the target actual hash value is the same as the preset hash value, acquiring a cascade attribute mark of a comparison circuit to which the message to be processed belongs in the same hash value;
if the cascade attribute marks are cascade matching, acquiring a cascade circuit of the cascade matching;
and determining that the matching index information is the matching index information of the cascade circuit according to the fact that the preset hash value corresponding to the cascade circuit is identical to the target actual hash value.
Preferably, the cascade matching includes intra-group cascade matching and inter-group cascade matching, and the determining process of the cascade circuit includes:
when the cascade attribute marks are cascade matching in the group, determining a continuous cascade circuit to which the message belongs according to the layer number and the type of each layer of the message, and respectively setting marks corresponding to the layer number of the message in the continuous cascade circuit to determine the cascade circuit;
adding cascade registers in each circuit of the hardware parallel circuit for marking cascade index values when the cascade attribute marks the cascade matching among groups;
determining a target cascade circuit to which the message belongs according to the layer number and the type of each layer of the message, wherein the cascade circuit in the target cascade circuit is a discontinuous cascade circuit;
And setting the corresponding cascade index value and marking the layer number of the message in the target cascade circuit to determine the cascade circuit.
Preferably, when each of the actual hash values is different from the corresponding preset hash value, the method further includes:
outputting a special mark value as the matching index information, wherein the special mark value is a mark value which is distinguished from serial numbers corresponding to the parallel paths of the hardware parallel circuit;
and returning to the step of acquiring the current data packet to acquire the next data packet.
In order to solve the technical problem, the present invention further provides a message processing device, including:
the acquisition module is used for acquiring matching subscript information of the message to be processed, wherein the matching subscript information is obtained through matching of hardware parallel circuits;
the identification module is used for identifying the message type of the message to be processed according to the matched subscript information;
the determining module is used for determining a corresponding processing strategy according to the relation between the message type and a preset requirement, wherein the preset requirement is a requirement for reorganizing a data packet corresponding to the message to be processed, and the processing strategy at least comprises one strategy of removing the data packet without analyzing a data header strategy, jumping to designate an offset address strategy, removing the data packet and jumping to designate the strategy;
And the extraction module is used for processing the message to be processed according to the message type and the corresponding processing strategy so as to determine a final target message.
In order to solve the technical problem, the present invention further provides a message processing device, including:
a memory for storing a computer program;
and a processor for implementing the steps of the message processing method as described above when executing the computer program.
To solve the above technical problem, the present invention further provides a computer readable storage medium, where a computer program is stored, where the computer program when executed by a processor implements the steps of the message processing method described above.
The invention provides a message processing method, which is used for acquiring matching subscript information of a message to be processed based on a data packet so as to identify the type of the message, and matching is performed through a hardware parallel circuit so as to improve the matching processing efficiency. Determining a corresponding processing strategy according to the relation between the message type and the preset requirement, and extracting fixed information relative to the existing message protocol processing, wherein the method and the device realize the recombination of the data packets through the preset requirement, and if the preset requirement is met, the corresponding data packet removal does not analyze the data header strategy or remove the data packet and jump the designated strategy; if the preset requirement is not met, the corresponding jump designates an offset address strategy. The processing strategy is at least one strategy of removing the data packet without analyzing the data head strategy, jumping to designate an offset address strategy, removing the data packet and jumping to designate the strategy, and corresponding information can be extracted in a rich way according to different strategies without extracting from a fixed place, such as the fixed place is extracted from an initial address 0, and the like, so that the flexibility of processing a message protocol is improved; meanwhile, for a new protocol message, a target message can be extracted based on different processing strategies, so that redesign of a hardware circuit is avoided, the extraction period is shortened, and the research and development cost is saved.
In addition, the invention also provides a message processing device and a medium, which have the same beneficial effects as the message processing method.
Drawings
For a clearer description of embodiments of the present invention, the drawings that are required to be used in the embodiments will be briefly described, it being apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to the drawings without inventive effort for those skilled in the art.
FIG. 1 is a flow chart of a message processing method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a hardware parallel circuit according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a GENEVE message format provided in this embodiment;
fig. 4 is a schematic diagram of a get header structure according to an embodiment of the present invention;
FIG. 5 is a flow chart of a circuit matching process according to an embodiment of the present invention;
FIG. 6 is a flowchart of a circuit packet matching method according to an embodiment of the present invention;
fig. 7 is a schematic diagram of the first 32 bits of an IP header according to an embodiment of the present invention;
fig. 8 is a schematic diagram of an ethernet header with multiple VLAN headers according to an embodiment of the present invention;
Fig. 9 is a schematic structural diagram of a VLAN Tag header according to an embodiment of the present invention;
FIG. 10 is a schematic diagram of a serial cascade of 4 parallel circuits according to an embodiment of the present invention;
FIG. 11 is a schematic diagram of a discontinuous set of 4 parallel circuit cascades according to an embodiment of the present invention;
FIG. 12 is a flow chart of a message processing for cascade matching according to an embodiment of the present invention;
fig. 13 is a flowchart of a network card update configuration packet according to an embodiment of the present invention;
FIG. 14 is a diagram illustrating a message processing apparatus according to an embodiment of the present invention;
fig. 15 is a block diagram of another message processing apparatus according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without making any inventive effort are within the scope of the present invention.
The core of the invention is to provide a message processing method, a device and a medium, which are used for solving the technical problems of the limitation of the existing hardware circuit for extracting fixed information, the increase of design period and research and development cost caused by the need of redesigning the corresponding hardware circuit when a new message is added, and poor flexibility.
In order to better understand the aspects of the present invention, the present invention will be described in further detail with reference to the accompanying drawings and detailed description.
It should be noted that, the integrated circuit of the hardware circuit is a circuit that integrates a certain number of common electronic components, such as resistors, capacitors, transistors, and the like, and the connection lines between these components together through a semiconductor process to have a specific function. ASIC chips are typically considered to be an integrated circuit designed for a specific purpose. A network processor, a special integrated circuit with high programmability. An important step in processing data packets by a network processor is to parse each incoming data packet according to rules and conditions actually applied, especially to parse a header or a certain part of the entire content of the packet, and then to perform various types of processing operations on the data packet. However, in the processing of the data packet, only for the fixed information, for example, resolving from the fixed offset address, it can be known that the offset addresses corresponding to different message protocols are different, and if a new message protocol needs to be resolved, since the hardware circuit is not provided with a processing mechanism corresponding to the new message protocol, it is necessary to design a hardware circuit for the new message protocol again, and applicability to any message protocol cannot be achieved. Therefore, the message processing method provided by the invention considers the technical problems.
Fig. 1 is a flowchart of a message processing method according to an embodiment of the present invention, as shown in fig. 1, where the method includes:
s11: obtaining matching subscript information of a message to be processed;
the matching subscript information is obtained through matching of hardware parallel circuits;
s12: identifying the message type of the message to be processed according to the matched subscript information;
s13: determining a corresponding processing strategy according to the relation between the message type and the preset requirement;
the processing strategy at least comprises a strategy for removing the data packet without analyzing the data header, a strategy for jumping to a specified offset address, a strategy for removing the data packet and a strategy for jumping to the specified address;
s14: processing the message to be processed according to the message type and the corresponding processing strategy to determine a final target message;
the method for determining the corresponding processing strategy according to the relation between the message type and the preset requirement comprises the following steps:
judging whether a message identifier of a message type meets a preset requirement or not;
if yes, determining that the processing strategy is a strategy for removing the data packet without analyzing the data header or a strategy for removing the data packet and skipping to the designated strategy;
if not, determining that the processing strategy is a jump specified offset address strategy.
Specifically, matching subscript information of the message to be processed is obtained, and the matching subscript information can be obtained based on hash value matching, or can be obtained through other matching modes (such as range comparison matching). And comparing and matching the corresponding ranges, distinguishing the hash value from a specific value, and splitting a plurality of byte digits to determine the corresponding target range for matching. As a matching subscript, the subscript value may be used for subsequent processing operations of the message type. For the matching beacon information, which is obtained by matching the hardware parallel circuits, fig. 2 is a schematic diagram of a hardware parallel circuit provided by the embodiment of the invention, as shown in fig. 2, by extracting offset addresses of 64 parallel circuits, and then comparing hash values, matching subscript information can be obtained by matching hits. The offset addresses corresponding to the 64 paths are different, but the number of bytes corresponding to the offset addresses in the corresponding paths is the same.
And identifying the corresponding message type according to the matched subscript information, namely, the matched subscript information is a specific sequence number, and if the sequence number is number 2, the corresponding message type is a preset number 2 mapped message type.
It should be noted that, the matching index information includes specific 64 paths of corresponding 0-63 serial numbers and special tag values, that is, two corresponding hit results are included, one hit is performed, and the other hit is not performed, where the matching index information outputs the special tag values under the condition that no hit is performed. The special tag value in this embodiment is mainly a tag (0-63 sequence numbers) that distinguishes the sequence numbers corresponding to the number of parallel paths of the hardware parallel circuit. Thus, successfully matched circuits have subscripts of 0-63 and all circuits do not match have a flag value of mismatch_flag. The mismatch_flag may be any value other than 0 to 63. A number with all binary bits of 1 can be taken without loss of generality.
1) If the "successful match index" bit width is 1 byte, then the mismatch_flag may take 0xFF;
2) If the "successful match index" bit width is 2 bytes, then the mismatch_flag may take 0xFFFF;
3) If the "successful match index" bit width is 4 bytes, then the mismatch_flag may take 0xFFFF_FFFF;
it is to be understood that the bit width in this embodiment may be set according to practical situations, and is not specifically limited. The subsequent processing of the corresponding matching index information in this embodiment is only for the case of one hit.
According to the relation between the message type and the preset requirement, the corresponding processing strategy is determined, and it should be noted that the preset requirement in this embodiment is a requirement for reorganizing the data packet corresponding to the message to be processed, whether the message type meets the preset requirement or not, the corresponding processing strategy can be determined, and the preset requirement is only a branch for determining the processing strategy. Correspondingly, the preset requirement can also be an application scene requirement set by a user according to actual conditions. The processing strategy at least comprises one strategy of three strategies, namely a strategy for removing the data packet without analyzing the data header, a strategy for jumping to a specified offset address, a strategy for removing the data packet and a strategy for jumping to the specified offset address.
As an embodiment, determining a corresponding processing policy according to a relation between a message type and a preset requirement includes:
judging whether a message identifier of a message type meets a preset requirement or not;
if yes, determining that the processing strategy is a strategy for removing the data packet without analyzing the data header or a strategy for removing the data packet and skipping to the designated strategy;
if not, determining that the processing strategy is a jump specified offset address strategy.
It should be noted that, the message identifier to which the message type belongs is used to identify what purpose the message to be processed is used for, and corresponds to different preset requirements, if the specific information of the message identifier is that the parsing header needs to be removed, and the preset requirements of the reorganization data packet are met, the processing policy is determined to be a policy for removing the data packet without parsing the data header or removing the data packet and a skip designating policy. Because both strategies in the embodiment can be selected, one strategy can be selected from the two strategies according to the priority, the priority can be that the processing speed is higher, the flow is less, and the strategy that the data packet is removed and the data head is not analyzed is selected; the priority may be more accurate, and the policy of removing the data packet and skipping the designated policy may be selected, which is not limited herein, and may be set according to the actual situation.
If the specific information of the message identifier is that the analysis head does not need to be removed and the preset requirement of the recombined data packet is not met, determining a processing strategy to be a jump specified offset address strategy. Processing is performed according to different processing strategies. The packet-removal-without-parse header policy is based on the fact that when parsing a packet, it is often necessary to remove some unnecessary header, e.g., in an internal switching network, a network card will often receive various packets, where there are normal header packets, but there may be abnormal packets with private header before the media access control (Media Access Control, MAC) address. This is common in switching networks of stacked (Stack) configured switches. In this case, removing the unresolved header becomes an important first step in parsing the data message. It should be noted that, removing the unresolved header of the packet means to reassemble the packet.
The traditional data packet analysis is based on the traditional data packet analysis from the two-layer header, and along with the continuous development of network protocols, the format of the data packet is more and more flexible, and various requirements exist, such as the analysis from the three-layer header and the four-layer header of the data packet; if the message with the tunnel header is added, the field of the tunnel header needs to be skipped for analysis; switching to a different tunneling protocol; besides returning the offset of the needed resolving header, the requirements such as data message type identification of the returned message are required, and the resolving is needed to jump to the appointed offset address for resolving according to various requirements, wherein the strategy of jumping to the appointed offset address in the embodiment is established on the basis.
The jump to the address of the specified offset, such as a data packet, including an internet protocol address (Internet Protocol Address, IP) packet, a two-layer header, a three-layer header, a four-layer header, a tunnel header, etc., may be changed based on different protocols, for example, the jump to the corresponding three-layer header or four-layer header according to the jump specified offset address policy is required. In addition, the specified offset address, including offset address 0 to any one layer of offset address, may be skipped, and only the specific address of the offset address may be determined.
The data packet removal and jump specifying strategy is aimed at the combination form of the two strategies, and is firstly removed and then jumped.
And processing the message to be processed according to the corresponding message type and the processing strategy to obtain a final target message. The processing of this embodiment is to extract the data packet in the data packet based on the relative offset address of the data packet type under the packet type, or may be the data packet aiming at the relative offset address of the data packet type under the packet type. It should be noted that, the existing extraction of the data message is based on how the extracted field is used, and the invention extracts the specific field or the message of the specific message type.
The extraction policy may be a specific field extracted from the data message based on the data message type under the corresponding message type, or may be based on the whole data extraction after the message type, which is not limited herein. For example, the a message type includes IP header, two-layer header, three-layer header, four-layer header, tunnel header, and the like, and one embodiment is data specific field extraction under each data message type.
The method for processing the message obtains the matching subscript information of the message to be processed based on the data packet so as to identify the type of the message, and improves the matching processing efficiency by matching through the hardware parallel circuit. Determining a corresponding processing strategy according to the relation between the message type and the preset requirement, and extracting fixed information relative to the existing message protocol processing, wherein the method and the device realize the recombination of the data packets through the preset requirement, and if the preset requirement is met, the corresponding data packet removal does not analyze the data header strategy or remove the data packet and jump the designated strategy; if the preset requirement is not met, the corresponding jump designates an offset address strategy. The processing strategy is at least a strategy for removing the data packet without analyzing the data header, a strategy for jumping to designate an offset address, a strategy for removing the data packet and a strategy for jumping to designate, corresponding information can be extracted in a rich way according to different strategies, extraction from a fixed place is not needed, and if the fixed places are all extracted from an initial address 0, the flexibility of processing a message protocol is improved; meanwhile, for a new protocol message, a target message can be extracted based on different processing strategies, so that redesign of a hardware circuit is avoided, the extraction period is shortened, and the research and development cost is saved.
As one embodiment, the determining process for removing the data packet without parsing the data header policy includes:
presetting a mapping relation between message types corresponding to each matched subscript information and unresolved message header offset information, wherein the number of the message types is the same as the parallel path number of a hardware parallel circuit;
obtaining a target message type;
determining a corresponding target offset address according to the type of the target message and the mapping relation;
and removing the message corresponding to the target offset address from the message to be processed to obtain the reorganized data packet so as to establish a strategy for removing the data packet without analyzing the data header.
Specifically, table 1 is a data structure table without parsing the header offset, and as shown in table 1, the number of the set mapping relationship between the message types corresponding to each matching subscript information and the header offset information is the same as the number of parallel paths of the hardware parallel circuit.
Table 1 data structure table without parsing header offset
And obtaining a target message type of the message to be processed from each message type, namely, the message type corresponding to the sequence number matched with the subscript information, determining a target OFFSET address according to the target message type and the corresponding mapping relation, wherein if the lower index value is 0, the corresponding target message type is 0 message type, and the resolved message header OFFSET address is REMOVE_HDR_OFFSET_0. And removing the message corresponding to the target offset address from the message to be processed, wherein the rest message information is the reorganized data packet.
And removing the private header to obtain correct data packet information so as to establish the strategy.
As one embodiment, the determining process for removing the data packet and the jump specifying policy includes:
presetting a mapping relation between message types corresponding to each matched subscript information and unresolved message header offset information, wherein the number of the message types is the same as the parallel path number of a hardware parallel circuit;
obtaining a target message type;
determining a corresponding target offset address according to the type of the target message and the mapping relation;
removing a message corresponding to the target offset address from the message to be processed to obtain a reorganized data packet;
presetting a mapping relation between a message type corresponding to each matched subscript information and specified analysis head offset information, wherein a first offset address of the specified analysis head offset information is a next bit offset address of an unresolved message head offset address and is relative offset information under the data message type;
presetting a mapping relation between each piece of matched subscript information and a data message type, wherein the message type at least comprises one data message type;
and pre-analyzing the recombined data packet according to the target message type to obtain the corresponding target data message type so as to establish a data packet removal and jump specifying strategy.
It should be noted that, the partial packet removal policy of the packet removal and jump specifying policy is the same as the packet removal and header resolution failure policy described above, and will not be described in detail herein, reference may be made to the above embodiments.
After obtaining the reassembled data packet, a mapping relationship between the message type corresponding to each matching subscript information and the specified parsing head offset information is preset, and it should be noted that the first offset address of the specified parsing head offset information is the next bit offset address of the non-parsed message head offset address in the above embodiment, and is the relative offset information under the data message type. The mapping relation between each matched subscript information and the data message type is preset, wherein the message type at least comprises one data message type. That is, a message type may include one or more data message types.
Table 2 is a table of specified parsing head offset information, as shown in table 2, and the subscript value, specified parsing head offset, and mapping relationship established by the data packet type. And pre-analyzing the recombined data packet by using the subscript value to obtain the target data packet type.
Table 2 specifies the parse header offset information table
As one embodiment, the determining process of the jump specifying offset address policy includes:
presetting a mapping relation between a message type corresponding to each matched subscript information and specified analysis head offset information, wherein the specified analysis head offset information is relative offset information under the data message type;
presetting a mapping relation between each piece of matched subscript information and a data message type, wherein the message type at least comprises one data message type;
and pre-analyzing the message to be processed according to the target message type and the specified analysis head offset information to obtain the corresponding target data message type.
It should be noted that, the jump specified offset address policy in this embodiment is based on the relative offset information under the data message type in the message to be processed, and is the same as the mapping relationship established in the above table 2, and will not be described herein.
Designating the offset of the analysis head to start to analyze the message to be processed from the offset of the data packet; the data message type is obtained according to partial pre-analysis of the message and is used for analysis by a subsequent module.
After determining the three processing strategies, extracting the message to be processed according to the message type and the corresponding processing strategy to determine a final target message, wherein the method comprises the following steps:
Acquiring an extraction keyword strategy, wherein the extraction keyword strategy is based on a data packet extraction strategy corresponding to a data structure under a message type;
and extracting the current message to be processed according to the message type, the corresponding processing strategy and/or the keyword extraction strategy to obtain the target message.
Specifically, the message to be processed is processed according to the corresponding message type and the processing strategy to obtain the final target message. The extraction process may be to extract the data message in the data packet based on the relative offset address of the data message type under the message type. It should be noted that, the extraction of the existing data message is based on how the extracted field is used, and the invention can extract the specific field.
The extraction policy is a specific field extracted from the data message based on the data message type under the corresponding message type. For example, the A message type comprises IP header, two-layer header, three-layer header, four-layer header, tunnel header and other data message types, and the invention extracts the data specific field under each data message type.
The extraction key policy is a field extraction that begins after an offset address under a specific field. And after the matching hit, extracting the data of the corresponding features based on the matching index information and the corresponding processing strategy.
In addition, the invention can extract the current message to be processed based on the message type and the corresponding processing strategy to obtain the target message. Without limitation, in this embodiment, extraction of the data specific field is not required to improve flexibility of obtaining the target message.
As one embodiment, the determination process for extracting the keyword policy includes:
the method comprises the steps of pre-establishing a mapping relation of data message types corresponding to each matched subscript information and extracting a specific offset address, wherein when a processing strategy corresponding to the message types is a data header removing strategy, the corresponding data message types are data structure types of the pre-established message types and the reorganized data packets, and the specific offset address is an offset address of a message protocol custom field of a message to be processed;
obtaining a target message type of a message to be processed, wherein when a processing strategy corresponding to the message type is a data head strategy which is not analyzed by a data packet removal method, the message to be processed is a reorganized data packet;
analyzing the message to be processed according to the target message type to obtain a corresponding target message type;
determining a corresponding feature extraction position according to the type of the target message and the mapping relation of the specific offset address;
And extracting according to the message data corresponding to the feature extraction position to establish an extraction keyword strategy.
Specifically, table 3 is a specific field table of the extracted packet, and as shown in table 3, a mapping relationship between the data packet type corresponding to each matching subscript information and the extracted specific offset address is established, and it should be noted that, if the extraction policy is based on the premise of removing the data packet without parsing the data header policy, the data packet type is the data structure type of the data packet after the packet is reassembled and the established packet type. Correspondingly, regarding byte addresses, this embodiment is only a preferred manner, but other bytes may be used, or the number of bytes in a specific field of each extracted message may be different, which is not limited herein.
Table 3 specific field table of extracted message
According to the embodiment of the invention, through the establishment process of each processing strategy and the extraction process of the corresponding extraction strategy, corresponding information can be extracted in a rich way according to different strategies, extraction from a fixed place is not needed, related data can be written into the network card firmware, and the network card can complete a new function by loading the firmware when restarting. The related processing flow of the message can be completed by simply upgrading the firmware. Therefore, the method has better flexibility and configurability, and can be widely suitable for continuous updating and development of network scenes. It can be understood that the specific offset address extracted in this embodiment is an offset address of a custom field of a message protocol of a message to be processed, which may be an offset address of a field corresponding to the message protocol, or may be an offset address based on a body characteristic of the message, such as an extension field or a header of the message, etc., which is not limited herein, and may be set according to an actual situation.
As an embodiment, the processing policy of removing the data packet and skipping the designated policy is combined with extracting the keyword policy to extract a specific field from the current message to be processed to obtain the target message, for example, the tunneling protocol message is in a relatively complex message format. The description herein is based on the current state-of-the-art open source data virtualization encapsulation (tunnel) generic network virtualization encapsulation protocol (Generic Network Virtualization Encapsulation, GENEVE). When the existing hardware analysis circuit is developed, the extracted message keyword information is fixed, and the information of a well-defined protocol header is provided in advance. Such as:
1. inner layer protocol: two-layer, three-layer, four-layer and other protocol header fields;
2. outer layer protocol: protocol fields of two layers, three layers, four layers and the like;
3. the tunneling protocol GENEVE itself has a fixed field in the protocol header portion.
For some special data fields, specific fields cannot be flexibly extracted, such as:
1. extension field of tunneling protocol: because this field is optional, variable length;
2. message body information: information of the payload specific location of the message body, etc.
Fig. 3 is a schematic diagram of a format of a get message provided in this embodiment, and as shown in fig. 3, a specific field of a parse message is the most core part of a network card. The performance of the network card is directly determined, usually as a performance bottleneck.
1) Analyzing an extension field of the tunnel protocol;
by combining codes of the unavailable protocol contents, when the message type is identified, the codes of the protocol to which the keywords belong and the position offset of the keywords relative to the protocol are appointed to be extracted, so that keyword extraction is completed, for example, the tunnel message extension header information is extracted, the protocol type can be appointed to the tunnel header, and the extraction of the extension header information is realized by setting an offset value.
Fig. 4 is a schematic diagram of a get header structure according to an embodiment of the present invention, as shown in fig. 4, and defined in RFC8926. Compared to the previous similar technology, a significant difference in GENEVE is: the metadata of the protocol itself is extensible. GENEVE provides an extensible GENEVE header, allowing more flexibility in business.
The Opt Len field, which occupies 6 binary digits in total, is used to indicate the length of the "variable length option" (Variable Length Options) of the header, i.e., the extracted portion for a particular field in this embodiment. Here one bit represents 4 bytes of "variable length option". Because there is only 6 bits, the maximum length of the "variable length option" is 252 bytes= (2≡6-1) 4 bytes. This variable length field presents a significant challenge to parsing a GENEVE message.
Traditional extraction and analysis are mainly based on hardware circuit extraction or software program extraction, for example, for hardware circuit extraction, the Opt Len field needs to be analyzed first, and the length information of the Opt Len is read;
2. because the length information represented by Opt Len may be 0-252 bytes, specific processing is required for each length of "variable length option".
This brings about the following disadvantages:
1. the hardware structure is complex: the complex parsing structure results in a hardware product with very large area and power consumption.
2. The stored data is large: since the location is not yet fixed, the header needs to be processed, and the header is first stored. Here, however, the structure to be stored may be very large.
After the hardware-based parallel circuit in this embodiment, the value of the Opt Len field is introduced into the message feature to determine the message type, and is a configurable item; in the same scenario, the "variable length option" of the data message is fixed within days. Therefore, the data (including the Opt Len field stored and used in the get message processing path) of the matching circuit can be set in advance to locate the correct start_paring_hdr_offset, so as to prepare for the subsequent extraction of the message characteristics. The method comprises the following specific steps:
1. Matching the GENEVE message;
2. extracting message characteristics, and identifying the message characteristics as a GENEVE message;
3. skipping the header and the variable length option, and returning to the correct start_paring_hdr_offset;
4. when the network message protocol is modified, the data used by the matching circuit is reconfigured, namely the Opt Len field in GENEVE and other field position offsets based on the data are reconfigured.
Based on the processing strategy, start_paring_hdr_offset and proco_type can be obtained, and the specific definition value of the variable length selection field of the GENEVE message is extracted. At this time, the relevant fields defined in the "variable length option" of the get may be acquired byte by byte as defined by the specific field table in table 3.
2) Analyzing the information of the message body of the message;
the data message is divided into two parts:
1. message header: also called protocol header. This section mainly describes some information of the transport protocol;
2. message body: also known as payload. This section mainly describes information that the message formally carries actual data.
In practice, the current network card usually uses some or all of the data in the 12-Tuple (12-Tuple), and the 12-Tuple specifically includes the following as a general feature describing the format of the data packet:
1. An input port: an input Port;
2. source MAC address: ether Source;
3. destination MAC address: ether Dst Address;
4. type of ethernet: ether Type;
5、VLAN ID:VLAN ID;
6. VLAN priority: VLAN Priority;
7. source IP address: IP Src Address;
8. destination IP address: IP Dst Address;
9. IP protocol: IP Proto;
10. IP TOS bit: IP ToS bits;
11. transmission control protocol (Transmission Control Protocol, TCP)/user datagram protocol (User Datagram Protocol, UDP) source port: TCP/UDP Src Port;
12. TCP/UDP destination port: TCP/UDP Dst Port;
these are mainly data fields of the message protocol header, which are predefined. And payload represents the data of the message body. Taking complicated protocols such as GENEVE as an example, the payload specific field of the L4 layer or L5 layer protocol of the internal message of the protocol message is extracted. This embodiment takes the hypertext transfer protocol (HyperText Transfer Protocol, HTTP) protocol of the L5 layer as an example:
the usual network card cannot parse specific bytes of the message body of these message protocols for the following reasons:
1. a typical network card generally can only parse the header of a data packet, thereby obtaining a specific value of a 12-tuple (12-tuple). However, because the payload part has too much content, the general network card cannot provide a parsing scheme for all the content at the time of design due to lack of configurability. Therefore, specific fields in the payload cannot be parsed;
2. The specific payload start position is not fixed: as the types of existing network protocols increase, the format of the message also becomes more and more complex. More fields of length appear in the header, and longer fields exist in the GENEVE protocol header as described above. This also makes the specific payload start position unfixed. Therefore, the existing network card fixed parsing circuit locates the starting address of the payload, and cannot extract the specific field specified by the payload.
Therefore, in this embodiment, the target message is obtained by removing the data packet and skipping the processing policy of the specified policy, and extracting the keyword policy to extract the specific field of the current message to be processed.
On the basis of the above embodiment, the determining process of the matching index information of the message to be processed in step S11 includes:
acquiring a current data packet;
and carrying out matching processing on the current data packet through a hardware parallel circuit to obtain matching subscript information.
As shown in fig. 2, a current data packet is obtained, in this embodiment, the current data packet is a data packet to be matched, parallel hash processing may be performed on the current data packet based on the hardware parallel circuit in fig. 2 to match subscript information, a matching manner different from a hash value may be adopted, a specific packet with the number of bytes or the number of bits extracted may be used to perform matching comparison in a specific range, or other matching algorithms may be used to perform comparison, which is not limited herein, as long as the message type corresponding to the current data packet can be identified by the matching subscript information.
As an embodiment, when considering that the given comparison packet type of the current data packet does not need all circuits of the hardware parallel circuit, in order to reduce the data matching time, when the number of packet types of the current data packet is smaller than the parallel circuit number of the hardware parallel circuit, matching processing is performed on the current data packet by the hardware parallel circuit to obtain matching subscript information, including:
determining a target matching circuit according to the number of message types of the current data packet and the hardware parallel circuit;
adding corresponding registers into each parallel circuit of the hardware parallel circuit;
determining corresponding register mark values according to the target matching circuit, the parallel circuits except the target matching circuit and the corresponding registers;
and performing matching processing on the current data packet by a target matching circuit which takes the register mark value as a matching mark to obtain matching subscript information.
Specifically, when the number of message types of the current data packet is smaller than the number of parallel paths of the hardware parallel circuit, the number of parallel paths of the currently set hardware parallel circuit is larger, and the matching process of the current data packet is more complex, so that a register and a corresponding judgment mode are required to be introduced on hardware, a target matching circuit which is actually required to be matched is determined according to the number of message types of the current data packet and the hardware parallel circuit, and a corresponding register is added in each parallel circuit of the hardware parallel circuit to set a register mark value. The specific register flag values set for the target matching circuit and the parallel circuits other than the target matching circuit are different, for example, 1 for the register flag value corresponding to the target matching circuit and 0 for the register flag values corresponding to the other parallel circuits.
The number of parallel paths of the hardware parallel circuit in fig. 2 is 64, and a 64-bit register can be correspondingly set, because a 32-bit register is common, 2 32-bit registers can be set based on the common 32-bit register, and 64 register bits correspond to 64 working circuits. Each binary bit of the register corresponds to whether one of the paths is active. 1, the circuit can work normally; and 0, the circuit is not operated. Fig. 5 is a flowchart of a circuit matching process according to an embodiment of the present invention, as shown in fig. 5:
s15: determining the serial number of the current matching circuit;
s16: judging whether the register bit of the current matching circuit is 1, if so, entering a step S17, and if not, entering a step S18;
s17: the current matching circuit is correct, and a matching result is returned;
s18: an error message is returned.
It will be appreciated that, after the current packet completes the determination, in order to facilitate matching of the next packet, a preset register flag value corresponding to the current packet is set to a default value so as to facilitate matching of the next packet.
As another embodiment, without a register determination process, marking and modifying by a marking value, when the number of packet types of the current data packet is smaller than the number of parallel paths of the hardware parallel circuit, performing matching processing on the current data packet by the hardware parallel circuit to obtain matching subscript information, including:
Determining a target matching circuit according to the number of message types of the current data packet and the hardware parallel circuit;
matching processing is carried out on the current data packet according to the hardware parallel circuit to obtain first matching subscript information;
determining successful matching index information except for the target matching circuit in the first matching index information, and taking the successful matching index information as second matching index information;
modifying the second matching index information to determine failure matching index information;
and updating the first matching index information with the failure matching index information obtained after modification to obtain final matching index information.
Specifically, the first matching subscript information is obtained according to a normal matching processing procedure, and it can be understood that, because the number of packet types of the current data packet is smaller than that of the hardware parallel circuits, taking a 64-way parallel circuit as an example, the current data packet uses only 10 ways, and the matching subscript information is output for the other 54 ways, and is included in the first matching subscript information. And screening successful matching index information from the first matching index information, wherein the successful matching index information is information obtained by error matching in the rest 54 paths, and the successful matching index information (second matching index information) needs to be modified into failure matching index information. And updating the first matching index information based on the failure matching index information to obtain final matching index information.
The 54-way parallel circuit is unchanged, and the following two cases can occur for the non-working circuit:
1. if the matching fails, directly returning a mismatch_flag to indicate the matching failure;
2. and if the matching is successful, a modified successful matching index, namely a mismatch_flag, is returned, which indicates that the matching is failed.
As an embodiment, the matching processing of the current data packet by the hardware parallel circuit to obtain matching subscript information includes:
determining the key words of each message corresponding to the current data packet and having the same parallel paths of the hardware parallel circuit;
carrying out hash calculation on the key words of each message to obtain corresponding actual hash values;
and comparing each actual hash value with a corresponding preset hash value to determine matching index information.
The specific hash processing mode comprises the following steps: from the input message, keys of 64×16 bytes (128 binary bits) are extracted, respectively. Where 64 represents 64 comparison passes and 16 bytes represent the feature values of 16 bytes extracted for each particular pass for subsequent hash value calculations. Specific extraction algorithm takes 16 continuous bytes for each offset address, and then the extraction of each 128-bit keyword can be completed.
It should be noted that, the 16 bytes in this embodiment are different from the 16 bytes of the extraction policy in the above embodiment, and the 16 bytes in the above embodiment are feature extraction for data in a packet, that is, a specific field of extraction, which may be feature extraction in an extension header, or may be feature not limited to 16 bytes.
Table 4 is a data structure table of the matching circuit, as shown in table 4, the mapping relationship among the circuit serial number, the starting byte address, the hash contrast value and the matching subscript information needs to be established in advance, so that each message keyword with the same parallel path number corresponding to the hardware parallel circuit can be determined according to the current data packet. The 16 bytes are only one preset byte number, and the hash calculation is carried out on each message keyword to obtain a corresponding actual hash value. The hash calculation at this time may be a hash algorithm in which all 64 paths are fixed, or may be hash algorithms which are all different, or may be hash algorithms which are partially identical and partially different, and is not limited herein.
Comparing each actual hash value with a corresponding preset hash value, if one of the actual hash values is the same, indicating that the target is hit, and determining the target information of the hit as matched target information. If they are not the same, a miss is indicated.
The comparison matching process can be based on the original hardware circuit parallel matching of fig. 2, and can be set according to practical situations by adding cascade matching corresponding to a plurality of bytes without limitation in consideration of the fact that the number of preset bytes is small and the message type of the finally obtained matching subscript information cannot be distinguished from other message types.
Fig. 6 is a flowchart of a circuit packet matching method according to an embodiment of the present invention, as shown in fig. 6, including:
s21: extracting 16 byte characteristic data keywords according to the byte start bit;
s22: calculating an actual hash value corresponding to each characteristic data keyword;
s23: judging whether each actual hash value is the same as the corresponding preset hash value, if so, entering a step S24, and if not, entering a step S25;
s24: determining hit to obtain matching subscript information;
s25: marking special information as matching subscript information;
s26: and outputting the matched subscript information.
As one embodiment, determining each message key corresponding to the current data packet and having the same number of parallel paths of the hardware parallel circuit includes:
acquiring a starting byte address and a preset byte number in each parallel circuit in the hardware parallel circuit, wherein the preset byte number is a continuous byte number or a discontinuous byte number starting from the starting byte address;
And extracting the current data packet according to the initial byte address and the preset byte number to obtain a corresponding message keyword.
Specifically, each message keyword is extracted by taking bytes of a preset byte number for the offset address of each start address, and it should be noted that the preset byte number may be continuous bytes or discontinuous bytes, which is not limited herein, for example, 16 bytes are taken by each path, and 16 bytes may be continuous or discontinuous 16 bytes, so as to increase flexibility and diversity of matching.
As an embodiment, performing hash computation on each message keyword to obtain a corresponding actual hash value, including:
based on the hash calculation modes corresponding to the parallel circuits in the hardware parallel circuits, wherein the types of the hash calculation modes are smaller than or equal to the number of paths of the parallel circuits;
and calculating the message key words corresponding to the belonging parallel circuits according to the corresponding hash calculation mode to obtain the corresponding actual hash values.
Specifically, the hash calculation method may be based on the same calculation method for each parallel circuit, or may be all different hash calculation methods, that is, in combination with fig. 2, 64 hash calculation methods may be adopted, or may be partially identical or partially different hash calculation methods, and in general, the types of the hash calculation methods are smaller than or equal to the number of paths of the parallel circuits.
And then, according to the corresponding hash calculation mode, calculating the message key words of the belonging parallel circuit to obtain the corresponding actual hash value. The specific hash algorithm is not limited, and may be based on an existing hash algorithm.
The preset hash value may be calculated according to a hash algorithm of the same way, or may be obtained by other different hash algorithms, which is not limited herein. The matching process provided by the embodiment defines different or the same hash algorithm, and ensures the accuracy of the matching process.
As one embodiment, determining each message key corresponding to the current data packet and having the same number of parallel paths of the hardware parallel circuit includes:
acquiring a starting byte address and a preset byte number in each parallel circuit in the hardware parallel circuit, wherein the preset byte number is a continuous byte number or a discontinuous byte number starting from the starting byte address;
processing the current data packet according to the initial byte address and the preset byte number to determine a target byte, wherein corresponding bit operation is stored in each byte in advance, and the bit operation comprises mask and shift number;
and carrying out operation processing on the current data packet according to the mask and the shift number corresponding to the target byte so as to determine the corresponding message key word.
Specifically, in the above embodiment, the key word is extracted by using a byte as a unit, and in this embodiment, the key word is extracted by using a bit as a unit, for example, for an IP packet, it is necessary to analyze the value of 0-3 bits of the IP header in the packet, so as to further determine whether the packet is an IPv4 packet or an IPv6 packet.
Fig. 7 is a schematic diagram of the first 32 bits of an IP header according to an embodiment of the present invention, as shown in fig. 7, where a version number needs to be determined:
1. the version number is 4 in decimal, namely, the binary number 0100B, and the message is expressed as IPv 4;
2. the version number is decimal 6, i.e. binary 0110B, and is represented as IPv6 message.
The above judgment only uses the lower 4 bits of the extracted corresponding characteristic byte, and a masking operation is needed. The mask operation in this embodiment is to simplify the extraction process on the basis of ensuring the correctness of the extraction process.
For the IP Header Length (IHL) field in fig. 7, the size of the IP Header is indicated in 4 bytes (32 bits). The length of the IP header is also referred to as length 4 bytes. When there is no option in the IP header, length is 5, i.e., 20 bytes. If the actual byte length of the field is obtained, masking and shift operations are needed, namely:
IHL_in_Byte=(Byte_0&0xF0)>>2;
The complete calculation process is as follows:
1. the mask extracts the upper 4 bits of the 1 st byte;
2. then shifting the left to 4 bits to obtain a high 4-bit numerical value;
3. then, the left shift is 2 bits again, which corresponds to multiplication by 4.
The merge looks like a mask, and moves 2 bits to the right.
In general, a specific message type to be matched is determined first to determine the corresponding byte number, then a characteristic value under the byte is found, a corresponding mask bit and shift data are determined according to the characteristic value, new data are obtained, and the data are compared with a hash value obtained by a hash algorithm to judge whether the matching is successful, so that matching subscript information is obtained.
Table 4 is a data structure table of the bitwise extracted features, as shown in table 4,
table 4 data structure table for extracting features by bit
In connection with table 4, it is possible to determine what byte characteristic value a specific matching circuit (parallel circuit) is, for example: byte_x_y, x represents a specific path of the parallel circuit, y represents a number of Byte characteristic values, and mask and shift operations are performed;
when shift_x_y is a positive integer (the highest sign bit is 0), this indicates that the operation result is shifted to the right. Returning to the values shown below:
(Byte_x_y&MASK_x_y)>>(SHIFT_x_y&0x7F);
when shift_x_y is a negative integer (the highest sign bit is 1), this indicates that the operation result is shifted left. Returning to the values shown below:
(Byte_x_y&MASK_x_y)<<(SHIFT_x_y&0x7F)。
The Verilog code is used as follows:
reg[7:0]Byte_x_y;
reg[7:0]Mask_x_y;
reg[7:0]SHIFT_x_y;
reg[7:0]dout;
always@(posedge clk)begin
if(SHIFT_x_y[7])
dout<=(Byte_x_y[7:0]&Mask_x_y[7:0])>>SHIFT_x_y[6:0];
else
dout<=(Byte_x_y[7:0]&Mask_x_y[7:0])<<SHIFT_x_y[6:0];
end
note that the shift in this embodiment is a logical shift, not an arithmetic shift, for example, a logical shift to the right, a left complement of 0, an arithmetic shift to the right, and a left complement of sign bits.
Taking the version of the extracted IP message header as an example, the type of the matched message is allocated as IPv4 or IPv6.
1. Matching the IPv4 message;
1) Pkt_offset_0_0 is 14 bytes (ethernet header length is 14 bytes, followed by IP header portion);
2) The mask_0_0 value is 0x0F (the last 4 binary bits are extracted);
3) Shift_0_0 has a value of 0x00 (no SHIFT operation is required);
4) The pre-calculation value of the pre-stored bit corresponding to the comparison Hash value is 0x04 (the extracted version field should represent the value of IPv4, i.e., 0x 04).
2. Matching the IPv6 message;
1) Pkt_offset_0_0 is 14 bytes (ethernet header length is 14 bytes, followed by IP header portion);
2) Mask_0_0 has a value of 0x0F (4 binary system bits of the lower bits are extracted);
3) Shift_0_0 has a value of 0x00 (no SHIFT operation is required);
4) The pre-calculation value of the pre-stored bit corresponding to the comparison Hash value is 0x06 (the extracted version field should be an IPv6 representation value, i.e., 0x 06).
In addition, combining the circuit function of the network card, realizing the matching of IPv4 and IPv6 messages, if the accurate IP_header_length_in_byte value is to be extracted, only setting PKT_OFFSET to be 14, namely extracting the first Byte of the IP message Header; the MASK value is set to 0xF0, and 4 binary digits of the high order are extracted; the SHIFT value is set to 0x82, i.e., 2 bits to the right.
The present embodiment provides direct operations and configuration information, and bit operations are implemented by circuits that extract features through configuration. The method is realized by programming again by the firmware under the condition that each message is not matched; implementation using hardware circuitry may be more efficient than software implementations.
As an embodiment, distinguishing from hash matching, performing matching processing on a current data packet by a hardware parallel circuit to obtain matching subscript information, including:
determining the key words of each message corresponding to the current data packet and having the same parallel paths of the hardware parallel circuit;
splitting the message keywords according to the register bit numbers corresponding to the register types to obtain split message keywords, wherein the register types comprise a selected keyword index register, a starting boundary register, an ending boundary register and an operation register, and the index value set by the selected keyword index register corresponds to the register under each register type;
Determining a corresponding target extracted message field according to the split message keywords and the index value;
determining an extraction interval range according to the values corresponding to the initial boundary register and the end boundary register;
determining that the target extraction message segment is matched in the extraction interval range according to the value of the operation register so as to determine a preset value;
carrying out matching calculation on the keywords of each message to obtain an actual value;
and comparing the actual value with the preset value to determine matching subscript information.
Specifically, in a common packet matching process, the packet matching process is a specific integer value, such as a hash value. However, specific byte number features can be easily ignored, so that the accuracy of the message types obtained by matching can be inaccurate. Therefore, in this embodiment, the message key is split into specific several bytes for matching.
And determining each message keyword corresponding to the current data packet and having the same parallel path number as that of the hardware parallel circuit, and splitting the message keyword based on the register bit number corresponding to the register type to obtain the split message keyword, such as the numerical values of an IP address field, a protocol number and the like in the five-tuple. The hash ALGORITHM-based identification register is set as an INDEX value of the INTERVAL COMPARISON ALGORITHM and is recorded as an interval_comparison_algorithm_index, and only one value which is not the same as the INDEX value of the normal hash ALGORITHM is selected, such as 0xFFFF_FFFF. The register is divided into 4 registers of 32 bits in combination with the 128-bit hash value in fig. 2, which are a selected key index register, a start boundary register, an end boundary register, and an operation register, respectively.
32-bit selection key index register: key_index_reg: the value is 0-3. Because the extracted key defaults to 128 bits, i.e., 4 32-bit registers. The key_index_reg takes values of 0-3, corresponding to 4 32-bit registers respectively. For example, when the value of key_index_reg is 1, a register of 32 bits to 63 bits is taken as matching data. 32-bit start boundary register: begin_value_reg: comparing the starting boundary values of the ranges; 32-bit end boundary register: end_value_reg: comparing the ending boundary value of the range; 32-bit operation bit register: operator_reg: specifically, a comparison scheme with a defined numerical range.
The values of the registers are not limited, and specific reference operations for operating the registers are given in this embodiment as follows:
1. the value is 0x00, which indicates that the key word needs to be equal to the value of the initial boundary register to be successfully matched;
2. the value of the key word is 0x01, which means that the key word needs to be unequal to the value of the initial boundary register to be successfully matched;
3. the value is 0x02, which indicates that the key word needs to be in the open interval of the value of the (start boundary register begin_value_reg and end boundary register end_value_reg) before being successfully matched;
4. the value is 0x03, which indicates that the key word needs to be in the closed interval of the value of [ initial boundary register begin_value_reg ] and end boundary register end_value_reg ] before successful matching;
5. The value of the key word is 0x04, which means that the key word can be successfully matched only if the key word is bitwise matched with the initial boundary register and the result is not 0;
6. the value of the key word is 0x05, which indicates that the key word can be successfully matched only if the key word is bitwise or with the initial boundary register and the result is not 0;
7. the value of the key word is 0x06, which indicates that the key word needs to be bitwise exclusive-or with the initial boundary register, and the result is not 0, so that the key word can be successfully matched;
8. the value of the key word is 0x07, which means that the key word needs to be inverted according to the bit, the result is not 0, so that the key word can be successfully matched, and other subsequent values can be defined according to the expansion function.
Referring to a specific virtual local area network (Virtual Local Area Network, VLAN), fig. 8 is a schematic diagram of an ethernet header with multiple VLAN headers according to an embodiment of the present invention, and as shown in fig. 8, VLAN Tag is composed of 4 bytes. Fig. 9 is a schematic structural diagram of a VLAN Tag header according to an embodiment of the present invention, where, as shown in fig. 9, the first two bytes are Tag protocol identifiers (Tag Protocol Identifier, TPIDs), and when the value is 0x8100, the VLAN Tag header represents a VLAN data frame of IEEE 802.1Q. The three types of packets in fig. 8 are distinguished by identifying the value of the TPID.
The preparation work is as follows:
1. setting an identification register of the hash ALGORITHM as an INDEX value interval_match_algorism_index of the INTERVAL COMPARISON ALGORITHM;
2. extracting continuous 8 bytes from 13 th byte of the message, namely, PKT_OFFSET_x_0-PKT_OFFSET_x_7 values are 13-20 bytes (the first byte of the message is counted from 1), and other bytes needed subsequently;
3. extracting the 1 st TPID message field:
1) The mask_x_0-mask_0_3 values are 0xFF, 0x00 and 0x00 in sequence;
2) The shift_x_0 to shift_0_3 values are 0x00, 0x00;
3) The key_index_reg takes a value of 0 to obtain a first 32-bit register;
4. extracting the 2 nd TPID message field:
1) The mask_x_4-mask_0_7 values are 0xFF, 0x00 and 0x00 in sequence;
2) The shift_x_4 to shift_0_7 values are 0x00, 0x00;
3) The key_index_reg takes a value of 1, resulting in a second 32-bit register.
Matching the related register values (corresponding to specific hash comparison values) based on three types of data packets;
a first passage: ethernet packets without VLANs are identified. I.e. the 1 st TPID message field value is not 0x8100, the matching is successful. The specific register values are:
1) key_index_reg=0, extract 1 st TPID message field;
2) begin_value_reg=0x8100, the value of end_value_reg is arbitrary, and the interval value is set;
3) operater_reg=0x00, meaning that the key needs to be equal to the value 0x8100 of the start boundary register in order to match successfully.
4) After the matching is successful, the returned START_PARSING_HDR_OFFSET is 12, namely the analysis of the Ethernet three-layer message is started from the 12 th byte;
and a second path, namely identifying the Ethernet packet with one layer of VLAN. I.e. the 1 st TPID message field value is 0x8100 (this condition has been confirmed to be true by the first pass filtering), and the 2 nd TPID message field value is not 0x8100:
1) key_index_reg=1, extract the 2 nd TPID message field;
2) begin_value_reg=0x8100, the value of end_value_reg is arbitrary, and the interval value is set;
3) operater_reg=0x01, meaning that the key needs to be unequal to the value 0x8100 of the start boundary register to match successfully.
4) After matching is successful, the returned start_paring_hdr_offset is 18, namely the analysis of the three-layer ethernet message STARTs from the 18 th byte;
third path: an ethernet packet with a two layer VLAN is identified. Namely, the 1 st TPID message field value is 0x8100, and the 2 nd TPID message field value is 0x8100 (after the first and second paths are filtered, the condition can be confirmed to be true):
If the previous two paths are not successfully matched, the path is successfully matched by default, the direct return start_paring_hdr_offset is 22, namely the analysis of the three-layer Ethernet message STARTs from the 22 th byte.
In the range comparison process provided by the embodiment, a plurality of bytes are split into different byte registers for comparison, so that the matching process is more accurate.
On the basis of the foregoing embodiment, as one embodiment, after performing hash computation on each message keyword to obtain a corresponding actual hash value, before comparing each actual hash value with a corresponding preset hash value to determine matching subscript information, the method further includes:
obtaining mask information corresponding to each parallel circuit;
performing mask operation processing on each actual hash value and corresponding mask information to obtain a masked hash value;
and comparing the masked hash value with a corresponding preset hash value to determine matching index information.
In the above embodiment, the matching mode adopted is accurate matching, so that the accurate matching is to compare one by one, and the invention considers the matching efficiency in the matching process, adds a mask, and realizes fuzzy matching of the specific mask field after configuration through bitwise and operation. After determining the key, some fields may support fuzzy match lookups. Therefore, after the actual hash value is calculated, before the comparison and matching, masking operation is performed, wherein the mask field has a value of 1 and the mask field has a value of 0 in the corresponding key words, and the field needing fuzzy matching corresponds to the mask field.
Table 5 is a fuzzy matching data structure table, as shown in table 5, for example, the actual hash value is 1100, the preset hash value is 1001, the mask information is set to 1010, and the two hash values are divided into the mask information setting and 1000, so that the matching index information can be determined.
Table 5 fuzzy matching data structure table
In the system, 64 mask values of 128 bits are used for 64 comparison circuit channels respectively.
Assuming that the extracted key is a 4-bit binary number, the stored hash value is also calculated from the 4-bit binary key.
The mask value for 4 bits is set to: 1010B (B at the end of the number is represented as a string of binary digits), which has the true meaning of a 0, 2 th bit fuzzy match, i.e., does not participate in the comparison. Only bits 1, 3 are compared, assuming:
1. the extracted keywords are as follows: 1100B;
2. the stored keywords are: 1001B.
The two are not identical by direct comparison. But all require masking operations that can be successfully matched. The specific calculation process is as follows:
1. the extracted key and mask are bitwise and operated on: 1100B &1010 b=1000b;
2. the stored key and mask are bitwise and operated on: 1001B &1010 b=1000b.
By performing the mask computation, fuzzy matching of specific bits can be achieved.
Regarding fuzzy matching, the most common usage scenario is matching of IP address segments. 192.168.0.0/24 is an IP address segment, i.e. 192.168.0.0-192.168.0.255. This can be difficult to achieve if the previous matches were employed. However, after adding the mask, only the corresponding byte mask of the IP address field in the feature value needs to be set to mask=0xff_ff_ff_00, so that the matching to 192.168.0.255 can be completed. While other feature fields may still match exactly or with ambiguity as normal.
As one embodiment, comparing each actual hash value with a corresponding preset hash value to determine matching index information includes:
acquiring a target actual hash value of a message to be processed;
when the target actual hash value is the same as the preset hash value, acquiring a cascade attribute mark of a comparison circuit to which the message to be processed belongs in the same hash value;
if the cascade attribute marks are cascade matching, acquiring a cascade circuit of cascade matching;
and determining the matching index information as the matching index information of the cascade circuit according to the fact that the preset hash value corresponding to the cascade circuit is identical to the target actual hash value.
It should be noted that, for a single 128-bit eigenvalue hit in fig. 2, it may not be possible to accurately characterize a class of data messages. Layer 2 MAC or layer 3 IP message types can be described when 16 bytes (128 bits) are used; when the message is classified by the layer 2 MAC information, a 16-byte (128-bit) keyword can be used, and then the message type can be identified by Hash calculation. However, when the message needs to be classified by layer 2 MAC plus layer 3 IP information, the 16 byte (128 bit) key word cannot be selected to cover the characteristics of the message. At this time, the network packet can accurately describe the packet layer 2 MAC, layer 3 IP and layer 4 TCP/UDP types using the first 64 bytes (4×128 bits). In practice, the message needs to be classified by layer 2 MAC plus layer 3 IP information, which is a relatively common case.
Therefore, in order to cope with the above-described situation, it is possible to employ data input inside hardware by concatenating a plurality of 16-byte Hash operation circuits. The characteristic value of the plurality of paths is hit, so that whether the message characteristic is hit or not can be determined more accurately. In the parallel computing process, the cascade attribute marks of the comparison circuit need to be checked, one is checked when the target actual hash value is the same as the preset hash value, and the other is checked when the matching is started correspondingly, the method is not limited in this embodiment, the former is selected, and when the cascade attribute marks are cascade matching on the premise that 16 bytes are adopted and whether the message characteristics hit or not cannot be determined, the cascade circuit is obtained, and the cascade circuit in the embodiment can be a group of N circuits, of course, the more and the better are not, and meanwhile, the processing efficiency of the circuit is considered.
For example, the cascade circuits are 0 and 1 circuits, which correspond to 16×2=32 bytes, and the predetermined hash values of the two cascade circuits are the same as the target actual hash value, and the cascade circuits are hit only if one of the two cascade circuits is different from the target actual hash value, and the cascade circuits are not hit. And outputting the matched index information in the case of hit.
The cascade circuit in this embodiment includes continuous and discontinuous cascade matching, and the corresponding cascade matching includes intra-group cascade matching and inter-group cascade matching, and as one embodiment, the determining process of the cascade circuit includes:
When the cascade attribute marks are intra-group cascade matching, determining a continuous cascade circuit to which the message belongs according to the number of layers and types of the layers of the message, and respectively setting marks corresponding to the number of layers of the message in the continuous cascade circuit to determine the cascade circuit;
when the cascade attribute is marked as inter-group cascade matching, adding a cascade register in each circuit of the hardware parallel circuit for marking a cascade index value;
determining a target cascade circuit to which the message belongs according to the layer number and the type of each layer of the message, wherein the cascade circuit in the target cascade circuit is a discontinuous cascade circuit;
and setting corresponding cascade index values in the target cascade circuit and marking the number of layers of the message to determine the cascade circuit.
Specifically, when cascade attribute is marked as intra-group cascade matching, 4 continuous parallel circuits are cascaded to form a group, fig. 10 is a schematic diagram of a group of continuous 4 parallel circuits, as shown in fig. 10, a cascade identifier is added in a 16-byte Hash calculation circuit, and when judging results, a plurality of 16-byte Hash results are jointly compared through the cascade identifier, so that message identification of more than 16B is realized. Considering the processing efficiency, because the data volume that can be processed by a single beat of a data path is 64 bytes, 4 Hash circuit cascades are needed to be realized at most, and the types of the message 2-layer MAC, the 3-layer IP and the 4-layer TCP/UDP are accurately matched. And is done within a group to achieve maximum processing efficiency.
In the case that cascade attribute is marked as inter-group cascade matching, no continuous parallel circuit is needed, fig. 11 is a schematic diagram of a group of discontinuous 4 parallel circuits cascade, as shown in fig. 11, an original 64-way comparison circuit is changed into a new comparison circuit whole, and every 4 original circuits are optionally cascade-connected together. There may be 64 or more cascaded index registers between groups. Each register is 32 bits, each 8 bits describe a circuit index, and the cascade matching is performed by using the matching results of 4 circuits.
For example, the following value of the inter-group cascade index register-0 is 0x00_02_0f_3f, i.e. the following 4 circuits are cascade-connected:
1. the highest 8-bit value is 0x00, namely the 0 th path;
2. the other 8 bits have a value of 0x02, namely the 2 nd path;
3. the value of the lower 8 bits is 0x0F, namely the 15 th path;
4. the lowest 8 bits have a value of 0x3F, i.e., path 63.
Only if all the message features represented by the 4 paths are successfully matched, the index register-0 is cascaded among groups, and the represented paths are successfully matched. The matching results of the paths represented by all the inter-group cascade index registers are finally delivered to a 'matched index selector after cascade consideration' for unified processing. Of course, the inter-group cascade INDEX register may be cascade connected by using fewer than 4 paths, and only the corresponding circuit bit needs to be set to a special flag value, which is denoted as no_match_index, and the specific value may be a value of 0xFF, which does not belong to the normal circuit INDEX.
Fig. 12 is a flow chart of processing a message in cascade matching according to an embodiment of the present invention, where, as shown in fig. 12, the flow chart includes:
s21: extracting 16 byte characteristic data keywords according to the byte start bit;
s22: calculating an actual hash value corresponding to each characteristic data keyword;
s23: judging whether each actual hash value is the same as the corresponding preset hash value, if so, entering a step S31, and if not, entering a step S32;
s31: judging whether to cascade with the previous stage, if yes, entering a step S33, and if not, entering a step S34;
s33: judging whether the previous stage hits or not, if yes, entering a step S34, and if not, entering a step S32;
s32: the matching subscript is error information;
s34: determining correct matched subscript information;
s35: and outputting the matched subscript information.
The cascade matching provided by the embodiment is based on the original parallel circuit, so that the circuit processing efficiency is improved, the limitation of circuit relation and the number of bytes processed by each circuit is not needed, and the flexibility is improved.
On the basis of the foregoing embodiment, as an embodiment, when each actual hash value is different from the corresponding preset hash value, the method further includes:
Outputting a special mark value as matched subscript information, wherein the special mark value is a mark value of a serial number corresponding to the number of parallel paths of the parallel circuits of the hardware;
returning to the step of acquiring the current data packet to acquire the next data packet.
Specifically, if each actual hash value is different from the corresponding preset hash value, it is indicated that there is no hit, i.e., all the ways are missed, and then a special tag value needs to be output as matching index information. It should be noted that, the matching index information is not used as the matching index information (serial numbers 0-63) of the subsequent processing of the present invention, that is, the tag value of the serial number corresponding to the parallel path number of the hardware parallel circuit is distinguished, and is only a record of output information, such as: 0xffff_ffff, and returns to the step of acquiring the current packet to acquire the next packet.
The output of the matched subscript information provided by the embodiment of the invention can timely stop the subsequent processing of the special mark value, thereby saving time. As an embodiment, the message processing method of the present invention may be applied to a network card firmware, and fig. 13 is a flowchart of a network card update configuration data packet provided by the embodiment of the present invention, as shown in fig. 13, where the method includes:
S41: the network card is connected to the development machine and enters a debug state;
the network card is accessed to a development host end, and the development host end carries out step S42;
s42: updating a data table in the firmware source code;
s43: the firmware is cross compiled.
Downloading and updating the compiled firmware, and entering step S44;
s44: downloading the compiled firmware to a Flash storage unit of the network card;
s45: the network card is accessed to a formal network environment;
s46: the network card starts boot, boot guide firmware starts;
s47: initializing firmware, namely configuring a data table to a storage unit corresponding to the network card;
s48: the network card works normally;
s49: and judging whether to exit the operation, if so, ending, and if not, returning to the step S48.
Updating a data table in a source code at a development host end, and performing cross compiling again to generate new firmware; taking out the network card from the formal network environment after power failure, connecting to a development machine, and entering a debug state; downloading the firmware obtained by cross compiling into Flash of the network card; re-accessing the network card to a formal network environment; when the network card is restarted, boot guide firmware is loaded first, and then formal function firmware is loaded; when the functional firmware is initialized, the data table compiled into the functional firmware is configured to the storage unit corresponding to the network card, so that the configuration data updating of the network card can be completed; the network card runs the new firmware program and the updated data set, so that the normal work can be continuously performed; when the working operation state of the network card is interrupted by certain conditions (such as shutdown, internal error and the like), the work is ended.
The invention further discloses a message processing device corresponding to the method on the basis of the detailed description of the embodiments corresponding to the message processing method, and fig. 14 is a block diagram of the message processing device provided by the embodiment of the invention. As shown in fig. 14, the message processing apparatus includes:
the acquiring module 11 is configured to acquire matching subscript information of a message to be processed, where the matching subscript information is obtained by matching with a hardware parallel circuit;
the identifying module 12 is configured to identify a message type of the message to be processed according to the matching subscript information;
the determining module 13 is configured to determine a corresponding processing policy according to a relationship between a packet type and a preset requirement, where the preset requirement is a requirement for reorganizing a data packet corresponding to a to-be-processed packet, and the processing policy at least includes a policy that the data packet is removed and the data header is not parsed, a policy that the data packet is removed and the policy is specified in a skip manner;
the extracting module 14 is configured to process the message to be processed according to the message type and the corresponding processing policy to determine a final target message;
the method for determining the corresponding processing strategy according to the relation between the message type and the preset requirement comprises the following steps:
Judging whether a message identifier of a message type meets a preset requirement or not;
if yes, determining that the processing strategy is a strategy for removing the data packet without analyzing the data header or a strategy for removing the data packet and skipping to the designated strategy;
if not, determining that the processing strategy is a jump specified offset address strategy.
Since the embodiments of the device portion correspond to the above embodiments, the embodiments of the device portion are described with reference to the embodiments of the method portion, and are not described herein.
For the description of the message processing device provided by the present invention, please refer to the above method embodiment, the description of the present invention is omitted herein, and the method has the same advantages as the above method.
Fig. 15 is a block diagram of another message processing apparatus according to an embodiment of the present invention, as shown in fig. 15, where the apparatus includes:
a memory 21 for storing a computer program;
a processor 22 for implementing the steps of the message processing method when executing the computer program.
Processor 22 may include one or more processing cores, such as a 4-core processor, an 8-core processor, or the like, among others. The processor 22 may be implemented in hardware in at least one of a digital signal processor (Digital Signal Processor, DSP), a Field programmable gate array (Field-Programmable Gate Array, FPGA), a programmable logic array (Programmable Logic Array, PLA). The processor 22 may also include a main processor, which is a processor for processing data in an awake state, also referred to as a central processor (Central Processing Unit, CPU), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 22 may be integrated with an image processor (Graphics Processing Unit, GPU) for use in responsible for rendering and rendering of content required for display by the display screen. In some embodiments, the processor 22 may also include an artificial intelligence (Artificial Intelligence, AI) processor for processing computing operations related to machine learning.
Memory 21 may include one or more computer-readable storage media, which may be non-transitory. Memory 21 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 21 is at least used for storing a computer program 211, where the computer program can implement the relevant steps of the message processing method disclosed in any of the foregoing embodiments after being loaded and executed by the processor 22. In addition, the resources stored in the memory 21 may further include an operating system 212, data 213, and the like, and the storage manner may be transient storage or permanent storage. The operating system 212 may include Windows, unix, linux, among other things. The data 213 may include, but is not limited to, data related to a message processing method, and the like.
In some embodiments, the message processing apparatus may further include a display 23, an input/output interface 24, a communication interface 25, a power supply 26, and a communication bus 27.
It will be appreciated by those skilled in the art that the structure shown in fig. 15 is not limiting of the message processing apparatus and may include more or fewer components than shown.
The processor 22 implements the message processing method provided in any of the above embodiments by calling instructions stored in the memory 21.
For the description of the message processing device provided by the present invention, please refer to the above method embodiment, the description of the present invention is omitted herein, and the method has the same advantages as the above method.
Furthermore, the present invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by the processor 22 implements the steps of the message processing method as described above.
It will be appreciated that the methods of the above embodiments, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored on a computer readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium for performing all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
For an introduction to a computer readable storage medium provided by the present invention, please refer to the above method embodiment, the present invention is not described herein, and the method has the same advantages as the above message processing method.
The above describes in detail a message processing method, a message processing device and a medium provided by the invention. In the description, each embodiment is described in a progressive manner, and each embodiment is mainly described by the differences from other embodiments, so that the same similar parts among the embodiments are mutually referred. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section. It should be noted that it will be apparent to those skilled in the art that various modifications and adaptations of the invention can be made without departing from the principles of the invention and these modifications and adaptations are intended to be within the scope of the invention as defined in the following claims.
It should also be noted that in this specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

Claims (19)

1. A method for processing a message, comprising:
obtaining matching subscript information of a message to be processed, wherein the matching subscript information is obtained through matching of a hardware parallel circuit;
identifying the message type of the message to be processed according to the matching subscript information;
determining a corresponding processing strategy according to the relation between the message type and a preset requirement, wherein the preset requirement is a requirement for reorganizing a data packet corresponding to the message to be processed, and the processing strategy at least comprises one strategy of removing the data packet without analyzing a data header strategy, jumping to designate an offset address strategy, removing the data packet and jumping to designate the strategy;
processing the message to be processed according to the message type and the corresponding processing strategy to determine a final target message;
the determining process of the matching subscript information of the message to be processed comprises the following steps:
acquiring a current data packet;
matching the current data packet through the hardware parallel circuit to obtain the matching subscript information;
the determining a corresponding processing strategy according to the relation between the message type and the preset requirement comprises the following steps:
judging whether the message identifier of the message type meets the preset requirement or not;
If yes, determining that the processing strategy is the data head removing strategy or the data packet removing and jump specifying strategy;
if not, determining that the processing strategy is the jump specified offset address strategy.
2. The method for processing a packet according to claim 1, wherein the determining process for removing the packet without parsing the header policy includes:
presetting a mapping relation between message types corresponding to each matched subscript information and unresolved message header offset information, wherein the number of the message types is the same as the parallel path number of the hardware parallel circuit;
obtaining a target message type;
determining a corresponding target offset address according to the target message type and the mapping relation;
and removing the message corresponding to the target offset address from the message to be processed to obtain a reorganized data packet so as to establish a strategy for removing the data packet without analyzing the data header.
3. The method for processing a packet according to claim 1, wherein the determining process for removing the packet and skipping the specified policy includes:
presetting a mapping relation between message types corresponding to each matched subscript information and unresolved message header offset information, wherein the number of the message types is the same as the parallel path number of the hardware parallel circuit;
Obtaining a target message type;
determining a corresponding target offset address according to the target message type and the mapping relation;
removing the message corresponding to the target offset address from the message to be processed to obtain a reorganized data packet;
presetting a mapping relation between a message type corresponding to each matched subscript information and specified analysis head offset information, wherein a first offset address of the specified analysis head offset information is a next offset address of an unresolved message head offset address and is relative offset information under a data message type;
presetting a mapping relation between each piece of matched subscript information and the data message type, wherein the message type at least comprises one data message type;
and pre-analyzing the recombined data packet according to the target message type to obtain a corresponding target data message type so as to establish the data packet removal and jump specifying strategy.
4. The method for processing a message according to claim 1, wherein the determining of the jump specifying offset address policy includes:
presetting a mapping relation between a message type corresponding to each matched subscript information and specified analysis head offset information, wherein the specified analysis head offset information is relative offset information under a data message type;
Presetting a mapping relation between each piece of matched subscript information and the data message type, wherein the message type at least comprises one data message type;
and pre-analyzing the message to be processed according to the target message type and the specified analysis head offset information to obtain the corresponding target data message type.
5. The method for processing a message according to any one of claims 1 to 4, wherein the processing the message to be processed according to the message type and the corresponding processing policy to determine a final target message includes:
acquiring an extraction keyword strategy, wherein the extraction keyword strategy is based on a data packet extraction strategy corresponding to a data structure under the message type;
and extracting the current message to be processed according to the message type, the corresponding processing strategy and/or the extraction keyword strategy to obtain the target message.
6. The method for processing a message according to claim 5, wherein the determining process of the keyword extraction strategy comprises:
the method comprises the steps of pre-establishing a mapping relation of data message types corresponding to each matched subscript information and extracting a specific offset address, wherein when a processing strategy corresponding to the message type is the data header removing strategy, the corresponding data message type is the pre-established data structure type of the message type and the reorganized data packet, and the specific offset address is the offset address of a message protocol custom field of the message to be processed;
Obtaining a target message type of the message to be processed, wherein when a processing strategy corresponding to the message type is the data head removing strategy, the message to be processed is the reorganized data packet;
analyzing the message to be processed according to the target message type to obtain a corresponding target message type;
determining a corresponding feature extraction position according to the target message type and the mapping relation of extracting the specific offset address;
and extracting according to the message data corresponding to the characteristic extraction position to establish the keyword extraction strategy.
7. The method for processing a packet according to claim 1, wherein when the number of packet types of the current packet is smaller than the number of parallel paths of the hardware parallel circuit, the matching processing, by the hardware parallel circuit, of the current packet to obtain the matching subscript information includes:
determining a target matching circuit according to the number of message types of the current data packet and the hardware parallel circuit;
adding corresponding registers in each parallel circuit of the hardware parallel circuit;
determining a corresponding register mark value according to the target matching circuit, parallel circuits except the target matching circuit and a corresponding register;
And carrying out matching processing on the current data packet based on the target matching circuit with the register mark value as a matching mark to obtain the matching index information.
8. The method for processing a packet according to claim 1, wherein when the number of packet types of the current packet is smaller than the number of parallel paths of the hardware parallel circuit, the matching processing, by the hardware parallel circuit, of the current packet to obtain the matching subscript information includes:
determining a target matching circuit according to the number of message types of the current data packet and the hardware parallel circuit;
matching the current data packet according to the hardware parallel circuit to obtain first matching subscript information;
determining successful matching index information except the target matching circuit in the first matching index information, and taking the successful matching index information as second matching index information;
modifying the second matching index information to determine failure matching index information;
and updating the first matching index information with the failure matching index information obtained after modification to obtain the final matching index information.
9. The method for processing a packet according to any one of claims 1, 7 and 8, wherein the matching processing, by the hardware parallel circuit, the current packet to obtain the matching index information includes:
determining each message keyword corresponding to the current data packet and having the same parallel path number as the hardware parallel circuit;
carrying out hash calculation on each message keyword to obtain a corresponding actual hash value;
and comparing each actual hash value with a corresponding preset hash value to determine the matching index information.
10. The method for processing a packet according to claim 9, wherein determining each packet key corresponding to the current packet and having the same number of parallel paths of the hardware parallel circuit includes:
acquiring a starting byte address and a preset byte number in each parallel circuit in the hardware parallel circuit, wherein the preset byte number is a continuous byte number or a discontinuous byte number starting from the starting byte address;
and extracting the current data packet according to the starting byte address and the preset byte number to obtain a corresponding message keyword.
11. The method for processing a packet according to claim 9, wherein determining each packet key corresponding to the current packet and having the same number of parallel paths of the hardware parallel circuit includes:
acquiring a starting byte address and a preset byte number in each parallel circuit in the hardware parallel circuit, wherein the preset byte number is a continuous byte number or a discontinuous byte number starting from the starting byte address;
processing the current data packet according to the initial byte address and the preset byte number to determine a target byte, wherein corresponding bit operation is stored in each byte in advance, and the bit operation comprises mask and shift number;
and carrying out operation processing on the current data packet according to the mask and the shift number corresponding to the target byte so as to determine a corresponding message keyword.
12. The method for processing a packet according to any one of claims 1, 7 and 8, wherein the matching processing, by the hardware parallel circuit, the current packet to obtain the matching index information includes:
determining each message keyword corresponding to the current data packet and having the same parallel path number as the hardware parallel circuit;
Splitting the message keywords according to the number of register bits corresponding to the register types to obtain split message keywords, wherein the register types comprise a selected keyword index register, a starting boundary register, an ending boundary register and an operation register, and index values set by the selected keyword index register correspond to registers under each register type;
determining a corresponding target extracted message field according to the split message keywords and the index value;
determining an extraction interval range according to the values corresponding to the initial boundary register and the end boundary register;
determining that the target extraction message segment is matched in the extraction interval range according to the value of the operation register so as to determine a corresponding preset value;
carrying out matching calculation on each message keyword to obtain a corresponding actual value;
and comparing the corresponding actual value with the corresponding preset value to determine the matching index information.
13. The method according to claim 9, wherein after performing hash computation on each of the message keywords to obtain a corresponding actual hash value, before comparing each of the actual hash values with a corresponding preset hash value to determine the matching index information, the method further comprises:
Obtaining mask information corresponding to each parallel circuit;
performing mask operation processing on each actual hash value and the corresponding mask information to obtain a masked hash value;
and comparing the masked hash value with the corresponding preset hash value to determine the matched index information.
14. The method according to claim 13, wherein comparing each of the actual hash values with a corresponding preset hash value to determine the matching index information comprises:
acquiring a target actual hash value of the message to be processed;
when the target actual hash value is the same as the preset hash value, acquiring a cascade attribute mark of a comparison circuit to which the message to be processed belongs in the same hash value;
if the cascade attribute marks are cascade matching, acquiring a cascade circuit of the cascade matching;
and determining that the matching index information is the matching index information of the cascade circuit according to the fact that the preset hash value corresponding to the cascade circuit is identical to the target actual hash value.
15. The method for processing a message according to claim 14, wherein the cascade matching includes intra-group cascade matching and inter-group cascade matching, and the determining process of the cascade circuit includes:
When the cascade attribute marks are cascade matching in the group, determining a continuous cascade circuit to which the message belongs according to the layer number and the type of each layer of the message, and respectively setting marks corresponding to the layer number of the message in the continuous cascade circuit to determine the cascade circuit;
adding cascade registers in each circuit of the hardware parallel circuit for marking cascade index values when the cascade attribute marks the cascade matching among groups;
determining a target cascade circuit to which the message belongs according to the layer number and the type of each layer of the message, wherein the cascade circuit in the target cascade circuit is a discontinuous cascade circuit;
and setting the corresponding cascade index value and marking the layer number of the message in the target cascade circuit to determine the cascade circuit.
16. The method for processing a message according to claim 9, wherein when each of the actual hash values is different from the corresponding preset hash value, further comprising:
outputting a special mark value as the matching index information, wherein the special mark value is a mark value which is distinguished from serial numbers corresponding to the parallel paths of the hardware parallel circuit;
And returning to the step of acquiring the current data packet to acquire the next data packet.
17. A message processing apparatus, comprising:
the acquisition module is used for acquiring matching subscript information of the message to be processed, wherein the matching subscript information is obtained through matching of hardware parallel circuits;
the identification module is used for identifying the message type of the message to be processed according to the matched subscript information;
the determining module is used for determining a corresponding processing strategy according to the relation between the message type and a preset requirement, wherein the preset requirement is a requirement for reorganizing a data packet corresponding to the message to be processed, and the processing strategy at least comprises one strategy of removing the data packet without analyzing a data header strategy, jumping to designate an offset address strategy, removing the data packet and jumping to designate the strategy;
the extraction module is used for processing the message to be processed according to the message type and the corresponding processing strategy so as to determine a final target message;
the determining process of the matching subscript information of the message to be processed comprises the following steps:
acquiring a current data packet;
matching the current data packet through the hardware parallel circuit to obtain the matching subscript information;
The determining a corresponding processing strategy according to the relation between the message type and the preset requirement comprises the following steps:
judging whether the message identifier of the message type meets the preset requirement or not;
if yes, determining that the processing strategy is the data head removing strategy or the data packet removing and jump specifying strategy;
if not, determining that the processing strategy is the jump specified offset address strategy.
18. A message processing apparatus, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the message processing method according to any one of claims 1 to 16 when executing said computer program.
19. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the message processing method according to any of claims 1 to 16.
CN202311518307.6A 2023-11-15 2023-11-15 Message processing method, device and medium Active CN117240947B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311518307.6A CN117240947B (en) 2023-11-15 2023-11-15 Message processing method, device and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311518307.6A CN117240947B (en) 2023-11-15 2023-11-15 Message processing method, device and medium

Publications (2)

Publication Number Publication Date
CN117240947A CN117240947A (en) 2023-12-15
CN117240947B true CN117240947B (en) 2024-02-02

Family

ID=89096985

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311518307.6A Active CN117240947B (en) 2023-11-15 2023-11-15 Message processing method, device and medium

Country Status (1)

Country Link
CN (1) CN117240947B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106254381A (en) * 2016-09-12 2016-12-21 全球能源互联网研究院 Protocol analysis method, device and comprise the Layer2 switching system of protocol analysis device
CN112217765A (en) * 2019-07-10 2021-01-12 深圳市中兴微电子技术有限公司 Message parsing method and device
CN115766620A (en) * 2022-09-26 2023-03-07 阿里巴巴(中国)有限公司 Message processing method, programmable network card device, physical server and storage medium

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106254381A (en) * 2016-09-12 2016-12-21 全球能源互联网研究院 Protocol analysis method, device and comprise the Layer2 switching system of protocol analysis device
CN112217765A (en) * 2019-07-10 2021-01-12 深圳市中兴微电子技术有限公司 Message parsing method and device
CN115766620A (en) * 2022-09-26 2023-03-07 阿里巴巴(中国)有限公司 Message processing method, programmable network card device, physical server and storage medium

Also Published As

Publication number Publication date
CN117240947A (en) 2023-12-15

Similar Documents

Publication Publication Date Title
US10764181B2 (en) Pipelined evaluations for algorithmic forwarding route lookup
US11677664B2 (en) Apparatus and method of generating lookups and making decisions for packet modifying and forwarding in a software-defined network engine
US10735221B2 (en) Flexible processor of a port extender device
US9606781B2 (en) Parser engine programming tool for programmable network devices
US11729300B2 (en) Generating programmatically defined fields of metadata for network packets
US20170053012A1 (en) High-performance bloom filter array
US7382777B2 (en) Method for implementing actions based on packet classification and lookup results
US20090207838A1 (en) Ternary content addressable memory embedded in a central processing unit
US7599364B2 (en) Configurable network connection address forming hardware
US9979648B1 (en) Increasing entropy across routing table segments
US7480300B2 (en) Content addressable memory organized to share entries between different entities such as ports of a network unit
US20140307736A1 (en) Method for providing a parser to extract information from fields of a data packet header
US20210185153A1 (en) Hybrid Fixed/Programmable Header Parser for Network Devices
CN117240947B (en) Message processing method, device and medium
US9135997B2 (en) System and method for filtering addresses
CN115883681A (en) Message analysis method and device, electronic equipment and storage medium
CN114079634B (en) Message forwarding method and device and computer readable storage medium
CN110933001B (en) Basic processing unit structure of extensible reconfigurable switch packet parser
US9032142B2 (en) System and method for storing integer ranges in a memory
CN116471344B (en) Keyword extraction method, device and medium for data message
WO2022057887A1 (en) Packet modification method and apparatus, computer device, and storage medium
EP4099644A1 (en) Packet processing method and related apparatus
CN116033044A (en) Method, device, equipment and storage medium for analyzing message segments

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant