CN115883681A - Message analysis method and device, electronic equipment and storage medium - Google Patents
Message analysis method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115883681A CN115883681A CN202310083400.2A CN202310083400A CN115883681A CN 115883681 A CN115883681 A CN 115883681A CN 202310083400 A CN202310083400 A CN 202310083400A CN 115883681 A CN115883681 A CN 115883681A
- Authority
- CN
- China
- Prior art keywords
- message
- protocol layer
- protocol processing
- processing unit
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 30
- 238000012545 processing Methods 0.000 claims abstract description 175
- 230000009471 action Effects 0.000 claims abstract description 80
- 238000000034 method Methods 0.000 claims abstract description 64
- 238000004891 communication Methods 0.000 claims description 15
- 239000000284 extract Substances 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 8
- 230000005540 biological transmission Effects 0.000 claims description 6
- 230000008569 process Effects 0.000 abstract description 17
- 238000010586 diagram Methods 0.000 description 5
- 102100033189 Diablo IAP-binding mitochondrial protein Human genes 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 101710101225 Diablo IAP-binding mitochondrial protein Proteins 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 101000650775 Boana raniceps Raniseptin-1 Proteins 0.000 description 1
- 101150082208 DIABLO gene Proteins 0.000 description 1
- MHABMANUFPZXEB-UHFFFAOYSA-N O-demethyl-aloesaponarin I Natural products O=C1C2=CC=CC(O)=C2C(=O)C2=C1C=C(O)C(C(O)=O)=C2C MHABMANUFPZXEB-UHFFFAOYSA-N 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000004148 unit process Methods 0.000 description 1
Images
Abstract
The embodiment of the invention provides a message analysis method, a message analysis device, electronic equipment and a storage medium, which are applied to a message analysis module, wherein the message analysis module comprises a plurality of protocol processing units, and the method comprises the following steps: receiving a message to be analyzed, generating a table look-up request according to the message, matching the message with a tcam rule table pre-configured by a target protocol processing unit in parallel according to the table look-up request, generating a matched action table, extracting key information of the message, stripping the message according to the length of a preset protocol layer in the action table, and obtaining data information of the message. The invention distributes a protocol processing unit for each layer of protocol independently, processes the protocol according to the sequence pipeline of the network protocol layer, matches the message with the tcam rule table pre-configured by the target protocol processing unit in parallel, strips the data information of the message, and the tcam rule table can be flexibly configured to support various protocols, thereby realizing the message analysis with linear speed and high efficiency.
Description
Technical Field
The invention relates to the technical field of data communication, and provides a message parsing method and device, electronic equipment and a storage medium.
Background
With the development of network technology and the rapid increase of data flow, the traditional network message method based on the TCP/IP protocol stack of the CPU not only consumes a large amount of CPU resources, but also affects the performance of the CPU, so that in a cloud service system for processing big data, some work of a CPU inner core is unloaded into an intelligent network card, so that the performance of data processing is improved, and the calculation overhead of the CPU is saved.
At present, the mainstream method is to offload the network forwarding function and the storage function of the CPU core to the smart network card, offload the OVS (OpenVswitch, high-quality virtual switch supporting multi-layer data forwarding) forwarding flow table entry to the DDR externally attached to the smart network card, analyze the key value of the table lookup by the smart network card chip, query the flow table entry in the DDR, obtain the final destination of the message, which needs to analyze the information of the message correctly, or else, if the error result is found by table lookup, cause a message forwarding error.
However, because the types of network protocol layers involved in the message are many, the current intelligent network card cannot enumerate various types of situations one by one, the first is that the circuit structure is very complex and the on-chip resources are difficult to support, and the second is that no protocol type application scenario is involved, which wastes resources. Therefore, for large-flow messages, the problems of low speed, low configuration flexibility and resource limitation on the hardware structure of the processing module exist in the message analysis performance of the current intelligent network card, so that the message analysis performance is influenced.
Disclosure of Invention
The embodiment of the invention provides a message analysis method, a message analysis device, electronic equipment and a storage medium, which are used for realizing high-performance and linear-speed processing on a message and achieving the purpose of efficiently analyzing the message.
In order to solve the above problem, in a first aspect, an embodiment of the present invention discloses a packet parsing method, which is applied to a packet parsing module shown in fig. 7, where the packet parsing module includes multiple protocol processing units, and the method includes:
receiving a message to be analyzed, and generating a table look-up request according to the message;
according to the table look-up request, the message is matched with a tcam rule table pre-configured by a target protocol processing unit in parallel to generate a matched action table, and key information of the message is extracted;
and stripping the message according to the length of a protocol layer preset in the action table to obtain the data information of the message.
Optionally, the receiving a message to be analyzed, and generating a table lookup request according to the message includes:
carrying out offset processing on the message according to preset bytes, and numbering the message;
and generating a table look-up request to inquire a pre-configured tcam rule table based on the message number, wherein the table look-up request comprises the message, the message number and the protocol layer type included by the message.
Optionally, the matching the message with a tcam rule table pre-configured by a target protocol processing unit in parallel according to the table look-up request to generate a matched action table, and extracting key information of the message includes:
according to the message, matching a tcam rule table configured in advance by the target protocol processing unit, wherein the tcam rule table comprises a key value, a mask and a protocol layer type configured according to a network protocol layer type;
if the table look-up request information is matched with the tcam rule table, generating an action table comprising a matching result, and extracting key information of the message according to the action table;
otherwise, the message is transmitted to the next target protocol processing unit.
Optionally, the stripping the packet according to the length of a protocol layer preset in the action table to obtain the data information of the packet includes:
stripping the key information from the message according to the length of a preset protocol layer in the action table;
transmitting the message with the key information stripped to the next target protocol processing unit;
and obtaining the data information of the message under the condition that the protocol layer information of the message is stripped.
Optionally, before receiving the message to be analyzed and generating a table lookup request according to the message, the method further includes:
determining a plurality of target protocol processing units corresponding to the protocol layer type according to the protocol layer information of the message, and controlling the plurality of target protocol processing units to be enabled and started;
and according to the type of a network protocol layer, extracting bit width configuration from the target protocol processing units.
Optionally, after the extracting, according to the network protocol layer type, the bit width configuration of the plurality of target protocol processing units, the method further includes:
configuring a tcam rule table for the target protocol processing units according to the network protocol layer type, wherein the tcam rule table is used for extracting message key information corresponding to the target processing units;
and configuring the protocol layer type of the next target protocol processing unit based on the action table generated by the tcam rule table.
Optionally, the stripping the packet according to the length of the protocol layer preset in the action table to obtain the data information of the packet further includes:
generating a checksum of the data information according to the data information of the message;
and verifying the checksum to determine that the message is analyzed effectively, and sending the data information to a message editing module.
In a second aspect, an embodiment of the present invention further discloses a packet parsing apparatus, which is applied to a packet parsing module, where the packet parsing module includes a plurality of protocol processing units, and the apparatus includes:
the message receiving module is used for receiving a message to be analyzed and generating a table look-up request according to the message;
the matching information module is used for matching the message with a tcam rule table pre-configured by a target protocol processing unit in parallel according to the table look-up request, generating a matched action table and extracting key information of the message;
and the message processing module is used for stripping the message according to the length of a preset protocol layer in the action table to obtain the data information of the message.
Optionally, the message receiving module includes:
the first processing submodule is used for carrying out offset processing on the message according to preset bytes and carrying out message numbering;
and the second processing submodule is used for generating a table look-up request to inquire a preconfigured tcam rule table based on the message number, wherein the table look-up request comprises the message, the message number and the protocol layer type included by the message.
Optionally, the matching information module includes:
the matching submodule is used for matching a tcam rule table which is configured in advance by the target protocol processing unit according to the message, wherein the tcam rule table comprises a key value, a mask and a protocol layer type which are configured according to a network protocol layer type;
a generating submodule, configured to generate an action table including a matching result if the table look-up request information matches the tcam rule table, and extract key information of the packet according to the action table;
and the first transmission submodule is used for transmitting the message to the next target protocol processing unit.
Optionally, the packet processing module includes:
the third processing submodule is used for stripping the key information from the message according to the length of a preset protocol layer in the action table;
the second transmission submodule is used for transmitting the message with the key information stripped out to the next target protocol processing unit;
and the fourth processing submodule is used for obtaining the data information of the message under the condition that the protocol layer information of the message is stripped.
Optionally, the apparatus further comprises:
the determining module is used for determining a plurality of target protocol processing units corresponding to the types of the protocol layers according to the protocol layer information of the message and controlling the plurality of target protocol processing units to be enabled and started;
and the first configuration module is used for extracting bit width configuration from the target protocol processing units according to the type of the network protocol layer.
Optionally, the apparatus further comprises:
a second configuration module, configured to configure a tcam rule table for the plurality of target protocol processing units according to the network protocol layer type, where the tcam rule table is used to extract message key information corresponding to the target processing unit;
and the third configuration module is used for configuring the protocol layer type of the next target protocol processing unit based on the action table generated by the tcam rule table.
Optionally, the apparatus further comprises:
the generating module is used for generating a checksum of the data information according to the data information of the message;
and the message sending module is used for determining that the message analysis is effective by verifying the checksum and sending the data information to the message editing module.
In a third aspect, an embodiment of the present invention further discloses an electronic device, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the message parsing method described above.
In a fourth aspect, the present invention further discloses a computer-readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the message parsing method.
The embodiment of the invention has the following advantages:
the message analysis method provided by the embodiment of the invention is applied to a message analysis module which comprises a plurality of protocol processing units, generates a table look-up request according to a message by receiving the message to be analyzed, and matches the message with a tcam rule table pre-configured by a target protocol processing unit in parallel according to the table look-up request to generate a matched action table, extracts key information of the message, and peels off the message according to the length of a protocol layer preset in the action table to obtain data information of the message. The invention allocates a protocol processing unit for each layer of protocol independently, processes the protocol according to the sequence pipeline of the network protocol layer without caching the message, saves the register resource in the module, and can meet the requirement of high performance.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart illustrating a step of a message parsing method according to an embodiment of the present invention;
fig. 2 is a flowchart of a method of step 102 of the message parsing method provided in the embodiment of the present invention in fig. 1;
fig. 3 is a second flowchart illustrating steps of a message parsing method according to an embodiment of the present invention;
fig. 4 is a third flowchart illustrating steps of a message parsing method according to an embodiment of the present invention;
fig. 5 is a flowchart of a message parsing method according to an embodiment of the present invention;
fig. 6 is a block diagram of a message parsing apparatus according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a message parsing apparatus according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.
Referring to fig. 1, a flowchart of a step of a message parsing method provided in an embodiment of the present invention is shown, and is applied to a message parsing module shown in fig. 7, where the message parsing module includes multiple protocol processing units, and the method may include:
In the embodiment of the invention, in order to realize the line speed and high-efficiency message processing performance, based on the idea of tcam fast table lookup, an OVS basic framework is adopted to analyze the message, wherein the OVS basic framework is composed of a message analysis module, a matching module and a message editing module, specifically, the schematic diagram of the message analysis module shown in fig. 7 is referred to.
It should be noted that the message parsing module is configured to find the destination and the action of the message according to the key information extracted from the message, and finally send the destination and the action to the message editing module for editing and forwarding. It should be noted that, the message parsing is implemented based on the tcam lookup concept, where each protocol processing unit of the message parsing module is stacked by a logic circuit, and does not need to be cached, and ends with two fixed clocks.
Specifically, the message parsing module receives a message to be parsed, and first, the message content is offset by taking one byte as a unit and numbered, for example, the first byte of the message is numbered as 0, the next byte is numbered as 1, and so on. And forming a table look-up request by numbering the last step and shifting the messages corresponding to the two bytes backwards and the cur _ type protocol layer types of the protocol processing units of the message analysis module, wherein the table look-up request queries a preconfigured tcam rule table under the condition that the protocol layer types are judged to be legal in the current protocol processing unit.
Specifically, in some embodiments, the step 101 of receiving a message to be analyzed and generating a table lookup request according to the message may include the following steps:
firstly, carrying out offset processing on a message according to preset bytes, and numbering the message;
in this embodiment, the preset byte may be determined according to a message format of the protocol processing unit, and the preset byte in this embodiment is described by taking 2 bytes as an example, which is only a specific example, and the preset byte of the message offset may also be set as other parameters in an actual using process, which is not described herein in detail. The message number is not limited in the embodiment, and in the actual use process, the message number can be any one of the identifiers capable of uniquely indicating the message.
Secondly, generating a table look-up request to inquire a preconfigured tcam rule table based on the message number, wherein the table look-up request comprises the message, the message number and the protocol layer type included by the message.
And step 102, according to the table look-up request, the message is matched with a tcam rule table pre-configured by the target protocol processing unit in parallel, a matched action table is generated, and key information of the message is extracted.
In the embodiment of the invention, a tcam rule table configured in advance by a target protocol processing unit is matched according to a table look-up request and a message. The tcam rule table comprises a key value, a mask and a protocol layer type configured according to the network protocol layer type. If the table lookup request information is matched with the tcam rule table, an action table comprising a matching result is generated, and the key information of the message is extracted according to the action table. If the table lookup request information is not matched with the tcam rule table, the message is transmitted to the next target protocol processing unit.
Note that, a TCAM (ternary content-addressable memory) is a dedicated high-speed memory, and can search data in a single clock cycle. The stored table entries are very flexible in length requirement, key table entries with any length can be stored in the same TCAM chip, each table entry in the TCAM rule table is stored in a form of < address and mask >, and the address and the mask occupy W bits respectively on the assumption that the length range of the address key is from 1 to W. For an entry with a key length of Y (1 ≦ Y ≦ W), the first Y bits of its mask are 1, while the remaining W-Y bits are 0, and the last W-Y bits of its address may be either 0 or 1.
In this embodiment, a target protocol processing unit corresponding to a message to be parsed is determined in a message parsing module, an input interface of each protocol processing unit has a register configuration interface, and a message input interface, a tcam rule configuration interface, a protocol layer type input interface of a previous protocol processing unit, and a key information input interface, and an output interface has a message output interface, a structure table of key information extracted by message parsing, and a protocol layer type of a next protocol processing unit.
Specifically, in response to the table lookup request, a key value key, a mask msk and a protocol layer type cur _ type in the tcam rule are read, wherein the mask supports the bit level, and the cur _ type is the protocol layer type analyzed by the current target protocol processing unit. Inquiring the information of a pre-configured tcam rule table through the message and the message number, and obtaining an action table of the rule if the information is hit. The action table includes the protocol type of the next target protocol processing unit, the length of the current protocol layer, and the offset and length of the key information of the message to be extracted, and extracts four key fields at most.
If the table lookup request information is matched and hit with the tcam rule table, extracting the key information of the message according to the offset and the length of the key information in the action table which generates the matching, sequentially putting the information into a queue with a byte bit width according to the sequential arrangement order, transmitting the information to the next target protocol processing unit, if the table lookup request information is not matched with the tcam rule table, filling 0 in the queue, and transmitting the message to the next target protocol processing unit.
And 103, stripping the message according to the length of the protocol layer preset in the action table to obtain the data information of the message.
In this embodiment, the message is stripped according to the preset protocol layer length of the target protocol processing unit in the action table, and the remaining message part is transmitted to the next target protocol processing unit for processing. Each target protocol processing unit processes one layer of protocol independently, and after the protocol layer processing is finished, the content of the current protocol layer is stripped and sent to the next target protocol processing unit to process the next layer of protocol. If the current target protocol processing unit encounters the unrecognized protocol type, the message is transmitted to the subsequent target protocol processing unit without processing, and so on, the whole series-connected target protocol processing unit is completed, and the analysis is completed, so that the linear speed processing is achieved.
Specifically, in some embodiments, the step 103 may peel off the packet according to the length of the protocol layer preset in the action table to obtain the data information of the packet, and includes the following steps:
firstly, stripping key information from a message according to the length of a preset protocol layer in an action table;
secondly, transmitting the message with the key information stripped to a next target protocol processing unit;
and finally, obtaining the data information of the message under the condition that the protocol layer information of the message is stripped.
The message analysis method provided by the embodiment of the invention is applied to a message analysis module which comprises a plurality of protocol processing units, a table lookup request is generated according to a message by receiving the message to be analyzed, the message is matched with a tcam rule table preset by a target protocol processing unit in parallel according to the table lookup request to generate a matched action table, key information of the message is extracted, and the message is stripped according to the length of a protocol layer preset in the action table to obtain data information of the message. The invention allocates a protocol processing unit for each layer of protocol independently, processes the protocol according to the sequence pipeline of the network protocol layer without caching the message, saves the register resource in the module, and can meet the requirement of high performance.
Specifically, referring to fig. 2, fig. 2 is a flowchart of a method of step 102 of the message parsing method provided in the embodiment of the present invention in fig. 1, where step 102 matches a message with a tcam rule table pre-configured by a target protocol processing unit in parallel according to a table lookup request, generates a matched action table, and extracts key information of the message, and includes:
The tcam rule table comprises a key value, a mask and a protocol layer type configured according to the network protocol layer type.
And step 1023, if not, the message is transmitted to the next target protocol processing unit.
In the steps 1021-1023, after the table lookup request of the message to be analyzed is obtained, the tcam rule table pre-configured by the target protocol processing unit is matched according to the table lookup request and the message content. The tcam rule table comprises a key value, a mask and a protocol layer type configured according to the network protocol layer type. If the table lookup request information is matched with the tcam rule table, generating an action table comprising a matching result, and extracting key information of the message according to the action table; if the table lookup request information is not matched with the tcam rule table, the message is transmitted to the next target protocol processing unit. It should be noted that, the target protocol processing unit, based on the general configuration register resource and 32 tcam rule resources, can flexibly configure the tcam rule table and the register enable switch, so that the target protocol processing unit becomes a two-layer, three-layer or other self-defined protocol processing unit, and realizes the message covering most protocol types, thereby avoiding the problem of modifying hardware codes for supporting a certain protocol, shortening the development period, and saving time and resource cost.
Referring to fig. 3, a second step flow chart of the message parsing method provided in the embodiment of the present invention is shown, where the method may include:
and step 104, determining a plurality of target protocol processing units corresponding to the protocol layer type according to the protocol layer information of the message, and controlling the plurality of target protocol processing units to be enabled and started.
In this embodiment, a message parsing module is initialized, initialization information is configured through a register, and a plurality of target protocol processing units corresponding to a protocol layer type are determined according to protocol layer information of a message, it should be noted that one register in the message parsing module is used for controlling enabling of the plurality of protocol processing units, and only when global enabling is turned on, the register can have authority to participate in querying a tcam rule table.
And 105, extracting bit width configuration of the target protocol processing units according to the type of the network protocol layer.
Specifically, bit width configuration is extracted from a plurality of target protocol processing units according to the type of a network protocol layer, because bit widths required to be extracted by each protocol layer are different according to the network protocol, if the protocol layers are designed according to the maximum bit width, for example, if sip + dip of IPV6 is already 256 bits, resource waste is caused, and meanwhile, the large bit width also causes that the chip timing sequence is difficult to converge, that is, it is very necessary to extract bit width configuration from a plurality of target protocol processing units, and key information extracted by each target protocol processing unit is put into a queue, so that message editing and forwarding are facilitated.
And step 102, according to the table look-up request, the message is matched with a tcam rule table pre-configured by the target protocol processing unit in parallel, a matched action table is generated, and key information of the message is extracted.
And 103, stripping the message according to the length of the protocol layer preset in the action table to obtain the data information of the message.
The above steps 101-103 are not described again with reference to the preamble discussion.
It should be noted that the present embodiment does not limit the specific execution positions of steps 104 and 105, and for convenience of understanding, the present embodiment will be described by taking the steps 104 and 105 as an example before step 101. In an actual using process, steps 104 and 105 may also be completed as one step, and details of each case are not described here.
Specifically, after the step 105 of extracting bit width configuration from the multiple target protocol processing units according to the network protocol layer type, the method may further include:
configuring a tcam rule table for a plurality of target protocol processing units according to the type of the network protocol layer, wherein the tcam rule table is used for extracting message key information corresponding to the target processing units;
and configuring the protocol layer type of the next target protocol processing unit based on the action table generated by the tcam rule table.
By way of example, the configuration process is: if the message to be analyzed is a seven-tuple (SMAC, DMAC, SIP, DIP, IP _ PROTOCOL, SP, DP), the number of the message layers is four, namely an MAC layer, an IP layer, a TCP layer and a message load layer. First, initialization configuration is performed, and it is determined that enable switches for configuring the target protocol processing unit stage1, the target protocol processing unit stage2, the target protocol processing unit stage3, and the target protocol processing unit stage10 are turned on, and other enable switches are turned off. Stage1 processes mac layer related information and configures bit width for extracting key information, and the extracted bit width can be configured to be 96 bits due to the fact that smac and dmac need to be extracted. Similarly, stage2 processes the IP layer related information, and since ipv4 is used, tag extraction bit width is configured to be 72 bits. Stage3 processes tcp layer information, and the configuration extraction bit width is 32 bits. Stage10 can automatically calculate checksum of message data information without any configuration.
The tcam rule table is next configured: the self-defined MAC TYPE of Stage1 is MAC _ TYPE, cur _ TYPE input by Stage1 is fixed into MAC _ TYPE, cur _ TYPE of protocol layer TYPE of tcam rule table is configured into MAC _ TYPE, and corresponding msk is filled with 0.Key0 is padded with 0x0800, offset is configured to 6, corresponding to msk position padded with 0, other keys 1, 2, 3 are padded with 0xffff, corresponding to msk each bit padded with 1, indicating no concern. The action table contains nxt _ TYPE to fill in IPV4_ TYPE, head _ len 14, rsp0 _offset0, rsp1 _offset6, and the rest of 0xff to indicate invalidity, so that smac/dmac can be extracted. Similarly, stage2 configuration will extract SIP, DIP, IP _ PROTOCOL correspondingly, and Stage3 configuration will extract SP, DP correspondingly.
On the basis of achieving the beneficial effects of the first embodiment, the embodiment of the invention can flexibly configure the register and the tcam rule table in the module, achieve the purpose of analyzing various protocol messages, avoid repeatedly modifying hardware codes, shorten the development period, and facilitate the subsequent conversion of the IC stream chip. And moreover, a common modularized design style of IC design is adopted, a complex network protocol is processed in a grading way according to a network protocol layer, and maintenance and reuse are facilitated.
Referring to fig. 4, a third step flow chart of the message parsing method provided in the embodiment of the present invention is shown, where the method may further include:
And step 102, according to the table look-up request, the message is matched with a tcam rule table pre-configured by the target protocol processing unit in parallel, a matched action table is generated, and key information of the message is extracted.
And 103, stripping the message according to the length of the protocol layer preset in the action table to obtain the data information of the message.
The above steps 101-103 are not described again with reference to the preamble discussion.
And 106, generating a checksum of the data information according to the data information of the message.
In this embodiment, the message parsing module is divided into a plurality of protocol processing units, each protocol processing unit completes processing of one layer of protocol, and the plurality of protocol processing units are connected in series, so that the pipeline completes processing of the whole message. The purpose of generating the checksum of the data information according to the data information of the message is to perform calculation without caching the whole packet when the udp/tcp checksum is recalculated by the message editing module at the rearmost of the intelligent network card, thereby playing a certain acceleration role.
And step 107, verifying the checksum to determine that the message is analyzed effectively, and sending the data information to a message editing module.
On the basis of achieving the beneficial effects of the first embodiment, the embodiment of the invention determines that the message analysis is effective by checking and verifying the message data information, and sends the data information to the message editing module for editing and forwarding, so that a plurality of protocol processing units can simultaneously analyze the network message and obtain the message analysis result, single-line waiting is not needed, the processing delay is greatly reduced, the analysis speed of the network message is obviously improved, meanwhile, the analysis of the subsequent message cannot be influenced by errors in the network message in the application, and the system fault tolerance is higher.
Referring to fig. 5, a flowchart of a message parsing method provided in the embodiment of the present invention is shown, where the method may include:
Specifically, the message is preprocessed, and the message content is offset and numbered according to a byte as a unit, for example, the first byte of the message is numbered as 0, the next byte is numbered as 1, and the message number is generated by analogy.
Specifically, the number of the previous step, the message corresponding to two bytes shifted backward, and the cur _ type protocol layer type of the protocol processing unit of the message parsing module form a table lookup request, and the table lookup request queries a preconfigured tcam rule table when the protocol layer type is judged to be legal in the current protocol processing unit.
Step 203, judging whether the message is matched with the tcam rule table.
Specifically, the tcam rule table pre-configured by the target protocol processing unit is matched according to the table look-up request and the message content. The tcam rule table includes a key value, a mask and a protocol layer type configured according to a network protocol layer type. If the table lookup request information is matched with the tcam rule table, entering step 204, generating an action table comprising a matching result, and extracting key information of the message according to the action table; otherwise, if the table lookup request information does not match the tcam rule table, step 206 is performed to pass the message to the next target protocol processing unit.
In this embodiment, the preconfigured tcam rule table information is queried through the message and the message number, and if the information is matched and hit, an action table of the rule is generated. The action table includes the protocol type of the next target protocol processing unit, the length of the current protocol layer, and the offset and length of the key information of the message to be extracted.
And step 205, stripping the message to obtain the data information of the message.
Specifically, the message is stripped according to the preset protocol layer length of the target protocol processing unit in the action table, and the rest message part is transmitted to the next target protocol processing unit for processing.
The message analysis method provided by the embodiment of the invention is applied to a message analysis module which comprises a plurality of protocol processing units, a table lookup request is generated according to a message by receiving the message to be analyzed, the message is matched with a tcam rule table preset by a target protocol processing unit in parallel according to the table lookup request to generate a matched action table, key information of the message is extracted, and the message is stripped according to the length of a protocol layer preset in the action table to obtain data information of the message. The invention independently allocates a protocol processing unit for each layer of protocol, processes the stream line according to the sequence of the network protocol layer without caching the message, saves the register resource in the module, and meets the requirement of high performance.
Referring to fig. 6, a block diagram of an apparatus for parsing a message according to an embodiment of the present invention is shown, and the apparatus is applied to a message parsing module shown in fig. 7, where the message parsing module includes a plurality of protocol processing units, and the apparatus includes:
a message receiving module 301, configured to receive a message to be analyzed, and generate a table lookup request according to the message;
a matching information module 302, configured to match the message with a tcam rule table preconfigured by the target protocol processing unit in parallel according to the table lookup request, generate a matched action table, and extract key information of the message;
and the message processing module 303 is configured to strip the message according to a preset protocol layer length in the action table to obtain data information of the message.
Further, the message receiving module 301 includes:
the first processing submodule is used for carrying out offset processing on the message according to preset bytes and carrying out message numbering;
and the second processing submodule is used for generating a table look-up request to inquire a preconfigured tcam rule table based on the message number, wherein the table look-up request comprises the message, the message number and the protocol layer type included by the message.
Further, the matching information module 302 includes:
the matching submodule is used for matching a tcam rule table which is configured in advance by the target protocol processing unit according to the message, wherein the tcam rule table comprises a key value, a mask and a protocol layer type which are configured according to a network protocol layer type;
a generating submodule, configured to generate an action table including a matching result if the table look-up request information matches the tcam rule table, and extract key information of the packet according to the action table;
and the first transmission submodule is used for transmitting the message to the next target protocol processing unit.
Further, the message processing module 303 includes:
the third processing submodule is used for stripping the key information from the message according to the length of a preset protocol layer in the action table;
the second transmission submodule is used for transmitting the message with the key information stripped out to the next target protocol processing unit;
and the fourth processing submodule is used for obtaining the data information of the message under the condition that the protocol layer information of the message is stripped.
Further, the apparatus further comprises:
the determining module is used for determining a plurality of target protocol processing units corresponding to the types of the protocol layers according to the protocol layer information of the message and controlling the plurality of target protocol processing units to be enabled and started;
and the first configuration module is used for extracting bit width configuration from the target protocol processing units according to the type of the network protocol layer.
Further, the apparatus further comprises:
a second configuration module, configured to configure a tcam rule table for the plurality of target protocol processing units according to the network protocol layer type, where the tcam rule table is used to extract key message information corresponding to the target processing unit;
and the third configuration module is used for configuring the protocol layer type of the next target protocol processing unit based on the action table generated by the tcam rule table.
Further, the apparatus further comprises:
the generating module is used for generating a checksum of the data information according to the data information of the message;
and the message sending module is used for determining that the message is analyzed effectively by verifying the checksum and sending the data information to the message editing module.
The message analysis device provided by the embodiment of the invention receives a message to be analyzed, generates a table look-up request according to the message, and generates a matched action table by matching the message with a tcam rule table configured in advance by a target protocol processing unit according to the table look-up request in parallel, extracts key information of the message, and peels off the message according to the length of a protocol layer preset in the action table to obtain data information of the message. The invention allocates a protocol processing unit for each layer of protocol independently, processes the protocol according to the sequence pipeline of the network protocol layer without caching the message, saves the register resource in the module, and can meet the requirement of high performance.
An embodiment of the present invention further provides an electronic device, as shown in fig. 8, including a processor 401, a communication interface 402, a memory 403, and a communication bus 404, where the processor 401, the communication interface 402, and the memory 403 complete mutual communication through the communication bus 404,
a memory 403 for storing a computer program;
the processor 401, when executing the program stored in the memory 403, may implement the following steps:
receiving a message to be analyzed, generating a table look-up request according to the message, matching the message with a tcam rule table configured in advance by a target protocol processing unit in parallel according to the table look-up request, generating a matched action table, extracting key information of the message, and stripping the message according to the length of a protocol layer preset in the action table to obtain data information of the message.
The communication bus mentioned in the above terminal may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this is not intended to represent only one bus or type of bus.
The communication interface is used for communication between the terminal and other equipment.
The Memory may include a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. Alternatively, the memory may be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.
In another embodiment of the present invention, a computer-readable storage medium is further provided, where instructions are stored in the computer-readable storage medium, and when the instructions are executed on a computer, the computer is caused to execute the message parsing method in any one of the above embodiments.
In another embodiment of the present invention, there is also provided a computer program product including instructions, which when run on a computer, cause the computer to perform the message parsing method according to any one of the above embodiments.
In the above embodiments, all or part of the implementation may be realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that includes one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), among others.
While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or terminal apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above detailed description is provided for the message parsing method, apparatus, electronic device and storage medium provided by the present invention, and specific examples are applied herein to explain the principle and implementation manner of the present invention, and the description of the above embodiments is only used to help understanding the method and core ideas of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (15)
1. A message analysis method is applied to a message analysis module, the message analysis module comprises a plurality of protocol processing units, and the method comprises the following steps:
receiving a message to be analyzed, and generating a table look-up request according to the message;
according to the table look-up request, the message is matched with a tcam rule table pre-configured by a target protocol processing unit in parallel to generate a matched action table, and key information of the message is extracted;
and stripping the message according to the length of a protocol layer preset in the action table to obtain the data information of the message.
2. The method of claim 1, wherein the receiving a message to be parsed and generating a table lookup request from the message comprises:
carrying out offset processing on the message according to preset bytes, and numbering the message;
and generating a table look-up request to inquire a preconfigured tcam rule table based on the message number, wherein the table look-up request comprises the message, the message number and the protocol layer type included by the message.
3. The method according to claim 1, wherein said matching said message with a tcam rule table pre-configured by a target protocol processing unit in parallel according to said table lookup request to generate a matched action table, and extracting key information of said message comprises:
according to the message, matching a tcam rule table configured in advance by the target protocol processing unit, wherein the tcam rule table comprises a key value, a mask and a protocol layer type configured according to a network protocol layer type;
if the table look-up request information is matched with the tcam rule table, generating an action table comprising a matching result, and extracting key information of the message according to the action table;
otherwise, the message is transmitted to the next target protocol processing unit.
4. The method according to claim 1, wherein the stripping the packet according to the length of the protocol layer preset in the action table to obtain the data information of the packet comprises:
stripping the key information from the message according to the length of a preset protocol layer in the action table;
transmitting the message with the key information stripped to the next target protocol processing unit;
and under the condition that the protocol layer information of the message is stripped, obtaining the data information of the message.
5. The method of claim 1, wherein before receiving the message to be parsed and generating a table lookup request from the message, the method further comprises:
determining a plurality of target protocol processing units corresponding to the protocol layer type according to the protocol layer information of the message, and controlling the plurality of target protocol processing units to be enabled and started;
and according to the type of a network protocol layer, extracting bit width configuration from the target protocol processing units.
6. The method according to claim 5, wherein after said extracting bit width configuration from said plurality of target protocol processing units according to network protocol layer type, further comprising:
configuring a tcam rule table for the target protocol processing units according to the network protocol layer type, wherein the tcam rule table is used for extracting message key information corresponding to the target protocol processing units;
and configuring the protocol layer type of the next target protocol processing unit based on the action table generated by the tcam rule table.
7. The method according to claim 1, wherein after the message is stripped according to a preset protocol layer length in the action table to obtain data information of the message, the method further comprises:
generating a checksum of the data information according to the data information of the message;
and verifying the checksum to determine that the message is analyzed effectively, and sending the data information to a message editing module.
8. A message parsing apparatus, applied to a message parsing module, where the message parsing module includes a plurality of protocol processing units, the apparatus includes:
the message receiving module is used for receiving a message to be analyzed and generating a table look-up request according to the message;
the matching information module is used for matching the message with a tcam rule table pre-configured by a target protocol processing unit in parallel according to the table look-up request, generating a matched action table and extracting key information of the message;
and the message processing module is used for stripping the message according to the length of a preset protocol layer in the action table to obtain the data information of the message.
9. The apparatus of claim 8, wherein the means for receiving messages comprises:
the first processing submodule is used for carrying out offset processing on the message according to preset bytes and carrying out message numbering;
and the second processing submodule is used for generating a table look-up request to inquire a pre-configured tcam rule table based on the message number, wherein the table look-up request comprises the message, the message number and the protocol layer type included by the message.
10. The apparatus of claim 8, wherein the matching information module comprises:
the matching submodule is used for matching a tcam rule table which is configured in advance by the target protocol processing unit according to the message, wherein the tcam rule table comprises a key value, a mask and a protocol layer type which are configured according to a network protocol layer type;
a generating submodule, configured to generate an action table including a matching result if the table look-up request information matches the tcam rule table, and extract key information of the packet according to the action table;
and the first transmission submodule is used for transmitting the message to the next target protocol processing unit.
11. The apparatus of claim 8, wherein the message processing module comprises:
the third processing submodule is used for stripping the key information from the message according to the length of a preset protocol layer in the action table;
the second transmission submodule is used for transmitting the message with the key information stripped out to the next target protocol processing unit;
and the fourth processing submodule is used for obtaining the data information of the message under the condition that the protocol layer information of the message is stripped.
12. The apparatus of claim 8, further comprising:
the determining module is used for determining a plurality of target protocol processing units corresponding to the types of the protocol layers according to the protocol layer information of the message and controlling the plurality of target protocol processing units to be enabled and started;
and the first configuration module is used for extracting bit width configuration from the target protocol processing units according to the type of the network protocol layer.
13. The apparatus of claim 12, further comprising:
a second configuration module, configured to configure a tcam rule table for the plurality of target protocol processing units according to the network protocol layer type, where the tcam rule table is used to extract message key information corresponding to the target protocol processing unit;
and the third configuration module is used for configuring the protocol layer type of the next target protocol processing unit based on the action table generated by the tcam rule table.
14. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing the communication between the processor and the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the steps of the message parsing method according to any one of claims 1 to 7 when executing a program stored in a memory.
15. A readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the message parsing method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310083400.2A CN115883681B (en) | 2023-02-08 | 2023-02-08 | Message parsing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310083400.2A CN115883681B (en) | 2023-02-08 | 2023-02-08 | Message parsing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115883681A true CN115883681A (en) | 2023-03-31 |
| CN115883681B CN115883681B (en) | 2023-06-23 |
Family
ID=85760874
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310083400.2A Active CN115883681B (en) | 2023-02-08 | 2023-02-08 | Message parsing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115883681B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116156026A (en) * | 2023-04-20 | 2023-05-23 | 中国人民解放军国防科技大学 | RMT-supporting parser, reverse parser, parsing method and switch |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1585379A (en) * | 2004-05-25 | 2005-02-23 | 华中科技大学 | Rapid analyzing method for data pack |
| CN108781185A (en) * | 2016-12-13 | 2018-11-09 | 甲骨文国际公司 | The system and method that programmable packet taxonomy model for the network equipment is provided |
| CN109672669A (en) * | 2018-12-03 | 2019-04-23 | 国家计算机网络与信息安全管理中心 | The filter method and device of traffic messages |
| CN111277612A (en) * | 2020-05-08 | 2020-06-12 | 常州楠菲微电子有限公司 | Network message processing strategy generation method, system and medium |
| CN112732241A (en) * | 2021-01-08 | 2021-04-30 | 烽火通信科技股份有限公司 | Programmable analyzer under multistage parallel high-speed processing and analysis method thereof |
| CN112929281A (en) * | 2021-02-04 | 2021-06-08 | 恒安嘉新（北京）科技股份公司 | Message processing method, device and equipment of network equipment based on FPGA |
| US20210185153A1 (en) * | 2019-12-13 | 2021-06-17 | Marvell Israel (M.I.S.L) Ltd. | Hybrid Fixed/Programmable Header Parser for Network Devices |
| CN113691486A (en) * | 2020-05-19 | 2021-11-23 | 中移(成都)信息通信科技有限公司 | Message modification method, device, equipment and storage medium |
-
2023
- 2023-02-08 CN CN202310083400.2A patent/CN115883681B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1585379A (en) * | 2004-05-25 | 2005-02-23 | 华中科技大学 | Rapid analyzing method for data pack |
| CN108781185A (en) * | 2016-12-13 | 2018-11-09 | 甲骨文国际公司 | The system and method that programmable packet taxonomy model for the network equipment is provided |
| CN109672669A (en) * | 2018-12-03 | 2019-04-23 | 国家计算机网络与信息安全管理中心 | The filter method and device of traffic messages |
| US20210185153A1 (en) * | 2019-12-13 | 2021-06-17 | Marvell Israel (M.I.S.L) Ltd. | Hybrid Fixed/Programmable Header Parser for Network Devices |
| CN111277612A (en) * | 2020-05-08 | 2020-06-12 | 常州楠菲微电子有限公司 | Network message processing strategy generation method, system and medium |
| CN113691486A (en) * | 2020-05-19 | 2021-11-23 | 中移(成都)信息通信科技有限公司 | Message modification method, device, equipment and storage medium |
| CN112732241A (en) * | 2021-01-08 | 2021-04-30 | 烽火通信科技股份有限公司 | Programmable analyzer under multistage parallel high-speed processing and analysis method thereof |
| CN112929281A (en) * | 2021-02-04 | 2021-06-08 | 恒安嘉新（北京）科技股份公司 | Message processing method, device and equipment of network equipment based on FPGA |
Non-Patent Citations (3)
| Title |
|---|
| 吉萌等: "高性能路由器上分布式协议栈的研究和实现" * |
| 李洪兰等: "基于TCAM的高速报文过滤卡设计", 科学技术与工程 * |
| 杨斌: "基于FPGA的低功耗高速网络报文解析系统设计", 电子技术与软件工程 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116156026A (en) * | 2023-04-20 | 2023-05-23 | 中国人民解放军国防科技大学 | RMT-supporting parser, reverse parser, parsing method and switch |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115883681B (en) | 2023-06-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7599364B2 (en) | Configurable network connection address forming hardware | |
| US8681819B2 (en) | Programmable multifield parser packet | |
| CN112491708A (en) | Routing header encapsulation method and device of IPv6 message | |
| EP1715631A1 (en) | Method and apparatus for managing a network flow in a high performance network interface | |
| RU2608874C2 (en) | Method and device for modifying and forwarding messages in data network | |
| JP2015165650A (en) | Apparatus and method of generating lookups and making decisions for packet modifying and forwarding in software-defined network engine | |
| WO2020087400A1 (en) | Header parsing apparatus and method | |
| CN111614580B (en) | Data forwarding method, device and equipment | |
| CN115883681A (en) | Message analysis method and device, electronic equipment and storage medium | |
| CN112929419B (en) | Data packet transmission method and device, electronic equipment and readable storage medium | |
| CN113794788A (en) | Gateway diversion method, system, device, equipment, storage medium and product | |
| CN114946167B (en) | Message parsing method and device | |
| US10185783B2 (en) | Data processing device, data processing method, and non-transitory computer readable medium | |
| CN112087389B (en) | Message matching table look-up method, system, storage medium and terminal | |
| CN114079634B (en) | Message forwarding method and device and computer readable storage medium | |
| CN115277553B (en) | Stream table storage method, device, equipment and computer readable storage medium | |
| CN106789440B (en) | IP packet header detection method and device | |
| CN105634999A (en) | Aging method and device for medium access control address | |
| WO2016101490A1 (en) | Update processing method and device | |
| CN116074404A (en) | Parsing device, message parsing method, forwarding chip and network equipment | |
| CN117240947B (en) | Message processing method, device and medium | |
| CN116471344B (en) | Keyword extraction method, device and medium for data message | |
| CN113098780B (en) | Message processing method of virtual network, electronic equipment and storage medium | |
| EP3993366B1 (en) | Network load balancer, request message distribution method, program product and system | |
| WO2022057887A1 (en) | Packet modification method and apparatus, computer device, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP02 | Change in the address of a patent holder |
Address after: 100007 room 205-32, floor 2, building 2, No. 1 and No. 3, qinglonghutong a, Dongcheng District, Beijing Patentee after: Tianyiyun Technology Co.,Ltd. Address before: 100093 Floor 4, Block E, Xishan Yingfu Business Center, Haidian District, Beijing Patentee before: Tianyiyun Technology Co.,Ltd. |
|
| CP02 | Change in the address of a patent holder |