CN112564856A

CN112564856A - Message processing method and device and computer readable storage medium

Info

Publication number: CN112564856A
Application number: CN201911207236.1A
Authority: CN
Inventors: 史济源; 陈庆; 张明礼
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2019-09-10
Filing date: 2019-11-29
Publication date: 2021-03-26

Abstract

The application discloses a message processing method, a message processing device and a computer readable storage medium, and belongs to the technical field of data coding. According to the method, the splicing processing is carried out on other original messages except the largest first message in the plurality of original messages, only when the size of the spliced message is smaller than that of the largest message, the spliced message is filled, and each other original message is not required to be filled, so that less data are filled in the filled equal-length data blocks, the network bandwidth occupied by FEC coding message transmission is reduced, and the waste of network resources is avoided.

Description

Message processing method and device and computer readable storage medium

The present application claims priority of chinese patent application with application number 201910854179.X entitled "method, apparatus and system for generating equal length coding blocks" filed on 10/09/2019, which is incorporated herein by reference in its entirety.

Technical Field

The present application relates to the field of data encoding technologies, and in particular, to a method and an apparatus for processing a packet, and a computer-readable storage medium.

Background

With the development of coding and decoding technology, in the process of transmitting data stream, a network device may perform Forward Error Correction (FEC) coding on a plurality of original messages in the data stream to obtain redundant messages, and send the original messages and the redundant messages to a decoding end.

At present, the process of encoding a plurality of original messages by a network device may be: the network equipment fills invalid data in other original messages except the maximum message in the plurality of original messages, so that the sizes of the filled messages are the size of the maximum message in the plurality of original messages, and the network equipment performs FEC coding on the filled messages and the maximum message to obtain redundant messages.

In the encoding process, since the network device fills the invalid data in each original message smaller than the maximum message, a large amount of invalid data is filled in the filled original message, and the data amount of the filled original message is large, when the network device sends the filled original message to the decoding end, the filled original message occupies a large amount of network bandwidth, which causes network resource waste.

Disclosure of Invention

The application provides a message processing method, a message processing device and a computer readable storage medium, which can reduce the network bandwidth occupied in the message sending process and avoid network resource waste. The technical scheme is as follows:

in a first aspect, a method for processing a packet is provided, where the method includes:

acquiring a plurality of original messages;

according to the size of a first message in the original messages, splicing other original messages except the first message in the original messages to obtain at least one spliced message, wherein the first message is the largest message in the original messages;

when the size of any spliced message in the at least one spliced message is smaller than that of the first message, filling the at least one spliced message to obtain equal-length data blocks, wherein the size of each data block in the equal-length data blocks is the size of the first message;

and carrying out Forward Error Correction (FEC) coding processing on the first message and the equal-length data block to obtain at least one redundant message.

According to the method, the splicing processing is carried out on other original messages except the largest first message in the plurality of original messages, only when the size of the spliced message is smaller than that of the largest message, the spliced message is filled, and each other original message is not required to be filled, so that less data are filled in the filled equal-length data blocks, the network bandwidth occupied by FEC coding message transmission is reduced, and the waste of network resources is avoided.

In a possible implementation manner, the splicing processing, according to a size of a first packet in the multiple original packets, other original packets except the first packet in the multiple original packets to obtain at least one spliced packet includes:

according to the size of a first message in the original messages, grouping other original messages except the first message in the original messages to obtain at least one message group, wherein each message group comprises at least one original message, and the sum of the sizes of all the original messages in each message group is smaller than or equal to the size of the first message;

and splicing each message group to obtain at least one spliced message, wherein each spliced message corresponds to one message group.

In a possible implementation manner, the splicing, according to a size of a first packet in the multiple original packets, the messages other than the first packet in the multiple original packets to obtain at least one spliced message includes:

the processor groups other original messages except the first message in the original messages according to the size of the first message in the original messages to obtain at least one message group, and generates a coding task according to the at least one message group, wherein the coding task comprises address information of each message group;

and the target hardware engine carries out splicing processing on each message group according to the coding task to obtain at least one spliced message.

In a possible implementation manner, the grouping, according to the size of a first packet in the multiple original packets, other original packets in the multiple original packets except the first packet to obtain at least one packet group includes:

sequencing the plurality of original messages according to the size of each original message in the plurality of original messages to obtain the sequence i of each original message, wherein the sequence i is an integer which is greater than or equal to zero, and the original message with the sequence of 0 is the first message;

when i is 1, dividing the original message with the sequence of 1 into a 1 st target message group, wherein the sequence of the 1 st target message group is 1;

when i is greater than 1, if the sum of the size of a jth target packet group and the size of an original packet with the sequence i is smaller than or equal to a target size, merging the jth target packet group and the original packet with the sequence i into an ith target packet group, wherein the target size is the size of the first packet, the jth target packet group is a packet group with the minimum sequence in at least one target packet group, the sequence of the jth target packet group is j, the sum of the size of each packet group in the at least one target packet group and the size of the original packet with the sequence i is smaller than or equal to the target size, and j is 0< i;

when i is greater than 1, if the jth target message group does not exist, adding an ith target message group, and dividing the original message with the order i into the ith target message group;

and after the last original message in the plurality of original messages is grouped, taking each current residual target message group as one message group in the at least one message group.

In a possible implementation manner, the performing the concatenation processing on each packet group to obtain the at least one concatenated packet includes:

when the sum of the number of the at least one message group and the number of first messages in the plurality of original messages reaches a first target number, and when other original messages except the plurality of original messages cannot be added to the at least one message group, splicing processing is performed on each message group, wherein the first target number is the dimension of a matrix to be coded when FEC coding is performed for one time; or the like, or, alternatively,

and when the number of the plurality of original messages is equal to a second target number, and the original messages with the second target number are divided into at least one message group, splicing each message group, wherein the second target number is the preset number of the original messages for carrying out FEC encoding once.

In one possible implementation, the method further includes:

and when the size of each spliced message in the at least one spliced message is equal to that of the first message, performing FEC (forward error correction) coding processing on the first message and the at least one spliced message to obtain at least one redundant message.

In a second aspect, a message processing apparatus is provided, which includes a plurality of functional modules, where the functional modules interact with each other to implement the method in the first aspect and the embodiments thereof. The functional modules can be implemented based on software, hardware or a combination of software and hardware, and the functional modules can be combined or divided arbitrarily based on specific implementation.

In a third aspect, a computer-readable storage medium is provided, in which instructions are stored, and the instructions are loaded and executed by a processor to implement the message processing method as described above.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required to be used in the description of the embodiments are briefly introduced below, the drawings in the following description are only some embodiments of the present application, and it is obvious for a person skilled in the art to obtain other drawings based on the drawings without creative efforts.

Fig. 1 is a schematic diagram of a message transmission system according to an embodiment of the present application;

fig. 2 is a flowchart of a message processing method according to an embodiment of the present application;

fig. 3 is a flowchart of a method for implementing message processing in a network device according to an embodiment of the present application;

fig. 4 is a schematic diagram of a packet group determining process according to an embodiment of the present application;

fig. 5 is a flowchart of a packet according to an embodiment of the present application;

fig. 6 is a schematic diagram of an original packet flow provided in an embodiment of the present application;

FIG. 7 is a diagram illustrating an encoding task provided by an embodiment of the present application;

fig. 8 is a schematic diagram of an FEC packet according to an embodiment of the present application;

fig. 9 is a schematic flowchart of internal interaction of a network device according to an embodiment of the present application;

fig. 10 is a flowchart of a codec provided in an embodiment of the present application;

fig. 11 is a schematic structural diagram of a message processing apparatus according to an embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

Fig. 1 is a schematic diagram of a message transmission system according to an embodiment of the present application, and referring to fig. 2, a first terminal 101, a first network device 102, a second network device 103, and a second terminal 104, where the first terminal 101 is configured to generate a data stream and send the data stream to the first network device 102. The first network device 102 splices a plurality of original messages in the data stream into an equal-length data block, the size of each data block in the equal-length data block is equal, performs FEC coding processing on the equal-length data block to obtain redundant messages of the plurality of original messages, adds the same FEC header to each data block and each redundant message in the equal-length data block, encapsulates the FEC header into a plurality of FEC messages, and then sends a coded stream composed of the FEC messages to the second network device 103. The second network device 303 decodes the original FEC message lost in the encoded stream, and recovers the lost original FEC message, where the original FEC message is any FEC message. The second network device 103 recovers the data stream based on the recovered original FEC packets and the FEC packets that are not lost, and sends the data stream to the second terminal 104. The first terminal 101 and the second terminal 104 may be mobile phones, notebook computers, etc., and the first network device 102 and the second network device 103 may be routers, switches, etc. The data stream may be a video stream, an audio stream, or a text stream composed of text data, and the type of the data stream is not specifically limited in the embodiments of the present application.

Fig. 1 is a schematic diagram illustrating a video stream as an example, when a first user and a second user perform a video session, the first user may record a video of the first user using a camera on a first terminal 302, and send an original message for forming the video to a first network device 102 to form a video stream (data stream), the first network device 102 performs FEC encoding on the original message in the video stream to obtain redundant messages, an FEC message header is added to each original message and each redundant message to obtain a corresponding FEC message, and the first network device 102 may send an encoded stream formed by FEC messages to the second network device 103 through a Wide Area Network (WAN); after the second network device 103 acquires the FEC messages in the encoded stream, it determines which original messages are lost according to the FEC message header, and recovers the lost original messages according to the received FEC messages (including the original messages and the redundant messages), the second network device 103 combines the recovered original messages and the original messages that are not lost into a data stream, and sends the data stream to the second terminals 104 in the local area networks a and B, respectively, and after the video streams are received by the plurality of second terminals 104, video playing is performed, so that the second user can view the video of the first user on the second terminal 304, thereby implementing the cross-local area network video session between the first user and the second user. In a possible implementation, the first terminal 101, the first network device 102, the second network device 103, and the second terminal 104 may be distributed in the same local area network or different local area networks.

In some possible implementations, the first network device may directly perform splicing processing on the original packet in the data stream through the processor to obtain equal-length data blocks, and then perform FEC encoding on the equal-length data blocks through the processor. In some possible implementations, the processor in the first network device may determine a splicing scheme according to the size of each original packet, send the splicing scheme to the target hardware engine, and perform splicing processing on the original packets in the data stream by the target hardware engine according to the splicing scheme provided by the processor to obtain equal-length data blocks, and perform FEC encoding on the equal-length data blocks. The processor may be a Network Processor (NP), a Central Processing Unit (CPU), or a combination of an NP and a CPU. The processor may further include a hardware chip, which may be an application-specific integrated circuit (ASIC), a Programmable Logic Device (PLD), or a combination thereof. The PLD may be a Complex Programmable Logic Device (CPLD), a field-programmable gate array (FPGA), a General Array Logic (GAL), or any combination thereof.

Optionally, embodiments of the present application further provide a computer-readable storage medium, such as a memory, including instructions, which are executable by a processor in a network device to perform the method provided in the following embodiments. For example, the computer-readable storage medium may be a read-only memory (ROM), a Random Access Memory (RAM), a compact disc-read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

For further explaining the process of encoding the original packet in the data stream by the first network device, refer to a flowchart of a packet processing method provided in this embodiment shown in fig. 2, where the method flow may specifically include the following steps 201 and 208. In a possible implementation, when the application describes the process shown in fig. 2, the steps included in the entire flow of fig. 2 are described first, and then detailed description is performed on each step.

201. The first network device obtains a plurality of original messages.

The first network device may obtain a plurality of data streams. The plurality of original messages are messages in a first data stream of the plurality of data streams, and are messages used for FEC encoding of the first network device. The first data stream is any one of a plurality of data streams, the first data stream may include a plurality of original messages, each original message is used to carry data, the data carried by the original message may be video data, audio data, text data, or the like, and the type of the data in the first data stream is not specifically limited in this embodiment of the application.

202. The processor groups other original messages except the first message in the plurality of original messages according to the size of the first message in the plurality of original messages to obtain at least one message group, wherein each message group comprises at least one original message, and the sum of the sizes of all the original messages in each message group is smaller than or equal to the size of the first message.

203. And the processor generates an encoding task according to the at least one message group, wherein the encoding task comprises the address information of each message group.

204. The processor sends the encoding task to the target hardware engine.

In some possible implementations, the processor may also issue the encoding task to the memory, and the target hardware engine reads the encoding task from the memory. In some possible implementations, the processor may also issue a coding task to the memory or the target hardware engine according to the priority of the data stream, and the processor may issue a coding task of a data stream with a high priority first and then issue a coding task of a data stream with a low priority.

205. And the target hardware engine carries out splicing processing on each message group according to the coding task to obtain at least one spliced message.

206. And when the size of any spliced message in the at least one spliced message is smaller than that of the first message, the target hardware engine performs filling processing on the at least one spliced message to obtain equal-length data blocks, wherein the size of each data block in the equal-length data blocks is the size of the first message.

The target hardware engine may fill target data in any of the spliced messages to obtain a data block of the size of the first message, where the target data may be data such as 0 or 1, and after the target hardware engine completes filling of at least one spliced message, an isometric data block is obtained, and each data block in the isometric data blocks corresponds to one spliced message, and the size of each data block is equal to the size of the first message.

In one possible implementation, this step 206 may be performed by a processor when the first network device encodes a plurality of original messages by the processor.

207. And the target hardware engine performs Forward Error Correction (FEC) coding processing on the first message and the equal-length data block to obtain at least one redundant message.

208. And when the size of each spliced message in the at least one spliced message is equal to that of the first message, the target hardware engine performs FEC coding processing on the first message and the at least one spliced message to obtain at least one redundant message.

A detailed description of the steps in fig. 2 begins below.

In step 201, the first network device may perform FEC encoding on a plurality of original packets in the data stream, and perform FEC encoding on another plurality of original packets in the data stream after one FEC encoding is completed, and in a possible implementation manner, step 201 may be implemented by the following process shown in step 2011-.

Step 2011, the first network device stores the received original message in the data stream into a memory of the first network device.

For example, as shown in fig. 3, a flow chart of a method for implementing packet processing in a network device according to an embodiment of the present invention is provided, where after a network interface of a first network device receives an original packet in each data stream, the original packet is sent to a Packet Parsing Engine (PPE), and after the PPE receives the original packet, the original packet is stored in a memory, and a storage address of the original packet is recorded, so that the PPE can collect the original packet in the data stream received by the network interface. The network interface may be a Gigabit Ethernet (GE) network port or a gigabit ethernet (GXE) network port.

Step 2012, if the message amount of the uncoded original messages of the data stream in the memory is equal to or greater than the second target number, the processor obtains the original messages of the second target number from the uncoded original messages of the data stream, where the second target number is the preset number of the original messages for FEC encoding once.

As the first network device encodes the uncoded original messages with the next second target number after FEC encoding is performed on each pair of original messages with the second target number, for the data stream, when the original messages with the second target number are newly cached in the memory of the first network device, the newly cached original messages with the second target number can be used as a plurality of original messages to be encoded next time.

The processor includes at least one of an NP and a CPU, and still taking fig. 3 as an example for description, after the PPE collects each first original packet, the first original packet is stored in the memory, and a storage completion message of the first original packet is sent to the NP, where the storage completion message of the first original packet may carry a stream identifier of a first data stream to which the first original packet belongs and a storage address of the first original packet, and the stream identifier may be a name or a number of the first data stream and is used to indicate that the first original packet belongs to the first data stream. The first original message is any original message in the first data stream.

In a possible implementation manner, the first network device sets a corresponding priority for each received data stream according to a service requirement, and the first network device may preferentially process the data stream with the high priority, and accordingly, the storage completion message may also carry the priority of the data stream, so that each module in the first network device may process the original packet in the data stream according to the priority. When the first network device receives a data stream, the service type of the data stream may be determined according to data carried in an original packet in the data stream, and the priority corresponding to the service type of the data stream may be determined from the priority information table, so that the first network device may set a corresponding priority for the data stream.

Still taking fig. 3 as an example for explanation, after receiving the storage completion message of the first original packet, the NP analyzes the first original packet in the memory according to the storage address of the first original packet carried by the storage completion message, obtains whether the first original packet needs to be encoded, if the first original packet needs to be encoded, the NP generates an encoding notification message, where the encoding notification message may carry at least one of the storage address of the first original packet, the size of the first original packet, the flow identifier of the first data flow, and the priority, and sends the encoding notification message to the protocol stack; the User Datagram Protocol (UDP) proxy may collect the encoding notification messages from the protocol stack, and each time the number of the encoding notification messages newly collected by the UPD proxy is the second target number, the newly collected encoding notification messages of the second target number may indicate that the original messages to be encoded of the second target number are newly stored in the memory, so that the UDP proxy may send the encoding notification messages of the second target number to the FEC software module (the process is also a process in which the FEC software module obtains the notification messages from the NP), and the FEC software module determines the splicing scheme of the original messages of the second target number.

In some possible implementations, the first network device does not limit the number of original messages to be FEC-encoded each time, but limits the number of original messages to be encoded each time FEC-encoded, where the number of original messages to be encoded is a dimension of a matrix to be encoded when FEC-encoded is performed, that is, the first network device does not preset a second target number, but preset a first target number, for this case, the UPD agent sends, every time it receives one encoding notification message, the encoding notification message to the FEC software module, so that the FEC software module can obtain multiple encoding notification messages, and when the FEC software module can obtain the sizes of multiple original messages from multiple encoding notification messages, the FEC software module can determine the largest first message from the multiple original messages, and according to the size of the largest message (which is recorded as the target size) and the sizes of messages other than the first message in the multiple original messages, determining a splicing scheme of a third target number (that is, a process of determining a splicing scheme of original messages by the FEC software module in fig. 3), where each splicing scheme is a splicing scheme of splicing at least two original messages into one spliced message, a size of each spliced message is smaller than or equal to a target size, a sum of the third target number and a number of first messages in the multiple original messages is equal to the first target number, and a process of determining a specific splicing scheme is referred to in step 202; and the subsequent FEC software module sends a splicing scheme to the target hardware engine, and the target hardware engine splices the original messages according to the splicing scheme.

The following description is made for step 202:

in step 202, the first packet is the largest packet in the plurality of original packets, each packet group may be regarded as a concatenation scheme, and the determining process of the packet group is also a concatenation scheme determining process. In order to make the size of each packet group smaller than or equal to the size of the first packet, the size of each packet group is also the size of all original packets in each packet group. The processor may first sort the plurality of original packets according to sizes of the plurality of original packets, and then determine at least one packet group according to the sorting of the plurality of original packets and the size of the first packet. In one possible implementation, the step 202 can be implemented by the following process 2021-2025. In one possible implementation, this step may be performed by an FEC software module in the processor.

Step 2021, the processor sorts the plurality of original packets according to the size of each original packet in the plurality of original packets, to obtain an order i of each original packet, where i is an integer greater than or equal to zero, and the original packet with the order of 0 is the first packet.

When the processor sorts a plurality of original messages, the sequence of the larger original message is smaller, and the sequence of the smaller original message is larger, because the first message is the largest message in the plurality of original messages, the sequence of the first message is the smallest and 0, and the sequence of the smallest original message in the plurality of original messages is the largest. For example, fig. 4 is a schematic diagram of a packet group determining process provided by the embodiment of the present application shown in fig. 4, where fig. 4 includes a left diagram, a middle diagram, and a right diagram, where the left diagram includes an original packet sequence, where the original packet sequence includes 5 original packets, which are packets P1-P5, and packet P1 is a first packet, and the processor sorts packets P1-P5 according to sizes of packets P1-P5, so as to obtain a sort of the 5 packets shown in the middle diagram: the order of the messages P1, P4, P3, P5 and P2 is 0-4 respectively.

Step 2022, when i is equal to 1, the processor divides the original packet with the order of 1 into a 1 st target packet group, where the order of the 1 st target packet group is 1.

Because the size of the first packet is the target size, the processor does not need to group the first packets, or can directly treat each first packet as a packet group. The processor may sequentially group the original messages except the first message according to a sequence of the original messages from low to high. Still taking the middle diagram in FIG. 4 as an example, with the order of packet P4 being 1, the processor may directly partition packet P4 into the 1 st target packet group.

Step 2023, when i >1, if the sum of the size of the jth target packet and the size of the original packet whose order is i is less than or equal to the target size, the processor merges the jth target packet and the original packet whose order is i into the ith target packet, where the target size is the size of the first packet, the jth target packet is the packet with the smallest order in at least one target packet, the order of the jth target packet is j, the sum of the size of any packet in the at least one target packet and the size of the original packet whose order is i is less than or equal to the target size, and 0< j < i.

The size of a target message group, namely the sum of the sizes of all messages in the target message group, the jth target message group is a message group where an original message with the sequence j is located, the ith target message group is a message group where an original message with the sequence i is located, namely the jth target message group is obtained after j original message groups are completed, and the ith target message group is obtained after i original message groups are completed. Since the processor groups a plurality of original packets, a plurality of target packet groups may exist in the grouping process, for convenience of description, each target packet group is assigned with an order, the order of the jth target packet group is j, the order of the ith target packet group is i, that is, the order of the ith target packet group is several.

Still taking the middle diagram in fig. 4 as an example, when i is 2, the processor groups the packet P3 with the order of 2, and at this time, the 1 st target packet group only includes the packet P4, then the sum of the size of the 1 st target packet group and the size of the packet P3 with the order of 2, that is, the sum of the sizes of the packets P4 and P3, if the sum of the sizes of the packets P4 and P3 is smaller than or equal to the size of the first packet P1 (that is, the target size), the processor merges the packets P4 and P3 into the 2 nd target packet group, and since the 1 st target packet group is merged into the 2 nd target packet group, only the 2 nd target packet group currently remains.

In some possible implementations, a plurality of target packet groups may be currently left, and if the sum of the size of each packet group in at least one of the plurality of target packet groups and the size of the original packet with the order i is smaller than or equal to the target size, the processor combines the original packet with the order i and the jth target packet group into the ith target packet group, where the jth target packet group is the packet group with the smallest order in the at least one target packet group.

Step 2024, when i is greater than 1, if there is no jth target packet group, the processor adds an ith target packet group, and divides the original packet with the order i into the ith target packet group.

If the sum of the size of each of the plurality of current remaining target packet groups and the size of the original packet with the order i is larger than the target size, it indicates that the jth target packet group does not exist in the plurality of target packet groups, and the processor may additionally add one more target packet group as the ith target packet group, and divide the original packet with the order i into the ith target packet group.

Still taking the middle graph in fig. 4 as an example, when i is 3, the current remaining target packet is the 2 nd target packet composed of the packets P4 and P3, and the sizes of the packets P4, P3 and the packet P5 with the order of 3 are smaller than or equal to the target size, which indicates that the 2 nd target packet is also the jth target packet in at least one target packet group, and the packets P4, P3 and P5 may be directly merged into the 3 rd target packet group. If the sizes of the packets P4, P3, and the packet P5 with the order of 3 are larger than the target size, which indicates that the jth target packet does not exist in the remaining target packet groups, the processor adds the 3 rd target packet group, and divides the packet P5 into the newly added 3 rd target packet group, and at this time, the 2 nd target packet group and the 3 rd target packet group remain.

Step 2025, after the last original packet in the plurality of original packets is completed, the processor uses each of the remaining target packet groups as one packet group in the at least one packet group.

After the processor completes the grouping of the last original packet, the currently remaining target packet group is also the finally determined packet group, so that the processor can use each currently remaining target packet group as one packet group of the at least one packet group.

Taking the right diagram in fig. 4 as an example, the processor combines the messages P4, P3, and P5 into the 3 rd destination packet group, when i is 4, the packet P2 with the order of 4 is the last packet in the packet P1-5, the sum of the sizes of the packets P4, P3, P5 and P2 is smaller than the target size, the processor merges the 3 rd destination packet group and packet P2 into a 4 th destination packet group, leaving only the 4 th destination packet group, thus, the 4 th destination packet is the final packet for packets P2-P5, the 4 th destination packet includes packets P2-P5, the 4 th destination packet is also a concatenation scheme, since each concatenation scheme is used to indicate the concatenation of a concatenation message of the size of the first message, subsequently if the size of the concatenation message is smaller than the size of the first message, data can be filled in the spliced message, so that the size of the spliced and filled message is equal to that of the first message.

When the number of the original messages is not set as the second target number by the processor, but is set as the first target number, the processor may group the original messages cached by the first network device within the preset time at intervals of preset time, where the grouping process may be the process shown in steps 21 to 25, the target size at this time may be the size of the largest message in the original messages cached by the first network device within the preset time, or may be a preset message size, and the preset message size may be the message of the largest message in the first data stream to which the plurality of original messages belong; when the original messages cached by the first network device in the preset time are just distributed into the message groups with the third target number, and the message groups with the third target number can not contain other messages, the grouping is finished; after the original messages cached by the first network device in the preset time are just grouped into at least one message group, if the number of the at least one message group is equal to the third target number, but the at least one message group can also contain other messages, the processor can continuously collect new original messages until the at least one message group can not contain new original messages, and the grouping is finished; if the number of at least one message group is less than the third target number, the processor continuously collects new original messages, and groups the collected new original messages until the message groups with the third target number are obtained, and the grouping is finished when the message groups with the third target number can not contain the new original messages.

For facilitating understanding of the process shown in steps 2021 and 2025, refer to a flowchart of a packet provided in the embodiment of the present application shown in fig. 5, where the flowchart includes steps 501 and 509.

Step 501, the processor sets the number of the original messages subjected to one-time sequencing as a second target number M, where M is a positive integer.

Step 502, the network interface accumulatively receives M original messages.

Step 503, the processor performs descending order on the M original messages according to the sizes of the M original messages to obtain an original message sequence including the M original messages, in the original message sequence, the larger the original message sequence of the M original messages is, the smaller the original message sequence is, the larger the original message sequence is, the smaller the original message sequence is

Step 504, the processor constructs a virtual message according to the size of the largest message of the M original messages, and inserts the virtual message into a virtual message sequence, where the size of the virtual message is the size of the largest message of the M original messages, and the virtual message sequence is used for placing the virtual message.

Each virtual message can be regarded as a virtual storage space, and the virtual storage size of the virtual storage space is the size of the maximum message of the M original messages, that is, a pseudo message can be regarded as a message group.

And 505, the processor traverses the messages Pk with the order of k in the original message sequence, wherein k is more than or equal to 0 and less than M.

Step 506, the processor queries whether a virtual message capable of containing the message Pk exists in the virtual message sequence, and if so, the processor stores the message Pk in a first virtual message capable of containing the message Pk in the virtual message sequence.

Step 507, if the virtual message does not exist, the processor creates a new virtual message, inserts the new virtual message into the virtual message sequence, and places the message Pk in the new virtual message.

Step 508, after placing the message Pk in the virtual message, the processor inquires whether there is any original message that has not been traversed in the original message, if yes, the step 505 is skipped, otherwise, the step 509 is executed.

Step 509, after all the original messages in the original message sequence are traversed, the processor regards each virtual message in the virtual message sequence as a final message group.

In some possible implementations, when a plurality of original messages are grouped, the processor may group part of data in one original message into one packet group and group other data in the original message into other packet groups. In a possible implementation manner, in a primary grouping process, if a processor divides a received first original message into a first message group, where the size of the first message group is a target size, where the target size may be any preset size, if the size of one original message is smaller than or equal to the target size, the processor divides the first original message into the first message group, if the size of the first original message is larger than the target size, the processor virtually divides the first original message into at least two data blocks, where the size of a last data block of the at least two data blocks is smaller than or equal to the target size, and the sizes of other data blocks except the last data block of the at least two data blocks are equal to the target size, and the processor divides each data block into one message group; when a processor receives a new original message in the following process, if the sum of the size of the new original message and the size of the last message group in all the current message groups is smaller than or equal to the target size, dividing the new original message into the last message group; if the sum of the size of a new original message and the size of the last message group in all current message groups is larger than a target size, virtually dividing the new original message into a first target data block and at least one second target data block, wherein the size of the first target data block is the difference value between the size of the last target and the current size of the last message, the size of the last data block in the at least one second target data block is smaller than or equal to the target size, the sizes of the data blocks except the last data block in the at least one second target data block are equal to the target size, the processor divides the first target data block into the last message group, adds at least one message group, and divides each second target data block into one added message group; and when the processor finishes dividing the original messages with the second target number, or when the message group with the first target number is determined and the message group with the first target number can not contain other original messages, finishing the grouping. For example, as shown in fig. 6, which is a schematic diagram of an original message grouping process provided in this embodiment of the present application, this time, original messages 1 to 8 are grouped, the size of each message group is fixed to a target size, a processor divides the original messages 1 to 2 and front part messages of the original message 4 into the message group 1, divides rear part messages of the original message 4, original messages 5 and front part messages of the original message 6 into the message group 2, divides rear part messages of the original message 6 and the original messages 7 to 8 into the message group 2, the size of the message group 3 is smaller than the target size, and subsequently, data may be filled in concatenated messages spliced from the messages in the messages 3, so that the size of the filled messages is the target size.

In a possible implementation, when the first network device directly performs FEC encoding on a plurality of original packets through the processor, the processor directly performs the following step 205, that is, performs concatenation processing on each packet group to obtain at least one concatenated packet, where each concatenated packet corresponds to one packet group. Without interacting with the target hardware engine, if the first network device FEC-encodes the multiple original packets through the target hardware engine, the processor performs step 203 described below.

The following is described for step 203:

in step 203, the encoding task is configured to instruct to splice the original packets in each packet group into one spliced packet, and encode a plurality of spliced packets, where address information of each packet group includes storage addresses of all the original packets in one packet group, and in some possible implementations, the address information of each packet group further includes a splicing identifier, for example, a schematic diagram of an encoding task provided in this embodiment shown in fig. 7, and the encoding task in fig. 7 includes a plurality of address information, and each address information includes one splicing identifier and a storage address of each original packet. The concatenation identifier may be used to indicate that all original messages in a message group are concatenated into a concatenated message, and the concatenation identifier may also be used to indicate that all storage addresses in the address information of a message group are in a group.

The address information of each packet group may be represented in a set form, for example, the packet group 2 includes

original packets

3 and 4, and if the set 1 includes the storage addresses of the

original packets

3 and 4 and the concatenation identifier, the set 1 is the address information of the packet group 2. In some possible implementations, the address information of each packet group may be represented in the form of a character string, still taking packet group 2 as an example, character string 1 is the "storage address of original packet 3; the storage address of the original message 4; the concatenation identification ", then the character string 1 is also the address information of the packet group 2. In some possible implementations, the address information of each packet group may be represented in a form of a table, and still taking packet group 2 as an example, see table 1 below, where the storage addresses of

original packets

3 and 4 in table 1 both correspond to the same concatenation identifier, and table 1 is also the address information of packet group 2.

TABLE 1

The encoding task may also carry a storage address of the first packet, so that a subsequent target hardware engine may obtain the first packet according to the storage address of the first packet. The encoding task may further carry at least one of an encoding parameter, an identifier of a data stream, and a priority identifier of a first data stream, where the first data stream is a data stream in which the plurality of original packets are located, and the encoding parameter may include a packet amount to be encoded (that is, a first target number), a redundant packet amount, and a target size, where the packet amount to be encoded is a sum of the number of the first packets in the plurality of original packets and the number of at least one packet group, so that a subsequent target hardware engine may encode the plurality of original packets according to the encoding parameter.

The following is described for step 205:

in step 205, the target hardware engine may obtain at least one packet group from the memory according to the address information in the encoding task, where an original packet in each packet group is a packet in a storage address in one address information; the target hardware engine splices the original messages in each message group into a spliced message, so that the target hardware engine can obtain at least one spliced message.

When a first network device encodes a plurality of original messages through a processor, if the sum of the number of at least one message group and the number of first messages in the plurality of original messages reaches a first target number, and the last message group in the at least one message group can not add other original messages except the plurality of original messages, the processor performs splicing processing on each message group, wherein the first target number is the dimension of a matrix to be encoded during one-time FEC encoding.

Or, when the number of the plurality of original messages is equal to the second target number and the original messages of the second target number are divided into at least one message group, the processor performs splicing processing on each message group, where the second target number is the preset number of the original messages used for performing FEC encoding once. The process of splicing each packet group by the processor is the same as the process of splicing each packet group by the target hardware engine, and here, the process of splicing each packet group by the processor is not described in detail in the embodiments of the present application.

The following is described for step 207:

in step 207, the target hardware engine may obtain the encoding parameters from the encoding task, and set a to-be-encoded packet amount in the encoding parameters as Q, a redundant packet amount as R, and a target size as L, where Q and R are positive integers, L is a numerical value greater than 0, the redundant packet amount R is the number of redundant packets obtained after FEC encoding Q to-be-encoded packets, and Q to-be-encoded original packets include a first packet and an equal-length data block in a plurality of original packets. The target hardware engine may obtain the first packet from the memory according to the storage address of the first packet carried by the encoding task, and use the obtained first packet and the equal-length data block obtained in step 206 as the packet to be encoded. The target hardware engine can firstly form Q messages to be coded into a Q x L matrix to be coded, and each row of the matrix to be coded is an equal-length data block or a first message; the target hardware engine constructs a generating matrix of (Q + R) Q according to the message quantity Q to be coded and the redundant message quantity R, wherein the generating matrix is composed of a first sub-matrix and a second sub-matrix, the first sub-matrix is a unit matrix of Q and Q, and the second sub-matrix is a Cauchy matrix of R and Q.

Wherein, the element of the ith row and the jth column in the second sub-matrix is

x_i-1With y_i-1And both are Galois Field (GF) (2)^w) Wherein i and j are each an integer greater than or equal to 0, and w may be 8; and the target hardware engine performs multiplication calculation on the generated matrix and the matrix to be encoded to obtain an (Q + R) × L encoding matrix, wherein the encoding matrix comprises the matrix to be encoded and an R × L check matrix, and each row of the check matrix is a redundant message.

In a possible implementation, the process shown in step 205-. This step 207 may be performed by the processor when the first network device encodes a plurality of raw messages by the processor.

The following is described for step 208:

in step 208, when the size of each of the at least one concatenation message is equal to the size of the first message, the at least one message may be regarded as an equal-length data block, and therefore, the target hardware engine may directly perform this step 208. In one possible implementation, when the first network device encodes multiple original messages through the processor, this step 207 may be performed by the processor.

When the target hardware engine acquires the at least one redundant message, storing the at least one redundant message to a memory, and sending a coding completion message to the processor, where the coding completion message may carry a storage address of the at least one redundant message, so that the processor may acquire the at least one redundant message from the memory. Or, the target engine directly sends the at least one redundant message to the processor, and the implementation of the present application does not specifically limit the manner in which the processor obtains the at least one redundant message.

The processor may obtain an original packet in each packet group from the memory, perform concatenation processing on each packet group to obtain at least one concatenated packet, perform padding processing on the at least one concatenated packet to obtain equal-length data blocks when the size of any concatenated packet in the at least one concatenated packet is smaller than the size of the first packet, and obtain the first packet in the plurality of original packets from the memory. Of course, the target hardware engine may also directly send the equal-length data block and the first packet to the processor, and the embodiment of the present application does not specifically limit the manner in which the equal-length data block and the first packet are obtained by the processor.

After the processor obtains the equal-length data block, the first message and at least one redundant message, adding an FEC message header to each data block in the equal-length data block and the first message to obtain a plurality of original FEC messages, and adding an FEC message header to each redundant message to obtain at least one redundant FEC message, wherein the FEC message header carries coding parameters. For example, an FEC message in the schematic diagram of an FEC message provided in the embodiment of the present application shown in fig. 8 includes the FEC message and a load message, where when the load message is one of a first message and an equal-length data block, the FEC message is an original FEC message, and when the load message of the FEC message is a redundant message, the FEC message is a redundant FEC message.

The FEC header is used to indicate the encoding condition of the multiple original packets. The FEC header carries a plurality of encoding parameters of the original packet, for example, the FEC header in fig. 8 carries the amount of the packet to be encoded, the amount of the redundant packet, and the size of the target. The FEC packet header may also carry a target identifier of the multiple original packets, where the target identifier is used to indicate the number of times that the data streams to which the multiple original packets belong are encoded. The FEC packet header may also carry sequence numbers of the multiple original packets in the data stream (that is, original sequence numbers of original packets before concatenation), sizes of the original packets, sequence numbers of each first packet and each data block in the equal-length data blocks in a to-be-encoded matrix, sequence numbers of each redundant packet in the encoded matrix, and a target concatenation identifier corresponding to each data block, where the target concatenation identifier is used to indicate that a data block is a concatenated data block.

After the processor obtains the original FEC message and the redundant FEC message, the processor may send the original FEC message and the redundant FEC message to the second network device. Of course, in some possible implementations, the target hardware engine may directly encapsulate the first packet and the equal-length data block into an original FEC packet, encapsulate the redundant packet into a redundant FEC packet, and send the original FEC packet and the redundant FEC packet to the second network device.

Referring to fig. 9, an embodiment of the present application provides a schematic flow diagram of internal interaction of a network device, where the flow specifically includes

steps

901 and 907. Fig. 9 illustrates the switching of each internal module when the first network device FEC-encodes multiple original packets.

Step 901, the network interface receives 1 original message from the network, and sends the original message to the PPE.

Step 902, PPE writes the received original message into memory.

Step 903, PPE sends the storage completion message of the original packet to NP to notify NP to process the original packet in the memory.

Step 904, after receiving the storage complete message, the NP parses the original message to determine whether the original message needs to be encoded, and if so, the NP sends an encoding notification message of the original message to the CPU.

The CPU may send the encoding completion message to the FEC software module through the protocol stack and the UDP proxy, and the specific process is described in step 2012 above, which is not described herein again.

Step 905, after the FEC software module in the CPU collects the coding notification messages of the second target number, it is equivalent to that the FEC software module collects all the original messages of the second target number that need to be coded, and the FEC software module sends a coding task of the original messages of the second target number to the target hardware engine based on the original messages of the second target number.

The encoding task corresponds to a notification message that informs the target hardware processor to perform FEC encoding based on the encoding task.

Step 906, the target hardware engine obtains the second target number of original messages from the memory according to the coding task, and performs FEC coding on the second target number of original messages to obtain at least one redundant message.

The process shown in step 906 is the same as the process shown in steps 205-207, and here, the process shown in step 906 is not described in detail in this embodiment of the application.

Step 907, the target hardware engine writes the at least one redundant message into the memory, and sends the storage address of the at least one redundant message to the FEC software module, so that the FEC software module obtains the at least one redundant message, and sends the original message with the second target number and the at least one redundant message to the second network device.

In a possible implementation manner, the FEC software module may be deployed in an NP or may also be deployed in a CPU, and the deployment manner of the FEC software module is not specifically limited in this embodiment.

According to the method provided by the embodiment of the application, the splicing processing is carried out on other original messages except the largest first message in the plurality of original messages, the spliced messages are filled only when the size of the spliced messages is smaller than that of the largest message, and each other original messages do not need to be filled, so that the data filled in the isometric data blocks after filling is less, the network bandwidth occupied during transmission of the FEC coding messages is reduced, and the waste of network resources is avoided.

When the second network device receives the FEC message sent by the first network device, if the original FEC message is lost in the transmission process, the second network device decodes the FEC message that is not lost, so as to recover the lost message. To further explain the encoding and decoding processes of multiple original messages, refer to fig. 10, which is a flowchart of encoding and decoding provided in the embodiment of the present application, and the flowchart includes the following steps 1001 and 1006.

Step 1001, after the FEC software module in the first network device collects the original messages with the second target number, determining the first message and at least one message group in the original messages with the second target number according to the original messages with the second target number, and sending an encoding task to the target hardware engine.

Before step 1001, the UDP proxy collects an original packet in a data stream sent by the first terminal, and sends a storage address of the collected original packet and a size of the original packet to the FEC software module, and when the FEC software module receives the storage address and the size of the second target number of original packets, the step 1001 is executed. Taking the second target number as 6 as an example, after the FEC software module collects the storage addresses and sizes of the original messages 1 to 6 from the UDP proxy, it is described that the FEC software module collects 6 original messages, the FEC software module uses the

original messages

1 and 6 as first messages respectively, divides the

original messages

2 and 4 into the message group 1, and divides the

original messages

3 and 5 into the message group 2. The FEC software module may further splice the

original messages

2 and 4 into a spliced message 1, the FEC software module may further splice the

original messages

3 and 5 into a spliced message 2, and when the size of any spliced message is smaller than the size of the original message 3, perform padding processing on any spliced message to obtain a padded message, otherwise, perform no padding processing on any spliced message. In fig. 10, for example, the size of any spliced message is the size of the original message 3, the FEC software module takes the

original messages

1 and 6 and the spliced messages 1-2 as a matrix to be encoded, in fig. 10, the original message is marked as message 1 in the matrix to be encoded, the spliced messages 1-2 are respectively marked as messages 2-3 in the matrix to be encoded, and the original message 6 is marked as message 4 in the matrix to be encoded.

Step 1002, the target hardware engine acquires

original messages

1 and 6 and a to-be-encoded matrix formed by splicing messages 1-2 according to an address set in the encoding task, constructs a corresponding generating matrix according to encoding parameters in the encoding task, performs FEC encoding according to the to-be-encoded matrix and the generating matrix to obtain redundant messages a, b and c, stores the redundant messages a, b and c in a memory, and sends storage addresses of the redundant messages a, b and c to an FEC software module.

Step 1003, the FEC software module obtains the redundant messages a, b, and c from the memory according to the storage addresses of the redundant messages a, b, and c, and adds an FEC message header to each of the messages 1 to 4 and the redundant messages a to c to obtain a plurality of FEC messages, and sends the encoded stream composed of the FEC messages to the UDP proxy, and the UDP proxy sends the encoded stream to the second network device through the WAN.

Step 1004, the UDP proxy of the second network device receives the encoded stream, where the

messages

1 and 3 in the encoded stream are lost, and when the FEC software module of the second network device collects the

messages

2 and 4 and any 2 redundant messages of the redundant messages a, b, and c, the FEC software module issues a decoding task to the target hardware engine in the second network device, where the decoding task carries the storage addresses and decoding parameters of the

messages

2 and 4 and any 2 redundant messages of the redundant messages a, b, and c, and the decoding parameters include the number of messages with lost encoding parameters, the position of the lost messages in the encoding matrix, and the target size.

Step 1005, the target hardware engine acquires the

messages

2 and 4 and any 2 redundant messages of the redundant messages a, b and c from the memory according to the storage address in the decoding task, forms a matrix to be decoded by the

messages

2 and 4 and any 2 redundant messages of the redundant messages a, b and c, constructs a corresponding decoding matrix according to the decoding parameters, performs FEC decoding by the target hardware engine based on the matrix to be decoded and the decoding matrix to obtain the lost

messages

1 and 3, stores the lost

messages

1 and 3 in the memory, and sends the storage addresses of the

messages

1 and 3 to the FEC software module.

Step 1006, the FEC software module obtains the storage addresses of the original message 1 and the concatenated message 1 from the memory according to the storage addresses of the original message 1 and the concatenated message 1, obtains the original message 2 and the concatenated message 2, splits the

original messages

2 and 3 from the concatenated message 1, splits the

original messages

3 and 5 from the concatenated message 2, restores the sequence of the original messages 1-6 in the data stream, and sends the original messages 1-6 to the UDP proxy according to the sequence to form a data stream, and the UDP proxy sends the data stream to the second terminal.

Fig. 11 is a schematic structural diagram of a message processing apparatus according to an embodiment of the present application, where the apparatus 1100 includes:

a processor 1101 for executing the step 201;

the processor 1101 is further configured to splice, according to a size of a first packet in the multiple original packets, other original packets except the first packet in the multiple original packets to obtain at least one spliced packet, where the first packet is a packet with a largest data amount in the multiple original packets;

the processor 1101 is further configured to, when the size of any one of the at least one concatenated packet is smaller than the size of the first packet, perform padding processing on the at least one concatenated packet to obtain equal-length data blocks, where the size of each data block in the equal-length data blocks is the size of the first packet;

the processor 1101 is further configured to perform forward error correction FEC coding processing on the first packet and the equal-length data block to obtain at least one redundant packet.

Optionally, the apparatus further comprises a memory for storing program instructions. The processor 1101 may be a CPU, configured to execute instructions stored in the memory, so as to implement the message processing method shown in fig. 2.

In one possible implementation, the processor 1101 is configured to:

the above step 202 is executed;

In a possible implementation manner, the message processing apparatus further includes a target hardware engine 1102;

the processor 1101 is further configured to execute the step 202 and the step 203;

the target hardware engine 1102 is configured to perform the step 205.

In one possible implementation, the processor 1101 is configured to:

In one possible implementation, the processor 1101 is further configured to:

and when the size of each spliced message in the at least one spliced message is equal to that of the first message, performing Forward Error Correction (FEC) coding processing on the first message and the at least one spliced message to obtain at least one redundant message.

In a possible implementation, the target hardware engine 1101 is further configured to perform the step 208.

All the above optional technical solutions may be combined arbitrarily to form the optional embodiments of the present disclosure, and are not described herein again.

In the message processing apparatus provided in the foregoing embodiment, when a message is processed, only the division of the functional modules is illustrated, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the embodiments of the message processing method provided in the foregoing embodiments belong to the same concept, and specific implementation processes thereof are described in detail in the embodiments of the method, which is not described herein again.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The above description is only for the purpose of illustrating the preferred embodiments of the present application and is not to be construed as limiting the present application, and any modifications, equivalents, improvements and the like that are made within the principles of the present application are intended to be included within the scope of the present application.

Claims

1. A message processing method is characterized by comprising the following steps:

acquiring a plurality of original messages;

2. The method according to claim 1, wherein the splicing the original messages except for the first message in the original messages according to the size of the first message in the original messages to obtain at least one spliced message comprises:

3. The method according to claim 1, wherein the splicing the packets, except for the first packet, in the plurality of original packets according to the size of the first packet in the plurality of original packets to obtain at least one spliced packet comprises:

4. The method according to claim 1, wherein the splicing each packet group to obtain the at least one spliced packet comprises:

5. The method of claim 1, further comprising:

6. A message processing apparatus, the apparatus comprising:

the processor is used for acquiring a plurality of original messages;

the processor is further configured to splice, according to a size of a first packet in the multiple original packets, other original packets except the first packet in the multiple original packets to obtain at least one spliced packet, where the first packet is a packet with a largest data amount in the multiple original packets;

the processor is further configured to perform padding processing on the at least one concatenated packet to obtain equal-length data blocks when the size of any concatenated packet in the at least one concatenated packet is smaller than the size of the first packet, where the size of each data block in the equal-length data blocks is the size of the first packet;

the processor is further configured to perform forward error correction FEC coding processing on the first packet and the equal-length data block to obtain at least one redundant packet.

7. The apparatus of claim 6, wherein the processor is configured to:

8. The apparatus of claim 7, wherein the apparatus further comprises a target hardware engine;

the processor is further configured to group, according to a size of a first packet in the multiple original packets, other original packets except the first packet in the multiple original packets to obtain at least one packet group, and generate an encoding task according to the at least one packet group, where the encoding task includes address information of each packet group;

and the target hardware engine is used for splicing each message group according to the coding task to obtain at least one spliced message.

9. The apparatus of claim 7, wherein the processor is further configured to:

10. The apparatus of claim 8, wherein the processor is further configured to:

11. A computer-readable storage medium having stored thereon instructions which are loaded and executed by a processor to implement the message processing method according to any one of claims 1 to 5.