CN111404872A

CN111404872A - Message processing method, device and system

Info

Publication number: CN111404872A
Application number: CN201911149626.8A
Authority: CN
Inventors: 何焕
Original assignee: Hangzhou Hikvision System Technology Co Ltd
Current assignee: Hangzhou Hikvision System Technology Co Ltd
Priority date: 2019-11-21
Filing date: 2019-11-21
Publication date: 2020-07-10

Abstract

The embodiment of the invention provides a message processing method, a device and a system, wherein the method of a sending end comprises the following steps: packaging data by using a preset DPDK-based message packaging format to obtain a message to be sent; adding a first identifier and a second identifier to a message to be sent, and generating data of the message to be sent; adding data of a message to be sent to a data queue, sequentially sending the message to be sent in each data to a receiving end realized based on a DPDK packaging library according to the sequence of each data in the data queue, and deleting the message to be sent which is successfully sent from the data queue; and judging whether to resend the message to be sent based on the first identifier and the second identifier contained in the data of the message to be sent aiming at the message to be sent which is sent and is not sent successfully. The embodiment of the invention can utilize the DPDK technology to reduce the message transmission delay on one hand, and improve the probability of successful message reception at the receiving end on the other hand.

Description

Message processing method, device and system

Technical Field

The present invention relates to the field of network communication technologies, and in particular, to a method, an apparatus, and a system for processing a packet.

Background

TCP (Transmission Control Protocol) and UDP (User datagram Protocol) are common transport protocols, and the above transport protocols are generally used for data interaction between local area networks.

When a TCP or UDP protocol is used to process a packet, for example, when a sending end sends a packet, if a data packet of the packet is a small data packet, that is, a data packet with a data amount smaller than a threshold (usually, a data packet with a single packet size smaller than 1500 bytes), the small data packet is buffered in a sending buffer, and the large data packet is sent after being merged with other small data packets smaller than the threshold into a large data packet.

Disclosure of Invention

The embodiment of the invention aims to provide a message processing method, a message processing device and a message processing system, so as to further reduce message transmission delay. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present invention provides a packet processing method, which is applied to a transmitting end implemented based on a data plane development kit DPDK package library, and the method includes:

packaging data by using a preset DPDK-based message packaging format to obtain a message to be sent, wherein the message to be sent carries a number for uniquely identifying the message to be sent, and the numbers of different messages to be sent are sequentially increased according to the packaging sequence;

adding a first identifier and a second identifier to the message to be sent, and generating data of the message to be sent, wherein the first identifier is used for identifying the last sending time of the message to be sent, and the second identifier is used for identifying the retransmission times of the message to be sent;

adding the data of the message to be sent to a data queue;

according to the sequence of each data in the data queue, sequentially sending messages to be sent in each data to a receiving end realized based on a DPDK packaging library so that the receiving end receives each message to be sent, and determining the sequence of each message to be sent based on the number;

deleting the data of the successfully transmitted message to be transmitted from the data queue;

and judging whether to resend the message to be sent based on the first identifier and the second identifier contained in the data of the message to be sent aiming at the message to be sent which is sent and is not sent successfully.

Optionally, the step of determining, for a message to be sent that has been sent and has not yet been sent successfully, whether to resend the message to be sent based on a first identifier and a second identifier included in data of the message to be sent includes:

acquiring the current time, and judging whether the difference between the current time and the last sending time corresponding to a first identifier in the data to be sent is smaller than a first preset time threshold value or not;

if the time is less than the first preset time threshold, determining not to resend the message to be sent;

if the retransmission time is greater than or equal to the first preset time threshold and the retransmission times corresponding to the second identifier of the data are less than the preset time threshold, retransmitting the message to be sent;

recording the resending time in the first identifier of the data corresponding to the message to be sent;

and adding one to the retransmission times corresponding to the second identifier of the data corresponding to the message to be sent.

Optionally, before determining whether to resend the message to be sent based on the first identifier and the second identifier included in the data of the message to be sent, for the message to be sent that has been sent and has not yet been sent successfully, the method further includes:

and if a response message which is returned by a receiving end and aims at the sent message to be sent is not received within a preset time length, determining that the message to be sent is not sent successfully, wherein the response message is used for indicating that the message to be sent is received successfully.

Optionally, the method further includes:

when the retransmission times corresponding to the second identification of the data are larger than or equal to a preset time threshold value, removing the data from the data queue;

and recording the data into a log file.

Optionally, the preset DPDK-based packet encapsulation format includes: the method comprises the steps of destination machine number, source machine number, message type identification, real data and check sum identification.

In a second aspect, an embodiment of the present invention further provides a packet processing method, which is applied to a receiving end implemented based on a data plane development kit DPDK package library, and the method includes:

receiving a message to be sent by a sending end, wherein the message to be sent is obtained by the sending end by packaging data by using a preset message packaging format based on a data plane development kit DPDK, the message to be sent carries a number for uniquely identifying the message, and the numbers of different messages to be sent are sequentially increased according to the packaging sequence;

and determining the sequence of each message to be sent according to the received serial number in each message to be sent.

Optionally, after determining the sequence of each message to be sent according to the number in each received message to be sent, the method further includes:

caching the messages to be sent into a cache queue according to the determined sequence;

if the message to be sent exists in the cache queue, the current time and the message receiving time of which are smaller than a second preset time threshold, the message to be sent and all messages in the cache queue before the message to be sent are moved from the cache queue to a receiving queue, wherein the second preset time threshold is larger than the product of the first preset time threshold and a preset time threshold.

In a third aspect, an embodiment of the present invention provides a packet processing apparatus, applied to a transmitting end implemented based on a data plane development kit DPDK package library, where the apparatus includes:

the encapsulation module is used for encapsulating data by utilizing a preset DPDK-based message encapsulation format to obtain a message to be sent, wherein the message to be sent carries a number for uniquely identifying the message to be sent, and the numbers of different messages to be sent are sequentially increased according to the encapsulation sequence;

an adding module, configured to add a first identifier and a second identifier to the message to be sent, and generate data of the message to be sent, where the first identifier is used to identify a last sending time of the message to be sent, and the second identifier is used to identify a number of times of retransmission of the message to be sent;

the adding module is used for adding the data of the message to be sent to a data queue;

a sending module, configured to send, to a receiving end implemented based on a DPDK package library, messages to be sent in each data in sequence according to a sequence of each data in the data queue, so that the receiving end receives each message to be sent, and determines the sequence of each message to be sent based on the number;

the deleting module is used for deleting the data of the message to be sent which is successfully sent from the data queue;

and the judging module is used for judging whether to resend the message to be sent based on the first identifier and the second identifier contained in the data of the message to be sent aiming at the message to be sent which is sent and still not sent successfully.

Optionally, the determining module includes:

the judgment submodule is used for acquiring the current time and judging whether the difference between the current time and the last sending time corresponding to the first identifier in the data to be sent is smaller than a first preset time threshold value;

the determining submodule is used for determining not to resend the message to be sent if the time is less than a first preset time threshold;

the retransmission submodule is used for retransmitting the message to be sent if the retransmission time is greater than or equal to a first preset time threshold and the retransmission times corresponding to the second identifier of the data are less than a preset time threshold;

the recording submodule is used for recording the resending time in the first identifier of the data corresponding to the message to be sent;

and the counting submodule is used for adding one to the retransmission times corresponding to the second identifier of the data corresponding to the message to be sent.

Optionally, the apparatus further comprises:

the first determining module is configured to determine that the message to be sent is not sent successfully if a response message, which is returned by the receiving end and is specific to the sent message to be sent, is not received within a preset time period, where the response message is used to indicate that the message to be sent is received successfully.

Optionally, the apparatus further comprises:

a removing module, configured to remove the data from the data queue when a retransmission number corresponding to the second identifier of the data is greater than or equal to a preset number threshold;

and the recording module is used for recording the data into a log file.

Optionally, the preset DPDK-based packet encapsulation format is: the method comprises the steps of occupying a destination machine number of 2 bytes, a source machine number of 2 bytes, a message number of 3 bytes, real data of 54-1490 bytes and a checksum identification of 2 bytes.

In a fourth aspect, an embodiment of the present invention further provides a packet processing apparatus, which is applied to a receiving end implemented based on a DPDK package library of a data plane development kit, where the apparatus includes:

the system comprises a receiving module, a sending end and a sending module, wherein the receiving module is used for receiving a message to be sent by the sending end, the message to be sent is obtained by the sending end by packaging data by using a preset message packaging format based on a data plane development kit DPDK, the message to be sent carries a number for uniquely identifying the message, and the numbers of different messages to be sent are sequentially increased according to the packaging sequence;

and the second determining module is used for determining the sequence of each message to be sent according to the number of each received message to be sent.

Optionally, the apparatus further comprises:

the buffer module is used for buffering the messages to be sent into a buffer queue according to the determined sequence;

and the moving module is used for moving the message to be sent and all messages in the cache queue before the message to be sent from the cache queue to a receiving queue if the message to be sent exists in the cache queue, wherein the current time and the message receiving time are smaller than a second preset time threshold, and the second preset time threshold is larger than the product of the first preset time threshold and a preset time threshold.

In a fifth aspect, an embodiment of the present invention provides a message processing system, including: a transmitting end and a receiving end realized based on a DPDK package library of a data plane development kit;

the sending end is used for packaging data by using a preset DPDK-based message packaging format to obtain a message to be sent; adding a first identifier and a second identifier to the message to be sent, and generating data of the message to be sent; adding the data of the message to be sent to a data queue; according to the sequence of each data in the data queue, sequentially sending messages to be sent in each data to a receiving end realized based on a DPDK packaging library so that the receiving end receives each message to be sent, and determining the sequence of each message to be sent based on the number; deleting the data of the successfully transmitted message to be transmitted from the data queue; judging whether to resend the message to be sent or not according to the message to be sent which is sent and still not sent successfully based on a first identifier and a second identifier contained in data of the message to be sent; the message to be sent carries a number for uniquely identifying the message, the numbers of different messages to be sent are sequentially increased according to the packaging sequence, the first identification is used for identifying the last sending time of the message to be sent, and the second identification is used for identifying the retransmission times of the message to be sent;

the receiving end is used for receiving a message to be sent by the sending end; and determining the sequence of each message to be sent according to the received serial number in each message to be sent.

In a sixth aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete mutual communication through the communication bus; the machine-readable storage medium stores machine-executable instructions executable by the processor, the processor being caused by the machine-executable instructions to: the method steps of the message processing method provided by the first aspect of the embodiment of the present invention are implemented.

In a seventh aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete mutual communication through the communication bus; the machine-readable storage medium stores machine-executable instructions executable by the processor, the processor being caused by the machine-executable instructions to: the method steps of the message processing method provided by the second aspect of the embodiment of the present invention are implemented.

In an eighth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and the computer program is executed by a processor to perform the method steps of the message processing method provided in the first aspect of the embodiment of the present invention.

In a ninth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and the computer program is executed by a processor to perform the method steps of the message processing method provided in the first aspect of the embodiment of the present invention.

When the message processing method, the device, the electronic device, and the storage medium provided in the embodiments of the present invention are applied to a sending end, after a preset packet encapsulation format based on DPDK is used to encapsulate data to obtain a packet to be sent, a first identifier for identifying a last sending time of the packet to be sent and a second identifier for identifying a sending frequency of the packet to be sent are added to the packet to obtain data corresponding to the packet and add the data to a data queue, so that after the packet to be sent corresponding to the data is sent, whether the packet to be sent corresponding to the data needs to be resent is determined according to the first identifier and the second identifier of the data, and the packet to be sent corresponding to the data needs to be resent when needed.

When the method is applied to the receiving end, after the receiving end receives the message to be sent, the receiving end can determine the sequence of each message to be sent according to the number in the message to be sent, so that the message received by the receiving end is prevented from being sent out in a disorder manner, and the possibility of errors in message receiving can be reduced.

Of course, it is not necessary for any product or method of practicing the invention to achieve all of the above-described advantages at the same time.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic flow chart of a message processing method according to an embodiment of the present invention;

fig. 2 is a schematic flowchart of step S106 in the message processing method according to the embodiment of the present invention;

fig. 3 is a schematic diagram of a message format according to an embodiment of the present invention;

fig. 4 is another schematic flow chart of the message processing method according to the embodiment of the present invention;

fig. 5 is a schematic diagram of a third flow of a message processing method according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram of a message processing apparatus according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of a determining module in a message processing apparatus according to an embodiment of the present invention;

fig. 8 is another schematic structural diagram of a message processing apparatus according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of a third message processing apparatus according to an embodiment of the present invention;

fig. 10 is a schematic diagram of a fourth structure of a message processing apparatus according to an embodiment of the present invention;

fig. 11 is a schematic structural diagram of a fifth message processing apparatus according to an embodiment of the present invention;

fig. 12 is a schematic structural diagram of a message processing system according to an embodiment of the present invention;

fig. 13 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;

fig. 14 is another schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

At present, data exchange between local area networks and inside large network systems usually employs TCP or UDP as a transport layer protocol for data exchange. When small data packets are transmitted, the transmission rate of TCP and UDP is far lower than the maximum transmission rate of a network card, and the main reason is that the transmission of TCP and UDP needs the following information besides data: ethernet header (14 bytes for recording source MAC, destination MAC, protocol type, etc.) IP header (20 bytes), TCP header (20 bytes), and UDP header (8 bytes) for example, the total size of a packet is 100 bytes, and the actual data size transmitted using TCP is: 100 bytes-14 bytes-20 bytes, i.e. 46 bytes, the real data percentage is 46%, and the smaller the data packet, the lower the real data percentage. In practical application, in TCP and UDP transmission, small data packets are buffered in a sending buffer, combined with other small packets to form a large packet sharing a header, and then sent, but actually, the small data packets are not transmitted in real time, and have partial delay, the performance is still far lower than the maximum transmission rate of a ten-gigabit network card, and the transmission rate of 100-byte data packets is still lower than 200 MB/s.

In view of this, as shown in fig. 1, an embodiment of the present invention provides a message processing method, which is applied to a sending end implemented based on a DPDK (Data Plane Development Kit), where the process may include the following steps:

s101, data is packaged by using a preset DPDK-based message packaging format, and a message to be sent is obtained.

The embodiment of the invention can be applied to a sending end for realizing a DPDK packaging library, namely, a network card driver of the sending end can use a DPDK user mode network card driver instead of a system kernel network card driver, the DPDK is developed by multiple companies such as 6WIND, Intel and the like, is mainly based on L inux system operation, is used for a function library and a driver set for fast data packet processing, can greatly improve the data processing performance and throughput, and is commonly used in the message transmission process in a local area network.

After obtaining data to be sent, the embodiment of the present invention may package the data using a preset packet package format based on DPDK, so as to obtain a packet to be sent, and the packet to be sent may carry a number for uniquely identifying the packet to be sent, where the numbers of different packets to be sent are sequentially increased according to a packaging sequence, for example, the number of a packet to be sent that is packaged first is 01, the number of a packet to be sent that is packaged later is 02, and so on.

It can be understood that, before sending a message to be sent, a sending end needs to perform an initialization process of a DPDK, which includes the following steps: initializing a DPDK environment, mainly responsible for detecting which lcores (command line parameters of the DPDK) of a DPDK program are available, analyzing the command line parameters, initializing each sub-module and the like; initializing a DPDK memory pool, namely applying for a memory in a kernel, and mapping the memory in the kernel to a user mode for storing data in a sending and receiving buffer area; initializing a data queue and a receiving queue, a sending port and a receiving port, setting the number of the data queue and the receiving queue, the size of an annular buffer zone, an ID (identity) of a network card port and the like, then starting a network card device, and setting the network card to be in a hybrid mode so that the network card can receive all data streams passing through the network card regardless of whether a destination address of the network card is the same or not; and creating a sending thread for circularly sending the message to be sent in the data queue, and creating a receiving thread for receiving message confirmation, namely response message, returned by the receiving end.

S102, adding a first identifier and a second identifier for the message to be sent, and generating data of the message to be sent.

In this embodiment of the present invention, a first identifier and a second identifier may be added to a message to be sent, and then data of the message to be sent is generated, where the first identifier is used to identify a last sending time of the message to be sent, an initial value is set to 0 when the message is sent for the first time, the second identifier is used to identify a retransmission frequency of the message to be sent, and the initial value may also be set to 0, that is, the generated data may include: the message to be sent, and the first identifier and the second identifier corresponding to the message to be sent.

S103, adding the data of the message to be sent to a data queue.

Before sending a message to be sent, the embodiment of the present invention may add data corresponding to the message to be sent to a data queue, where as shown in table 1, a plurality of pieces of data are recorded in the data queue, and each piece of data corresponds to one message to be sent.

TABLE 1 contents of records in data queue

Number of data strips	Information contained in each piece of data
		The first piece of data	{ message to be sent + first identifier + second identifier }
Second piece of data	{ message to be sent + first identifier + second identifier }
		The third piece of data	{ message to be sent + first identifier + second identifier }
……	……
		Fourth data	{ message to be sent + first identifier + second identifier }

And S104, sequentially sending the message to be sent in each data to a receiving end realized based on the DPDK packaging library according to the sequence of each data in the data queue.

The embodiment of the invention can adopt a circulating transmission mode to transmit each message to be transmitted in the data queue, and can add the message which is not successfully transmitted into the data queue again to be transmitted, so that the receiving end receives each message to be transmitted, and determines the sequence of each message to be transmitted based on the number. Of course, the receiving end may also be implemented based on the DPDK package library.

And S105, deleting the data of the successfully transmitted message to be transmitted from the data queue.

When the message to be sent is successfully sent, the successfully sent data can be deleted from the data queue. In the data queue according to the embodiment of the present invention, a plurality of pieces of data may be sequentially arranged, and each piece of data includes the first identifier and the second identifier of each message to be sent.

And S106, for the message to be sent which has been sent and has not yet been sent successfully, judging whether to resend the message to be sent based on the first identifier and the second identifier included in the data of the message to be sent.

It can be understood that some messages to be sent may not be sent successfully once, and for those messages to be sent that have been sent and have not yet been sent successfully, the embodiment of the present invention may determine whether to resend the message to be sent based on the first identifier and the second identifier included in the data of the message to be sent, for example, if the first identifier and/or the second identifier in the data of the message to be sent satisfy a preset condition, the sending end determines that the message to be sent is sent successfully, and if the first identifier and/or the second identifier in the data of the message to be sent determines that the message to be sent is sent unsuccessfully, the message to be sent needs to be resent, so as to improve the probability that the message. The specific judgment process can be seen in the following examples.

As an optional implementation manner of the embodiment of the present invention, as shown in fig. 2, the step S106 may include:

s1061, obtaining a current time, and determining whether a difference between the current time and a last sending time corresponding to the first identifier in the data corresponding to the message to be sent is smaller than a first preset time threshold.

The embodiment of the present invention may cyclically acquire data in a data queue, and for a message to be sent that has not yet been sent successfully, each time the message to be sent is retransmitted, a sending end may acquire a current time, which is denoted as t2, and it can be understood that, if the message to be sent has been sent once, a last sending time, which is denoted as t1, corresponding to a first identifier in data corresponding to the message to be sent may be acquired, so that t2 is compared with t1, and whether a difference between t2 and t1 is smaller than a first preset time threshold is determined.

S1062, if the time is less than the first preset time threshold, determining not to resend the message to be sent.

Still taking the above t2 and t1 as examples, if the first preset time threshold is 20us (microseconds), if t2-t1 is less than 20us, the sending end determines that the message to be sent does not need to be resent, and continues to process the next piece of data of the data corresponding to the message to be sent in the data queue. Of course, the first preset time threshold may be set according to an actual transmission requirement, which is not limited in the embodiment of the present invention.

S1063, if the time is greater than or equal to the first preset time threshold and the retransmission time corresponding to the second identifier of the data is less than the preset time threshold, resending the message to be sent.

And if the t2-t1 is greater than or equal to 20us, and the retransmission times corresponding to the second identifier of the data are smaller than a preset time threshold, for example, the retransmission times are smaller than 3 times, retransmitting the message to be sent. In a general case, the transmission delay of a DPDK small packet is generally less than 10us, so that the transmission delay can be reduced.

S1064, recording the retransmission time in the first identifier of the data corresponding to the message to be transmitted.

It can be understood that, after the message to be sent is retransmitted, the retransmission time may be recorded in the first identifier of the data corresponding to the message to be sent, and of course, the current time t2 may be directly assigned to t1 as the retransmission time, which is more convenient.

S1065, adding one to the retransmission times corresponding to the second identifier of the data corresponding to the message to be sent.

After the message to be sent is retransmitted, one may be added to the retransmission number corresponding to the second identifier of the data corresponding to the message to be sent, for example, if the added original retransmission number is 1, the retransmission number becomes 2 after retransmission.

Optionally, the current data may be taken out from the data queue, if the retransmission number is less than 3, the current message is retransmitted, and then the current time t2 is assigned to t1, and the retransmission number is increased by 1, so as to facilitate writing of the data.

As an optional implementation manner in this embodiment of the present invention, the sending method at the sending end in this embodiment of the present invention may further include:

and when the retransmission times corresponding to the second identification of the data are greater than or equal to the preset time threshold, removing the data from the data queue and recording the data into the log file.

Illustratively, if the number of retransmissions corresponding to the second identifier of the data is greater than or equal to 3, which indicates that the message to be sent is still unsuccessful after being sent for multiple times, at this time, the data corresponding to the message to be sent may be removed from the data queue and recorded in a log file, so that a worker can check and analyze the data queue to find out the reason of unsuccessful transmission, thereby improving the usability of the embodiment of the present invention. Of course, the preset number threshold may be set according to an actual transmission requirement, which is not limited in the embodiment of the present invention.

As an optional implementation manner of the embodiment of the present invention, if the sending end does not receive a response message that the receiving end really aims at a certain message to be sent within a preset time length after sending the certain message to be sent, the sending end may determine that the message to be sent is not sent successfully. The response message may carry a response identifier, for example, a response identifier 1, which indicates that the message is a response message. Of course, the preset time period may be set according to an actual transmission requirement, which is not limited in the embodiment of the present invention.

As an optional implementation manner of the embodiment of the present invention, a message format of the embodiment of the present invention may be as shown in fig. 3, and includes: the method comprises the steps of occupying a destination machine number of 2 bytes, a source machine number of 2 bytes, a message number of 3 bytes, a message type identifier of 1 byte, real data of 54-1490 bytes and a checksum identifier of 2 bytes. The number of the target machine may be a number predefined to the machine in each local area network, and the number of the target machine may be set as: the upper byte is represented by 0, and the lower byte is represented by the last byte of the destination machine IP address, for example, if the destination machine IP is 192.168.1.10, the destination machine number is 000A, where 00 is the upper byte for later expansion, and 0A is the lower byte for identifying the destination machine, i.e., the receiving end. Of course, if the destination machine IP is 192.168.1.11, the destination machine is numbered 000B, and so on.

The source machine is numbered, the high-order byte is 0, the low-order byte can adopt the last bit of the IP of the source machine, and the identification method refers to the identification method of the target machine, which is not described herein again.

The message number is used for giving a number to each message, so that the problem of message disorder is reduced conveniently at a receiving end.

The message type identifier is used to identify the type of the message, for example, 0 is a message sending identifier, and 1 is a message response identifier.

The MTU (Maximum Transmission Unit ) of the embodiment of the present invention is generally 1500, and the minimum length of each packet Transmission is generally 64 bytes, so the real data length len is generally: 54 ≦ len ≦ 1490 bytes, i.e., the real data size is greater than or equal to 54 and less than or equal to 1490 bytes, and if the data is greater than 1490 bytes, the data may be split for multiple transmissions. The checksum identification is used for verifying data, and the correctness of the data transmission process is verified by adopting a real data accumulation sum mode for real data, and the last two bits of the accumulation sum result are usually taken, so that the two bytes are occupied. Of course, the way of implementing data verification is not limited to the implementation shown in the illustrated example, and all the ways of implementing data verification belong to the scope of protection of the embodiments of the present invention.

In the message processing method provided by the embodiment of the present invention, after data is encapsulated by using a preset packet encapsulation format based on DPDK to obtain a packet to be sent, a first identifier for identifying a last sending time of the packet to be sent and a second identifier for identifying a sending frequency of the packet to be sent are added to the packet to obtain data corresponding to the packet and add the data to a data queue, so that after the packet to be sent corresponding to the data is sent, whether the packet to be sent corresponding to the data needs to be resent or not is determined according to the first identifier and the second identifier of the data, and the packet to be sent corresponding to the data needs to be resent when needed.

As shown in fig. 4, an embodiment of the present invention provides a packet processing method, which is applied to a receiving end implemented based on a DPDK package library, and the process may include the following steps:

s201, receiving a message to be sent by a sending end.

The embodiment of the invention can be applied to a receiving end realized by a DPDK packaging library, namely, a network card driver of the receiving end can use a DPDK user mode network card driver instead of a system kernel network card driver. The message received by the receiving end is obtained by encapsulating data by the sending end by using a preset DPDK-based message encapsulation format.

It can be understood that the receiving end also needs to perform an initialization process of the DPDK before receiving the message to be sent, and the initialization process is the same as that of the transmitting end, which is not described herein again. Of course, the receiving end in the embodiment of the present invention may also forward the message to be sent after receiving the message to be sent, that is, the receiving end and the sending end may be converted to each other. For convenience of description, the message to be sent by the sending end is collectively referred to as the message to be sent in the embodiment of the present invention.

S202, determining the sequence of each message to be sent according to the number of each received message to be sent.

Because the message to be sent received by the receiving end carries the number for uniquely identifying the message to be sent and the numbers of different messages to be sent are sequentially increased according to the packaging sequence, the receiving end can determine the sequence of each message to be sent according to the number in each received message to be sent.

Illustratively, the numbers of 4 messages to be sent by the sending end are 001, 002, 003 and 004 in sequence, and after the receiving end receives the 4 messages to be sent, the receiving end can determine the sequence of the messages to be sent according to the numbers of the messages to be sent, so as to avoid the problem of disorder.

As an optional implementation manner in the embodiment of the present invention, after determining the sequence of each message to be sent according to the received number in each message to be sent, as shown in fig. 5, the message processing method according to the embodiment of the present invention further includes:

s301, caching the messages to be sent into a cache queue according to the determined sequence.

The embodiment of the invention can firstly buffer the message to be sent into the buffer queue according to the determined sequence, and the buffer queue is usually positioned in the buffer area, so the data processing stability of the receiving end can be improved.

S302, if there exists a message to be sent in the buffer queue whose current time and message receiving time are less than a second preset time threshold, moving the message to be sent and all messages in the buffer queue before the message to be sent from the buffer queue to the receiving queue.

It can be understood that after the sending end sends the message to be sent, the receiving end may not receive the message to be sent immediately, and usually waits for a period of time before receiving the message to be sent, so a receiving buffer time period may be set for the receiving end, that is, the receiving buffer time period is the second preset time threshold, the second preset time threshold is greater than the product of the first preset time threshold and the preset time threshold, for example, the first preset time threshold is 20us, the preset time threshold is 3 times, the product of the two is 60 times, and the second preset time threshold is 100 us. The receiving end compares the difference between the current time and the receiving time of each message to be sent with a second preset time threshold value, so that when a message meeting the conditions appears, the message to be sent and all messages to be sent in the cache queue before the message to be sent can be moved from the cache queue to the receiving queue, and the messages in the range can not be out of order. Of course, the second preset time threshold may be set according to an actual transmission requirement, which is not limited in the embodiment of the present invention.

Optionally, after the receiving end successfully receives a certain message to be sent, a response message may be returned to the sending end, where the message type in the response message is 1.

According to the message processing method provided by the embodiment of the invention, after the receiving end receives the message to be sent, the receiving end can determine the sequence of each message to be sent according to the number in the message to be sent, so that the message received by the receiving end is prevented from being sent out in a disorder manner, and the possibility of errors in message receiving can be reduced.

A specific embodiment of a message processing apparatus according to an embodiment of the present invention corresponds to the flow shown in fig. 1, and with reference to fig. 6, fig. 6 is a schematic structural diagram of a message processing apparatus according to an embodiment of the present invention, which is applied to a transmitting end implemented based on a data plane development kit DPDK package library, and the apparatus includes:

the encapsulating module 401 is configured to encapsulate data by using a preset packet encapsulation format based on DPDK, to obtain a packet to be sent, where the packet to be sent carries a number for uniquely identifying the packet to be sent, and the numbers of different packets to be sent are sequentially increased according to an encapsulation sequence.

An adding module 402, configured to add a first identifier and a second identifier to a message to be sent, and generate data of the message to be sent, where the first identifier is used to identify a last sending time of the message to be sent, and the second identifier is used to identify a number of times of retransmission of the message to be sent.

An adding module 403, configured to add data of a message to be sent to a data queue.

A sending module 404, configured to send, according to the sequence of each data in the data queue, a to-be-sent message in each data to a receiving end implemented based on the DPDK package library in sequence, so that the receiving end receives each to-be-sent message, and determines the sequence of each to-be-sent message based on the number.

A deleting module 405, configured to delete, from the data queue, data of a message to be sent that is successfully sent.

The determining module 406 is configured to determine, for a to-be-sent message that has been sent and has not yet been sent successfully, whether to resend the to-be-sent message based on the first identifier and the second identifier included in the data of the to-be-sent message.

As shown in fig. 7, the determining module 406 includes:

the determining submodule 4061 is configured to obtain a current time, and determine whether a difference between the current time and a last transmission time corresponding to the first identifier in the data to be transmitted is smaller than a first preset time threshold.

The determining submodule 4062 is configured to determine not to resend the message to be sent if the time is smaller than the first preset time threshold.

The retransmission submodule 4063 is configured to resend the message to be sent if the retransmission time is greater than or equal to the first preset time threshold and the retransmission frequency corresponding to the second identifier of the data is less than the preset frequency threshold.

The recording sub-module 4064 is configured to record a retransmission time in the first identifier of the data corresponding to the message to be sent.

The counting sub-module 4065 is configured to add one to the retransmission times corresponding to the second identifier of the data corresponding to the message to be sent.

On the basis of the device structure shown in fig. 6 or fig. 7, as shown in fig. 8, the message processing device according to the embodiment of the present invention may further include:

a first determining module 501, configured to determine that a message to be sent is not sent successfully if a response message, which is returned by the receiving end and is specific to the sent message to be sent, is not received within a preset time period, where the response message is used to indicate that the message to be sent is received successfully.

On the basis of the apparatus structure shown in fig. 8, as shown in fig. 9, the message processing apparatus according to the embodiment of the present invention may further include:

a removing module 601, configured to remove the data from the data queue when the number of retransmissions corresponding to the second identifier of the data is greater than or equal to a preset number threshold.

A recording module 602, configured to record data into a log file.

The preset DPDK-based packet encapsulation format is as follows: the method comprises the steps of occupying a destination machine number of 2 bytes, a source machine number of 2 bytes, a message number of 3 bytes, real data of 54-1490 bytes and a checksum identification of 2 bytes.

In the message processing apparatus provided in the embodiment of the present invention, after data is encapsulated by using a preset packet encapsulation format based on DPDK to obtain a packet to be sent, a first identifier for identifying a last sending time of the packet to be sent and a second identifier for identifying a sending frequency of the packet to be sent are added to the packet to obtain data corresponding to the packet and add the data to a data queue, so that after the packet to be sent corresponding to the data is sent, whether the packet to be sent corresponding to the data needs to be resent is determined according to the first identifier and the second identifier of the data, and the packet to be sent corresponding to the data needs to be resent when needed.

A specific embodiment of a message processing apparatus according to an embodiment of the present invention corresponds to the flow shown in fig. 4, with reference to fig. 10, fig. 10 is a schematic structural diagram of a message processing apparatus according to an embodiment of the present invention, and is applied to a receiving end implemented based on a data plane development kit DPDK package library, where the apparatus includes:

the receiving module 701 is configured to receive a to-be-sent message sent by a sending end, where the to-be-sent message is obtained by the sending end by encapsulating data using a preset message encapsulation format based on a data plane development kit DPDK, and the to-be-sent message carries a number for uniquely identifying the to-be-sent message, and the numbers of different to-be-sent messages are sequentially increased according to an encapsulation sequence.

A second determining module 702, configured to determine, according to the number in each received message to be sent, a sequence of each message to be sent.

In addition to the structure of the apparatus shown in fig. 10, as shown in fig. 11, the apparatus further includes:

the caching module 801 is configured to cache the messages to be sent into the caching queue according to the determined sequence.

The moving module 802 is configured to, if a to-be-sent message exists in the cache queue, where the current time and the message receiving time are smaller than a second preset time threshold, move the to-be-sent message and all messages in the cache queue before the to-be-sent message from the cache queue to the receiving queue, where the second preset time threshold is greater than a product of the first preset time threshold and a preset time threshold.

According to the message processing device provided by the embodiment of the invention, after the receiving end receives the message to be sent, the receiving end can determine the sequence of each message to be sent according to the number in the message to be sent, so that the message received by the receiving end is prevented from being sent out in a disorder manner, and the possibility of errors in message receiving can be reduced.

As shown in fig. 12, an embodiment of the present invention further provides a message processing system, including: a transmitting end 901 and a receiving end 902 realized based on a data plane development kit DPDK package library, where the transmitting end 901 and the receiving end 902 are communicatively connected through a DPDK network, where,

the sending terminal is used for packaging data by using a preset DPDK-based message packaging format to obtain a message to be sent; adding a first identifier and a second identifier to a message to be sent, generating data of the message to be sent, adding the data of the message to be sent to a data queue, sequentially sending the message to be sent in each data to a receiving end based on DPDK packaging library according to the sequence of each data in the data queue, and removing data corresponding to the message to be sent which is successfully sent from the data queue, wherein the data queue comprises a plurality of pieces of data, and each piece of data comprises one message and the first identifier and the second identifier corresponding to the message; and judging whether to resend the message to be sent based on a first identifier and a second identifier contained in data of the message to be sent aiming at the message to be sent which is sent and is not sent successfully, wherein the message to be sent carries a number for uniquely identifying the message, the numbers of different messages to be sent are sequentially increased according to the packaging sequence, the first identifier is used for identifying the last sending time of the message to be sent, and the second identifier is used for identifying the retransmission times of the message to be sent;

the receiving end is used for receiving a message to be sent by the sending end; and determining the sequence of each message to be sent according to the number of each received message to be sent.

In the message processing system provided by the embodiment of the present invention, after a sending end packages data by using a preset DPDK-based message package format to obtain a message to be sent, a first identifier for identifying the last sending time of the message to be sent and a second identifier for identifying the sending times of the message to be sent are added to the message to obtain data corresponding to the message and add the data to a data queue, so that after the message to be sent corresponding to the data is sent, whether the message to be sent corresponding to the data needs to be resent or not is determined by using the first identifier and the second identifier of the data, and the message needs to be resent when needed, on one hand, the transmission delay of the message can be reduced by using a DPDK technique, and on the other hand, the probability of successful receiving of the message at a receiving end is improved; after the receiving end receives the message to be sent, the receiving end can determine the sequence of each message to be sent according to the number in the message to be sent, so that the message received by the receiving end is prevented from being sent out in a disorder manner, and the possibility of errors in message receiving can be reduced.

The embodiment of the present invention further provides an electronic device, as shown in fig. 13, including a processor 1001, a communication interface 1002, a memory 1003 and a communication bus 1004, where the processor 1001, the communication interface 1002 and the memory 1003 complete mutual communication through the communication bus 1004,

a memory 1003 for storing a computer program;

the processor 1001 is configured to implement the following steps when executing the program stored in the memory 1003:

adding a first identifier and a second identifier to a message to be sent, wherein the first identifier is used for identifying the last sending time of the message to be sent, and the second identifier is used for identifying the number of times of retransmission of the message to be sent;

adding a message to be sent and a first identifier and a second identifier of the message to be sent to a data queue, sequentially sending the message to be sent in each data to a receiving end realized based on a DPDK packaging library according to the sequence of each data in the data queue, and removing data corresponding to the successfully sent message to be sent from the data queue, wherein the data queue comprises a plurality of pieces of data, and each piece of data comprises: a message to be sent, and a first identifier and a second identifier corresponding to the message to be sent;

According to the electronic device provided by the embodiment of the invention, after the data is encapsulated by using the preset DPDK-based message encapsulation format to obtain the message to be sent, the first identifier for identifying the last sending time of the message to be sent and the second identifier for identifying the sending times of the message to be sent are added to the message to obtain the data corresponding to the message and add the data to the data queue, so that after the message to be sent corresponding to the data is sent, whether the message to be sent corresponding to the data needs to be resent or not is judged according to the first identifier and the second identifier of the data, and the message to be sent corresponding to the data needs to be resent when needed.

An embodiment of the present invention further provides an electronic device, as shown in fig. 14, including a processor 1101, a communication interface 1102, a memory 1103 and a communication bus 1104, where the processor 1101, the communication interface 1102 and the memory 1103 complete mutual communication through the communication bus 1104,

a memory 1103 for storing a computer program;

the processor 1101 is configured to implement the following steps when executing the program stored in the memory 1103:

and determining the sequence of each message to be sent according to the number of each received message to be sent.

According to the electronic device provided by the embodiment of the invention, after the message to be sent is received, the sequence of each message to be sent can be determined according to the number in the local message, so that the message received by the receiving end is prevented from being sent out in a disorder manner, and the possibility of errors in message receiving can be reduced.

The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the electronic equipment and other equipment.

The Memory may include a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.

An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, and is configured to execute the following steps:

According to the computer-readable storage medium provided by the embodiment of the invention, after data is encapsulated by using a preset DPDK-based message encapsulation format to obtain a message to be sent, a first identifier for identifying the last sending time of the message to be sent and a second identifier for identifying the sending times of the message to be sent are added to the message to obtain data corresponding to the message and add the data to a data queue, so that after the message to be sent corresponding to the data is sent, whether the message to be sent corresponding to the data needs to be resent or not is judged according to the first identifier and the second identifier of the data, and the message to be sent corresponding to the data needs to be resent when needed.

According to the computer-readable storage medium provided by the embodiment of the invention, after the messages to be sent are received, the sequence of each message to be sent can be determined according to the number in the message to be sent, so that the messages received by the receiving end are prevented from being sent out in a disorder manner, and the possibility of errors in message receiving can be reduced.

For the apparatus/electronic device/storage medium embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to part of the description of the method embodiment.

It should be noted that the apparatus, the electronic device, and the storage medium according to the embodiments of the present invention are respectively an apparatus, an electronic device, and a storage medium to which the message processing method is applied, and all embodiments of the message processing method are applicable to the apparatus, the electronic device, and the storage medium, and can achieve the same or similar beneficial effects.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims

1. A message processing method is characterized in that the method is applied to a sending end realized based on a data plane development kit DPDK packaging library, and the method comprises the following steps:

adding the data of the message to be sent to a data queue;

2. The method according to claim 1, wherein the step of determining whether to resend the message to be sent based on the first identifier and the second identifier included in the data of the message to be sent for the message to be sent that has been sent and has not yet been sent successfully comprises:

3. The method according to claim 1 or 2, wherein before determining whether to resend the message to be sent based on the first identifier and the second identifier included in the data of the message to be sent, for the message to be sent that has been sent and has not yet been sent successfully, the method further comprises:

4. The method of claim 2, further comprising:

and recording the data into a log file.

5. The method of claim 1, wherein the preset DPDK-based packet encapsulation format comprises: the method comprises the steps of destination machine number, source machine number, message type identification, real data and check sum identification.

6. A message processing method is characterized in that the method is applied to a receiving end realized based on a DPDK package library of a data plane development kit, and the method comprises the following steps:

7. The method according to claim 6, wherein after determining the sequence of each message to be sent according to the number in each received message to be sent, the method further comprises:

8. A message processing device is applied to a sending end realized based on a data plane development kit DPDK packaging library, and the device comprises:

9. The apparatus of claim 8, wherein the determining module comprises:

10. The apparatus of claim 8 or 9, further comprising:

11. The apparatus of claim 9, further comprising:

and the recording module is used for recording the data into a log file.

12. The apparatus of claim 8, wherein the preset DPDK-based packet encapsulation format is: the method comprises the steps of occupying a destination machine number of 2 bytes, a source machine number of 2 bytes, a message number of 3 bytes, real data of 54-1490 bytes and a checksum identification of 2 bytes.

13. A message processing device is applied to a receiving end realized based on a DPDK package library of a data plane development kit, and the device comprises:

14. The apparatus of claim 13, further comprising:

15. A message processing system, comprising: a transmitting end and a receiving end realized based on a DPDK package library of a data plane development kit;