CN115334169B

CN115334169B - Communication protocol coding method capable of saving network bandwidth

Info

Publication number: CN115334169B
Application number: CN202210461679.9A
Authority: CN
Inventors: 蒋春风; 肖成虎; 何发
Original assignee: Shenzhen Securities Communication Co ltd
Current assignee: Shenzhen Securities Communication Co ltd
Priority date: 2022-04-28
Filing date: 2022-04-28
Publication date: 2023-06-06
Anticipated expiration: 2042-04-28
Also published as: CN115334169A

Abstract

The invention discloses a communication protocol coding method capable of saving network bandwidth, which comprises a coding flow and a decoding flow, wherein the coding flow comprises the step of coding data contents into data packets; grouping fields to be encoded; each field is preprocessed as follows according to different data types; arranging the field codes after grouping into a communication packet; performing conventional treatment; the decoding flow includes parsing the data packet; analyzing the fields of the packet; conventional processing is performed. The invention comprises the following steps: (1) saving network traffic by about 35%. (2) The method has the advantages of flexible processing of the whole data range, avoiding the change of the communication protocol caused by a small amount of data exceeding the range, and improving the adaptability of the communication protocol and software. (3) Compared with compression coding, the coding method has no obvious consumption on CPU and has stronger practicability.

Description

Communication protocol coding method capable of saving network bandwidth

Technical Field

The invention relates to a communication protocol coding method capable of saving network bandwidth, belonging to the technical field of software design and development in the field of network communication.

Background

In network communication, before transmitting data, the data needs to be encoded in a certain format and then transmitted to a network; after the other party receives the data, the data is decoded by a method opposite to the encoding method, so that the real data of the other party is obtained.

When data coding is performed, each field needs to be arranged one by one, and common coding methods for communication protocols include:

(1) Fixed-length text mode coding;

(2) Variable length text mode coding with separator;

(3) Binary encoding;

(4) XML format, etc.;

(5) And compressed and transmitted.

Different coding methods are generally selected for different occasions. Each method has respective advantages and disadvantages, for example, binary system saves network bandwidth compared with a text mode, but manual reading is difficult; variable length saves bandwidth over fixed length, but variable length is more complex to process for computer programming. XML is highly human readable, but consumes almost the largest amount of network bandwidth and generally must rely on third party libraries to convert entities.

In order to save network bandwidth, the data is compressed and then transmitted, so that the bandwidth is saved. Tested, this approach is essentially not feasible for real-time communication due to the too much CPU resources consumed by compressed data. Thus, little is done in a compressed manner in real-time communication applications, except in non-real-time communications.

Disclosure of Invention

The present invention is directed to a communication protocol encoding method capable of saving network bandwidth, so as to solve the problems set forth in the background art.

In order to achieve the above purpose, the present invention provides the following technical solutions: a communication protocol encoding method for saving network bandwidth, the method comprising an encoding process comprising steps 1 to 5 and a decoding process comprising steps 6 to 8:

step 1, data contents are encoded into the data packets;

step 2, grouping fields to be coded into groups of four, wherein the fields with less than four tail groups are grouped;

step 3, respectively preprocessing each field as follows according to different data types;

a. shaping with symbols;

if the integer range is [ -128,127], i.e., 1 byte can be represented, then a forced conversion to a signed integer of one byte;

if the integer range is not within the above range, but is within [ -32768,32767], i.e., 2 bytes, then a forced conversion to a signed integer of two bytes is possible;

if the integer range is not within the above range, but is within [ -2147483648,2147483647], i.e., 4 bytes, then a forced conversion to a signed integer of four bytes is possible;

wherein, if int64 is not in the above range, no conversion is performed;

b. unsigned integer:

if the integer range is [0,0xFF ], i.e. 1 byte can be represented, then the conversion is forced to an unsigned integer of one byte;

if the integer range is [0x100,0xFFFF ], i.e. 2 bytes can be represented, a forced conversion into an unsigned integer of two bytes;

if the integer range is [0x10000,0x FFFFFFFF ], i.e. 4 bytes can be represented, then a forced conversion into an unsigned integer of four bytes is performed;

wherein, the uint64 greater than 0xFFFFFFFF does not perform conversion;

c. character string type:

the type will be arranged in a manner that uses length + string content; for the length thereof, the same manner as the above-described unsigned integer is adopted for storage;

d. floating point type:

directly using binary data;

step 4, arranging the field codes after grouping into the communication packet, specifically:

a. writing a binary integer of one byte, and the byte order of the size end is the same for the integer of one byte, so that conversion is not needed, and the byte order is called grouping information; since each byte has 8 bits, these 8 bits are equally divided into four parts, each 2 bits representing the length of the value in the corresponding field of the subsequent set, that is:

bits 1 to 2 represent the length of the value in field 1 in the group that follows;

bits 3 to 4 represent the length of the value in the 2 nd field in the group that follows;

bits 5 to 6 represent the length of the value in the 3 rd field in the group that follows;

bits 7 to 8 represent the length of the value in the 4 th field in the group that follows;

wherein each 2 bits can only represent four possible values of 0, 1, 2, 3, which means:

0: indicating that the value length in the subsequent corresponding field is 1 byte;

1: indicating that the value length in the subsequent corresponding field is 2 bytes;

2: indicating that the value length in the subsequent corresponding field is 4 bytes;

3: indicating that the value in the subsequent corresponding field is 8 bytes in length;

for the last group, if the number of fields is less than 4, the corresponding position can be filled with 0;

b. the four fields of the set are then written in sequence, each field having the following format:

if the character string is of the character string type, writing the length and the character string content which are already preprocessed by the method in sequence;

if the floating point type is adopted, directly writing binary contents of the corresponding big-end byte order;

if the integer type is the integer type comprising signed integer and unsigned integer, writing the integer which is pretreated by the method;

and is subjected to conventional processing.

Preferably, the conventional processing of step 5 includes, but is not limited to, adding a checksum field to the end of the packet.

Preferably, the conventional processing of step 8 includes, but is not limited to, parsing a check field.

Compared with the prior art, the invention has the beneficial effects that:

(1) Network traffic is saved by about 35%.

It is assumed that 70% of the memory space in a packet is integer, the others are floating points or strings. Since in most cases, the integer is small number and the character string is relatively short, especially in the financial industry, the integer can be usually represented by not more than 1 byte or 2 bytes, and in the computer, we often use a length of 4 bytes or 8 bytes to represent, resulting in more than half of bytes being wasted. Assuming that 50% of the space is wasted for the integer (in the present invention, although one byte is supplemented before each group, this one byte is insignificant for a large number of integers that are wasted more than half the length), the overall waste rate is 70% by 50% = 35%. That is, with the present invention, the packet size can be reduced by 35% in most cases, or the network traffic, or bandwidth consumption, can be saved by 35%.

Such as price of quotation, number of claims, etc. in the financial industry, the data size will generally not exceed 65535 after the decimal point is removed, but we cannot completely exclude the possibility of exceeding 65535. For example, the stock price of 95% does not exceed 655 yuan, but the standard of Guizhou Maotai exceeds 1000 yuan, if 4 bytes are used for representing at one time in network transmission, the method is very wasteful, and a great deal of bandwidth can be saved by adopting the coding method of the communication protocol.

(2) Compared with a field coding method with fixed length, the coding method is flexible in processing the integer data range, avoids changing a communication protocol caused by a small amount of data exceeding the range, and improves the adaptability of the communication protocol and software.

The traditional fixed-length field cannot expand the length of data, and in case of encountering an unexpected data range in the future, the communication protocol becomes invalid, and only the communication protocol can be redesigned, so that both communication parties need to redevelop software, and a great deal of cost is caused. The coding method is unified for the integer data represented by a few bytes, and the problems of redesign of a communication protocol and recoding of software caused by the expansion of the data range in use are avoided.

(3) Compared with compression coding, the coding method has no obvious consumption on CPU and has stronger practicability. Compression coding is not practical in real-time communication.

Drawings

Fig. 1 is a schematic diagram of packet information according to the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely, and it is apparent that the described embodiments are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

The invention provides a technical scheme that: binary encoding is used. The communication protocol packet header is temporarily not considered, the packet header is generally short, and the conventional mode is adopted. Only the coding scheme of individual fields is considered here.

1. The specific coding flow is as follows:

the coding refers to packaging the data before transmission to form a data stream, and finally transmitting the data stream through a network.

Step 1: and the data packet is encoded with the contents such as a protocol header.

Step 2: the fields to be encoded are grouped in groups of four each, with fewer than four fields at the end being a group.

Step 3: each field is respectively preprocessed as follows according to different data types:

(1) Signed integer (whether 1, 2, 4, or 8 byte integer):

(1) if the integer range is [ -128,127], i.e., 1 byte can be represented, then a forced conversion to a signed integer of one byte;

(2) if the integer range is not within the above range, but is within [ -32768,32767], i.e., 2 bytes, then a forced conversion to a signed integer of two bytes is possible;

(3) if the integer range is not within the above range, but is within [ -2147483648,2147483647], i.e., 4 bytes, then a forced conversion to a signed integer of four bytes is possible;

(4) in other cases (referring to int64 not in the above range), no conversion is performed.

(2) Unsigned integer (whether 1, 2, 4, or 8 byte integer):

(1) if the integer range is [0,0xFF ], i.e. 1 byte can be represented, then the conversion is forced to an unsigned integer of one byte;

(2) if the integer range is [0x100,0xFFFF ], i.e. 2 bytes can be represented, a forced conversion into an unsigned integer of two bytes;

(3) if the integer range is [0x10000,0x FFFFFFFF ], i.e. 4 bytes can be represented, then a forced conversion into an unsigned integer of four bytes is performed;

(4) in other cases (referring to uint64 greater than 0 xFFFFFFFF), no conversion is performed.

(3) Character string type: this type will be arranged in a manner that uses length + string content. For the length thereof, storage is performed in the same manner as the above-described unsigned integer.

(4) Floating point type (including float single precision and double precision): binary data is directly used without conversion.

Step 4: the mode of arranging the four fields into a group is encoded into a communication packet, and the specific flow is as follows:

(1) Binary one byte integers (the same for one byte integer, the size end endian, so no conversion is necessary) are written first, we refer to packet information. As shown in fig. 1. Since there are 8 bits per byte, these 8 bits are equally divided into four parts, each 2 bits representing the length of the value in the corresponding field of the subsequent set. That is to say:

bits 1 to 2 represent the length of the value in the 1 st field in the group that follows.

Bits 3 to 4 represent the length of the value in the 2 nd field in the group that follows.

Bits 5 to 6 represent the length of the value in the 3 rd field in the group that follows.

Bits 7 to 8 represent the length of the value in the 4 th field in the group that follows.

0: indicating that the value in the corresponding field that follows is 1 byte in length.

1: indicating that the value in the corresponding field that follows is 2 bytes in length.

2: indicating that the value in the corresponding field that follows is 4 bytes in length.

3: indicating that the value in the subsequent corresponding field is 8 bytes in length.

For the last group, if there are less than 4 fields, the corresponding position may be filled with 0.

(2) The four fields of the set are then written in sequence, each field having the following format:

(1) if the character string is of the character string type, the length (big-end byte order representation) and the character string content which are already preprocessed by the method are written in sequence.

(2) If the floating point type (including float single precision and double precision) binary contents of the corresponding big-end byte order are directly written.

(3) If of integer type, both signed and unsigned, the integer already preprocessed by the above method (big-end endian) is written.

Step 5: other conventional processing such as adding a checksum field to the end of the packet, etc. And is not in the scope of the present invention due to conventional processing, and therefore, will not be described in detail.

The specific decoding flow is as follows:

decoding refers to the process of unpacking after receiving data from the network, thereby obtaining the original information.

Step 1: and analyzing the contents such as the packet header and the like for the data packet. And are not described in detail herein, as they do not belong to the present invention.

Step 2: each four-field packet is circularly analyzed, and the flow is as follows:

(1) First parse one byte of packet information:

For the last group, if there are less than 4 fields, the remainder may be ignored.

(2) Sequentially analyzing the 1 st, 2 nd, 3 rd and 4 th fields in the group.

The data type of each field needs to be determined according to the conventions of both parties. At the same time, care should be taken to translate to native endian. For the storage length of the local integer, the storage length can only be larger than or equal to the received length value, otherwise, the storage length is always wrong.

If the last packet is less than four fields, the remainder is ignored.

Step 3: other conventional processing such as parsing the check fields of the trailer, etc.

The invention has practical value in real-time communication application, reduces network flow, saves about 35% of network bandwidth compared with the common binary coding method under the condition of no information loss, and improves the adaptability of the protocol to various data ranges.

Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims

1. A communication protocol encoding method for saving network bandwidth, the method comprising an encoding process and a decoding process, wherein the encoding process comprises steps 1 to 5, and the decoding process comprises steps 6 to 8: