CN111193696B

CN111193696B - Data transmitting method, data receiving method, data transmitting device, data receiving device, computer equipment and storage medium

Info

Publication number: CN111193696B
Application number: CN201910707619.9A
Authority: CN
Inventors: 毛江云; 杨涛; 陈志博
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2019-08-01
Filing date: 2019-08-01
Publication date: 2021-09-07
Anticipated expiration: 2039-08-01
Also published as: CN111193696A

Abstract

The application relates to a data sending method, a data receiving device, a computer device and a storage medium, wherein the data receiving method comprises the following steps: acquiring target data to be sent; generating description data corresponding to the target data, wherein the description data comprises starting description information and ending description information; segmenting to obtain multiple data according to the description data and the target data to form a data sequence, wherein the data sequence comprises first data and second data, the starting description information is located at the head of the first data, the ending description information is located at the tail of the second data, the first data is the head data of the data sequence, and the second data is the tail data of the data sequence; and sending the data sequence to a receiving party. The data transmission method enables the receiver to receive the transmitted data more conveniently and accurately.

Description

Data transmitting method, data receiving method, data transmitting device, data receiving device, computer equipment and storage medium

Technical Field

The present application relates to the field of communications, and in particular, to data transmission and data reception methods, apparatuses, computer devices, and storage media.

Background

Applications of a computer device typically need to obtain data from other devices or other processes while running. For example, processes between the same devices may communicate data by way of interprocess communication. At present, when receiving data, a receiving party often finds that the received data is incomplete data or mixed with other data, so that complete or independent data cannot be obtained.

Disclosure of Invention

In view of the above, it is necessary to provide a data transmission method, a data reception method, an apparatus, a computer device, and a storage medium for solving the above-mentioned problem of incomplete or independent data.

A method of data transmission, the method comprising: acquiring target data to be sent; generating description data corresponding to the target data, wherein the description data comprises starting description information and ending description information; segmenting to obtain multiple data according to the description data and the target data to form a data sequence, wherein the data sequence comprises first data and second data, the starting description information is located at the head of the first data, the ending description information is located at the tail of the second data, the first data is the head data of the data sequence, and the second data is the tail data of the data sequence; and sending the data sequence to a receiving party.

A data transmission apparatus, the apparatus comprising: the target data acquisition module is used for acquiring target data to be sent; the description data generation module is used for generating description data corresponding to the target data, and the description data comprises starting description information and ending description information; the segmentation module is configured to segment multiple data according to the description data and the target data to form a data sequence, where the data sequence includes first data and second data, the start description information is located at a head of the first data, the end description information is located at a tail of the second data, the first data is head data of the data sequence, and the second data is tail data of the data sequence; and the data sequence sending module is used for sending the data sequence to a receiving party.

In some embodiments, the slitting module comprises: the splicing unit is used for splicing the description data and the target data to obtain spliced data, wherein the starting description information is positioned at the head of the spliced data, and the ending description information is positioned at the tail of the spliced data; and the segmentation unit is used for segmenting the spliced data to obtain multiple data and form a data sequence.

In some embodiments, the description data includes data length information corresponding to the target data, and the splicing unit is configured to splice the start description information, the end description information, the data length information, and the target data according to a preset rule to obtain spliced data, where the data length information is located before the target data.

In some embodiments, the description data includes service-related information corresponding to the target data, and the splicing unit is configured to splice the start description information, the service-related information, the target data, and the termination description information according to a preset rule to obtain spliced data, where the service-related information is located before the target data.

In some embodiments, the slicing unit is to: and when the data volume of the spliced data is larger than the size of a data sending buffer area, segmenting the spliced data according to the size of the data sending buffer area to obtain multiple data, and forming a data sequence.

In some embodiments, the apparatus further comprises: the communication connection establishing module is used for establishing inter-process communication connection based on the socket with the process of the receiving party; the slitting module is used for: acquiring the data residual space of a data sending buffer corresponding to the interprocess communication connection; and segmenting according to the description data and the target data to obtain the first data, wherein the length of the first data is less than or equal to the capacity of the data residual space, so that the first data is added into the data sending buffer area.

A computer device comprising a memory and a processor, the memory having stored therein a computer program which, when executed by the processor, causes the processor to carry out the steps of the data transmission method described above.

A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, causes the processor to carry out the steps of the above-mentioned data transmission method.

According to the data sending method, the data sending device, the computer equipment and the storage medium, when the target data needs to be sent, the description data corresponding to the target data can be generated, the description data comprises the initial description information and the termination description information, the multiple data are obtained through segmentation according to the description data and the target data and form a data sequence, therefore, the data are convenient to send, the initial description information is located at the head of the first data, and the termination description information is located at the tail of the last data, so that even if the multiple data are split into multiple data packets or are mixed with other data to be sent during sending, a receiving party can obtain complete and independent information according to the initial description information and the termination description information, and the receiving party can receive the sent data more conveniently and accurately.

A method of data reception, the method comprising: receiving data sent by a sender to obtain received data, wherein the received data comprises a data sequence consisting of a plurality of data shares; analyzing the received data, and determining to receive complete data when initial description information and termination description information exist in the received data after analyzing; and splicing the data in the data sequence according to the position of the starting description information and the position of the ending description information to obtain complete target data.

A data receiving apparatus, the apparatus comprising: the data receiving module is used for receiving data sent by a sender to obtain received data, and the received data comprises a data sequence formed by a plurality of data; the analysis module is used for analyzing the received data, and when the received data is analyzed to obtain initial description information and termination description information, the received complete data is determined; and the target data obtaining module is used for splicing the data in the data sequence according to the position of the starting description information and the position of the ending description information to obtain complete target data.

In some embodiments, the parsing module is to: analyzing the received data to obtain the initial description information; determining a first reading position according to the position of the starting description information, and reading data length information from the received data according to the first reading position; and skipping the data with the corresponding length in the received data according to the data length information, and reading to obtain the termination description information.

In some embodiments, the target data is data obtained after serialization, and the apparatus further includes: a service related information extraction module, configured to extract service related information corresponding to the target data from the received data; the data processing determining module is used for determining a data processing module corresponding to the target data according to the service related information; and the target data sending module is used for sending the serialized target data to the corresponding data processing module.

A computer device comprising a memory and a processor, the memory having stored therein a computer program which, when executed by the processor, causes the processor to carry out the steps of the above-mentioned data receiving method.

A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, causes the processor to carry out the steps of the above-mentioned data receiving method.

According to the data receiving method, the data receiving device, the computer equipment and the storage medium, when the data are received, the received data are analyzed, when the initial description information and the termination description information exist in the received data obtained through analysis, the received complete data are determined, and the data in the data sequence are spliced according to the position of the initial description information and the position of the termination description information, so that complete target data are obtained. Therefore, even if the data is split into a plurality of data packets or is mixed with other data to be sent during sending, the receiving party can acquire complete and independent information according to the positions of the initial description information and the termination description information, and convenience and accuracy are achieved.

Drawings

FIG. 1 is a diagram of an application environment for a method of data transmission provided in some embodiments;

FIG. 2 is a flow diagram of a method of data transmission in some embodiments;

FIG. 3 is a diagram of a data format of stitching data in some embodiments;

FIG. 4 is a schematic illustration of stitching data in some embodiments;

FIG. 5 is a flow chart of a method of data reception in some embodiments;

FIG. 6 is a flow diagram of parsing start description information and end description information from received data in some embodiments;

FIG. 7 is a block diagram of an application receiving data in some embodiments;

FIG. 8 is a schematic diagram of sending data and receiving data through a UDS pipe in some embodiments;

FIG. 9 is a block diagram of a data transmission device in some embodiments;

FIG. 10 is a block diagram of a data receiving device in some embodiments;

FIG. 11 is a block diagram of the internal architecture of a computing device in some embodiments.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

It will be understood that, as used herein, the terms "first," "second," and the like may be used herein to describe various elements, but these elements are not limited by these terms unless otherwise specified. These terms are only used to distinguish one element from another. For example, a first process may be termed a second process, and, similarly, a second process may be termed a first process, without departing from the scope of the present application.

Fig. 1 is an application environment diagram of a data transmission method in some embodiments, as shown in fig. 1, in the application environment, the application environment includes a first process 110 and a second process 120, where the first process 110 and the second process 120 may be processes located on the same computer device, the computer device may be a terminal or a server, and the first process 110 and the second process 120 may communicate with each other through a UDS (universal Domain Socket) communication pipe. When the first process 110 needs to send data to the second process 120, the first process 110 may execute the data sending method provided in the embodiment of the present application to send the data to the second process 120, and the second process 120 may execute the data receiving method provided in the embodiment of the present application to extract complete data.

It is to be understood that the above application scenario is only an example, and does not limit the data transmission method provided in the present application, for example, the first process 110 and the second process 120 may communicate with each other through other inter-process communication manners.

The terminal 110 may be, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, and the like. The application is not limited thereto.

As shown in fig. 2, in some embodiments, a data transmission method is proposed, and this embodiment is mainly exemplified by applying the method to the first process 110 in fig. 1. The method specifically comprises the following steps:

step S202, target data to be sent is acquired.

Specifically, the target data is data that needs to be sent, and the target data may include at least one of data such as pictures, text, or voice. For example, the target data may be one video or may be a plurality of pictures.

In some embodiments, the target data is obtained according to a service data sending request, where the service data sending request is used to request to send service data corresponding to the target service, and the service data sending request may carry an identifier of data to be sent and may also carry the target data to be sent. Therefore, the first process can acquire the target data according to the service data sending request.

In some embodiments, the target data may be a picture. The target service may be, for example, a picture processing task or a picture storage service. When the first process determines that the picture needs to be processed, the picture can be sent to the second process so as to be processed through the second process. Or the picture is distributed to a module for picture processing through the second process, and the picture is processed through the picture processing module.

Step S204, generating description data corresponding to the target data, wherein the description data comprises starting description information and ending description information.

In particular, the description data may be used to describe the target data. For example, the description data may further include information describing a length of the target data and information describing a type of the target data. The start description information is used to describe a start position of the target data, and the end description information is used to describe an end position of the target data. The start description information may be a preset character string, and the end description information may also be a preset character string. For example, the start description Information may be "(" may be expressed in hexadecimal as 0 × 28. the end description Information may be in ASCII Code ") in ASCII (American Standard Code for Information Interchange Code) Code, and may be expressed in hexadecimal as 0 × 29. Of course, the start description information and the end description information may also be other character strings, which are specifically defined according to needs, for example, the start description information may be "\ r \ n".

In some embodiments, the length of the start description information and the end description information may be preset, for example, the number of bytes occupied by the start description information and the end description information may be 2.

And step S206, segmenting to obtain multiple data according to the description data and the target data to form a data sequence, wherein the data sequence comprises first data and second data, the initial description information is located at the head of the first data, the termination description information is located at the tail of the second data, the first data is the head data of the data sequence, and the second data is the tail data of the data sequence.

Specifically, the data sequence includes a plurality of copies of data, for example, 3, which may be different according to actual situations. The data in the data sequence is arranged in order. The head refers to the forwardmost position and the tail refers to the rearwardmost position. The first data is the head data in the data sequence, i.e. the first part of data in the data sequence, and the second data is the tail data in the data sequence, i.e. the last part of data in the data sequence. By placing the start description information at the head of the header data and the end description information at the tail of the second data, the receiver reads the data in sequence when receiving the data sequence, and can determine that the new data is received through the start description information and determine that the data has been received through the end description information. Therefore, complete data can be extracted, and corresponding business operation is executed according to the extracted complete target data. It will be appreciated that the data sequence may comprise other data in addition to the first data and the second data, for example third data, the third data being data in the data sequence that is ordered after the first data and before the second data.

In some embodiments, the description data may further include at least one of an identification of the data correspondence, a timestamp of the data correspondence, a type of the data, or a length of the data. The identification corresponding to the data is used for identifying the target data. The time stamp may be used to describe the time when the data transmission request is received or the time when the target data is generated. The type of the data may be, for example, a face recognition picture or a face recognition result. Specifically, the determination is based on actual services. The face recognition picture is a picture for face recognition.

In some embodiments, the target data may be segmented, and then start description information is added to a head of the segmented first data, and end description information is added to a tail of the second data.

In some embodiments, the description data and the target data may be spliced to obtain spliced data, and then the spliced data may be segmented.

In some embodiments, the rule of splitting may be set as needed, for example, the number of data copies or the size of the sub data obtained by splitting may be set.

In some embodiments, the sorted information of the data in the data sequence may be added to the sliced data.

In some embodiments, the slicing may be based on the size of the data transmit buffer. For example, the data may be sliced into data having a data length equal to or smaller than the size of the data transmission buffer.

In some embodiments, when the data volume of the spliced data is larger than the size of the data sending buffer, the spliced data is segmented according to the size of the data sending buffer to obtain multiple pieces of data, and the data sequence is formed.

Specifically, the data transmission buffer refers to a buffer for storing data to be transmitted, and the data in the data transmission buffer are sequentially transmitted in sequence. When the data volume of the spliced data is larger than the size of the data sending buffer area, the spliced data is segmented, otherwise, the spliced data can be sent under the condition of no segmentation. The concatenated data may be divided into a plurality of data portions according to the size of the data transmission buffer, each data portion being smaller than or equal to the size of the data transmission buffer. For example, assuming that the data transmission buffer is 10000 bytes and the splicing data occupies 26000 bytes, the splicing data may be divided into 3 parts, the first part and the second part are 10000 bytes, and the third part is 6000 bytes.

Step S208, the data sequence is sent to the receiving party.

Specifically, the receiving party refers to the party receiving the data, and may be, for example, the second process. After obtaining the data sequence, the first process may send the data sequence to the recipient. The transmission may be through interprocess communication. For example, a socket-based interprocess communication connection may be established, and the data sequence may be added to a data sending buffer corresponding to the interprocess communication connection, so that the data sequence may be sent to the second process by interprocess communication. The interprocess communication means may be, for example, a UDS means.

According to the data transmission method, when the target data needs to be transmitted, the description data corresponding to the target data can be generated, the description data comprises the initial description information and the termination description information, a plurality of data sets are obtained through segmentation according to the description data and the target data, and a data sequence is formed, so that the data transmission method is convenient to transmit, the initial description information is located at the head of the first data set, and the termination description information is located at the tail of the last data set, so that even if the plurality of data sets are split into a plurality of data packets to be transmitted or are mixed with other data to be transmitted during transmission, a receiving party can obtain complete and independent information according to the initial description information and the termination description information, and the receiving party can receive the transmitted data more conveniently and accurately.

In some embodiments, in step S206, the multiple data sets are obtained by segmenting according to the description data and the target data, and the forming the data sequence includes: splicing the description data and the target data to obtain spliced data, wherein the head part of the spliced data and the termination description information are positioned at the tail part of the spliced data; and segmenting the spliced data to obtain multiple data to form a data sequence.

Specifically, the concatenation data is data obtained by concatenating the description data and the target data. In the concatenated data, the start description information is located before the target data, and the end description information is located after the target data, and in the concatenated data, the start description information and the end description information may be adjacent to the target data or may not be adjacent to each other. For example, if the start description information is a, the target data is b, and the end description information is c, the splicing data may be abc. For another example, if the start description information is a, the target data is b, the end description information is c, and the description data further includes a data type, which is denoted as d, then the concatenation data may be adbc. After the spliced data is obtained, the spliced data can be segmented, and the spliced data is segmented into a plurality of data to form a data sequence. For example, if the start description information is "a 1a2a 3", the target data is "b 1b2b3b 4", and the end description information is "c 1c 2", the spliced data is "a 1a2a3b1b2b3b4c1c 2", and the split data may be three data sets, i.e., "a 1a2a3b 1", "b 2b 3", and "b 4c1c 2". The first data is "a 1a2a3b 1", and the second data is "b 4c1c 2".

In some embodiments, after the splicing data is obtained by splicing, the splicing data can be serialized. Serialization processing refers to the conversion of object data into a byte stream that can be stored or transmitted. For example, the concatenated data is converted into data in binary representation. Deserialization refers to the conversion of a byte stream into object data. For example, assuming that the data is "20577", it is represented as "0101000001100001" after being serialized into binary data.

In some embodiments, the description data may further include data length information corresponding to the target data, and the splicing the description data and the target data to obtain spliced data includes: and splicing the initial description information, the termination description information, the data length information and the target data according to a preset rule to obtain spliced data, wherein the data length information is positioned in front of the target data.

Specifically, the data length information is used to indicate the length of the target data. For example, assuming that the target data occupies 20577 bytes, the data length information may be 20577. The preset rule sets the corresponding positions of the data during splicing, and the preset rule comprises the following steps: the start description information is located at the head of the splicing data, the end description information is located at the tail of the splicing data, and the data length information is located before the target data. In the embodiment of the application, the data length information is added to the spliced data and is positioned in front of the target data, so that the receiving party can skip the target data according to the length of the target data and analyze whether the termination description information exists, the received data does not need to be analyzed in sequence, whether the termination description information exists is judged, and the data analysis efficiency is improved.

For example, if there is no data length information in the concatenated data, assuming that the concatenated data is "a 1a2a3b1b2b3b4c1c 2", i.e. there is no data length information in the concatenated data, after the start description information "a 1a2a 3" is identified, the second process needs to compare "b 1b 2" with the end description information to determine whether the concatenated data is a preset end character string "c 1c 2". If not, the step of comparing … … the obtained "b 2b 3" with the termination description information is continued until "c 1c 2" in the spliced data is obtained, and it cannot be determined that the received data includes the termination description information, but the data size of the target data is generally large, so that the identification time for identifying the termination description information is long, and the efficiency is low. On the other hand, if the concatenation data is "a 1a2a3eb1b2b3b4c1c 2", e represents the length of the target data, for example, 20577 ", after the start description information is obtained by identification, e, that is, the data length information, can be read, the second process can directly skip 20577 bytes of data to obtain" c1c2 ", and it can be determined that the end description information exists, so that the time for obtaining the end description information is reduced.

In some embodiments, the number of bytes occupied by the data length information may be preset. For example, it may be 4 bytes. In this way, when the data length information is analyzed, data of a corresponding byte length can be acquired as the data length information based on the data length information.

In some embodiments, the description data further includes service related information corresponding to the target data, and the splicing the description data and the target data to obtain spliced data includes: and splicing the initial description information, the service related information, the target data and the termination description information according to a preset rule to obtain spliced data, wherein the service related information is positioned in front of the target data.

Specifically, the service-related information is information related to a service corresponding to the target data. For example, one or more of the type of data, the type of traffic, or critical fields of the target data may be included. By adding the service related information into the spliced data and positioning the service related information in front of the target data, when a receiver receives the data, the receiver can determine the service related information related to the received target data before reading the target data, so as to obtain a data processing mode of the target data, and the target data is processed according to the data processing mode. For example, it is assumed that the service-related information includes a data type, if the data type is a face recognition picture, the data processing mode is to perform face recognition, and if the data type is a face recognition result, the data processing mode is to store. When the type of the target data read from the received data by the second thread is a face recognition result, the serialized face recognition result can be directly stored without performing deserialization processing on the serialized face recognition result. Therefore, under the scenes of various services and various data types, basic information of data, such as the data types, can be obtained by deserializing service related information, so that the data processing efficiency is improved.

In some embodiments, the splicing data may further include length information of the service related information, which is used to describe the length of the service related information, so that the service related information of the corresponding length may be obtained according to the length information of the service related information.

In some embodiments, the number of bytes occupied by the length information of the service-related information may be preset. For example, it may be 4 bytes. Therefore, when analyzing the length information of the service related information, the data of the corresponding byte can be acquired according to the number of bytes occupied as the length information of the service related information.

In some embodiments, the splicing data includes start description information, length information of the service-related information, data length information corresponding to the target data, the service-related information, the target data, and termination description information. And sequentially splicing the initial description information, the length information of the service related information, the data length information corresponding to the target data, the service related information, the target data and the termination description information according to the sequence to obtain spliced data. Fig. 3 is a schematic diagram of a data format of the concatenated data in some embodiments. The service related information is stored in a header (header), and the header length information may be length information corresponding to the service related information, and of course, the header may also include other information, such as a timestamp and a data identifier of the target data. The start symbol indicates start description information, the data body indicates target data, and the end symbol indicates end description information. The number of bytes occupied by the start symbol, the header length information, the body length information, and the end symbol may be preset, for example, 1 byte occupied by the start symbol and the end symbol, and 4 bytes occupied by the header length information and the body length information.

Fig. 4 is a schematic diagram of the concatenated data in some embodiments. In fig. 4, "[" indicates start description information, and occupies 1 byte. "0000000000001010" is header length information indicating that the header length is 10 bytes, and "0101000001100001" is data length information of the target data indicating that the target data length is 20577 bytes.

In some embodiments, before sending data, a socket-based interprocess communication connection may be established with a recipient process, and multiple pieces of data are obtained by splitting according to description data and target data, and forming a data sequence includes: acquiring data residual space of a data sending buffer corresponding to interprocess communication connection; and segmenting according to the description data and the target data to obtain first data, wherein the length of the first data is less than or equal to the capacity of the data residual space, so that the first data is added into the data sending buffer.

Specifically, the socket-based inter-process communication connection means that the communication connection is established based on a socket (socket) and is used for inter-process communication. Inter-Process Communication (IPC) refers to the transfer of data or signals between at least two processes or threads. When establishing a socket-based interprocess communication connection, the sender may apply for a data send buffer for sending data. The data transmission buffer is used for buffering data to be transmitted, the data in the data transmission buffer are sequentially transmitted according to a sequence, and the transmission can be a byte stream. The data remaining space is the size of the remaining capacity of the data transmission buffer. For the buffer area of the socket connection, under a general condition, the data can be sent when the buffer area is full, so that when the remaining space of the data is not enough to accommodate the description data and the target data, part of the data can be segmented to obtain first data, and the first data is sent to the data sending buffer area, so that the data can be sent to a receiving party in time, and the sending efficiency is improved.

It can be understood that after the first data is obtained by segmentation, the remaining space of the data transmission buffer area can be continuously monitored, and if the size of the remaining space of the data buffer area exceeds the preset length, the next data can be segmented and added into the data buffer area. The step of monitoring the size of the sending buffer area can be repeated, if the size of the residual space of the data buffer area exceeds the preset length, the next data is cut, and the next data is added into the data buffer area until the sending of the splicing data is completed. The preset length may be set as desired, for example, 0.3 megabytes. When the size of the residual space of the data buffer area exceeds the preset length, the next data is cut, so that the data can be sent to a receiver in time, and frequent data cutting is avoided.

As shown in fig. 5, in some embodiments, a data receiving method is proposed, and this embodiment is mainly exemplified by applying the method to the second process 120 in fig. 1. The method specifically comprises the following steps:

step S502, receiving data sent by a sender to obtain received data, wherein the received data comprises a data sequence formed by a plurality of data shares.

Specifically, the sender refers to a party that sends data, and may be, for example, a first process. The received data may include a data sequence formed of a plurality of copies of the data. For example, the received data may include three pieces of data, i.e., "a 1a2a3b 1", "b 2b 3", and "b 4c1c 2", and may also include data corresponding to other services. For example, data of a plurality of services are stuck together during transmission, and the received data may include "a 1a2a3b 1", "b 2b 3", "b 4c1c 2", and "h 1h2h 3". The received data may also be received in multiple portions. For example, the first time "a 1a2a3b 1" is received, only the start description information indicates that the complete data was not received, and the second process may continue to wait until "b 4c1c 2" is received, including the stop description information.

Step S504, analyzing the received data, and determining to receive complete data when the initial description information and the termination description information exist in the received data obtained by analyzing.

Specifically, after the received data is obtained, the receiver may analyze the data, and determine whether the received data includes the start description information and the end description information, for example, may determine whether the received data includes a preset start character string and an end character string. When it is determined that the start description information and the end description information exist in the received data, it is determined that the complete data is received, and thus it is possible to start the step of extracting the complete target data.

And step S506, splicing the data in the data sequence according to the position of the initial description information and the position of the termination description information to obtain complete target data.

Specifically, after the position of the start description information and the position of the end description information are obtained, the data of the data sequence after the position of the start description information and before the position of the end description information may be spliced to obtain complete target data.

According to the data receiving method, when data is received, the received data is analyzed, when initial description information and termination description information exist in the received data obtained through analysis, the received complete data is determined, and the data in the data sequence is spliced according to the position of the initial description information and the position of the termination description information, so that complete target data is obtained. Therefore, even if the data is split into a plurality of data packets or is mixed with other data to be sent during sending, the receiving party can acquire complete and independent information according to the positions of the initial description information and the termination description information, and convenience and accuracy are achieved.

In some embodiments, as shown in fig. 6, parsing the start description information and the end description information from the received data includes:

step S602, analyzing the received data to obtain the initial description information.

Specifically, after the received data is obtained, the receiving party, for example, the second process may analyze the received data, for example, determine whether the received data includes a preset start character string, and if so, analyze the start description information.

Step S604, determining a first reading position according to the position of the start description information, and reading data length information from the received data according to the first reading position.

Specifically, the first reading position refers to a position of data length information. When the receiver splices the data, the data length information and the initial description information are spliced according to a preset rule, and the relative positions are fixed, so that the position of the data length information can be determined according to the position of the initial description information. For example, if the data length information is located after the start description information in the preset rule, after the start description information is obtained through parsing, the data of the preset byte after the start description information may be determined to be the data length information.

Step S606, according to the data length information, skipping the data with the corresponding length in the received data, and reading the termination description information.

Specifically, after the data length information is obtained, since the termination description information is located after the target data, the data of the corresponding length can be skipped according to the data length information, and the obtained termination description information can be read. As an actual example, assuming that the concatenation data is "a 1a2a3eb1b2b3b4c1c 2", e represents the length of the target data, for example, 20577, after the start description information is identified, e, i.e., the data length information, can be read, the second process may directly skip 20577 bytes of data, i.e., skip "b 1b2b3b 4", obtain "c 1c 2", determine that the termination description information exists, and therefore reduce the time for obtaining the termination description information.

In some embodiments, the target data is serialized data, and the data transmission method further includes: extracting service related information corresponding to the target data from the received data; determining a data processing module corresponding to the target data according to the service related information; and sending the serialized target data to the corresponding data processing module.

Specifically, the data processing manner may be, for example, multiple manners such as uploading a picture, storing a result, filtering data, or beautifying a picture, and the data processing modules corresponding to different data processing manners may be different according to specific business determination. The corresponding relationship between the service-related information and the data processing module may be stored in advance, for example, the data processing module corresponding to each data type is stored, and after the service-related information is analyzed from the received data, the corresponding data processing module may be determined, the serialized target data is directly obtained, and the serialized data is sent to the corresponding data processing module, that is, the data may be processed without performing deserialization on the data. The resource of the receiver is saved, and the data processing efficiency is improved.

For example, as shown in fig. 7, the application program for receiving data may include a receiving module and N data processing modules, where N is greater than or equal to 2. The receiving module in the application can receive the data, the corresponding data processing module is determined according to the service related information, and the receiving module directly distributes the received serialized target data to the corresponding data processing module. Therefore, the service scene with the data distribution requirement is greatly facilitated through the service related information. For example, if data can be divided into N types according to types, data modules corresponding to different types of data are different, target data may reach several megabits, tens of megabits or even hundreds of megabits, if data itself is deserialized and then a data processing mode is determined each time, time consumption may reach tens of milliseconds or even hundreds of milliseconds, and service related information is often only tens of kb (kilobyte), so that the deserialization of the part of content only needs several milliseconds or even less, and a corresponding data processing module can be determined according to the deserialization service related information, so that deserialization time is reduced, processing speed of a receiving module is increased, and data throughput of a data receiving system is increased.

In some embodiments, when the receiving side sends data, the data is spliced according to a preset rule to obtain spliced data. Therefore, the sender can also analyze the received data according to the preset rule, that is, the analyzing process is the reverse process of the splicing and the segmentation of the sender. For example, when the start descriptor, the length information of the service-related information, the data length information corresponding to the target data, the service-related information, the target data, and the end descriptor are sequentially spliced in the receiving side, the sending side may read the start symbol, and then deserialize the data header length information to obtain the length len of the data header, with the character string with a preset number of bits after the start symbol as the data header length information_hThen, after the data header length information, the character string of the preset bit is used as the data length information, and the length len of the target data is obtained by deserialization_b. Then, the length is len after the data length information_hThe data of (2) is used as a data header and deserialized. After which len can be skipped_bAnd acquiring data with a preset byte length by using the data of the bits to obtain a terminator. Therefore, the receiver can obtain complete data and quickly acquire the service information related to the target data by deserializing the initial description information, the termination description information and the service related information.

The method provided by the embodiment of the present application will be described below by taking as an example the case where data is transmitted and received in the UDS inter-process communication connection by using the data transmission method and the data reception method provided by the embodiment of the present application. The method comprises the following steps:

1. and the first process receives the data sending request and establishes UDS connection with the second process according to the data sending request.

Specifically, the UDS is a mode developed on a socket for inter-process communication of the same device, and can copy application layer data from one process to another process through a network protocol stack without the processes of packing and unpacking, calculating a checksum, maintaining a sequence number, responding and the like. UDS is a common inter-process communication method in linux System, which is a UNIX (unified information and Computing System, U.S.) operating System. A UDS connection is established between the first process and the second process.

2. And the first process acquires target data corresponding to the target service as target data to be sent.

For example, if the target service is to distribute picture data, a picture to be distributed is taken as data to be transmitted.

3. The first process generates description data corresponding to the target data.

For example, the first process generates the description data after obtaining the target data. The description data may include start description information, length information of the service related information, data length information corresponding to the target data, service related information, and termination description information.

4. And the first process splices the description data and the target data to obtain spliced data.

For example, the first process sequentially splices the start description information, the length information of the service related information, the data length information corresponding to the target data, the service related information, the target data, and the end description information in sequence to obtain spliced data.

5. The first process divides the spliced data to obtain multiple data and form a data sequence.

For example, assuming that the data obtained by splicing is "abcdefghijk" and the capacity of the data transmission buffer is 1000 character strings, the "abcdefghijk" may be divided into three data sets, i.e., "abc", "defg", and "hijk", where a denotes a start symbol and k denotes a termination symbol.

6. The first process sends the data sequence to the second process through the UDS connection.

7. And the second process receives the data sent by the first process.

For example, the data received by the second process may include "xxx", "abc", "defg", and "hijk".

8. The second process analyzes whether the received data has the starting description information and the ending description information.

For example, when the second resolution is that the received data includes the start character a and the end character k, it is determined that the complete data is received.

9. And the second process splices the data in the data sequence according to the position of the initial description information and the position of the termination description information to obtain complete target data.

For example, the second process concatenates "c", "defg", and "hij", to obtain the completed target data "cdefghij".

The data receiving method and the device corresponding to the data sending method can be deployed on equipment with a Linux operating system and service requirements for interprocess communication, and the interprocess communication mode can also be a Transmission Control Protocol (TCP) or a pipeline mode. When the UDS is adopted to connect and send data, when the data is sent through the UDS, the condition that the data sent for many times are bonded together often exists, or the condition that the data sent for one time is too large and is read by a receiving party for many times, so that the received data is incomplete is caused.

For example, as shown in fig. 8, when the sender needs to send target data, description information, such as a start string, denoted as "0 x 28", and a stop string, denoted as "0 x 29", may be acquired. And then splicing the description data and the target data into spliced data according to a preset sending data format, sending the spliced data to a receiving party through a USD communication pipeline, and after receiving the data sent according to the preset sending data format, reading the received data to obtain information such as a starting character string, a terminating character string, a data head length, a data head and the like.

Table 1:

in some embodiments, the algorithm executed by the sender when sending the target data may be represented as algorithm 1, and the name of the algorithm is "cooludssend", as shown in table 1. The input data includes fd, head data (head), and body data (body). In the embodiment of the present application, the service-related information is referred to as header data. fd denotes an instance handle of unix domain socket. Buffer represents a Buffer area, nullptr represents a null pointer, head _ len represents the length of a data header, body _ len represents the length of a data body, start _ sig represents start description information, and end _ sign represents end description information. Pack indicates that the data is obtained by splicing. Min represents taking the minimum value, and MAX _ SOKET _ BUFFER _ SIZE represents the maximum length of the data BUFFER. send _ size indicates the length of the transmission data.

In some embodiments, the algorithm executed by the receiving party at the time of receiving may be represented as algorithm 2, which is the inverse process of algorithm 1 and is named "pool uds receive" as shown in table 2. The input of the algorithm 2 is the instance handle of unix domain socket, and the output is the head data and the text data. Wherein the ReadDataFromUdsBuffer functions of lines 2 and 8 are shown in table 3.

START _ SIGN _ LEN indicates the length of START description information, HEAD _ LEN indicates the length of header length information, BODY _ LEN indicates the length of length information of a data BODY, END _ SIGN _ LEN indicates the length of termination description information, bytetoxint indicates that data stored in byte form is deserialized, for example, deserialized data stored in binary. As can be seen from table 2, line 6 shows reading a binary string of the start symbol, the header length information, and the data body length information, and deserializing the start symbol start _ sign, the header length value head _ len, and the data body length value body _ len. Line 12 shows reading binary strings of data headers, data bodies and terminators and listing the data headers and terminators in reverse order, as for the data bodies, whether or not deserialization is to be performed can be selected according to specific business requirements.

Therefore, when the receiver receives data, it first reads the previous START _ SIGN _ LEN bit data, and determines whether to parse to obtain the START symbol, i.e. the START description information. Reading the data of the HEAD _ LEN _ LEN bits to obtain the length of a data HEAD, and deserializing to obtain a numerical value LEN_hThen, the subsequent BODY _ LEN bit data is read to obtain the data text length LEN_b. Since the data head length is len_hThus, len after data body length information can be determined_hBit data is a data header, and service-related information, such as data type, can be obtained through serialization. Since it is necessary to determine whether there is termination description information, the data body length is len_b. So len can be skipped_bAnd bit data, reading the data of the END _ SIGN _ LEN bit byte according to the END _ SIGN _ LEN, and determining whether the bit data is termination description information. When it is determined that the termination description information is obtained, it may be determined whether or not to include a len preceding the start description information according to the service-related information_bThe data is deserialized, that is, whether the target data is serialized or not is determined, and since the data text is often very large in many scenes, the time consumption of deserialization is relatively large, and the specific content of the data is not required in the receiving module, the data can be delayed to be deserialized in the subsequent processing module.

Table 2:

TABLE 3

As shown in fig. 9, in some embodiments, a data transmission apparatus is provided, which may be integrated in the above-mentioned computer device, for example, a terminal or a server, and specifically may include a target data obtaining module 902, a description data generating module 904, a dividing module 906, and a data sequence transmitting module 908.

A target data obtaining module 902, configured to obtain target data to be sent.

And a description data generating module 904, configured to generate description data corresponding to the target data, where the description data includes start description information and end description information.

The segmentation module 906 is configured to segment multiple data according to the description data and the target data to form a data sequence, where the data sequence includes first data and second data, the start description information is located at a head of the first data, the end description information is located at a tail of the second data, the first data is head data of the data sequence, and the second data is tail data of the data sequence.

A data sequence sending module 908 for sending the data sequence to the receiving side.

In some embodiments, the singulation module 906 includes:

and the splicing unit is used for splicing the description data and the target data to obtain spliced data, wherein the starting description information is positioned at the head of the spliced data, and the ending description information is positioned at the tail of the spliced data.

And the segmentation unit is used for segmenting the spliced data to obtain multiple data and form a data sequence.

In some embodiments, the description data includes service related information corresponding to the target data, and the splicing unit is configured to splice the start description information, the service related information, the target data, and the termination description information according to a preset rule to obtain spliced data, where the service related information is located before the target data.

In some embodiments, the slicing unit is to: and when the data volume of the spliced data is larger than the size of the data sending buffer area, segmenting the spliced data according to the size of the data sending buffer area to obtain multiple data to form a data sequence.

In some embodiments, the data transmission apparatus further comprises: and the communication connection establishing module is used for establishing the inter-process communication connection based on the socket with the process of the receiving party.

The slicing module 906 is configured to: acquiring data residual space of a data sending buffer corresponding to interprocess communication connection; and segmenting according to the description data and the target data to obtain first data, wherein the length of the first data is less than or equal to the capacity of the data residual space, so that the first data is added into the data sending buffer.

As shown in fig. 10, in some embodiments, a data receiving apparatus is provided, which may be integrated in the above-mentioned computer device, for example, a terminal or a server, and specifically may include a data receiving module 1002, a parsing module 1004, and a target data obtaining module 1006.

The data receiving module 1002 is configured to receive data sent by a sender to obtain received data, where the received data includes a data sequence formed by multiple pieces of data.

The parsing module 1004 is configured to parse the received data, and determine that complete data is received when the received data obtained through parsing has the start description information and the end description information.

And a target data obtaining module 1006, configured to splice data in the data sequence according to the position of the start description information and the position of the end description information to obtain complete target data.

In some embodiments, the parsing module is to: and analyzing the received data to obtain initial description information. Determining a first reading position according to the position of the initial description information, and reading data length information from the received data according to the first reading position; and skipping the data with the corresponding length in the received data according to the data length information, and reading to obtain the termination description information.

In some embodiments, the target data is data obtained after serialization, and the data receiving apparatus further includes:

and the service related information extraction module is used for extracting the service related information corresponding to the target data from the received data.

And the data processing determining module is used for determining the data processing module corresponding to the target data according to the service related information.

And the target data sending module is used for sending the serialized target data to the corresponding data processing module.

FIG. 11 illustrates an internal block diagram of a computer device in some embodiments. The computer device may specifically be a terminal or a server. As shown in fig. 11, the computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program that, when executed by the processor, causes the processor to implement at least one of a data transmission method or a data reception method. The internal memory may also have a computer program stored therein, which when executed by the processor, causes the processor to perform at least one of the data transmission method or the data reception method.

Those skilled in the art will appreciate that the architecture shown in fig. 11 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In some embodiments, the data transmission apparatus provided herein may be implemented in the form of a computer program that is executable on a computer device such as that shown in fig. 11. The memory of the computer device may store various program modules constituting the data transmission apparatus or the data reception apparatus, such as a target data acquisition module 902, a description data generation module 904, a segmentation module 906, and a data sequence transmission module 908 shown in fig. 9. The computer program constituted by the respective program modules causes the processor to execute the steps in the data transmission method of the respective embodiments of the present application described in the present specification.

For example, the computer device shown in fig. 11 can acquire target data to be transmitted through the target data acquisition module 902 in the data transmission apparatus shown in fig. 9. Description data corresponding to the target data is generated by the description data generation module 904, and the description data includes start description information and end description information. The data sequence is composed by the segmentation module 906 according to the description data and the target data, the data sequence includes a first data and a second data, the start description information is located at the head of the first data, the stop description information is located at the tail of the second data, the first data is the head data in the data sequence, and the second data is the tail data in the data sequence. The data sequence is transmitted to the receiving side through the data sequence transmitting module 908.

In some embodiments, a computer device is provided, comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of the data transmission method described above. Here, the steps of the data transmission method may be steps in at least one of the data transmission method or the data reception method of the above-described respective embodiments.

In some embodiments, a computer-readable storage medium is provided, in which a computer program is stored, which, when executed by a processor, causes the processor to perform the steps of the above-described data transmission method. Here, the steps of the data transmission method may be steps of at least one of the data transmission method and the data reception method of the above-described embodiments.

It should be understood that, although the steps in the flowcharts of the embodiments of the present application are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in various embodiments may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware related to instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, the computer program can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A method of data transmission, the method comprising:

acquiring target data to be sent;

generating description data corresponding to the target data, wherein the description data comprises starting description information and ending description information;

splicing the initial description information, the length information of the service related information, the data length information of the target data, the service related information, the target data and the termination description information in sequence to obtain spliced data, wherein the initial description information is positioned at the head of the spliced data, and the termination description information is positioned at the tail of the spliced data;

segmenting the spliced data to obtain multiple data to form a data sequence, wherein the data sequence comprises first data and second data, the starting description information is located at the head of the first data, the ending description information is located at the tail of the second data, the first data is the head data of the data sequence, and the second data is the tail data of the data sequence;

sending the data sequence to a receiving party so that the receiving party performs the following steps:

analyzing to obtain the initial description information in the data sequence, taking a character string with a preset number of bits behind the initial description information as length information of the service related information, and deserializing the length information of the service related information to obtain the length of the service related information;

taking a character string with preset digits after the length information of the service related information as the data length information of the target data, and deserializing the data length information of the target data to obtain the length of the target data;

after the data length information of the target data, the data of the length of the service related information is used as the service related information, and deserialization is carried out;

skipping the data with the length of the target data, and acquiring the data with the preset byte length as the termination description information;

when the termination description information is determined to be obtained, determining whether to perform deserialization on the target data according to the service related information obtained by deserialization;

and if not, sending the serialized target data to a corresponding data processing module for processing.

2. The method of claim 1, wherein the segmenting the concatenated data to obtain multiple data, and the forming a data sequence comprises:

and when the data volume of the spliced data is larger than the size of a data sending buffer area, segmenting the spliced data according to the size of the data sending buffer area to obtain multiple data, and forming a data sequence.

3. The method of claim 1, further comprising:

adding the sorting information in the data sequence to the segmented data.

4. The method of claim 1, further comprising:

establishing a socket-based interprocess communication connection with a receiver process;

the splicing data is segmented to obtain multiple data, and the data sequence is composed of:

acquiring the data residual space of a data sending buffer corresponding to the interprocess communication connection;

and segmenting the spliced data to obtain the first data, wherein the length of the first data is less than or equal to the capacity of the data residual space, so that the first data is added into the data sending buffer area.

5. The method of claim 4, wherein the segmenting the concatenated data to obtain multiple data, and the forming a data sequence comprises:

and after the first data is obtained by segmentation, when the data residual space of the data sending buffer area is detected to exceed a preset length, the spliced data is continuously segmented.

6. The method of claim 1, wherein the service-related information comprises one or more of a type of data, a type of service, or a key field of target data.

7. A method of data reception, the method comprising:

receiving data sent by a sender to obtain received data, wherein the received data comprises a data sequence formed by multiple data shares, the data sequence is obtained by segmenting splicing data, and starting description information of target data, length information of service related information, data length information of the target data, the service related information, the target data and termination description information of the target data are spliced in sequence to obtain the spliced data;

8. The method of claim 7, wherein sending the serialized target data to a corresponding data processing module for processing comprises:

determining a data processing module corresponding to the target data according to the service related information;

and sending the serialized target data to a corresponding data processing module.

9. The method of claim 7, wherein the service-related information comprises one or more of a type of data, a type of service, or a key field of target data.

10. A data transmission apparatus, the apparatus comprising:

the target data acquisition module is used for acquiring target data to be sent;

the description data generation module is used for generating description data corresponding to the target data, and the description data comprises starting description information and ending description information;

a segmentation module, configured to sequentially splice the start description information, length information of service-related information, data length information of the target data, the service-related information, the target data, and the termination description information to obtain spliced data, where the start description information is located at a head of the spliced data, and the termination description information is located at a tail of the spliced data; segmenting the spliced data to obtain multiple data to form a data sequence, wherein the data sequence comprises first data and second data, the starting description information is located at the head of the first data, the ending description information is located at the tail of the second data, the first data is the head data of the data sequence, and the second data is the tail data of the data sequence;

a data sequence sending module, configured to send the data sequence to a receiving party, so that the receiving party performs the following steps:

11. The apparatus of claim 10, wherein the slicing module is configured to:

12. The apparatus of claim 10, wherein the apparatus is further configured to:

adding the sorting information in the data sequence to the segmented data.

13. The apparatus of claim 10, further comprising:

the communication connection establishing module is used for establishing inter-process communication connection based on the socket with the process of the receiving party;

the slitting module is used for:

14. The apparatus of claim 13, wherein the slicing module is configured to:

15. The apparatus of claim 10, wherein the traffic-related information comprises one or more of a type of data, a type of traffic, or a key field of target data.

16. A data receiving apparatus, the apparatus comprising:

the data receiving module is used for receiving data sent by a sender to obtain received data, wherein the received data comprises a data sequence formed by a plurality of pieces of data, the data sequence is obtained by segmenting spliced data, and starting description information of target data, length information of service related information, data length information of the target data, the service related information, the target data and termination description information of the target data are spliced in sequence to obtain the spliced data;

the analysis module is used for analyzing and obtaining the initial description information in the data sequence, taking a character string with a preset number of bits behind the initial description information as the length information of the service related information, and deserializing the length information of the service related information to obtain the length of the service related information;

the apparatus is further configured to:

17. The apparatus of claim 16, wherein sending the serialized target data to a corresponding data processing module for processing comprises:

18. The apparatus of claim 16, wherein the traffic-related information comprises one or more of a type of data, a type of traffic, or a key field of target data.

19. A computer device comprising a memory and a processor, the memory having stored therein a computer program that, when executed by the processor, causes the processor to perform the steps of at least one of the data transmission method of any one of claims 1 to 6 or the data reception method of any one of claims 7 to 9.

20. A computer-readable storage medium, having a computer program stored thereon, which, when executed by a processor, causes the processor to perform the steps of at least one of the data transmission method of any one of claims 1 to 6 or the data reception method of any one of claims 7 to 9.