CN117692528A

CN117692528A - Data transmission method, device, computer equipment and storage medium

Info

Publication number: CN117692528A
Application number: CN202211069854.6A
Authority: CN
Inventors: 吴波
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2022-09-02
Filing date: 2022-09-02
Publication date: 2024-03-12

Abstract

The present application relates to a data transmission method, apparatus, computer device, storage medium and computer program product. The method comprises the following steps: acquiring target data comprising key data and non-key data; carrying out data message encapsulation on target data to generate a first type message stream carrying a first type stream identifier and a plurality of second type message streams respectively carrying different second type stream identifiers, wherein data carried by data messages in the first type message stream belong to key data, and data carried by data messages in the second type message stream belong to non-key data; transmitting each data message according to the first type message stream and the second type message stream, and receiving a confirmation response message aiming at the data message; and retransmitting key data carried by the lost data message in the first type message stream when the confirmation response message characterizes that the data message is lost and the lost data message belongs to the first type message stream. By adopting the method, the quality and the time efficiency of the transmitted data can be considered.

Description

Data transmission method, device, computer equipment and storage medium

Technical Field

The present application relates to the field of communications technologies, and in particular, to a data transmission method, apparatus, computer device, storage medium, and computer program product.

Background

With the development of network transmission technologies, existing network transmission technologies are generally divided into two types, one is based on TCP (Transport Control Protocol, transmission control protocol) transmission and the other is based on UDP (User Datagram Protocol ) transmission. The transmission technology based on TCP can ensure the reliability of the service, namely if packet loss occurs, the transmitting end can retransmit the lost message, the integrity of the data of the receiving end is ensured, but phenomena such as delay and blocking occur easily, the transmission technology based on UDP can furthest meet the real-time performance of the audio and video service, and the real-time performance is replaced by the cost of a small amount of data.

However, both TCP-based and UDP-based transmission do not allow for both quality and timeliness issues of data transmission.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a data transmission method, apparatus, computer device, computer readable storage medium, and computer program product that can achieve both quality and timeliness of data transmission.

In a first aspect, the present application provides a data transmission method. The method comprises the following steps:

acquiring target data comprising key data and non-key data;

Carrying out data message encapsulation on the target data to generate a first type message stream carrying a first type stream identifier and a plurality of second type message streams respectively carrying different second type stream identifiers, wherein data carried by data messages in the first type message stream belong to the key data, and data carried by data messages in the second type message stream belong to the non-key data;

transmitting each data message according to the first type message stream and the second type message stream, and receiving a confirmation response message aiming at the data message;

and retransmitting key data carried by the lost data message in the first type message stream when the confirmation response message characterizes that the data message is lost and the lost data message belongs to the first type message stream.

In a second aspect, the present application further provides a data transmission device. The device comprises:

the data acquisition module is used for acquiring target data containing key data and non-key data;

the message flow generating module is used for carrying out data message encapsulation on the target data to generate a first type message flow carrying a first type flow identifier and a plurality of second type message flows respectively carrying different second type flow identifiers, wherein data carried by data messages in the first type message flow belong to the key data, and data carried by data messages in the second type message flow belong to the non-key data;

The message receiving and transmitting module is used for sending each data message according to the first type message stream and the second type message stream and receiving a confirmation response message aiming at the data message;

and the key data retransmission module is used for retransmitting key data carried by the lost data message in the first type message stream when the confirmation response message represents that the data message is lost and the lost data message belongs to the first type message stream.

In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor which when executing the computer program performs the steps of:

acquiring target data comprising key data and non-key data;

In a fourth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:

acquiring target data comprising key data and non-key data;

In a fifth aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of:

acquiring target data comprising key data and non-key data;

According to the data transmission method, the device, the computer equipment, the storage medium and the computer program product, the target data containing the key data and the non-key data are obtained, the data message package is carried out on the target data, the first type message stream for transmitting the key data and the plurality of second type message streams for transmitting the non-key data are generated, the first type message stream carries the first type stream identification, the plurality of second type message streams respectively carry different second type stream identifications, the key data and the non-key data can be sent in a shunting mode through different stream identifications, meanwhile, the non-key data can be scattered in the sending process, and when the lost data message belongs to the first type message stream, the key data carried by the lost data message in the first type message stream are retransmitted, so that the key data can be retransmitted timely, reliable transmission of the key data is ensured, and the condition that the receiving end of the data message is lost can be effectively reduced based on the fact that the non-key data is scattered by the plurality of second type message streams, the condition that the receiving end of the data message is blocked is favorable for realizing high-efficiency transmission of the non-key data, and further, the quality of the target data is guaranteed, and the quality of the time-efficient transmission is guaranteed, and the quality of the target data is guaranteed.

Drawings

FIG. 1 is a diagram of an application environment for a data transmission method in one embodiment;

FIG. 2 is a diagram of an application environment of a data transmission method in another embodiment;

FIG. 3 is a flow chart of a data transmission method in one embodiment;

FIG. 4 is a schematic diagram of an interaction process for acquiring target data in one embodiment;

FIG. 5 is a schematic diagram of an interaction process for acquiring target data according to another embodiment;

FIG. 6 is a diagram of data messages belonging to different message flows in one embodiment;

FIG. 7 is a schematic diagram illustrating a sequence of received data messages according to one embodiment;

FIG. 8 is a schematic diagram of the structure of a data packet in one embodiment;

FIG. 9 is a diagram of acquiring target data and description fields from an application layer in one embodiment;

FIG. 10 is a diagram of acquiring target data and description fields from a CDN server in one embodiment;

FIG. 11 is a schematic diagram of the impact range of video frame loss for different categories of video data in one embodiment;

FIG. 12 is a flow chart of reporting data in a data packet with a loss in one embodiment;

FIG. 13 is a flow chart of a data transmission method in another embodiment;

FIG. 14 is a flow chart of reporting data after receiving a retransmitted data packet in one embodiment;

FIG. 15 is a block diagram of a data transmission device in one embodiment;

fig. 16 is an internal structural view of a computer device in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.

Cloud technology (Cloud technology) refers to a hosting technology for integrating hardware, software, network and other series resources in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. Cloud technology (Cloud technology) is based on the general terms of network technology, information technology, integration technology, management platform technology, application technology and the like applied by Cloud computing business models, and can form a resource pool, so that the Cloud computing business model is flexible and convenient as required. Cloud computing technology will become an important support. Background services of technical networking systems require a large amount of computing, storage resources, such as video websites, picture-like websites, and more portals. Along with the high development and application of the internet industry, each article possibly has an own identification mark in the future, the identification mark needs to be transmitted to a background system for logic processing, data with different levels can be processed separately, and various industry data needs strong system rear shield support and can be realized only through cloud computing.

In recent years, the network live broadcast industry has rapidly developed, and audio and video service flows still become important components of internet traffic. The audio and video services have various expression forms, such as real-time audio and video call, audio and video live broadcast service, audio and video on demand service, network game, virtual reality, simultaneous interpretation, "meta universe" and the like, each expression form has own specific service requirement, and research on network transmission technology adapting to the internet audio and video service becomes a research hotspot in recent years.

The existing network transmission technology facing the audio and video service is wholly divided into two types: one is based on TCP transmission and one is based on UDP transmission. The transmission technology based on TCP can ensure the reliability of the service, namely if packet loss occurs, the transmitting end can retransmit the lost message, the integrity of the data of the receiving end is ensured, and from a great amount of researches, the method cannot well utilize the available bandwidth resources in the network, so that the transmission efficiency of the audio and video traffic is not high, and the phenomena of delay, blocking and the like of the audio and video service of the client are caused. In contrast, the transmission technology based on UDP can maximally meet the real-time performance of audio and video services, if the packet loss phenomenon occurs, retransmission or selective retransmission is not performed, the real-time performance is replaced by the cost of a small amount of video quality, and the typical application is real-time audio and video call. However, TCP or UDP based techniques do not adequately adapt to the important characteristics of the audio-video service, i.e. tolerance to lossy transmission, i.e. loss of a small number of messages has limited impact on the quality of the audio-video service, even the user side cannot perceive changes in the quality of the audio-video. In addition, the audio and video service has different packet loss tolerance to different audio and video contents, namely, different audio and video frames have different influences or different importance degrees on the watching experience of the user side.

The embodiment of the application provides a data transmission method which can be applied to an application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The data storage system may store target data that the server 104 needs to send. The data storage system may be integrated on the server 104 or may be located on the cloud or other servers. The server 104 responds to the target data sending event, acquires target data containing key data and non-key data, packages the target data into a first type message stream for transmitting the key data and a plurality of second type message streams for transmitting the non-key data, then sends each data message according to the first type message stream and the second type message stream, receives a confirmation response message aiming at the data message to the terminal 102, feeds back a confirmation application message aiming at the data message to the server 104 after the terminal 102 receives the data message, and when the fact that the confirmation response message represents that the data message is lost and the lost data message belongs to the first type message stream is detected, retransmits the key data carried by the lost data message in the first type message stream to the terminal 102.

For the terminal 102, the terminal 102 sorts the received data messages according to the internal sequence of the flow identifier of the message flow, and sends the sorted complete data to the application layer, i.e. if the data messages in the same message flow are lost, only the data messages which are not blocked can be sent to the application layer. For the case that the lost data message belongs to the first type of message flow, the server can retransmit the key data carried in the lost data message in time so as to ensure that the key data can be normally sent to the application layer player. For the case that the lost data message belongs to the second class message stream, because the data content carried by each data message in the second class message stream is non-critical data and the non-critical data is scattered and sent through a plurality of second class messages, the data message loss occurs in one second class message stream, and the reporting of the subsequent data messages in the second class message stream is only affected, but the other second class message streams are not affected. The server 104 only needs to retransmit the key data, so that timeliness of data transmission can be ensured. And the terminal 102 is ensured to receive each key data and most of non-key data, so that the data quality is ensured, the transmission quality of the target data is ensured, the transmission timeliness of the target data is ensured, and the data transmission effect of both quality and timeliness is achieved.

The terminal 102 may be, but not limited to, various desktop computers, notebook computers, smart phones, tablet computers, internet of things devices, and portable wearable devices, where the internet of things devices may be smart speakers, smart televisions, smart air conditioners, smart vehicle devices, and the like. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, basic cloud computing services such as big data and artificial intelligent platforms. The terminal may be, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc. The terminal and the server may be directly or indirectly connected through wired or wireless communication, which is not limited herein.

As shown in fig. 2, the application scenario involved in the present application includes audio-video interaction between the cloud server 202 and the user terminal 204, including but not limited to real-time audio-video communication, audio-video on demand, and audio-video live broadcast. Specifically, after receiving the audio-video traffic request sent by the user terminal 204, the cloud server 202 identifies the type of the audio-video traffic to be sent, and performs reliable transmission through the first type of message flow for the traffic of the key data in the audio-video traffic, that is, performs data retransmission after the missing data message appears in the first type of message flow, and performs network transmission through the second type of message flow for the traffic of the non-key data in the audio-video traffic, so that the missing data message in the second type of message flow may not perform data retransmission or may perform data retransmission after the missing data message.

In one embodiment, as shown in fig. 3, a data transmission method is provided, and a transmitting end of the method applied to data transmission is taken as an example and described, where the transmitting end may be specifically a server in fig. 1, and the data transmission method specifically includes the following steps:

in step 302, target data including critical data and non-critical data is obtained.

Wherein critical data is data that needs to ensure reliable transmission, and non-critical data is data that allows unreliable transmission. Critical data and non-critical data may be determined based on default data class definition rules or may be defined by the data transmission service provider or by the data demander itself.

The default data class definition rule may be a definition of critical data and non-critical data for different data classes in advance. For example, audio data, video I frames, operation instructions, text messages may be defined as critical data, and video B frames and video P frames as non-critical data. The data transmission service provider and the data demander can also define key data and non-key data according to actual data transmission scenes, for example, in video-audio scenes containing audio and video at the same time, such as audio-video communication, audio-video on demand, audio-video live broadcast and the like, from the quantity, the audio frames in the video-audio scenes are fewer, the video frames are more, the loss of a small number of audio frames possibly brings more obvious "jamming" to users, and the loss of a small number of video frames does not bring visual "jamming". That is, from a business experience perspective, the importance of an audio frame is greater than that of a video frame, the packet loss tolerance of an audio frame is low, and the packet loss tolerance of a video frame is high, based on which audio data can be defined as critical data and video data can be defined as non-critical data. For another example, in an interactive scenario that includes both an operation instruction and a text message, the operation instruction may be defined as critical data when the importance of the operation instruction is greater than the importance of the text message, the text message may be defined as non-critical data, and the text message may be defined as critical data when the importance of the operation instruction is less than the importance of the text message, and the operation instruction may be defined as non-critical data.

Specifically, a specific manner of acquiring target data including critical data and non-critical data by the transmitting end may be to acquire, in response to a transmission request, target data requested by the transmission request. The target data may be obtained from an application layer of the server or may be received from other interactive objects.

In a specific application, taking a cloud server as a transmitting end to provide target data as an example, the cloud server includes, but is not limited to, a CDN (Content Delivery Network ) server, and a system including a CDN server gateway server. As shown in fig. 4, when the cloud server is only a CDN server, an application layer of the CDN server transfers target data to be sent to a transport layer, so that the transport layer performs transmission of the target data. As shown in fig. 5, when the cloud server is a system formed by a CDN server and a gateway server, the CDN server transmits target data to the gateway server through a streaming media transmission protocol (such as HTTP, RTMP, etc.), so that the gateway server transmits the target data.

Step 304, encapsulating the target data by data packets, and generating a first type of packet stream carrying a first type of stream identifier and a plurality of second type of packet streams respectively carrying different second type of stream identifiers, wherein the data carried by the data packets in the first type of packet stream belong to key data, and the data carried by the data packets in the second type of packet stream belong to non-key data.

The message encapsulation refers to a process of encapsulating a data packet with a specified data structure into a data message. A data message is a data unit exchanged and transmitted in a network, one data unit comprising a data block to be transmitted at a time. The data packet encapsulation of the target data is essentially to divide the target data into a plurality of data units, and then use each data unit as the data content that can be carried by one data packet to obtain the processing procedure of the plurality of data packets. Because the data units in the target data have a sequence, each data message generated based on each data unit can also have a message number for representing the sequence of the data units in the target data. For example, the message number of the data message in which the data unit sequenced before in the target data is located may be smaller than the message number of the data message in which the data unit sequenced after in the target data is located. Based on the message number, the receiving end of the data message can quickly splice the data units in each data message to obtain complete and correctly-ordered target data.

The message Stream refers to a Stream (Stream) formed by a plurality of data messages having the same Stream identifier. The flow identifier is an identity identifier for distinguishing different message flows. Each data message in the same message flow has the same flow identification, and each data message in different message flows has different flow identifications. For example, as shown in fig. 6, 7 data packets (numbers: 1, 2, …, 7) are sequentially sent to the receiving end, and among the 7 data packets, the Stream id=1 of the data packet numbered 1, 4, 5; stream id=2 for data packet numbered 2, stream id=3 for data packet numbered 3, stream id=4 for data packet numbered 6, and Stream id=5 for data packet numbered 7. Based on the flow identification, it can be determined that the data messages with the numbers 1, 4 and 5 belong to the same message flow, and the data messages with the numbers 2, 3, 6 and 7 respectively belong to different message flows.

The critical data and the non-critical data in the target data can alternately appear, or a plurality of critical data continuously appear and a plurality of non-critical data continuously appear, depending on the dividing rule of the critical data and the non-critical data in the target data. For example, for target data having obvious constituent units, the target data may be divided by taking one constituent unit as an object, for example, the video data includes a plurality of video frames, each of the video frames may be classified, and whether the video frame is critical data or non-critical data may be determined. For another example, for target data without obvious constituent units, the same kind of data that continuously appears may be divided into one constituent unit, and then it is determined whether each constituent unit is critical data or non-critical data. However, in either way, there may be cases where a single critical data or a single non-critical data cannot be transmitted through the same data message. Generally, the size of data that can be transmitted by one data packet is about 1.4Kb, and when the data size of a single critical data or a single non-critical data is greater than 1.4Kb, it is necessary to transmit the single critical data or the single non-critical data through multiple data packets, so that the data carried by the data packet includes at least a portion of the critical data or the non-critical data.

Specifically, the sending end sequentially encapsulates each data unit in the target data, adds the flow identifier of the first type of message flow to the data message where the data unit is located when the currently processed data unit belongs to the key data, and adds the flow identifier of the second type of message flow to the data message where the data unit is located when the currently processed data unit belongs to the non-key data. Further, each time a data unit belonging to the key data is processed, the added flow identifier may be the same flow identifier. For a plurality of continuous data units belonging to the same non-critical data, the added flow identifier can be the same flow identifier, and the flow identifier needs to be different from the flow identifier of the data message where the critical data is located. For a plurality of data units belonging to different non-critical data, the added flow identity may be a plurality of different flow identities that are different from the flow identity of the data message in which the critical data is located.

In a specific application, as shown in fig. 7, a sending end encapsulates a target data to obtain 7 data packets with numbers 1, 2, … and 7, where data carried by the data packets with numbers 1, 4 and 5 all belong to key data, data packet Stream id=1 of the data packet with numbers 1, 4 and 5, packet Stream id=1 belongs to a first type of packet Stream, data carried by the data packets with numbers 2, 3, 6 and 7 all belong to non-key data, and the data packets can be transmitted through different packet streams, stream id=2 of the data packet with number 2, stream id=3 of the data packet with number 3, stream id=4 of the data packet with number 6, and Stream id=5 of the data packet with number 7. The Stream id=2, 3, 4, 5 streams are a plurality of different streams, but belong to the second class of streams.

Step 306, each data message is sent according to the first type message stream and the second type message stream, and a confirmation response message for the data message is received.

Each data message in the target data is sequentially generated according to the arrangement sequence of the data units carried by the data messages in the target data, so that the sending sequence of each data message can be sent based on the generation sequence of the data message, namely, the data message generated first is sent, and then the data message generated later is sent. In the process of sending the data messages, because each data message belongs to different message flows, the data messages can be sent based on the message flows to which each data message belongs. For example, the data packet Stream IDs of numbers 1 and 5 are 1=1, the data packet streams of numbers 2,3, and 4 are 2=2, the data packets of numbers 1 and 5 are sequentially transmitted through the packet Stream 1, and the data packets of numbers 2,3, and 4 are sequentially transmitted through the packet Stream 2.

The acknowledgement message is a transmission type control character sent to the transmitting end by the receiving end in data communication, and indicates the receiving condition of the sent data. For example, the transmitting end sends 3 data messages with message numbers of 1,2 and 3 to the receiving end, and the receiving end informs the transmitting end that the data message 1 and the data message 3 are received and the data message 2 is not received through the acknowledgement message.

Specifically, the acknowledgement message may be fed back after a preset number of data messages are received in each accumulation, so that the server can timely determine whether there is a lost data message, and determine a specific policy to be adopted for the lost data message, for example, when data carried by the lost data message belongs to key data, a retransmission policy of fast retransmission is adopted, and when data carried by the lost data message belongs to non-key data, the adopted retransmission policy is not retransmission or delayed retransmission.

Step 308, retransmitting the key data carried by the lost data message in the first type message stream when the acknowledgement message indicates that the data message is lost and the lost data message belongs to the first type message stream.

The data message loss refers to a phenomenon that a sending end sends out the data message, but a receiving end does not receive the data message. For the lost data message, the sending end needs to further determine the message flow to which the lost data message belongs according to the lost data message, and determine the specific strategy to be adopted based on the message flow to which the data message belongs. Retransmission refers to a data processing procedure of retransmitting data of data content originally carried in a lost data packet.

Specifically, after receiving the acknowledgement message fed back by the receiving end, the sending end analyzes the acknowledgement message, determines whether the transmitted data message has a message loss condition and a message stream to which the lost data message belongs, and when the lost data message belongs to the first type of message stream, the sending end locates the position of key data carried by the lost data message in the target data through the acknowledgement message and retransmits the key data to the receiving end.

According to the data transmission method, the target data containing the key data and the non-key data are obtained, the data message package is carried out on the target data, the first type message stream for transmitting the key data and the plurality of second type message streams for transmitting the non-key data are generated, the first type message stream carries the first type stream identification, the plurality of second type message streams respectively carry different second type stream identifications, the key data and the non-key data can be sent in a split mode through different stream identifications, meanwhile, the non-key data can be scattered in the sending process, and when the lost data message belongs to the first type message stream, the key data carried by the lost data message in the first type message stream are retransmitted, so that the key data can be retransmitted in time, reliable transmission of the key data is ensured, and the non-key data can be effectively transmitted based on the fact that the receiving end of the data message is blocked due to the loss of the data message is scattered, thereby being beneficial to realizing high-efficiency transmission of the non-key data, and further ensuring the quality of the target data and the time-efficient transmission effect.

In some embodiments, data packet packaging is performed on target data, and a first class packet stream carrying a first class stream identifier and a plurality of second class packet streams respectively carrying different second class stream identifiers are generated, including: based on at least one target flow identifier in the first type flow identifiers, carrying out data message encapsulation on a plurality of key data in the target data to obtain at least one first type message flow; and based on a plurality of stream identifiers with the same number as that of the non-critical data in the second type stream identifiers, carrying out data message encapsulation on the same non-critical data in the target data by using the same stream identifiers in the plurality of stream identifiers to obtain a plurality of second type message streams corresponding to the plurality of non-critical data one by one.

The first class of flow identifiers are used for constructing the first class of message flows, and the second class of flow identifiers are used for constructing the second class of message flows. The first class of flow identifiers may be a preconfigured fixed flow identifier, and the number of the fixed flow identifiers may be one or more, the second class of flow identifiers may be a non-fixed flow identifier different from the fixed flow identifier, and the number of the non-fixed flow identifiers may be more, for example, any flow identifier other than the first class of flow identifiers may be used as the second class of flow identifiers.

For example, the first class Stream identifier may be configured as Stream id=1, 2, 3, and other Stream identifiers than the first class Stream identifier may be used as the second class Stream identifier, for example, stream id=4, 5, 6, … …, N (N is a positive integer greater than or equal to 6). When generating the message Stream, a first type Stream identifier with Stream id=1 can be selected, and data message packaging is performed on a plurality of key data in the target data, so as to obtain the first type message Stream with Stream id=1. And selecting different second class stream identifiers for each different non-critical data to carry out data message encapsulation. For example, data packet encapsulation is performed by using the second type Stream identifier with Stream id=4 for the non-critical data 1, data packet encapsulation is performed by using the second type Stream identifier with Stream id=5 for the non-critical data 2, data packet encapsulation is performed by using the second type Stream identifier with Stream id=6 for the non-critical data 3, and so on.

Specifically, before data packet encapsulation is performed on target data, a sending end obtains a first class of flow identifier and a second class of flow identifier which are configured in advance. For the key data, the transmitting end extracts at least one target flow identifier matched with the message configuration condition from the first class flow identifiers based on the message flow configuration condition of the key data, for example, when the message flow configuration condition is that one message flow is used for transmitting the key data, one target flow identifier is extracted from the first class flow identifiers, and when the message flow configuration condition is that the message flows are respectively transmitted according to the data types of the key data, a plurality of target flow identifiers with the same number as the data types of the key data are extracted from the first class flow identifiers. And the sending end performs data message encapsulation on a plurality of key data in the target data according to the message stream configuration conditions and at least one target stream identifier extracted from the first type stream identifier to obtain at least one first type message stream.

For non-critical data, the sending end extracts a plurality of stream identifiers with the same quantity of the non-critical data from the second class stream identifiers according to the quantity of the non-critical data in the target data, so that the stream identifiers added in a plurality of data messages distributed by the same non-critical data are the same second class stream identifiers, and the stream identifiers added in a plurality of data messages distributed by different non-critical data are different second class stream identifiers, thereby obtaining a plurality of second class message streams corresponding to the plurality of non-critical data one by one.

In a specific application, the target data includes 7 constituent units, where constituent units 1, 4, and 5 belong to critical data, constituent units 2, 3, 6, and 7 belong to non-critical data, and the data size of each constituent unit is smaller, a sending end may send the data content of one constituent unit through one data packet, and obtain a first type of Stream identifier and a second type of Stream identifier configured in advance, where the first type of Stream identifier includes Stream id=1, 2, and 3, the second type of Stream identifier includes Stream id=4, 5, 6, … …, and N (N is a positive integer greater than or equal to 6), and when the message Stream configuration condition is that one message Stream is used to send the critical data, one Stream identifier is selected from the first type of Stream identifiers, such as Stream id=1, and according to the arrangement sequence of the critical data 1, 4, and 5, the first type of Stream identifier is sequentially added to the critical data 1, 4, and 5 by using Stream id=1, to obtain a first type of message Stream including the critical data 1, 4, and 5. Stream id=4 is adopted to perform Stream identification addition on non-critical data 2 to obtain a second type message Stream comprising the non-critical data 2, stream id=5 is adopted to perform Stream identification addition on non-critical data 3 to obtain a second type message Stream comprising the non-critical data 3, stream id=6 is adopted to perform Stream identification addition on the non-critical data 6 to obtain a second type message Stream comprising the non-critical data 6, and Stream id=7 is adopted to perform Stream identification addition on the non-critical data 7 to obtain a second type message Stream comprising the non-critical data 7.

In this embodiment, by adding the flow identifiers to the critical data and the non-critical data by using the first type flow identifier and the second type flow identifier, so that the critical data is sent through at least one first type message flow, and each non-critical data is sent through a different second type message flow, scattering of the non-critical data can be achieved, the situation that a receiving end of a data message is blocked due to loss of the data message carrying the non-critical data can be effectively reduced, efficient transmission of the non-critical data is facilitated, and further transmission quality of the target data is ensured while transmission timeliness of the target data is ensured.

In some embodiments, based on at least one target flow identifier in the first type flow identifier, performing data packet encapsulation on key data in the target data to obtain at least one first type packet flow, including: selecting a target flow identifier from the first class of flow identifiers; and adding the target flow identifier to the data message of each key data to obtain a first type message flow carrying the target flow identifier.

The selecting a target Stream identifier from the first type of Stream identifier may be sequentially selected, for example, selecting Stream id=1, or may be randomly selected, for example, selecting Stream id=3, where a specific selecting manner of the target Stream identifier may be configured according to an actual application scenario, which is not limited herein. When the sending end adds the target flow identifier, the target flow identifier may be sequentially added to the data message where each key data is located according to the arrangement sequence of each key data in the target data, or may be sequentially added to the data message where each key data is located according to the generation sequence of the data message where each key data is located, and generally, the arrangement sequence of the key data in the target data is the same as the generation sequence of the data message where the key data is located. The data message corresponding to the key data with the front arranged position in the target data is also generated in advance.

Specifically, taking a Data packet as a Stream Frame as an example, the Data format of the Stream Frame is shown in fig. 8, and includes a Frame Type identifier field for indicating a Data Type, a Stream identifier field for indicating a Stream to which the Data packet belongs, a Data Offset field for indicating an Offset of the Data content in the whole target Data, a Data Length field for indicating a Length of the actual Data content, and an actual application Data field. For example, when it is determined that the Stream identifier to be added to the data packet is streamid=1, 1 is added to the streamid field of the data packet to characterize the Stream identifier of the data packet as 1.

In this embodiment, the sending end adds the target stream identifier to the data message in which each piece of key data is located, so that the stream identifier carried by the data message in which each piece of key data is located in the target data is the same, when data transmission is performed, the same message stream is used for transmitting, so that the sequence of the transmitted pieces of key data can be ensured, accurate acquisition of the target data by the receiving end is facilitated, for example, synchronization of audio and pictures, synchronization of audio and subtitles and the like are ensured, for the receiving end, all pieces of key data in the target data can be received based on the same message stream, once the data message in which the key data is located is lost, the situation of data message loss can be accurately identified based on the same message stream, and retransmission processing is requested by the sending end, so that the transmission reliability of the key data is effectively improved.

In some embodiments, based on at least one target flow identifier in the first type flow identifier, performing data packet encapsulation on key data in the target data to obtain at least one first type packet flow, including: determining the data category to which each of a plurality of key data in the target data belongs; based on a plurality of target stream identifiers with the same number as the data categories of the key data in the first type stream identifiers, adding the same target stream identifier in the plurality of target stream identifiers to the data messages of the same type of key data in the target data to obtain a plurality of first type message streams corresponding to each type of key data one by one.

In particular, when key data transmission is performed, the transmitted key data may include data of various data categories, such as I frames including text data, audio data, video data, and the like. For the key data of different data categories, the key data of multiple data categories can be transmitted through the same message flow, or the key data of multiple data categories can be transmitted by adopting multiple message flows according to the data categories. For example, the first column Stream of Stream id=1 is used for text data, the first column Stream of Stream id=2 is used for audio data, and the first column Stream of Stream id=3 is used for I-frames of video data.

Specifically, before sending multiple pieces of key data in the target data, the sending end determines the data category to which each piece of key data belongs, so as to select multiple target Stream identifiers corresponding to the data categories one by one from the first type of Stream identifiers, for example, selecting Stream id=1 for text data, selecting Stream id=2 for audio data, and selecting Stream id=3 for I-frame of video data. The number of stream identifiers selected from the first type of stream identifiers as target stream identifiers is the same as the number of data categories of key data, and the more the number of data categories of the key data is, the more the number of stream identifiers selected from the first type of stream identifiers as target stream identifiers is. Aiming at each type of key data in the target data, a sending end adds a target flow identifier matched with the type of key data to each data message of the type of key data to obtain a first type of message flow corresponding to the type of key data. Further, each time the transmitting end identifies key data of a new data category, a target flow identifier which is not selected in the interactive connection process is selected from the first type flow identifiers, a matching relation between the target flow identifier and the key data is established, and when the key data of the data category is identified in subsequent processing, the target flow identifier is directly added based on the matching relation.

In this embodiment, the sending end adds the target flow identifier according to the data type of the key data, so that the key data of different data types can be transmitted through different message flows, and the transmission efficiency of the key data can be effectively improved.

In some embodiments, the target data is combined data comprising multiple classes of data; the method further comprises the steps of: acquiring category configuration data defining at least one of a key data category set and a non-key data category set, and identifying data categories to which data units contained in target data belong; and matching each data category with at least one category set in the key data category set and the non-key data category set, and determining key data and non-key data in the target data based on a matching result.

The category configuration data is configuration data for defining key data and non-key data according to data categories, and the category configuration data can be standard configuration data suitable for any target data to be sent, or can be personalized configuration data suitable for sending the target data in a fixed application scene. In a specific application, standard configuration data and personalized configuration data can be configured at the same time, the selection priority of the personalized configuration data is higher than that of the standard configuration data, when a sending end receives a sending request aiming at target data, a specific application scene of the target data is determined first, when the personalized configuration data is configured aiming at the specific application scene, the personalized configuration data is selected, and when the personalized configuration data is not configured aiming at the specific application scene, the standard configuration data is selected.

A set of critical data categories is a set of data categories defined with belonging to critical data, and a set of non-critical data categories is a set of data categories defined with belonging to non-critical data. For example, the set of key data categories includes audio data, video I frames, operational instructions, text messages, etc., and the set of non-key data categories includes video B frames, video P frames, etc. The category configuration data may define only a set of critical data categories, or only a set of non-critical data categories when only critical data and non-critical data need to be distinguished for data of different data categories, and may define both the set of critical data categories and the set of non-critical data categories when the classification of data of different data categories includes three categories of critical data, non-critical data, and other data.

For example, in the case that only the critical data and the non-critical data need to be distinguished for the data of different data types, taking the class set configured in the class configuration data as an example, when the data type to which the data unit belongs to the critical data class set, the data unit is judged to be the critical data, and when the data type to which the data unit belongs does not belong to the critical data class set, the data unit is judged to be the non-critical data;

For another example, taking a category set configured in the category configuration data as a non-critical data category set as an example, when the data category to which the data unit belongs to the non-critical data category set, determining that the data unit is non-critical data; and when the data category to which the data unit belongs does not belong to the non-critical data category set, judging the data unit as critical data.

For another example, when the classification of data for different data categories includes three categories of critical data, non-critical data, and other data, the set of categories configured in the category configuration data includes both the set of critical data categories and the set of non-critical data categories. Specifically, the transmitting end determines the data type of the data unit in the target data, when the data type belongs to the key data type set, the data unit is determined to be the key data, when the data type does not belong to the key data type set, whether the data type belongs to the non-key data type set is further determined, if yes, the data unit is determined to be the non-key data, if not, the data unit is determined to be other data, and the data is transmitted by adopting other set transmission modes. It will be appreciated that in other embodiments, it may be determined whether the data category belongs to a non-critical data category set, and then whether the data category belongs to a critical data category set.

In this embodiment, by acquiring category configuration data defining at least one of a set of critical data categories and a set of non-critical data categories, and matching the data category to which each data unit included in the target data belongs with the at least one category set, the critical data and the non-critical data in the target data can be quickly and accurately identified, and the identification efficiency and accuracy of the critical data and the non-critical data in the target data can be effectively improved.

In some embodiments, identifying the data category to which each of the data units included in the target data belongs includes: respectively acquiring a data type field from respective description fields of each data unit in target data; and determining the data category represented by the data type field as the data category of the data unit represented by the description field where the data type field is located.

The data unit is a minimum data structure in the target data, which can be directly sent through the data content of the data message, and the data field is a field for describing the attribute of the data unit, and may specifically include a data type field for characterizing the data category to which the data unit belongs. The specific parameters of the data type field are used to characterize the data class of the data unit.

Specifically, the respective description field of each data unit in the target data may be embedded in each data unit, and may be acquired at the same time as the data unit is acquired, or may be transmitted as separate data. In a specific application, as shown in fig. 9, when the sending end is a CDN server and the sent target Data is audio/video Data, an application layer of the CDN server transfers the audio/video Data to a transport layer, a description field is carried, where the description field includes a field data_type (length is 8 bits), a Data Offset value data_offset (64 bits) of the audio/video Data, and a Data length data_len (64 bits), which indicate that the Type of Data between the data_offset and the data_offset+data_len of the audio/video Data to be transmitted is data_type.

As shown in fig. 10, when the sender is a system formed by a CDN server and a gateway server, the CDN server carries a description field when delivering audio/video Data to the gateway server through a streaming media transmission protocol (such as HTTP, RTMP, etc.), where the description field includes a field data_type (length is 8 bits), an Offset value data_offset (64 bits) of the audio/video Data, and a Data length data_len (64 bits), and the Type of Data between the data_offset to the data_offset+data_len that indicates the audio/video Data to be transmitted is data_type.

In a specific application, taking target data as an FLV file containing audio and video data and encoded by FLV (flash video, streaming media format) as an example, the data type of each data unit in the target data may be determined according to a relevant field in the audio and video data, where the FLV audio and video data includes a typeflag audio field and a typeflag video field, and if the typeflag audio field value is 1, it is indicated that there is audio in the data; if the value of the TypeFlagsVideo field is 1, the video is indicated to exist in the data, specifically, in the FLV audio/video data, each data unit corresponds to an FLV Tag field, and if the value of the TagType in the FLV Tag field is 0x08, the data unit is characterized as audio data; if the TagType value is 0x09, the data unit is characterized as video data; if the TagType value is 0x12, the data unit is characterized as script data.

In this embodiment, the transmitting end obtains the data type field from the description field of the data unit, determines the data type represented by the data type field as the data type of the data unit represented by the description field where the data type field is located, and can quickly and accurately identify whether each data unit is critical data or non-critical data, so as to add the corresponding flow identifier and transmit the data according to the affiliated message flow.

In some embodiments, the target data comprises a target video; the method further comprises the steps of: identifying the video frame category to which each video frame in the target video belongs; when the video frame category to which the video frame belongs is a video I frame, judging the video frame as key data; when the video frame category to which the video frame belongs is a video B frame or a video P frame, judging the video frame as non-key data.

The video I frame is an intra-frame (intra picture), which is also called full-frame compression coding frame. The I-frame represents a key frame, typically the first frame of each GOP (a video compression technique used by MPEG), which is moderately compressed as a reference point for random access, and can be regarded as a still image. The video P-frame (predictive-frame) represents a difference frame, which represents the difference between the frame and the previous key frame or the previous video P-frame, and the video P-frame has no complete picture data, and when decoding, the difference defined by the previous frame needs to be overlapped by the previously buffered picture, so as to generate a final picture, and the video P-frame is a forward predictive encoded frame, and usually, the time redundancy information of the previously encoded frame in the image sequence is sufficiently removed to compress the encoded image of the transmission data amount. The video B frame (bi-directional interpolated prediction frame) represents a bi-directional difference frame, recorded as the difference between the present frame and the preceding and following frames, is a bi-predictive interpolation encoded frame, which compresses an encoded image of the amount of transmission data taking into account both the encoded frame preceding the source image sequence and the temporal redundancy information between the encoded frames following the source image sequence, also referred to as a bi-predictive frame.

Specifically, the video I frame itself may be decompressed into a single complete video picture by a video decompression algorithm, where the video I frame removes redundant information of the video frame in the spatial dimension, the video P frame needs to be decoded into a complete video picture by referring to the previous video I frame or the video P frame, the video B frame needs to be decoded into a complete video picture by referring to the previous video I frame or the video P frame and the video P frame that follows the previous video I frame or the video P frame, and the video P frame and the video B frame remove redundant information of the video frame in the temporal dimension.

Taking the h.264 video coding and decoding standard in network live broadcast as an example, in the transmission process of video data, the video blocking phenomenon may be obvious due to the loss of video I frames, the slight blocking may be caused by the loss of video P frames, and the loss of video B frames will not be affected basically. As shown in fig. 11, the video I frame packet loss causes the client to miss 9 frame contents, so that the jamming is obvious, the video P frame packet loss causes the client to miss 5 frame contents, so that the jamming is slight, and the video B frame packet loss causes the client to miss 1 frame content, so that the jamming is not felt. The importance level of the video frames is as follows from high to low in sequence: video I frame > video P frame > video B frame. In network transmission, the transmission reliability of the I frame should be ensured as much as possible. In addition, the video is blocked in the live audio network, which has a more obvious effect on the user experience. From the number, fewer audio frames and more video frames, the loss of a small number of audio frames may bring more obvious "jamming" to the user, while the loss of a small number of video frames may not bring intuitive "jamming".

Specifically, for video data in target data, the sending end is composed of a plurality of video frames, the sending end can identify the video frame type of each video frame in the target video, when the video frame type of each video frame is a video I frame, the video frame is judged to be key data, when the video frame type of each video frame is a video B frame or a video P frame, the video frame is judged to be non-key data, and further reliable transmission of the video I frame in the video data can be guaranteed, unreliable transmission of the video B frame or the video P frame is allowed, occurrence of message blocking at the receiving end caused by loss of a data message of the video B frame or the video P frame is reduced or avoided, timeliness of transmission is ensured, and effects of sending timeliness and quality of the video data are achieved.

In some embodiments, the message data carries a message number that characterizes the sending order of the data message; when the confirmation response message characterizes that the data message is lost and the lost data message belongs to the first type message stream, retransmitting key data carried by the lost data message in the first type message stream, wherein the key data comprises: determining a target message number of the lost data message and a message flow to which the lost data message belongs based on the confirmation response message; when the message stream to which the lost data message belongs is a first type message stream, determining lost key data carried by the lost data message based on a target message number; retransmitting the data message carrying the lost key data through the first message stream.

The message numbers are numbers used for representing the sending sequence of the data messages, and the message numbers of the sequentially sent data messages are sequentially increased, such as message numbers 1, 2, 3, 4, … and N (N is a positive integer more than or equal to 6). Specifically, the transmitted data messages are divided into two types, the first type is the first transmitted data message, and the second type is the data message retransmitted under the condition that the data message is lost. For the data message sent for the first time, the generation sequence of the data message is corresponding to the sequence of the data content carried by the data message in the target data, the earlier the sequence of the data content carried by the data message in the target data is, the smaller the message number value of the data message is, and the later the sequence of the data content carried by the data message in the target data is, the larger the message number value of the data message is. For the retransmitted data message, the message number is determined based on the retransmission time of the data message, and the retransmitted data message can be inserted into the data message sent for the first time for transmission.

For the transmitting end, the data content carried by the data message represented by the message number can be found through the message number, and when the transmitting end receives a confirmation response message fed back by the receiving end of the data message, the confirmation response message represents that the data message is lost and the lost data message belongs to a first type of message flow, namely, when the key data is lost, the lost key data needs to be retransmitted. Specifically, the content included in the acknowledgement message may be a missing message number, or may be a message number of the received data message. For the case that the content contained in the acknowledgement message comprises the lost message number, the receiving end of the data message determines whether the lost data message exists according to whether the message numbers of the received data message are continuous.

Further, the plurality of data messages sequentially sent based on the same message stream also have an in-stream sequence, and the receiving end of the data message can determine the message stream to which the lost data message belongs according to the in-stream sequence of the received message stream, and synchronously feed back the message stream to which the lost data message belongs to the sending end of the data message through the confirmation response message, so that the sending end of the data message can directly determine whether the lost data message belongs to the first type of message stream based on the confirmation response message. When the confirmation response message only feeds back the message number of the lost data message, but does not feed back the message flow to which the lost data message belongs, the sending end can determine the lost key data carried by the lost data message according to the message number of the lost data message based on the recorded data of the sent data message.

In a specific application, the message number of the data message received by the receiving end of the data message is 1, 2, 3, 5, 6, 8, and based on the condition that the message numbers of the data messages sent by the sending end are sequentially increased, the receiving end can determine that the data message with the message number of 4 and the message number of 7 is lost. And feeding back the lost message numbers 4 and 7 to the sending end of the data message through the confirmation response message. The sending end can send recorded data based on the data message, further judge whether the lost message numbers 4 and 7 belong to the first type of message stream, if the message numbers 4 and 7 belong to the first type of message stream, determine the lost key data based on the message numbers 4 and 7 respectively, and then retransmit the data message carrying the lost key data and having the message numbers 9 and 10 through the first type of message stream.

In another specific application, the message number of the data message received by the receiving end of the data message is 1, 2, 3, 5, 6 and 8, and the received data message with the message number of 1, 2, 3, 5, 6 and 8 is directly fed back to the sending end of the data message through the acknowledgement message. The transmitting terminal judges that the data messages with the message numbers of 4 and 7 are lost based on the condition that the message numbers of the transmitted data messages are sequentially increased, further judges whether the lost message numbers of 4 and 7 belong to the first type of message streams, determines lost key data based on the message numbers of 4 and 7 respectively if the lost message numbers of 4 and 7 belong to the first type of message streams, and retransmits the data messages with the lost key data and the message numbers of 9 and 10 through the first type of message streams.

In this embodiment, by determining, based on the acknowledgement packet, the target packet number of the lost data packet and the packet flow to which the lost data packet belongs, it is possible to quickly determine whether the packet flow to which the lost data packet belongs is the first type packet flow, and in the case that the lost data packet belongs to the first type packet flow, it is possible to accurately locate the lost key data carried by the lost data packet based on the target packet number, and retransmit the data packet carrying the lost key data through the first type packet flow, so that reliable transmission of the lost key data can be ensured, and further retransmission processing of the lost key data can be quickly, accurately and reliably implemented.

In some embodiments, when the acknowledgement message indicates that there is a data message loss and the lost data message belongs to a first type of message flow, retransmitting key data carried by the lost data message in the first type of message flow, including: when the confirmation response message indicates that the data message is lost and the lost data message belongs to the first type of message flow, suspending the data message transmission of the data to be transmitted in the target data; and retransmitting key data carried by the lost data message in the first type message stream, and restarting the data message transmission of the data to be transmitted in the target data after the retransmission is finished.

Specifically, the data to be transmitted in the target data is data to be transmitted in the target data, and the data to be transmitted may be at least a part of the data not transmitted in the target data, and the data not transmitted is data other than the data already transmitted in the target data. When the transmitting end recognizes that the lost data message belongs to the first type of message stream, namely, the data carried by the lost data message is the key data in the target data, the transmitting end pauses the data message transmission of the data to be transmitted in the target data and retransmits the lost key data because the position of the lost key data in the target data is in front of the data to be transmitted, so that the receiving end of the data message can receive the retransmitted key data before the data to be transmitted, and the reliability of key data transmission is ensured.

In a specific application, a sending end encapsulates a target data to obtain data messages with numbers of 1, 2, … and N, and sequentially sends the data messages to a receiving end, taking a currently sent data message including data messages with message numbers of 1-7 as an example, wherein data carried by the data messages with numbers of 1, 4 and 5 all belong to key data, data carried by the data messages with numbers of 2, 3, 6 and 7 belong to non-key data, when the data messages received by the receiving end are the data messages with numbers of 1, 2, 5, 6 and 7, the receiving end sorts received first class message streams according to the internal sequence of stream identification, and sends the sorted complete data to an application layer, if the data messages with the key data are lost, a blocking condition occurs, and a transmission layer can only send the unblocked message to the application layer, as shown in fig. 12, the transmission layer can only send the message with the message number of 1 to the application layer, and the message number of 5 can not be sent to the application layer due to the blocking condition that the message number of 4 is lost.

At this time, the receiving end feeds back the missing situation of the message numbers 3 and 4 to the sending end through the confirmation response message, the sending end recognizes that the message stream to which the data message with the message number 4 belongs is a first type message stream, at this time, the message number of the last data message which is currently sent is M, the sending of the data message of the data to be sent in the target data is suspended, the key data carried by the missing data message is retransmitted as the data message with the message number of M+1, then the data message of the data to be sent in the target data is continuously sent, and at this time, the message number of the data message which is sent in the first data message of the data to be sent in the target data is M+2.

In this embodiment, the transmission of the data message of the data to be transmitted in the target data is suspended to retransmit the key data carried by the data message lost in the first type of message stream, and the transmission of the data message of the data to be transmitted in the target data is restarted after the retransmission is finished, so that the retransmission processing of the lost key data can be quickly and reliably implemented.

In some embodiments, after the sending end receives the acknowledgement message fed back by the receiving end, the data transmission method further includes: when the confirmation response message indicates that the data message is lost and the lost data message belongs to the second class message flow, waiting for ending the data transmission to be sent in the target data; and retransmitting non-key data carried by the lost data message in the second-class message stream after the data message of the data to be transmitted is transmitted.

Specifically, the influence of the loss of the non-critical data on the data display result of the application layer of the receiving end is small, so that the retransmission of the non-critical data can be delayed, the timely transmission of the data message of the data to be transmitted of the transmitting end is prioritized, and the efficiency of data transmission is improved.

The data message of the non-key data is transmitted through the second type message stream, and the situation that one non-key data is blocked due to the fact that the message in the stream of the other non-key data is lost is avoided. The data packet with the packet number 3 is lost, because Stream IDs of the data packets with the packet numbers 2, 6 and 7 are respectively 2, 4 and 5, and belong to different second-class packet flows, so that packet loss of the data packet with the packet number 3 does not cause blocking of the data packets with the packet numbers 2, 6 and 7, and in this case, the data packets with the packet numbers 2, 6 and 7 can be sent to an application layer by a transmission layer without waiting to re-accept retransmission about the packet number 3 from a sending end to send the data packets with the packet numbers 2, 6 and 7 to the application layer. It should be noted that, in other embodiments, when the acknowledgement message indicates that the data message is lost and the lost data message belongs to the second class message stream, the retransmission processing task for the lost data message belonging to the second class message stream may be directly canceled, that is, retransmission is not required.

In a specific application, when the receiving end feeds back the message numbers 3 and 4 to the sending end through the acknowledgement message, the sending end recognizes that the message stream to which the data message with the message number 3 belongs is the second type message stream, at this time, the message number of the last data message currently sent by the sending end is M, the data to be sent in the target data can be packaged into 3 data messages, the data to be sent is sent to the receiving end through m+1, m+2 and m+3, and after the data message with the message number m+3 is sent, the key data carried by the lost data message is retransmitted as the data message with the message number m+4.

In this embodiment, by canceling or postponing retransmission of key data carried by a data packet lost in the first type packet stream, timely transmission of the data packet of the data to be transmitted of the transmitting end can be prioritized, efficiency of data transmission of non-transmitted data is improved, and timeliness is improved.

In one embodiment, the data message carries a data offset value and a data length; and the retransmission data message is used for enabling the receiving end of the retransmission data message to determine the ordering position of the key data carried by the retransmission data message in the target data based on the data offset value and the data length carried by the retransmission data message.

Specifically, the Data packet carries a description field of the Data content included in the Data packet, where the description field includes a field data_type (8 bits in length), a Data Offset value data_offset (64 bits), and a Data length data_len (64 bits), which can identify the Type of Data, and indicates that the Type of Data between the Data unit to be transmitted from data_offset to data_offset+data_len is data_type.

In this embodiment, since the message number of the retransmitted data packet after retransmission is different from the message number sent for the first time, for example, the receiving end of the retransmitted data packet can accurately determine the accurate position of the data content in the retransmitted data packet, and the receiving end can determine the ordering position of the key data carried in the retransmitted data packet in the target data based on the data offset value and the data length carried in the retransmitted data packet, so as to accurately implement the recombination of the retransmitted key data and other received data.

The application also provides an application scene for transmitting the audio and video data to the terminal by the cloud server, and the application scene applies the data transmission method. Specifically, as shown in fig. 13, the application of the data transmission method in the application scenario is as follows:

The cloud server receives a request from a terminal, and issues audio and video Data to be sent to a transmission layer through an application layer, wherein the audio and video Data to be sent carries a description field, the description field comprises a field data_type (with the length of 8 bits) capable of identifying the Type of the Data, a Data Offset value data_offset (64 bits) and a Data length data_len (64 bits) of the audio and video Data, and the description field can represent the Type of the Data between the data_offset and the data_offset+data_len of the audio and video Data to be transmitted is data_type.

The cloud server identifies the traffic class of the audio and video Data to be transmitted through a field data_type of the identification Data Type, acquires a key Data class set and a non-key Data class set which are defined by a cloud service provider or an end user, and determines whether the audio and video Data to be transmitted is key Data or non-key Data based on the key Data class set and the non-key Data class set and the traffic class of the audio and video Data to be transmitted. Specifically, the key audio-video data categories include, but are not limited to, audio data, video I-frames, operational instructions, text messages, and the like; non-critical video data categories include, but are not limited to, video B frames and P frames.

The audio and video data is an FLV file coded by FLV (flash video, streaming media format), the data type of each data unit in the audio and video data can be judged according to the relevant field in the audio and video data, the FLV audio and video data comprises a TypeFlagnosudio field and a TypeFlagnosudio field, and if the TypeFlagnosudio field value is 1, the audio is indicated in the audio and video data; if the TypeFlagsvideo field value is 1, the fact that video exists in the audio-video data is indicated, specifically, in the FLV audio-video data, each data unit corresponds to an FLV Tag field, and if the TagType value in the FLV Tag field is 0x08, the data unit is characterized as audio data; if the TagType value is 0x09, the data unit is characterized as video data; if the TagType value is 0x12, the data unit is characterized as script data.

And the cloud server sends the audio and video data to the user terminal after acquiring the audio and video data to be sent and the data type of the audio and video data. Specifically, the data message of the key data is sent according to a fixed stream identifier, namely a first type stream identifier; meanwhile, the data message aiming at the non-key data is sent according to the flow identifier which is not used in the data connection process, namely the second type flow identifier.

For example, as shown in fig. 14, the audio/video data may be encapsulated into 7 audio/video data messages (numbered 1, 2, …, 7) that are sequentially sent from the cloud server to the user terminal, where the messages numbered 1, 4, 5 are key data messages; the messages with the numbers of 2, 3, 6 and 7 are non-key data messages; the key data messages (numbered 1, 4 and 5) are sent to the user terminal through a fixed Stream identifier (Stream id=1); non-critical data messages (numbered 2, 3, 6, 7) are sent to the user terminal through non-fixed and non-appeared flow identifiers; if the message number 2 is sent by Stream id=2, the message number 3 is sent by Stream id=3, the message number 6 is sent by Stream id=4, and the message number 7 is sent by Stream id=5.

After receiving the audio and video data messages sent by the cloud server, the terminal reorders the messages in the same message Stream according to the Stream ID field and the Stream offset field in the QUIC protocol, and sends the ordered data to an application layer; meanwhile, the terminal feeds back a message feedback acknowledgement message (namely an ACK message) to the cloud server, and feeds back the receiving or losing condition of the data message to the cloud server.

The terminal sorts the key data messages according to the internal sequence of the flow mark, and sends the sorted complete data to the application layer player, namely if the key data is lost, the blocked message is sent to the application layer, if the message number 4 is lost, the message number 1 can only be sent to the application layer, and the message number 5 is blocked because the message number 4 is lost, so that the blocked message can not be sent to the application layer;

the non-critical data message uses the flow identification which is not fixed and is not used for transmission, so that the situation that one non-critical data is blocked due to the fact that the message in the flow of the other non-critical data is lost cannot occur, for example, when the message number 3 is lost, because the message numbers 2, 6 and 7 respectively use different flow identifications, the packet loss of the message number 3 cannot cause the blocking of the message numbers 2, 6 and 7, and in this case, the message numbers 2, 6 and 7 can be sent to the application layer player by the transmission layer.

And the cloud server preferably retransmits the lost key data message after receiving the message feedback information from the user terminal. The messages with the numbers 3 and 4 are lost in the network transmission process successively, after the cloud server receives the message loss information with the numbers 3 and 4, the cloud server identifies the packet loss of key data (namely the message with the number 4 is lost), the data message with the number 4 is transmitted back to the user terminal preferentially, the user terminal receives the retransmitted data message from the cloud server, the retransmitted data message is sent to the application layer player, and after the user terminal receives the retransmitted data message with the number 4 in sequence, the user terminal sends the unblocked message (namely the data message with the number 5) to the application layer player.

The cloud server sends the key data message through identifying the type of the audio and video traffic to be sent and through a fixed flow identifier in a QUIC (Quick UDP Internet Connection, UDP-based low-delay Internet transport layer protocol) protocol, so as to realize reliable transmission; and aiming at the non-key data message, transmitting through the unused flow mark so as to realize unreliable transmission. The blocking phenomenon caused by packet loss in the same stream identifier (stream) in the QUIC protocol is improved, namely, the problem that the non-key data message is lost and blocked is prevented from being reported to an application layer player is avoided.

By adopting the data transmission method to carry out data transmission, under the condition of not modifying a user terminal protocol, reliable transmission of key data and unreliable transmission of non-key data are realized by changing the packet sending and retransmission strategy of the cloud server, and QoE (Quality of Experience, user experience quality) indexes such as the blocking of the user terminal are improved. Compared with the existing network transmission protocol, the method has the advantages of being strong in deployment, high in adaptation degree of audio and video services, capable of remarkably improving terminal jamming and the like. By setting different stream identification transmission methods for audio and video data of different types or priorities, the adaptation of a user terminal on a protocol level is avoided, the reliable transmission of a message of key data is ensured, the unreliable transmission of a non-key data message is ensured, the reporting of the key data message to an application layer is not blocked due to the loss of the non-key data message, and the transmission performance of data traffic is improved.

It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

Based on the same inventive concept, the embodiment of the application also provides a data transmission device for realizing the above related data transmission method. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation in one or more embodiments of the data transmission device provided below may refer to the limitation of the data transmission method hereinabove, and will not be repeated herein.

In one embodiment, as shown in fig. 15, there is provided a data transmission apparatus including: a data acquisition module 1502, a message flow generation module 1504, a message transceiver module 1506, and a key data retransmission module 1508, wherein:

a data acquisition module 1502 for acquiring target data including critical data and non-critical data;

a message flow generating module 1504, configured to perform data message encapsulation on the target data, generate a first type of message flow carrying a first type of flow identifier and a plurality of second type of message flows respectively carrying different second type of flow identifiers, where data carried by a data message in the first type of message flow belongs to the key data, and data carried by a data message in the second type of message flow belongs to the non-key data;

a message transceiver module 1506, configured to send each of the data messages according to the first type message stream and the second type message stream, and receive a confirmation response message for the data message;

and the key data retransmission module 1508 is configured to retransmit key data carried by the lost data packet in the first type packet stream when the acknowledgement packet indicates that the data packet is lost and the lost data packet belongs to the first type packet stream.

In one embodiment, the message flow generating module includes a first type message flow generating unit and a second type message flow generating unit.

The first type message stream generating unit is used for carrying out data message encapsulation on a plurality of key data in the target data based on at least one target stream identifier in the first type stream identifier to obtain at least one first type message stream;

the second class message stream generating unit is configured to perform data message encapsulation on the same non-critical data in the target data by using the same flow identifier in the plurality of flow identifiers based on a plurality of flow identifiers with the same number as the non-critical data in the second class flow identifiers, so as to obtain a plurality of second class message streams corresponding to the plurality of non-critical data one to one.

In one embodiment, the first class packet stream generating unit is further configured to select a target stream identifier from the first class stream identifiers; and respectively adding the target flow identifier to the data message of each key data to obtain a first type message flow carrying the target flow identifier.

In one embodiment, the first class packet stream generating unit is further configured to determine a data class to which each of the plurality of key data in the target data belongs; based on a plurality of target stream identifiers with the same number as the data categories of the key data in the first type stream identifiers, adding the same target stream identifier in the plurality of target stream identifiers to the data messages of the same type of key data in the target data to obtain a plurality of first type message streams corresponding to each type of key data one by one.

In one embodiment, the target data is combined data comprising multiple classes of data; the data transmission device also comprises a key data and non-key data identification module, which is used for acquiring category configuration data defining at least one category set of a key data category set and a non-key data category set and identifying the data category to which each data unit contained in the target data belongs; and matching each data category with at least one category set in the key data category set and the non-key data category set, and determining key data and non-key data in the target data based on a matching result.

In one embodiment, the critical data and non-critical data identification modules include a data category identification unit, configured to obtain a data type field from a description field of each data unit in the target data; and determining the data category represented by the data type field as the data category of the data unit represented by the description field where the data type field is located.

In one embodiment, the target data comprises a target video; the key data and non-key data identification module is further used for identifying the video frame category to which each video frame in the target video belongs; when the video frame category to which the video frame belongs is a video I frame, judging the video frame as key data; and when the video frame category to which the video frame belongs is a video B frame or a video P frame, judging that the video frame is non-key data.

In one embodiment, the message data carries a message number representing a sending sequence of the data message; the key data retransmission module is further configured to determine, based on the acknowledgement message, a target message number of a lost data message and a message flow to which the lost data message belongs; when the message stream to which the lost data message belongs is a first type message stream, determining lost key data carried by the lost data message based on the target message number; retransmitting the data message carrying the lost key data through the first type message stream.

In one embodiment, the key data retransmission module is further configured to suspend, when the acknowledgement packet indicates that there is a data packet loss and the lost data packet belongs to the first type packet flow, data packet transmission of data to be transmitted in the target data; and retransmitting key data carried by the lost data message in the first type message stream, and restarting the data message transmission of the data to be transmitted in the target data after the retransmission is finished.

In one embodiment, the key data retransmission module is further configured to monitor whether transmission of the data to be sent is finished when the acknowledgement packet indicates that there is a data packet loss and the lost data packet belongs to the second class packet flow; and retransmitting non-key data carried by the lost data message in the second class message stream after the data message of the data to be transmitted in the target data is transmitted.

In one embodiment, the data packet carries a data offset value and a data length; the retransmission data message is configured to enable a receiving end of the retransmission data message to determine an ordering position of key data carried in the retransmission data message in the target data based on a data offset value and a data length carried by the retransmission data message.

According to the data transmission device, the target data containing the key data and the non-key data are obtained, the data message package is carried out on the target data, the first type message stream for transmitting the key data and the plurality of second type message streams for transmitting the non-key data are generated, the first type message stream carries the first type stream identification, the plurality of second type message streams respectively carry different second type stream identifications, the key data and the non-key data can be sent in a split mode through different stream identifications, meanwhile, the non-key data can be scattered in the sending process, and when the lost data message belongs to the first type message stream, the key data carried by the lost data message in the first type message stream are retransmitted, so that the key data can be retransmitted in time, reliable transmission of the key data is ensured, and the non-key data can be effectively transmitted by scattering based on the plurality of second type message streams, so that the condition that the receiving end of the data message is blocked due to the loss is effectively reduced, the efficient transmission of the non-key data can be realized, the quality of the target data is ensured, and the time-efficient effect of the target data is achieved, and the time-efficient transmission effect is achieved.

The respective modules in the above-described data transmission apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In one embodiment, a computer device is provided, which may be a server, and the internal structure of which may be as shown in fig. 16. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing target data to be transmitted. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a data transmission method.

It will be appreciated by those skilled in the art that the structure shown in fig. 16 is merely a block diagram of a portion of the structure associated with the present application and is not limiting of the computer device to which the present application is applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, carries out the steps of the method embodiments described above.

In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.

It should be noted that, the user information (including, but not limited to, user equipment information, user personal information, etc.) and the data (including, but not limited to, data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data are required to comply with the related laws and regulations and standards of the related countries and regions.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The above examples only represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.

Claims

1. A method of data transmission, the method comprising:

acquiring target data comprising key data and non-key data;

2. The method of claim 1, wherein the encapsulating the target data to generate a first type of packet stream carrying a first type of stream identifier and a plurality of second type of packet streams respectively carrying different second type of stream identifiers comprises:

based on at least one target flow identifier in the first type flow identifiers, carrying out data message encapsulation on a plurality of key data in the target data to obtain at least one first type message flow;

and based on a plurality of stream identifiers with the same number as the non-critical data in the second type stream identifiers, carrying out data message encapsulation on the same non-critical data in the target data by using the same stream identifier in the plurality of stream identifiers to obtain a plurality of second type message streams which are in one-to-one correspondence with the plurality of non-critical data.

3. The method according to claim 2, wherein the performing data packet on the key data in the target data based on at least one target flow identifier in the first class flow identifier to obtain at least one first class packet flow includes:

selecting a target flow identifier from the first class of flow identifiers;

and respectively adding the target flow identifier to the data message of each key data to obtain a first type message flow carrying the target flow identifier.

4. The method according to claim 2, wherein the performing data packet on the key data in the target data based on at least one target flow identifier in the first class flow identifier to obtain at least one first class packet flow includes:

determining the data category to which each of a plurality of key data in the target data belongs;

based on a plurality of target stream identifiers with the same number as the data categories of the key data in the first type stream identifiers, adding the same target stream identifier in the plurality of target stream identifiers to the data messages of the same type of key data in the target data to obtain a plurality of first type message streams corresponding to each type of key data one by one.

5. The method of claim 1, wherein the target data is combined data comprising multiple classes of data; the method further comprises the steps of:

acquiring category configuration data defining at least one of a key data category set and a non-key data category set, and identifying data categories to which data units contained in the target data belong;

and matching each data category with at least one category set in the key data category set and the non-key data category set, and determining key data and non-key data in the target data based on a matching result.

6. The method of claim 5, wherein identifying the data categories to which the data units included in the target data each belong comprises:

respectively acquiring a data type field from respective description fields of each data unit in the target data;

and determining the data category represented by the data type field as the data category of the data unit represented by the description field where the data type field is located.

7. The method of claim 1, wherein the target data comprises a target video; the method further comprises the steps of:

Identifying the video frame category to which each video frame in the target video belongs;

when the video frame category to which the video frame belongs is a video I frame, judging the video frame as key data;

and when the video frame category to which the video frame belongs is a video B frame or a video P frame, judging that the video frame is non-key data.

8. The method according to claim 1, wherein the message data carries a message number indicating a transmission order of the data message;

and retransmitting key data carried by the lost data message in the first type message stream when the confirmation response message indicates that the data message is lost and the lost data message belongs to the first type message stream, wherein the key data comprises the following components:

determining a target message number of a lost data message and a message flow to which the lost data message belongs based on the confirmation response message;

when the message stream to which the lost data message belongs is a first type message stream, determining lost key data carried by the lost data message based on the target message number;

retransmitting the data message carrying the lost key data through the first type message stream.

9. The method according to claim 1, wherein retransmitting key data carried by the lost data message in the first type of message stream when the acknowledgement message indicates that there is a data message loss and the lost data message belongs to the first type of message stream, includes:

the data message sending of the data to be sent in the target data is paused when the confirmation response message indicates that the data message is lost and the lost data message belongs to the first type message flow;

and retransmitting key data carried by the lost data message in the first type message stream, and restarting the data message transmission of the data to be transmitted in the target data after the retransmission is finished.

10. The method according to claim 1, wherein the method further comprises:

when the confirmation response message represents that the data message is lost and the lost data message belongs to the second class message flow, canceling the retransmission processing task of the lost data message belonging to the second class message flow; or alternatively, the first and second heat exchangers may be,

and when the confirmation response message indicates that the data message is lost and the lost data message belongs to the second type message stream, waiting for the transmission of the data to be transmitted in the target data to finish, and retransmitting non-critical data carried by the lost data message in the second type message stream after the transmission of the data message of the data to be transmitted is finished.

11. The method according to any one of claims 1 to 10, wherein the data message carries a data offset value and a data length;

the retransmitted data message is configured to enable a receiving end of the retransmitted data message to determine an ordering position of key data carried in the retransmitted data message in the target data based on a data offset value and a data length carried by the retransmitted data message.

12. A data transmission apparatus, the apparatus comprising:

13. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 11 when the computer program is executed.

14. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 11.

15. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the steps of the method of any one of claims 1 to 11.