CN112995685B

CN112995685B - Data transmitting method and device, data receiving method and device, medium and equipment

Info

Publication number: CN112995685B
Application number: CN202110164316.4A
Authority: CN
Inventors: 王兴鹤; 阮良; 陈功
Original assignee: Hangzhou Netease Zhiqi Technology Co Ltd
Current assignee: Hangzhou Netease Zhiqi Technology Co Ltd
Priority date: 2021-02-05
Filing date: 2021-02-05
Publication date: 2023-02-17
Anticipated expiration: 2041-02-05
Also published as: CN112995685A

Abstract

The disclosure provides a data sending method and device, a data receiving method and device, a medium and equipment, and relates to the technical field of data transmission. The data transmission method comprises the following steps: determining a target reference frame corresponding to a frame to be transmitted currently from a preset reference frame cache; the target reference frame is obtained by successfully framing a transmitted frame corresponding to the current frame to be transmitted; coding the current frame to be transmitted according to the target reference frame, and subpackaging the coded current frame to be transmitted to obtain a plurality of sub data packets; writing the first frame number of the current frame to be transmitted and the second frame number of the target reference frame into each sub-data packet to obtain a plurality of real-time transmission protocol packets; and sending the real-time transmission protocol packet to a data receiving end. The method and the device have the advantages that the redundant code rate is not required to be adopted, so that the problem of traffic waste caused by transmission of the redundant code rate is remarkably reduced.

Description

Data transmitting method and device, data receiving method and device, medium and equipment

Technical Field

Embodiments of the present invention relate to the field of data transmission, and more particularly, to a data transmission method, a data transmission apparatus, a data reception method, a data reception apparatus, a computer-readable storage medium, and an electronic device.

Background

This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.

When video data is transmitted in a network, if network packet loss or jitter occurs, a receiving end will be stuck when playing the video data.

In order to avoid blocking, in some technical schemes, for persistent packet loss, some redundancy code rates are increased during transmission, so that a receiving end can recover packet loss data by using the received redundancy code rates in combination with original code rates; however, for bursty packet loss, if the method of the redundant code rate is always adopted, there is a problem of traffic waste.

Disclosure of Invention

However, in some technologies, there is a waste of traffic. Therefore, an improved data transmission method is highly needed to avoid the problems of congestion caused by forward error correction and retransmission and traffic waste caused by redundant code rate for burst packet loss.

In this context, embodiments of the present invention are intended to provide a data transmission method, a data transmission apparatus, a data reception method, a data reception apparatus, a computer-readable storage medium, and an electronic device.

According to an aspect of the present disclosure, there is provided a data transmitting method including:

determining a target reference frame corresponding to a frame to be transmitted currently from a preset reference frame cache; the target reference frame is obtained by successfully framing a transmitted frame corresponding to the current frame to be transmitted;

coding the current frame to be transmitted according to the target reference frame, and subpackaging the coded current frame to be transmitted to obtain a plurality of sub-data packets;

writing the first frame number of the current frame to be transmitted and the second frame number of the target reference frame into each sub-data packet to obtain a plurality of real-time transmission protocol packets;

and sending the real-time transmission protocol packet to a data receiving end.

In an exemplary embodiment of the present disclosure, determining a target reference frame corresponding to the current frame to be transmitted from a preset reference frame buffer includes:

and when the confirmation frame is determined not to be received, determining the reference frame closest to the current time node in the reference frame cache as the target reference frame corresponding to the current frame to be transmitted.

In an exemplary embodiment of the present disclosure, determining a target reference frame corresponding to the current frame to be transmitted from a preset reference frame buffer further includes:

and when determining that a frame acknowledgement frame is received, determining the acknowledgement frame as a target reference frame corresponding to the current frame to be transmitted.

and when the multi-frame acknowledgement frame is determined to be received, determining a target reference frame corresponding to the current frame to be transmitted according to the time difference between the receiving time node of each acknowledgement frame and the coding time node of the current frame to be transmitted.

In an exemplary embodiment of the present disclosure, the data transmission method further includes:

and updating the preset reference frame buffer according to the confirmation frame.

In an exemplary embodiment of the present disclosure, updating the preset reference frame buffer according to the acknowledgement frame includes:

writing the confirmation frame into the idle buffer space when determining that the idle buffer space exists in the preset reference frame buffer;

and when the preset reference frame cache is determined to have no free cache space, covering the confirmation frame to a reference frame except for a reference frame closest to the current time node in the reference frame cache.

In an exemplary embodiment of the present disclosure, determining a target reference frame corresponding to the current frame to be transmitted from the reference frame buffer further includes:

calculating the frame distance between the current frame to be transmitted and the target reference frame according to the first frame number of the current frame to be transmitted and the second frame number of the target reference frame;

and when the frame distance is determined to be greater than a preset threshold value, determining a last frame transmitted frame corresponding to the current frame to be transmitted as a target reference frame corresponding to the current frame to be transmitted.

According to an aspect of the present disclosure, there is provided a data receiving method including:

receiving a plurality of real-time transmission protocol packets which are sent after the current frame to be transmitted is coded through a preset coding rule and the coded frame to be transmitted is packetized; the preset coding rule comprises the steps of determining a target reference frame corresponding to the current frame to be transmitted from a preset reference frame cache, wherein the target reference frame is obtained by successfully framing a transmitted frame corresponding to the current frame to be transmitted;

framing each real-time transmission protocol packet according to a first frame number of the current frame to be transmitted and a second frame number of the target reference frame, wherein the first frame number and the second frame number are included in each real-time transmission protocol packet, so as to obtain the current frame to be transmitted;

and when the frame grouping success of the current frame to be transmitted is determined, generating feedback information according to the first frame number and the second frame number, and decoding and playing the current frame to be transmitted.

In an exemplary embodiment of the present disclosure, the data receiving method further includes:

and sending the feedback information to the data sending end so that the data sending end determines target reference frames of other frames to be transmitted corresponding to the current frame to be transmitted according to the feedback information.

In an exemplary embodiment of the present disclosure, framing each real-time transport protocol packet according to a first frame number of the current frame to be transmitted and a second frame number of the target reference frame included in each real-time transport protocol packet includes:

determining whether each real-time transmission protocol packet has packet loss;

and when determining that no packet loss exists in each real-time transmission protocol packet, framing each real-time transmission protocol packet according to a first frame number of the current frame to be transmitted and a second frame number of the target reference frame included in each real-time transmission protocol packet.

and when determining that any real-time transmission protocol packet is lost, stopping framing each real-time transmission protocol packet, and framing each real-time transmission protocol packet after receiving the retransmitted lost real-time transmission protocol packet.

and when determining that any real-time transmission protocol packet is lost but the packet of the frame to be transmitted of the next frame corresponding to the current frame to be transmitted is successfully packaged, stopping retransmitting the lost real-time transmission protocol packet.

According to an aspect of the present disclosure, there is provided a data transmission apparatus including:

the target reference frame determining module is used for determining a target reference frame corresponding to the current frame to be transmitted from a preset reference frame buffer; the target reference frame is obtained by successfully framing a transmitted frame corresponding to the current frame to be transmitted;

the frame coding module is used for coding the current frame to be transmitted according to the target reference frame and subpackaging the coded current frame to be transmitted to obtain a plurality of sub data packets;

a frame number writing module, configured to write a first frame number of the current frame to be transmitted and a second frame number of the target reference frame into each sub-data packet, so as to obtain a plurality of real-time transport protocol packets;

and the protocol packet sending module is used for sending the real-time transmission protocol packet to a data receiving end.

In an exemplary embodiment of the present disclosure, the data transmitting apparatus further includes:

and the frame buffer updating module is used for updating the preset reference frame buffer according to the confirmation frame.

when determining that an idle cache space exists in the preset reference frame cache, writing the confirmation frame into the idle cache space;

According to an aspect of the present disclosure, there is provided a data receiving apparatus including:

the data receiving module is used for receiving a plurality of real-time transmission protocol packets which are sent after the frames to be transmitted are coded according to a preset coding rule and the coded frames to be transmitted are packetized; the preset coding rule comprises the steps of determining a target reference frame corresponding to the current frame to be transmitted from a preset reference frame cache, wherein the target reference frame is obtained by successfully framing a transmitted frame corresponding to the current frame to be transmitted;

a first protocol packet framing module, configured to frame each real-time transmission protocol packet according to a first frame number of the current frame to be transmitted and a second frame number of the target reference frame, which are included in each real-time transmission protocol packet, to obtain the current frame to be transmitted;

and the feedback information generating module is used for generating feedback information according to the first frame number and the second frame number when the frame grouping success of the current frame to be transmitted is determined, and decoding and playing the current frame to be transmitted.

In an exemplary embodiment of the present disclosure, the data receiving apparatus further includes:

and when determining that no packet loss exists in each real-time transmission protocol packet, framing each real-time transmission protocol packet according to a first frame number of the current frame to be transmitted and a second frame number of the target reference frame, which are included in each real-time transmission protocol packet.

and the second protocol packet framing module stops framing each real-time transmission protocol packet when determining that any real-time transmission protocol packet is lost, and frames each real-time transmission protocol packet after receiving the retransmitted lost real-time transmission protocol packet.

and the retransmission stopping module is used for stopping retransmitting the lost real-time transmission protocol packet when determining that the packet of any real-time transmission protocol packet is lost but the packet of the frame to be transmitted of the next frame corresponding to the current frame to be transmitted is successfully packaged.

According to an aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the data transmission method of any of the above, and the data reception method of any of the above.

According to an aspect of the present disclosure, there is provided an electronic device including:

a processor; and

a memory for storing executable instructions of the processor;

wherein the processor is configured to perform the data transmission method of any one of the above and the data reception method of any one of the above via execution of the executable instructions.

According to the data transmission method and the data transmission device provided by the embodiment of the invention, the target reference frame corresponding to the current frame to be transmitted can be determined from the preset reference frame buffer; the target reference frame is obtained by successfully framing a transmitted frame corresponding to the current frame to be transmitted; coding the current frame to be transmitted according to the target reference frame, and subpackaging the coded current frame to be transmitted to obtain a plurality of sub data packets; writing the first frame number of the current frame to be transmitted and the second frame number of the target reference frame into each sub-data packet to obtain a plurality of real-time transmission protocol packets; and finally, the real-time transmission protocol packet is sent to a data receiving end without adopting a redundant code rate, so that the problem of flow waste caused by transmission of the redundant code rate is remarkably reduced, the problem of video blockage caused by time delay existing in forward error correction is reduced, and better experience is brought to a user.

Drawings

The above and other objects, features and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description read in conjunction with the accompanying drawings. Several embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which:

fig. 1 schematically shows a flow chart of a data transmission method according to an example embodiment of the present disclosure;

fig. 2 schematically illustrates a block diagram of a video transmission system according to an example embodiment of the present disclosure;

fig. 3 schematically illustrates an example of an acknowledgement scenario for a target reference frame according to an example embodiment of the present disclosure;

fig. 4 schematically illustrates another example view of an acknowledgement scenario for a target reference frame according to an example embodiment of the present disclosure;

fig. 5 schematically illustrates another example view of an acknowledgement scenario for a target reference frame according to an example embodiment of the present disclosure;

FIG. 6 schematically illustrates an example view of an update scenario of a reference frame buffer according to an example embodiment of the present disclosure;

fig. 7 schematically shows a flow chart of a data receiving method according to an example embodiment of the present disclosure;

fig. 8 schematically illustrates a flow chart of a video transmission method according to an example embodiment of the present disclosure;

fig. 9 schematically shows a block diagram of a data transmission apparatus according to an example embodiment of the present disclosure;

fig. 10 schematically shows a block diagram of a data receiving apparatus according to an example embodiment of the present disclosure;

fig. 11 schematically illustrates a computer-readable storage medium for storing the above-described data transmission method and data reception method according to an exemplary embodiment of the present disclosure;

fig. 12 schematically shows an electronic device for implementing the above-described data transmission method and data reception method according to an example embodiment of the present disclosure.

In the drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

Detailed Description

The principles and spirit of the present invention will be described with reference to a number of exemplary embodiments. It is understood that these embodiments are given solely for the purpose of enabling those skilled in the art to better understand and to practice the invention, and are not intended to limit the scope of the invention in any way. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

As will be appreciated by one skilled in the art, embodiments of the present invention may be embodied as a system, apparatus, device, method, or computer program product. Accordingly, the present disclosure may be embodied in the form of: entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.

According to an embodiment of the present invention, a data transmission method, a data transmission device, a data reception method, a data reception device, a computer-readable storage medium, and an electronic apparatus are provided.

In this document, it is to be understood that any number of elements in the figures are provided by way of illustration and not limitation, and any nomenclature is used for differentiation only and not in any limiting sense.

The principles and spirit of the present invention are explained in detail below with reference to several representative embodiments of the invention.

Summary of The Invention

The inventor finds that, when video data is transmitted in a network, if packet loss or jitter occurs, a receiving end will be jammed when playing. In some technical solutions, common techniques for packet loss resistance include retransmission of lost data packets and packet loss resistance using forward error correction.

The retransmission is to notify the sending end to retransmit when data loss is detected, the code rate utilization efficiency of the technology is high, but when the network delay is large, the delay sacrifice is very large, and the technology is not suitable for a communication scene;

the forward error correction refers to increasing some redundant code rates during transmission, and the receiving end recovers the packet-lost data by combining the received redundant code rates with the original code rates. Although forward error correction does not introduce additional delay, there is additional rate overhead; moreover, the redundancy of the forward error correction is dynamically changed according to the network conditions, and is mainly related to the packet loss rate and the network delay. When packet loss of the network is detected, forward error correction data is added during the next frame of coding, the greater the packet loss rate is, the greater the redundancy is, the greater the network delay is, and the greater the redundancy is.

Further, the network packet loss type includes burst type packet loss and continuous type packet loss; for continuous packet loss, forward error correction is adopted, so that the packet loss resistance performance is better; and, with forward error correction redundancy, after a period of convergence, it will be in a stable state. However, for burst-type packet loss, due to a certain convergence time of forward error correction and retransmission, the experience of the receiving end is still stuck in the convergence process; in addition, the redundancy code rate is brought in the normal network, the flow is increased, and the congestion control is not facilitated.

Based on this, the present disclosure provides a data transmission method, in one aspect, a target reference frame corresponding to a current frame to be transmitted is determined from a preset reference frame buffer; then, coding the current frame to be transmitted according to the target reference frame, and subpackaging the coded current frame to be transmitted to obtain a plurality of sub-data packets; writing the first frame number of the current frame to be transmitted and the second frame number of the target reference frame into each sub-data packet to obtain a plurality of real-time transmission protocol packets; finally, the real-time transmission protocol packet is sent to a data receiving end, and as the target reference frame is obtained by successfully framing the transmitted frame corresponding to the current frame to be transmitted, the target reference frame can be determined to have no packet loss, so that the problem that the current frame to be transmitted has no packet loss can be ensured to the maximum extent; on the other hand, because the current frame to be transmitted is coded according to the target reference, and only the first frame number and the second frame number are written in each sub data packet, the problem of flow waste caused by the need of writing the redundant code rate is avoided.

Having described the general principles of the invention, various non-limiting embodiments of the invention are described in detail below.

Exemplary method

A data transmission method according to an exemplary embodiment of the present invention is described below with reference to fig. 1, where the method may be executed in a terminal device, a server cluster or a cloud server, etc.; of course, those skilled in the art may also operate the method of the present invention on other platforms as needed, and this is not particularly limited in this exemplary embodiment. Referring to fig. 1, the data transmission method may include the steps of:

s110, determining a target reference frame corresponding to a current frame to be transmitted from a preset reference frame buffer; the target reference frame is obtained by successfully framing a transmitted frame corresponding to the current frame to be transmitted;

s120, coding the current frame to be transmitted according to the target reference frame, and subpackaging the coded current frame to be transmitted to obtain a plurality of sub data packets;

s130, writing a first frame number of the current frame to be transmitted and a second frame number of the target reference frame into each sub-data packet to obtain a plurality of real-time transmission protocol packets;

and S140, sending the real-time transmission protocol packet to a data receiving end.

In the data transmission method, a target reference frame corresponding to a current frame to be transmitted can be determined from a preset reference frame buffer; the target reference frame is obtained by successfully framing a transmitted frame corresponding to the current frame to be transmitted; coding the current frame to be transmitted according to the target reference frame, and subpackaging the coded current frame to be transmitted to obtain a plurality of sub data packets; writing the first frame number of the current frame to be transmitted and the second frame number of the target reference frame into each sub-data packet to obtain a plurality of real-time transmission protocol packets; and finally, the real-time transmission protocol packet is sent to a data receiving end without adopting a redundant code rate, so that the problem of flow waste caused by transmission of the redundant code rate is remarkably reduced, the problem of video blockage caused by time delay existing in forward error correction is reduced, and better experience is brought to a user.

First, terms referred to in example embodiments of the present disclosure are explained and explained.

NACK: the abbreviation of Negative Acknowledgement is a Negative feedback, and the receiver only notifies the sender when no data is received.

And ACK: the abbreviation of Acknowledgement, which is a forward feedback, is used to inform the sender of a reply message after the receiver receives the data.

RTT: the abbreviation of Round-trip time, which is the time consumed by one Round trip in the network, is used for evaluating the index of network delay.

RTP: abbreviation of Real-time Transport Protocol, real-time Transport Protocol.

RTCP: abbreviation of Real-time Transport Control Protocol, real-time Transport Control Protocol.

The Jitter Buffer functions to remove Jitter that may be caused by the network or introduced by retransmissions. Compared with a fixed Buffer Jitter elimination mode, the dynamic Jitter Buffer can better adapt to the Jitter and packet loss conditions of the previous network, and has lower delay.

Next, the video transmission system of the exemplary embodiment of the present disclosure is explained and explained. Referring to fig. 2, the video transmission system may include a data transmitting end 210 and a data receiving end 220, which are connected via a network. The data sending end may include an encoder 211, a packetization encapsulation module 212, a feedback receiving module 213, a reference frame scheduling module 214, an image quality detection module 215, and a data acquisition module 216; the data receiving end may include a packet loss resilience module 221, a framing module 222, a feedback information generation module 223, a de-jitter module 224, and a decoding playing module 225.

Specifically, after receiving feedback information sent by the feedback information generation module through a real-time transmission control protocol, the feedback information receiving module sends the feedback information to the reference scheduling module, and the reference scheduling module decides a target reference frame according to an internal preset algorithm and sends the target reference frame to the encoder; the encoder encodes a current frame to be transmitted of the video data acquired by the data acquisition module according to the target reference frame and transmits the encoded current frame to be transmitted to the sub-packet encapsulation module; the sub-packet packaging module writes a first frame number of a current frame to be transmitted and a second frame number of a target reference frame into a plurality of sub-data packets obtained according to the coded current frame to be transmitted to obtain a plurality of real-time transmission protocol packets, and sends each real-time transmission protocol packet to the anti-packet loss repairing module; when the packet loss resisting repair module detects that packet loss does not exist, the frame grouping module is informed to carry out frame grouping, and after the frame grouping module successfully groups frames, the feedback information generation module is informed to generate feedback information and send the feedback information to the feedback information receiving module; further, after the framing is successful, if the dithering needs to be removed, the dithering removal module is notified to remove the dithering, and the frame to be transmitted at present after the dithering removal is decoded and played through the decoding and playing module.

It should be noted that, in order to avoid the problem that the image quality is sacrificed due to the excessively large distance between the target reference frame and the current frame to be transmitted, the image quality detection module needs to compare the image quality of the acquired video data with the image quality of the target reference frame, and if the image quality is too poor and there is no packet loss in a certain time, a maximum reference distance may be determined according to the number of video frames transmitted in the certain time, and then the maximum reference distance is fed back to the encoder. By the method, the problem that the user experience is poor due to poor image quality can be avoided.

In an exemplary data transmission method of the present disclosure:

in step S110, a target reference frame corresponding to a frame to be transmitted currently is determined from a preset reference frame buffer; the target reference frame is obtained by successfully framing a transmitted frame corresponding to the current frame to be transmitted.

In this exemplary embodiment, in order to encode a current frame to be encoded, a target reference frame corresponding to the current frame needs to be determined first. Specifically, the target reference frame corresponding to the frame to be transmitted currently may be determined from a preset reference frame buffer. A fixed number of reference frame buffers (which may include eight frames, for example) may be maintained inside the encoder, each buffer stores one frame of data, and each frame of data may also be marked as a stored frame that is a long-term reference frame or a short-term reference frame (i.e., a forward reference), which is not limited in this example. Meanwhile, the disclosure explains and explains that all buffers are marked as long-term reference frames; furthermore, a frame fed back by the receiving end can be marked, which is called an ACK frame (acknowledgement frame), and the ACK frame is a reference frame preferentially selected by the current coding; if the buffer is full, the oldest frame except the ACK frame is preferentially overwritten.

The target reference frame corresponding to the current frame to be transmitted is determined from a preset reference frame buffer, and the determination can be performed in one of the following ways:

the first method is that when it is determined that no acknowledgement frame is received, a reference frame closest to a current time node in the reference frame cache is determined as a target reference frame corresponding to the current frame to be transmitted. For example, referring to fig. 3, when 1 to 4 frames (frames to be transmitted currently) are encoded, if no ACK frame is received, the I frame (the reference frame closest to the current time node) continues to be referred to.

And secondly, when a frame acknowledgement frame is determined to be received, determining the frame acknowledgement frame as a target reference frame corresponding to the current frame to be transmitted. For example, referring to fig. 4, when encoding frame 14, an ACK for frame 7 is received, and the ACK may be used as a target reference frame for frame 14 (a frame to be currently transmitted).

And thirdly, when the multi-frame acknowledgement frame is determined to be received, determining a target reference frame corresponding to the current frame to be transmitted according to the time difference between the receiving time node of each acknowledgement frame and the coding time node of the current frame to be transmitted. For example, as shown in fig. 5, when encoding frames 5 to 8 and receiving ACKs of frames 2 to 4, the ACK frame at each time is set as the target reference frame. That is, the time difference between the receiving time node of the ACK and the encoding time node of each frame is calculated first, and then the ACK with the smallest time difference is selected as the target reference frame of each frame.

It should be added that the principle of the above coding scheduling scheme is to always use the ACK frame fed back by the reference receiving end as the reference frame of the current coding frame, so that the decoding failure does not exist at the receiving end under the normal condition of the coded video, the capability of adapting to network packet loss is very strong, and the playing can be continued as long as the framing at the receiving end is successful. However, considering that the image quality is sacrificed due to the larger reference distance, the reference frame scheduling module needs to receive the maximum reference distance given by the image quality detection module, and if the current ACK is too late and the reference distance exceeds the maximum value, the normal encoding mode is returned to, i.e., the previous frame is reference-encoded. Therefore, a target reference frame corresponding to the frame to be transmitted currently is determined from the preset reference frame buffer, and the following fourth scheme also exists:

fourthly, firstly, calculating the frame distance between the current frame to be transmitted and the target reference frame according to the first frame number of the current frame to be transmitted and the second frame number of the target reference frame; and secondly, when the frame distance is determined to be larger than a preset threshold value, determining a last frame transmitted frame corresponding to the current frame to be transmitted as a target reference frame corresponding to the current frame to be transmitted. The preset threshold may be determined according to actual needs, for example, eight frames, ten frames, or sixteen frames, and the like, which is not particularly limited in this example. It should be further added here that the target reference frame may also be determined in other ways according to actual needs, and the illustrated scheme of the present disclosure is only for exemplary purposes, but is not limited thereto.

Further, in order to determine a target reference frame corresponding to a frame to be transmitted currently from a preset reference frame buffer, the preset reference frame buffer needs to be updated according to the confirmation frame. The method specifically comprises the following steps: writing the confirmation frame into the idle buffer space when determining that the idle buffer space exists in the preset reference frame buffer; and when the condition that the idle cache space does not exist in the preset reference frame cache is determined, covering the confirmation frame to a reference frame except for a reference frame which is closest to the current time node in the reference frame cache. Specifically, as shown in fig. 6, when the frames 9 to 13 are encoded, there is no ACK and the preset reference frame Buffer is full, the division frame 4 (the reference frame closest to the current time node) is not covered, and the ACK frame is used as the reference object. By the method, the packet loss rate is further reduced.

In step S120, the current frame to be transmitted is encoded according to the target reference frame, and the encoded current frame to be transmitted is packetized to obtain a plurality of sub-packets.

In step S130, the first frame number of the current frame to be transmitted and the second frame number of the target reference frame are written into each sub-data packet, so as to obtain a plurality of real-time transport protocol packets.

In step S140, the real-time transport protocol packet is sent to a data receiving end.

Hereinafter, step S120 to step S140 will be explained. Specifically, firstly, encoding (compressing) a current frame to be transmitted according to a frame rate, a code rate, a resolution ratio and the like adopted by a target reference frame in an encoding process, and then subpackaging the encoded current frame to be transmitted to obtain a plurality of sub-data packets; and then writing the first frame number and the second frame number into each sub-data packet to obtain a plurality of real-time transmission protocol packets, and sending the real-time transmission protocol packets to a data receiving end so that the data receiving end performs framing on each real-time transmission protocol packet according to the first frame number and the second frame number, and decoding and playing. It should be noted that, because the target reference frame is obtained by successfully framing the transmitted frame corresponding to the current frame to be transmitted, and only the first frame number and the second frame number need to be written into each sub-packet, the packet loss is avoided, the redundant bit rate is also avoided, the traffic utilization rate is improved, and the data transmission efficiency is also improved.

In an exemplary data receiving method of the present disclosure, as shown with reference to fig. 7, the data receiving method may include:

in step S710, receiving a plurality of real-time transport protocol packets that are transmitted after encoding a current frame to be transmitted according to a preset encoding rule and packetizing the encoded current frame to be transmitted; the preset coding rule comprises the step of determining a target reference frame corresponding to the current frame to be transmitted from a preset reference frame cache, wherein the target reference frame is obtained by successfully framing a transmitted frame corresponding to the current frame to be transmitted.

In step S720, framing each real-time transport protocol packet according to a first frame number of the current frame to be transported and a second frame number of the target reference frame included in each real-time transport protocol packet, to obtain the current frame to be transported;

in step S730, when it is determined that the current frame to be transmitted is successfully framed, feedback information is generated according to the first frame number and the second frame number, and the current frame to be transmitted is decoded and played.

Further, after the feedback information is obtained, the feedback information needs to be sent to the data sending end, so that the data sending end determines the target reference frames of other frames to be transmitted corresponding to the current frame to be transmitted according to the feedback information.

In the data receiving method schematically shown in fig. 7, on one hand, since the target reference frame is obtained by successfully framing the transmitted frame corresponding to the current frame to be transmitted, it can be determined that the target reference frame has no packet loss, and thus the problem that the current frame to be transmitted has no packet loss can be ensured to the maximum extent; on the other hand, the current frame to be transmitted is coded according to the target reference, and the first frame number and the second frame number are only written in each sub-data packet, so that the problem of flow waste caused by the fact that redundant code rates need to be written is solved.

In an exemplary embodiment, in order to perform successful framing, it is further required to determine whether there is a packet loss in each real-time transport protocol packet. Specifically, firstly, determining whether each real-time transport protocol packet has a packet loss; then, when it is determined that no packet loss exists in each real-time transmission protocol packet, framing each real-time transmission protocol packet according to a first frame number of the current frame to be transmitted and a second frame number of the target reference frame included in each real-time transmission protocol packet.

In an example embodiment, when it is determined that any of the real-time transport protocol packets is lost, framing of each of the real-time transport protocol packets is stopped, and after a retransmitted lost real-time transport protocol packet is received, framing of each of the real-time transport protocol packets is performed. And when determining that any real-time transmission protocol packet is lost but the packet of the frame to be transmitted of the next frame corresponding to the current frame to be transmitted is successfully packaged, stopping retransmitting the lost real-time transmission protocol packet.

For example, when the packet loss prevention repair module detects that the real-time transmission protocol packet has packet loss, the frame grouping module is notified to be in a waiting state, the packet loss prevention repair module waits for recovery through retransmission and forward error correction, and framing is performed after the recovery is successful; meanwhile, if the frame after the frame where the current packet is lost is framed in advance, the framing module notifies the packet loss resistant repair module to stop retransmission so as to save retransmission code rate.

Hereinafter, the video transmission method of the present disclosure is further explained and explained with reference to fig. 8. Referring to fig. 8, the video transmission method may include the steps of:

step S810, confirming a target reference frame; wherein the target reference frame is a long-term reference frame;

step S820, encoding the current frame to be transmitted based on the target reference frame;

step S830, performing packet packaging on the coded current frame to be transmitted to obtain a plurality of real-time transmission protocol packets, and sending each real-time transmission protocol packet to a data receiving end;

step 840, the data receiving end frames each real-time transmission protocol packet;

step S850, after the framing is determined to be successful, generating feedback information and sending the feedback information to a data sending end;

and step S860, decoding and playing the video frame after the framing is successful.

It should be added that when the current frame to be transmitted is encoded based on the target reference frame, it is further required to determine whether the frame distance between the current frame to be transmitted and the target reference frame exceeds the maximum reference distance based on the maximum reference distance given by the image quality detection module, and if so, the encoding mode is returned to the normal encoding mode, that is, the encoding is referred to the previous frame.

According to the data transmission method provided by the disclosure, on one hand, the current frame to be transmitted is encoded through the feedback-based long reference frame, so that the network burst packet loss can be effectively resisted; meanwhile, compared with the traditional packet loss resisting technology, the scheme provided by the disclosure establishes the coordination of coding and decoding through feedback, adaptively adjusts the reference distance to adapt to network packet loss, effectively achieves the purpose of resisting burst packet loss, does not sacrifice delay, does not increase redundant code rate, and further has lower delay and less code rate under the same packet loss capability; on the other hand, the reference frame scheduling design breaks through the limitation of the maximum 16-frame reference distance, and the design of the reserved ACK frame enables extremely smooth experience to be achieved under the condition that definition is allowed; on the other hand, after the coding reference distance exceeds a certain degree, the coder returns back to a forward reference strategy, and effective balance is made between smooth flow and clear flow; meanwhile, the maximum reference distance returned by the encoder is dynamically self-adaptive according to the scene, and the smoothness is maximally improved while the definition is met.

Exemplary devices

Having described the method of the exemplary embodiment of the present invention, next, a data transmission apparatus of the exemplary embodiment of the present invention will be described with reference to fig. 9.

Referring to fig. 9, the data transmission apparatus may include a target reference frame determination module 910, a frame encoding module 920, a frame number writing module 930, and a protocol packet transmission module 940. Wherein:

the target reference frame determining module 910 may be configured to determine a target reference frame corresponding to a frame to be currently transmitted from a preset reference frame buffer; the target reference frame is obtained by successfully framing a transmitted frame corresponding to the current frame to be transmitted;

the frame coding module 920 may be configured to code the current frame to be transmitted according to the target reference frame, and perform packetization on the coded current frame to be transmitted to obtain a plurality of sub-packets;

the frame number writing module 930 may be configured to write a first frame number of the current frame to be transmitted and a second frame number of the target reference frame into each sub-data packet, so as to obtain a plurality of real-time transport protocol packets;

the protocol packet sending module 940 may be configured to send the real-time transport protocol packet to a data receiving end.

Fig. 9 schematically illustrates an example embodiment, in one aspect, a target reference frame corresponding to a frame to be transmitted currently is determined from a preset reference frame buffer; then, coding the current frame to be transmitted according to the target reference frame, and subpackaging the coded current frame to be transmitted to obtain a plurality of sub-data packets; writing the first frame number of the current frame to be transmitted and the second frame number of the target reference frame into each sub-data packet to obtain a plurality of real-time transmission protocol packets; finally, the real-time transmission protocol packet is sent to a data receiving end, and as the target reference frame is obtained by successfully framing the transmitted frame corresponding to the current frame to be transmitted, the target reference frame can be determined to have no packet loss, so that the problem that the current frame to be transmitted has no packet loss can be ensured to the maximum extent; on the other hand, because the current frame to be transmitted is coded according to the target reference, and only the first frame number and the second frame number are written in each sub data packet, the problem of flow waste caused by the need of writing the redundant code rate is avoided.

and when the condition that the idle cache space does not exist in the preset reference frame cache is determined, covering the confirmation frame to a reference frame except for a reference frame which is closest to the current time node in the reference frame cache.

The present disclosure also provides a data receiving apparatus. Referring to fig. 10, the data receiving apparatus may include a data receiving module 1010, a first protocol packet framing module 1020, and a feedback information generating module 1030. Wherein:

the data receiving module 1010 may be configured to receive a plurality of real-time transport protocol packets, which are sent after encoding a current frame to be transmitted according to a preset encoding rule and packetizing the encoded current frame to be transmitted; the preset coding rule comprises the steps of determining a target reference frame corresponding to the current frame to be transmitted from a preset reference frame cache, wherein the target reference frame is obtained by successfully framing a transmitted frame corresponding to the current frame to be transmitted;

the first protocol packet framing module 1020 may be configured to frame each real-time transmission protocol packet according to a first frame number of the current frame to be transmitted and a second frame number of the target reference frame included in each real-time transmission protocol packet, so as to obtain the current frame to be transmitted;

the feedback information generating module 1030 may be configured to generate feedback information according to the first frame number and the second frame number when it is determined that the current frame to be transmitted is successfully framed, and decode and play the current frame to be transmitted.

In the data receiving apparatus schematically shown in fig. 10, on one hand, since the target reference frame is obtained by successfully framing the transmitted frame corresponding to the current frame to be transmitted, it can be determined that the target reference frame has no packet loss, and thus it can be ensured that the current frame to be transmitted has no packet loss to the maximum extent; on the other hand, because the current frame to be transmitted is coded according to the target reference, and only the first frame number and the second frame number are written in each sub data packet, the problem of flow waste caused by the need of writing the redundant code rate is avoided.

Exemplary storage Medium

Having described the data transmission method and apparatus, and the data reception method and apparatus according to exemplary embodiments of the present invention, a storage medium according to an exemplary embodiment of the present invention will be described with reference to fig. 11.

Referring to fig. 11, a program product 1100 for implementing the above method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. The readable signal medium may also be any readable medium other than a readable storage medium.

Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user computing device, partly on the user device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN).

Exemplary electronic device

Having described the storage medium of an exemplary embodiment of the present invention, next, an electronic device of an exemplary embodiment of the present invention will be described with reference to fig. 7.

The electronic device 1200 shown in fig. 12 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.

As shown in fig. 12, the electronic device 1200 is embodied in the form of a general purpose computing device. The components of the electronic device 1200 may include, but are not limited to: the at least one processing unit 1210, the at least one memory unit 1220, the bus 1230 connecting the various system components (including the memory unit 1220 and the processing unit 1210), and the display unit 1240.

Wherein the storage unit 1220 stores program codes that can be executed by the processing unit 1210 such that the processing unit 1210 performs the steps according to various exemplary embodiments of the present invention described in the above section "exemplary method" of the present specification. For example, the processing unit 1210 may perform steps S120-S140 as shown in fig. 1 and steps S710-S730 as shown in fig. 6.

The storage unit 1220 may include volatile storage units such as a random access memory unit (RAM) 12201 and/or a cache memory unit 12202, and may further include a read only memory unit (ROM) 12203.

Storage unit 1220 may also include a program/utility 12204 having a set (at least one) of program modules 12205, such program modules 12205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

Bus 1230 may include a data bus, an address bus, and a control bus.

The electronic device 1200 may also communicate with one or more external devices 1300 (e.g., keyboard, pointing device, bluetooth device, etc.) via an input/output (I/O) interface 1250. Also, the electronic device 1200 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 1260. As shown, the network adapter 1260 communicates with the other modules of the electronic device 1200 via the bus 1230. It should be appreciated that although not shown, other hardware and/or software modules may be used in conjunction with the electronic device 1200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

It should be noted that although in the above detailed description several modules or sub-modules of the pop-up window processing apparatus are mentioned, such division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more of the units/modules described above may be embodied in one unit/module according to embodiments of the invention. Conversely, the features and functions of one unit/module described above may be further divided into embodiments by a plurality of units/modules.

It should be noted that although in the above detailed description several units/modules or sub-units/modules of the apparatus are mentioned, such a division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more of the units/modules described above may be embodied in one unit/module according to embodiments of the invention. Conversely, the features and functions of one unit/module described above may be further divided into embodiments by a plurality of units/modules.

Moreover, while the operations of the method of the invention are depicted in the drawings in a particular order, this does not require or imply that the operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.

While the spirit and principles of the invention have been described with reference to several particular embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, nor is the division of aspects, which is for convenience only as the features in such aspects may not be combined to benefit. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A method of data transmission, comprising:

updating a preset reference frame cache according to the confirmation frame, and determining a target reference frame corresponding to the current frame to be transmitted from the preset reference frame cache; the target reference frame is obtained by successfully framing a transmitted frame corresponding to the current frame to be transmitted; the method comprises the following steps: calculating the frame distance between the current frame to be transmitted and the target reference frame according to the first frame number of the current frame to be transmitted and the second frame number of the target reference frame; when the frame distance is determined to be greater than a preset threshold value, determining a last frame transmitted frame corresponding to the current frame to be transmitted as a target reference frame corresponding to the current frame to be transmitted;

coding the current frame to be transmitted according to the target reference frame, and subpackaging the coded current frame to be transmitted to obtain a plurality of sub data packets;

and sending the real-time transmission protocol packet to a data receiving end.

2. The data transmission method according to claim 1, wherein determining the target reference frame corresponding to the current frame to be transmitted from a preset reference frame buffer includes:

3. The data transmission method according to claim 1, wherein the determining a target reference frame corresponding to the current frame to be transmitted from a preset reference frame buffer further comprises:

4. The data transmission method according to claim 1, wherein the determining a target reference frame corresponding to the current frame to be transmitted from a preset reference frame buffer further comprises:

5. The data transmission method according to claim 1, wherein updating the preset reference frame buffer according to the acknowledgment frame comprises:

6. A data receiving method, comprising:

receiving a plurality of real-time transmission protocol packets which are sent after the current frame to be transmitted is coded through a preset coding rule and the coded frame to be transmitted is packetized; updating a preset reference frame cache according to a confirmation frame, and determining a target reference frame corresponding to the current frame to be transmitted from the preset reference frame cache, wherein the target reference frame is obtained by successfully framing a transmitted frame corresponding to the current frame to be transmitted; the determining, from a preset reference frame buffer, a target reference frame corresponding to the current frame to be transmitted includes: calculating the frame distance between the current frame to be transmitted and the target reference frame according to the first frame number of the current frame to be transmitted and the second frame number of the target reference frame; when the frame distance is determined to be larger than a preset threshold value, determining a last frame transmitted frame corresponding to the current frame to be transmitted as a target reference frame corresponding to the current frame to be transmitted;

7. The data receiving method of claim 6, wherein the data receiving method further comprises:

and sending the feedback information to a data sending end so that the data sending end determines target reference frames of other frames to be transmitted corresponding to the current frame to be transmitted according to the feedback information.

8. The data receiving method according to claim 6, wherein framing each real-time transport protocol packet according to a first frame number of the current frame to be transmitted and a second frame number of the target reference frame included in each real-time transport protocol packet comprises:

9. The data receiving method as claimed in claim 8, wherein the data receiving method further comprises:

10. The data receiving method as claimed in claim 9, wherein the data receiving method further comprises:

11. A data transmission apparatus comprising:

the frame buffer updating module is used for updating the preset reference frame buffer according to the confirmation frame;

the target reference frame determining module is used for determining a target reference frame corresponding to the current frame to be transmitted from a preset reference frame buffer; the target reference frame is obtained by successfully framing a transmitted frame corresponding to the current frame to be transmitted; the method comprises the following steps: calculating the frame distance between the current frame to be transmitted and the target reference frame according to the first frame number of the current frame to be transmitted and the second frame number of the target reference frame; when the frame distance is determined to be greater than a preset threshold value, determining a last frame transmitted frame corresponding to the current frame to be transmitted as a target reference frame corresponding to the current frame to be transmitted;

12. The data transmission apparatus according to claim 11, wherein determining the target reference frame corresponding to the current frame to be transmitted from a preset reference frame buffer includes:

13. The data transmission apparatus according to claim 11, wherein the determining a target reference frame corresponding to the current frame to be transmitted from a preset reference frame buffer further comprises:

and when determining that a frame confirmation frame is received, determining the confirmation frame as a target reference frame corresponding to the current frame to be transmitted.

14. The data transmission apparatus according to claim 11, wherein the determining a target reference frame corresponding to the current frame to be transmitted from a preset reference frame buffer further comprises:

15. The data transmission apparatus according to claim 11, wherein updating the preset reference frame buffer according to the acknowledgment frame comprises:

16. A data receiving device, comprising:

the data receiving module is used for receiving a plurality of real-time transmission protocol packets which are sent after the frames to be transmitted are coded according to a preset coding rule and the coded frames to be transmitted are packetized; updating the preset reference frame cache according to a confirmation frame, and determining a target reference frame corresponding to the current frame to be transmitted from the preset reference frame cache, wherein the target reference frame is obtained by successfully framing a transmitted frame corresponding to the current frame to be transmitted; the determining, from a preset reference frame buffer, a target reference frame corresponding to the current frame to be transmitted includes: calculating the frame distance between the current frame to be transmitted and the target reference frame according to the first frame number of the current frame to be transmitted and the second frame number of the target reference frame; when the frame distance is determined to be greater than a preset threshold value, determining a last frame transmitted frame corresponding to the current frame to be transmitted as a target reference frame corresponding to the current frame to be transmitted;

17. The data receiving apparatus according to claim 16, wherein the data receiving apparatus further comprises:

18. The data receiving device according to claim 17, wherein framing each of the real-time transport protocol packets according to a first frame number of the current frame to be transmitted and a second frame number of the target reference frame included in each of the real-time transport protocol packets comprises:

19. The data receiving apparatus according to claim 18, wherein the data receiving apparatus further comprises:

20. The data receiving apparatus according to claim 19, wherein the data receiving apparatus further comprises:

21. A computer-readable storage medium on which a computer program is stored which, when executed by a processor, implements the data transmission method of any one of claims 1 to 5 and the data reception method of any one of claims 6 to 10.

22. An electronic device, comprising:

a processor; and

a memory for storing executable instructions of the processor;

wherein the processor is configured to perform the data transmission method of any one of claims 1-5 and the data reception method of any one of claims 6-10 via execution of the executable instructions.