CN110557677A - Video transmission method and device - Google Patents
Video transmission method and device Download PDFInfo
- Publication number
- CN110557677A CN110557677A CN201910924268.7A CN201910924268A CN110557677A CN 110557677 A CN110557677 A CN 110557677A CN 201910924268 A CN201910924268 A CN 201910924268A CN 110557677 A CN110557677 A CN 110557677A
- Authority
- CN
- China
- Prior art keywords
- data packet
- server
- retransmission request
- video frame
- lost
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
- H04N21/23106—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion involving caching operations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
- H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
- H04N21/44209—Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/637—Control signals issued by the client directed to the server or network components
- H04N21/6375—Control signals issued by the client directed to the server or network components for requesting retransmission, e.g. of data packets lost or corrupted during transmission from server
Abstract
The application provides a video transmission method and a video transmission device, which are used for a client, wherein the method comprises the following steps: receiving video frames sent by a server, wherein each video frame comprises a plurality of data packets; analyzing the received video frame, and sending a first retransmission request to a server under the condition that the video frame is determined to be lost, so that the server can resend the lost data packet; and if the first retransmission request is determined to fail, sending a second retransmission request to the server, so that the server sends the complete video frame again after the video frame is sent, thereby realizing double guarantee for retransmission of lost packets, solving the problem of lost packets and reducing network delay.
Description
Technical Field
The present application relates to the field of internet technologies, and in particular, to a method and an apparatus for video transmission, a computing device, and a computer-readable storage medium.
Background
In many games, the word network delay is often present, with some players high and some players low. Network latency refers to the round-trip time for a data packet to be sent from the user's computer to the web server and then immediately returned from the web server to the user's computer.
In the cloud game in the prior art, a large amount of video and audio data transmission exists between the game server and the client, which is different from the common network game, and because the transmission of the cloud game is bidirectional, the tolerance of the cloud game to the transmission delay is greatly lower than that of the common network game, and how to reduce the network delay to the maximum extent under the limitation of the existing network bandwidth condition becomes one of the technical bottlenecks of the cloud game.
Disclosure of Invention
In view of the above, embodiments of the present application provide a method and an apparatus for video transmission, a computing device, and a computer-readable storage medium, so as to solve the technical defects in the prior art.
The embodiment of the application provides a video transmission method, which is used for a client, and comprises the following steps:
receiving video frames sent by a server, wherein each video frame comprises a plurality of data packets;
Analyzing the received video frame, and sending a first retransmission request to a server under the condition that the video frame is determined to be lost, so that the server can resend the lost data packet;
And if the first retransmission request is determined to fail, sending a second retransmission request to the server so that the server sends the complete video frame again after the video frame is sent.
optionally, analyzing the received video frame includes:
acquiring the serial number of a data packet of the video frame, and judging whether the serial number of the data packet is continuous or not;
If so, determining that the received video frame is complete;
And if not, the data packet corresponding to the sequence number of the missing data packet is the missing data packet.
optionally, sending a first retransmission request to a server to cause the server to resend the lost data packet includes:
And writing the serial number of the missing data packet into the first retransmission request and sending the first retransmission request to the server so that the server searches the data packet corresponding to the serial number of the missing data packet and resends the data packet corresponding to the serial number of the missing data packet.
Optionally, the method further comprises:
Detecting a network receiving the video frame;
When the network jitter is detected to be larger than a set threshold value, informing a server to reduce the default code rate or the default resolution of the video frame to the processed code rate or the processed resolution;
And under the condition that the network jitter is detected to be less than or equal to the set threshold value, informing a server to restore the processed code rate or the processed resolution of the video frame to the default code rate or the default resolution.
Optionally, detecting a network receiving the video frame includes: and detecting whether the fluctuation rate of the round trip delay of the network is greater than a set threshold value.
optionally, before sending the first retransmission request to the server, the method further includes:
Judging whether the lost data packet can be recovered according to the received data packet;
If so, performing Forward Error Correction (FEC) processing on the received data packet, and recovering the lost data packet;
And if not, sending a first retransmission request to the server.
optionally, the method further comprises: and if the first retransmission request is successful, integrating the retransmitted data packet with the received data packet to generate a finished video frame.
optionally, the determining that the first retransmission request fails includes:
the number of lost packets is greater than a loss threshold;
The lost data packet is not found in the cache of the server; and/or
the request time of the first retransmission request exceeds a time threshold.
Optionally, the second retransmission request is an automatic retransmission request ARQ.
the embodiment of the application provides a video transmission's device, sets up in the client, the device includes:
A receiving module configured to receive video frames transmitted by a server, wherein each video frame comprises a plurality of data packets;
A first retransmission request sending module configured to analyze the received video frame, and send a first retransmission request to a server to enable the server to resend a lost data packet if it is determined that the video frame is lost;
And the second retransmission request sending module is configured to send a second retransmission request to the server if the first retransmission request is determined to be failed, so that the server sends the complete video frame again after the video frame is sent.
Optionally, the first retransmission request sending module is specifically configured to: acquiring the serial number of a data packet of the video frame, and judging whether the serial number of the data packet is continuous or not;
If so, determining that the received video frame is complete;
if not, determining that the data packet corresponding to the sequence number of the missing data packet is the missing data packet.
optionally, the first retransmission request sending module is specifically configured to: and writing the serial number of the missing data packet into the first retransmission request and sending the first retransmission request to the server so that the server searches the data packet corresponding to the serial number of the missing data packet and resends the data packet corresponding to the serial number of the missing data packet.
Optionally, the apparatus further comprises:
a detection module configured to detect a network receiving the video frame;
The first processing module is configured to inform a server to reduce the default code rate or the default resolution of the video frame to a processed code rate or a processed resolution under the condition that the network jitter is detected to be larger than a set threshold;
and the second processing module is configured to notify the server to restore the processed bitrate or the processed resolution of the video frame to the default bitrate or the default resolution when the network jitter is detected to be less than or equal to a set threshold.
optionally, the detection module is configured to: and detecting whether the fluctuation rate of the round trip delay of the network is greater than a set threshold value.
Optionally, the apparatus further comprises:
A judging module configured to judge whether the lost data packet can be recovered according to the received data packet, if so, execute the recovering module, and if not, execute the first retransmission request sending module;
A recovery module configured to perform forward error correction, FEC, processing on the received data packets, and recover the lost data packets;
The first retransmission request sending module is configured to send a first retransmission request to a server.
Optionally, the apparatus further comprises: and the integration module is configured to integrate the retransmitted data packet with the received data packet to generate a finished video frame if the first retransmission request is successful.
Embodiments of the present application provide a computing device comprising a memory, a processor, and computer instructions stored on the memory and executable on the processor, wherein the processor executes the instructions to implement the steps of the method for video transmission as described above.
embodiments of the present application provide a computer-readable storage medium storing computer instructions, which when executed by a processor, implement the steps of the method for video transmission as described above.
according to the method and the device for video transmission, the received video frames are analyzed, and the first retransmission request is sent to the server under the condition that the packet loss of the video frames is determined, so that the server can resend the lost data packet, and the retransmission of the data packet is carried out under the condition that the packet loss is found; and under the condition that the first retransmission request fails, a second retransmission request is sent to the server, so that the server sends the complete video frame again, the retransmission of packet loss is doubly guaranteed, the problem of packet loss is solved, and network delay is reduced.
secondly, the network receiving the video frame is detected, and the server is informed to reduce the default code rate or the default resolution of the video frame to the processed code rate or the processed resolution under the condition that the network jitter is detected to be larger than the set threshold value, so that the network congestion is relieved.
in the video transmission method of this embodiment, after the packet loss of the video frame is found and before the first retransmission request is sent to the server, the received data packet may be subjected to forward error correction FEC processing to recover the lost data packet, so that the original packet may be checked and recovered by decoding when the packet loss is within the correctable range, thereby improving the reliability of transmission.
drawings
FIG. 1 is a schematic block diagram of a computing device according to an embodiment of the present application;
FIG. 2 is a flow chart illustrating a method of video transmission according to an embodiment of the present application;
FIG. 3 is a schematic flow chart diagram illustrating a method for video transmission according to another embodiment of the present application;
FIG. 4 is a schematic flow chart diagram illustrating a method for video transmission according to yet another embodiment of the present application;
fig. 5 is a schematic structural diagram of an apparatus for video transmission according to an embodiment of the present application.
Detailed Description
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is capable of implementation in many different ways than those herein set forth and of similar import by those skilled in the art without departing from the spirit of this application and is therefore not limited to the specific implementations disclosed below.
The terminology used in the description of the one or more embodiments is for the purpose of describing the particular embodiments only and is not intended to be limiting of the description of the one or more embodiments. As used in one or more embodiments of the present specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present specification refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It will be understood that, although the terms first, second, etc. may be used herein in one or more embodiments to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first can also be referred to as a second and, similarly, a second can also be referred to as a first without departing from the scope of one or more embodiments of the present description. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
First, the noun terms to which one or more embodiments of the present invention relate are explained.
RTT (Round-Trip Time): round trip delay. It is an important performance indicator in computer networks, which indicates the total delay experienced from the time the sender sends data to the time the sender receives an acknowledgment from the receiver (the receiver sends an acknowledgment immediately after receiving the data).
FEC (Forward Error Correction): at a sending end, data to be sent is subjected to block coding, N-k redundancy check information packets are added to form a network data packet with the length of N, and after the network data packet is transmitted to a receiving end, if errors are in a correctable range, the original packet can be checked and recovered through decoding.
ARQ (Automatic Repeat-reQuest, Automatic Repeat reQuest): if in a protocol the sender waits for a positive acknowledgement before preparing the next data item, this protocol is called automatic repeat request protocol. Recovering an erroneous data packet by requesting a sender to retransmit the erroneous data packet by a receiver is one of methods for handling errors caused by a channel in communication, and is sometimes called Backward Error Correction (BEC).
network jitter: if the network becomes congested, queuing delays will affect end-to-end delays and cause packets transmitted over the same connection to vary in delay, and network jitter is used to describe the degree to which such a delay varies. Therefore, network jitter is an important parameter for real-time transmission.
in the present application, a method and an apparatus for video transmission, a computing device and a computer-readable storage medium are provided, which are described in detail in the following embodiments one by one.
Fig. 1 is a block diagram illustrating a configuration of a computing device 100 according to an embodiment of the present specification. The components of the computing device 100 include, but are not limited to, memory 110 and processor 120. The processor 120 is coupled to the memory 110 via a bus 130 and a database 150 is used to store data.
computing device 100 also includes access device 140, access device 140 enabling computing device 100 to communicate via one or more networks 160. Examples of such networks include the Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the internet. Access device 140 may include one or more of any type of network interface (e.g., a Network Interface Card (NIC)) whether wired or wireless, such as an IEEE802.11 Wireless Local Area Network (WLAN) wireless interface, a worldwide interoperability for microwave access (Wi-MAX) interface, an ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a bluetooth interface, a Near Field Communication (NFC) interface, and so forth.
in one embodiment of the present description, the above-described components of computing device 100 and other components not shown in FIG. 1 may also be connected to each other, such as by a bus. It should be understood that the block diagram of the computing device architecture shown in FIG. 1 is for purposes of example only and is not limiting as to the scope of the description. Those skilled in the art may add or replace other components as desired.
Computing device 100 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), a mobile phone (e.g., smartphone), a wearable computing device (e.g., smartwatch, smartglasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. Computing device 100 may also be a mobile or stationary server.
Wherein the processor 120 may perform the steps of the method shown in fig. 2. Fig. 2 is a schematic flow chart illustrating a method of video transmission according to an embodiment of the present application, for a client, including steps 201 to 203:
201. And receiving video frames sent by the server, wherein each video frame comprises a plurality of data packets.
it should be noted that each video frame includes a frame header and a frame data area, and the data packet is stored in the frame data area. The video frames may include packets that are the same as each other or different from each other.
202. Analyzing the received video frame, and sending a first retransmission request to a server under the condition that the video frame is determined to be lost, so that the server can resend the lost data packet.
the step 202 of analyzing the received video frame includes the following steps S2021 to S2023:
S2021, acquiring sequence numbers of the data packets of the video frame, and determining whether the sequence numbers of the data packets are consecutive, if yes, performing step S2022, and if no, performing step S2023.
s2022, determining that the received video frame is complete.
s2023, the data packet corresponding to the sequence number of the missing data packet is the missing data packet.
wherein, the sending a first retransmission request to a server in step 202 to enable the server to resend a lost data packet includes:
and writing the serial number of the missing data packet into the first retransmission request and sending the first retransmission request to the server so that the server searches the data packet corresponding to the serial number of the missing data packet and resends the data packet corresponding to the serial number of the missing data packet.
For example, if the sequence numbers of the packets of the acquired video frame are 1, 2, 4, and 7, the missing sequence numbers 3, 5, and 6 are written in the first retransmission request. And the server searches the missing data packets with the serial numbers 3, 5 and 6, adds retransmission packet identifiers to the corresponding data packets and sends the data packets to the client.
203. And if the first retransmission request is determined to fail, sending a second retransmission request to the server so that the server sends the complete video frame again after the video frame is sent.
wherein the condition for determining that the first retransmission request fails comprises:
The number of lost packets is greater than a loss threshold;
The lost data packet is not found in the cache of the server; and/or
the request time of the first retransmission request exceeds a time threshold.
In a specific case, the loss threshold is set to 4, that is, in the case that the number of lost packets is less than or equal to 4, the first retransmission request is successful, otherwise, the first retransmission request fails.
In another specific case, the data packets of the video frames sent by the server are all buffered in the buffer of the server. After receiving the first retransmission request, the server searches whether a corresponding data packet exists in the cache, and if not, a failure notification is returned, and the first retransmission request fails.
In another specific case, the time threshold is set to 10s, that is, in case the request time of the first retransmission request exceeds 10s, the first retransmission request fails.
And after determining that the first retransmission request fails, sending a second retransmission request to the server, so that the server sends the complete video frame again after the video frame is sent, and the problems of screen splash and screen blackout caused by frame loss are solved.
it should be noted that the first retransmission request is different from the second retransmission request, the request object of the first retransmission request is a data packet of a video frame, and the request object of the second retransmission request is a complete video frame.
The format of the second retransmission request is various, and in this embodiment, the second retransmission request is preferably an automatic retransmission request ARQ.
According to the video transmission method, the received video frames are analyzed, and under the condition that the packet loss of the video frames is determined, a first retransmission request is sent to the server, so that the server can resend the lost data packets, and the retransmission of the data packets is carried out under the condition that the packet loss is found; and under the condition that the first retransmission request fails, a second retransmission request is sent to the server, so that the server sends the complete video frame again, the retransmission of packet loss is doubly guaranteed, the problem of packet loss is solved, and network delay is reduced.
Another embodiment of the present application discloses a video transmission method, referring to fig. 3, for a client, including the following steps 301 to 308:
301. And receiving video frames sent by the server, wherein each video frame comprises a plurality of data packets.
302. And detecting a network receiving the video frame, and detecting whether the network jitter is greater than a set threshold, if so, executing step 303, and if not, executing step 304.
There are various methods for detecting the network receiving the video frame, and this embodiment preferably detects whether the fluctuation rate of the round trip delay of the network is greater than a set threshold.
Wherein the set threshold may be set according to actual requirements, for example, set to ± 60%. And detecting each round trip time RTT of the network, and calculating the fluctuation rate according to the round trip time of the network and the round trip time of the network. For example, if the previous RTT is 10ms and the current RTT is 15ms, the current fluctuation rate is 50%; the previous RTT is 10ms, the current RTT is 5ms, and the current fluctuation rate is-50%.
The fluctuation rate may be calculated from the most recent RTT of M times, for example, when the most recent RTT of four times is 10ms, 12ms, 15ms, and 22ms, respectively, the current fluctuation rate is (22ms-10ms)/22ms, which is 54.5%.
In this embodiment, it is preferable to calculate the fluctuation rate by the round trip delay of the latest M times, so as to avoid frequent switching of the code rate or the resolution.
303. And under the condition that the network jitter is detected to be larger than a set threshold value, informing a server to reduce the default code rate or the default resolution of the video frame to the processed code rate or the processed resolution.
when the network jitter is detected to be larger than the set threshold, the network delay is judged to be high, and the server side needs to be immediately informed to reduce the code rate or the resolution ratio so as to relieve the network congestion.
the default code rate and the processed code rate may be set according to actual requirements, for example, the default code rate is 400Kbps, and the processed code rate is 300 Kbps.
The default resolution and the processed resolution may be set according to actual requirements, for example, the default resolution is 1080P, and the processed resolution is 720P.
304. and under the condition that the network jitter is detected to be less than or equal to the set threshold value, informing a server to restore the processed code rate or the processed resolution of the video frame to the default code rate or the default resolution.
and when the network jitter is detected to be less than or equal to the set threshold, the network delay is low, and the client informs the server to recover the default code rate or the default resolution.
305. Analyzing the received video frame, and sending a first retransmission request to a server under the condition that the video frame is determined to be lost, so that the server can resend the lost data packet.
for a detailed explanation of step 305, refer to the detailed explanation of step 202 in the foregoing embodiment, and are not repeated here.
306. And judging whether the first retransmission request is successful, if so, executing step 307, and if not, executing step 308.
307. and if the first retransmission request is successful, integrating the retransmitted data packet with the received data packet to generate a finished video frame.
308. and if the first retransmission request fails, sending a second retransmission request to a server so that the server sends the complete video frame again after the video frame is sent.
For a detailed explanation of step 308, refer to the detailed explanation of step 203 in the foregoing embodiment, and are not repeated here.
according to the video transmission method, the received video frames are analyzed, and under the condition that packet loss of the video frames is determined, a first retransmission request is sent to the server, so that the server can resend lost data packets, and retransmission of the data packets is carried out under the condition that packet loss is found; and under the condition that the first retransmission request fails, a second retransmission request is sent to the server, so that the server sends the complete video frame again, the retransmission of packet loss is doubly guaranteed, the problem of packet loss is solved, and network delay is reduced.
Secondly, the network receiving the video frame is detected, and the server is informed to reduce the default code rate or the default resolution of the video frame to the processed code rate or the processed resolution under the condition that the network jitter is detected to be larger than the set threshold value, so that the network congestion is relieved.
the application further discloses a video transmission method, referring to fig. 4, including the following steps 401 to 410:
401. and receiving video frames sent by the server, wherein each video frame comprises a plurality of data packets.
402. And detecting whether the fluctuation rate of the round trip delay of the network is larger than a set threshold value, if so, executing a step 403, and if not, executing a step 404.
403. And under the condition that the network jitter is detected to be larger than a set threshold value, informing a server to reduce the default code rate or the default resolution of the video frame to the processed code rate or the processed resolution.
404. And under the condition that the network jitter is detected to be less than or equal to the set threshold value, informing a server to restore the processed code rate or the processed resolution of the video frame to the default code rate or the default resolution.
for steps 401 to 404, the same as steps 301 to 304 of the previous embodiment, please refer to the previous embodiment for the detailed explanation of steps 401 to 404, which will not be repeated herein.
405. Analyzing the received video frame, and under the condition that the video frame is determined to be lost, judging whether the lost data packet can be recovered according to the received data packet, if so, executing step 406, and if not, executing step 407.
406. And carrying out Forward Error Correction (FEC) processing on the received data packet, and recovering the lost data packet.
for example, in the case where the sequence numbers of the packets corresponding to the received video frames are 1, 3, and 5, it is determined that the packets of sequence numbers 2 and 4 are lost. Then, FEC processing is performed according to the data packets with sequence numbers 1, 3, and 5, the 2 nd data packet is recovered according to the 1 st data packet and the 3 rd data packet, and the 4 th data packet is recovered according to the 3 rd data packet and the 5 th data packet.
For another example, when the sequence numbers of the packets corresponding to the received video frames are 1, 3, and 7, it is determined that the packets with sequence numbers 2, 4, 5, and 6 are lost. For the 2 nd packet, the recovery can be performed according to the 1 st packet and the 3 rd packet, and for the 4 th to 6 th packets, because too many lost packets are lost, the FEC processing cannot be performed, in which case step 407 needs to be performed.
407. And sending a first retransmission request to a server so that the server retransmits the lost data packet.
408. And judging whether the first retransmission request is successful, if so, executing step 409, and if not, executing step 410.
409. and if the first retransmission request is successful, integrating the retransmitted data packet with the received data packet to generate a finished video frame.
410. And if the first retransmission request fails, sending a second retransmission request to a server so that the server sends the complete video frame again after the video frame is sent.
For steps 407 to 410, the same as steps 305 to 308 of the previous embodiment, and for the detailed explanation of steps 407 to 410, please refer to the previous embodiment, which is not repeated herein.
It should be noted that the first retransmission request is different from the second retransmission request, the request object of the first retransmission request is a data packet of a video frame, and the request object of the second retransmission request is a complete video frame.
The format of the second retransmission request is various, and in this embodiment, the second retransmission request is preferably an automatic retransmission request ARQ.
according to the video transmission method, the received video frames are analyzed, and under the condition that packet loss of the video frames is determined, a first retransmission request is sent to the server, so that the server can resend lost data packets, and retransmission of the data packets is carried out under the condition that packet loss is found; and under the condition that the first retransmission request fails, a second retransmission request is sent to the server, so that the server sends the complete video frame again, the retransmission of packet loss is doubly guaranteed, the problem of packet loss is solved, and network delay is reduced.
secondly, the network receiving the video frame is detected, and the server is informed to reduce the default code rate or the default resolution of the video frame to the processed code rate or the processed resolution under the condition that the network jitter is detected to be larger than the set threshold value, so that the network congestion is relieved.
In the video transmission method of this embodiment, after the packet loss of the video frame is found and before the first retransmission request is sent to the server, the received data packet may be subjected to forward error correction FEC processing to recover the lost data packet, so that the original packet may be checked and recovered by decoding when the packet loss is within the correctable range, thereby improving the reliability of transmission.
another embodiment of the present application discloses a video transmission apparatus, configured at a client, and referring to fig. 5, the apparatus includes:
a receiving module 501 configured to receive video frames sent by a server, wherein each video frame comprises a plurality of data packets;
A first retransmission request sending module 502, configured to analyze the received video frame, and send a first retransmission request to a server in a case that it is determined that the video frame is lost, so that the server resends the lost data packet;
The second retransmission request sending module 503 is configured to send a second retransmission request to the server if it is determined that the first retransmission request fails, so that the server sends the complete video frame again after the video frame is sent.
optionally, the first retransmission request sending module 502 is specifically configured to: acquiring the serial number of the data packet of the video frame, and judging whether the serial number of the data packet is continuous or not;
If so, determining that the received video frame is complete;
If not, determining that the data packet corresponding to the sequence number of the missing data packet is the missing data packet.
optionally, the first retransmission request sending module 502 is specifically configured to: and writing the serial number of the missing data packet into the first retransmission request and sending the first retransmission request to the server so that the server searches the data packet corresponding to the serial number of the missing data packet and resends the data packet corresponding to the serial number of the missing data packet.
optionally, the apparatus further comprises:
a detection module configured to detect a network receiving the video frame;
The first processing module is configured to inform a server to reduce the default code rate or the default resolution of the video frame to a processed code rate or a processed resolution under the condition that the network jitter is detected to be larger than a set threshold;
And the second processing module is configured to notify the server to restore the processed bitrate or the processed resolution of the video frame to the default bitrate or the default resolution when the network jitter is detected to be less than or equal to a set threshold.
Optionally, the detection module is configured to: and detecting whether the fluctuation rate of the round trip delay of the network is greater than a set threshold value.
optionally, the apparatus further comprises:
A judging module configured to judge whether the lost data packet can be recovered according to the received data packet, if so, execute the recovering module, and if not, execute the first retransmission request sending module;
A recovery module configured to perform forward error correction, FEC, processing on the received data packets, and recover the lost data packets;
The first retransmission request sending module is configured to send a first retransmission request to a server.
optionally, the apparatus further comprises: and the integration module is configured to integrate the retransmitted data packet with the received data packet to generate a finished video frame if the first retransmission request is successful.
Wherein determining that the first retransmission request fails comprises:
The number of lost packets is greater than a loss threshold;
The lost data packet is not found in the cache of the server; and/or
The request time of the first retransmission request exceeds a time threshold.
Optionally, the second retransmission request is an automatic retransmission request ARQ.
according to the video transmission device, the received video frames are analyzed, and under the condition that the packet loss of the video frames is determined, a first retransmission request is sent to the server, so that the server can resend the lost data packets, and the retransmission of the data packets is carried out under the condition that the packet loss is found; and under the condition that the first retransmission request fails, a second retransmission request is sent to the server, so that the server sends the complete video frame again, the retransmission of packet loss is doubly guaranteed, the problem of packet loss is solved, and network delay is reduced.
The above is a schematic scheme of an apparatus for video transmission of the present embodiment. It should be noted that the technical solution of the video transmission apparatus and the technical solution of the video transmission method belong to the same concept, and details that are not described in detail in the technical solution of the video transmission apparatus can be referred to the description of the technical solution of the video transmission method.
An embodiment of the present application also provides a computer readable storage medium storing computer instructions, which when executed by a processor, implement the steps of the method for video transmission as described above.
The above is an illustrative scheme of a computer-readable storage medium of the present embodiment. It should be noted that the technical solution of the storage medium belongs to the same concept as the technical solution of the above-mentioned video transmission method, and for details that are not described in detail in the technical solution of the storage medium, reference may be made to the description of the technical solution of the above-mentioned video transmission method.
The computer instructions comprise computer program code which may be in the form of source code, object code, an executable file or some intermediate form, or the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
It should be noted that, for the sake of simplicity, the above-mentioned method embodiments are described as a series of acts or combinations, but those skilled in the art should understand that the present application is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
The preferred embodiments of the present application disclosed above are intended only to aid in the explanation of the application. Alternative embodiments are not exhaustive and do not limit the invention to the precise embodiments described. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the application and the practical application, to thereby enable others skilled in the art to best understand and utilize the application. The application is limited only by the claims and their full scope and equivalents.
Claims (18)
1. A method of video transmission, for a client, the method comprising:
receiving video frames sent by a server, wherein each video frame comprises a plurality of data packets;
Analyzing the received video frame, and sending a first retransmission request to a server under the condition that the video frame is determined to be lost, so that the server can resend the lost data packet;
and if the first retransmission request is determined to fail, sending a second retransmission request to the server so that the server sends the complete video frame again after the video frame is sent.
2. the method of claim 1, wherein analyzing the received video frames comprises:
Acquiring the serial number of a data packet of the video frame, and judging whether the serial number of the data packet is continuous or not;
If so, determining that the received video frame is complete;
And if not, the data packet corresponding to the sequence number of the missing data packet is the missing data packet.
3. the method of claim 2, wherein sending a first retransmission request to a server to cause the server to retransmit lost data packets comprises:
And writing the serial number of the missing data packet into the first retransmission request and sending the first retransmission request to the server so that the server searches the data packet corresponding to the serial number of the missing data packet and resends the data packet corresponding to the serial number of the missing data packet.
4. The method of claim 1, 2 or 3, further comprising:
Detecting a network receiving the video frame;
when the network jitter is detected to be larger than a set threshold value, informing a server to reduce the default code rate or the default resolution of the video frame to the processed code rate or the processed resolution;
And under the condition that the network jitter is detected to be less than or equal to the set threshold value, informing a server to restore the processed code rate or the processed resolution of the video frame to the default code rate or the default resolution.
5. The method of claim 4, wherein detecting the network receiving the video frame comprises: and detecting whether the fluctuation rate of the round trip delay of the network is greater than a set threshold value.
6. The method of claim 1 or 3, wherein prior to sending the first retransmission request to the server, further comprising:
judging whether the lost data packet can be recovered according to the received data packet;
If so, performing Forward Error Correction (FEC) processing on the received data packet, and recovering the lost data packet;
And if not, sending a first retransmission request to the server.
7. the method of claim 1 or 3, further comprising:
And if the first retransmission request is successful, integrating the retransmitted data packet with the received data packet to generate a finished video frame.
8. The method of claim 1, wherein determining the condition that the first retransmission request failed comprises:
The number of lost packets is greater than a loss threshold;
The lost data packet is not found in the cache of the server; and/or
The request time of the first retransmission request exceeds a time threshold.
9. The method of claim 1, wherein the second retransmission request is an automatic repeat request (ARQ).
10. An apparatus for video transmission, provided at a client, the apparatus comprising:
A receiving module configured to receive video frames transmitted by a server, wherein each video frame comprises a plurality of data packets;
A first retransmission request sending module configured to analyze the received video frame, and send a first retransmission request to a server to enable the server to resend a lost data packet if it is determined that the video frame is lost;
And the second retransmission request sending module is configured to send a second retransmission request to the server if the first retransmission request is determined to be failed, so that the server sends the complete video frame again after the video frame is sent.
11. the apparatus of claim 10, wherein the first retransmission request sending module is specifically configured to: acquiring the serial number of a data packet of the video frame, and judging whether the serial number of the data packet is continuous or not;
if so, determining that the received video frame is complete;
If not, determining that the data packet corresponding to the sequence number of the missing data packet is the missing data packet.
12. the apparatus of claim 11, wherein the first retransmission request sending module is specifically configured to: and writing the serial number of the missing data packet into the first retransmission request and sending the first retransmission request to the server so that the server searches the data packet corresponding to the serial number of the missing data packet and resends the data packet corresponding to the serial number of the missing data packet.
13. The apparatus of claim 10, 11 or 12, further comprising:
A detection module configured to detect a network receiving the video frame;
The first processing module is configured to inform a server to reduce the default code rate or the default resolution of the video frame to a processed code rate or a processed resolution under the condition that the network jitter is detected to be larger than a set threshold;
And the second processing module is configured to notify the server to restore the processed bitrate or the processed resolution of the video frame to the default bitrate or the default resolution when the network jitter is detected to be less than or equal to a set threshold.
14. The apparatus of claim 13, wherein the detection module is configured to: and detecting whether the fluctuation rate of the round trip delay of the network is greater than a set threshold value.
15. the apparatus of claim 10 or 12, wherein the apparatus further comprises:
A judging module configured to judge whether the lost data packet can be recovered according to the received data packet, if so, execute the recovering module, and if not, execute the first retransmission request sending module;
a recovery module configured to perform forward error correction, FEC, processing on the received data packets, and recover the lost data packets;
The first retransmission request sending module is configured to send a first retransmission request to a server.
16. The apparatus of claim 10 or 12, further comprising:
and the integration module is configured to integrate the retransmitted data packet with the received data packet to generate a finished video frame if the first retransmission request is successful.
17. A computing device comprising a memory, a processor, and computer instructions stored on the memory and executable on the processor, wherein the processor implements the steps of the method of any one of claims 1-9 when executing the instructions.
18. A computer-readable storage medium storing computer instructions, which when executed by a processor, perform the steps of the method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910924268.7A CN110557677A (en) | 2019-09-27 | 2019-09-27 | Video transmission method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910924268.7A CN110557677A (en) | 2019-09-27 | 2019-09-27 | Video transmission method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110557677A true CN110557677A (en) | 2019-12-10 |
Family
ID=68741800
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910924268.7A Pending CN110557677A (en) | 2019-09-27 | 2019-09-27 | Video transmission method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110557677A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111262653A (en) * | 2020-01-07 | 2020-06-09 | 安徽文香信息技术有限公司 | Data packet loss resisting method, sending end, receiving end and related server |
| CN112713969A (en) * | 2020-12-30 | 2021-04-27 | 北京字跳网络技术有限公司 | Data transmission method and device and system using same |
| CN113300818A (en) * | 2021-02-08 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Data transmission system and method |
| CN113300819A (en) * | 2021-04-13 | 2021-08-24 | 中国科学技术大学 | Robust hop-by-hop reliable data transmission method, device and system |
| CN113422923A (en) * | 2021-06-22 | 2021-09-21 | 苏州市科远软件技术开发有限公司 | Method and device for determining retransmission times, electronic equipment and storage medium |
| CN114025389A (en) * | 2021-11-01 | 2022-02-08 | 网易（杭州）网络有限公司 | Data transmission method and device, computer equipment and storage medium |
| CN114422823A (en) * | 2021-12-27 | 2022-04-29 | 炫彩互动网络科技有限公司 | Improved cloud game stream stable transmission method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101656747A (en) * | 2009-09-25 | 2010-02-24 | 深圳创维数字技术股份有限公司 | Method and system for transmitting streaming media data |
| US8301795B1 (en) * | 2003-10-22 | 2012-10-30 | Sprint Spectrum L.P. | Method and system for managing abnormal disconnects during a streaming media session |
| CN105933319A (en) * | 2016-05-30 | 2016-09-07 | 贵阳朗玛信息技术股份有限公司 | Method and device for transmitting and receiving video data packets |
| CN107231328A (en) * | 2016-03-23 | 2017-10-03 | 福建星网锐捷通讯股份有限公司 | Method for real-time video transmission, device, equipment and system |
| CN108965775A (en) * | 2017-05-25 | 2018-12-07 | 中兴通讯股份有限公司 | Method of adjustment, device and the storage medium of data packetloss processing strategie |
| CN109120383A (en) * | 2017-06-26 | 2019-01-01 | 深圳市道通智能航空技术有限公司 | Unmanned plane and its earth station, data transmission method |
-
2019
- 2019-09-27 CN CN201910924268.7A patent/CN110557677A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8301795B1 (en) * | 2003-10-22 | 2012-10-30 | Sprint Spectrum L.P. | Method and system for managing abnormal disconnects during a streaming media session |
| CN101656747A (en) * | 2009-09-25 | 2010-02-24 | 深圳创维数字技术股份有限公司 | Method and system for transmitting streaming media data |
| CN107231328A (en) * | 2016-03-23 | 2017-10-03 | 福建星网锐捷通讯股份有限公司 | Method for real-time video transmission, device, equipment and system |
| CN105933319A (en) * | 2016-05-30 | 2016-09-07 | 贵阳朗玛信息技术股份有限公司 | Method and device for transmitting and receiving video data packets |
| CN108965775A (en) * | 2017-05-25 | 2018-12-07 | 中兴通讯股份有限公司 | Method of adjustment, device and the storage medium of data packetloss processing strategie |
| CN109120383A (en) * | 2017-06-26 | 2019-01-01 | 深圳市道通智能航空技术有限公司 | Unmanned plane and its earth station, data transmission method |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111262653A (en) * | 2020-01-07 | 2020-06-09 | 安徽文香信息技术有限公司 | Data packet loss resisting method, sending end, receiving end and related server |
| CN112713969A (en) * | 2020-12-30 | 2021-04-27 | 北京字跳网络技术有限公司 | Data transmission method and device and system using same |
| CN113300818A (en) * | 2021-02-08 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Data transmission system and method |
| CN113300818B (en) * | 2021-02-08 | 2023-04-28 | 阿里巴巴集团控股有限公司 | Data transmission system and method |
| CN113300819A (en) * | 2021-04-13 | 2021-08-24 | 中国科学技术大学 | Robust hop-by-hop reliable data transmission method, device and system |
| CN113300819B (en) * | 2021-04-13 | 2022-09-06 | 中国科学技术大学 | Robust hop-by-hop reliable data transmission method, device and system |
| CN113422923A (en) * | 2021-06-22 | 2021-09-21 | 苏州市科远软件技术开发有限公司 | Method and device for determining retransmission times, electronic equipment and storage medium |
| CN114025389A (en) * | 2021-11-01 | 2022-02-08 | 网易（杭州）网络有限公司 | Data transmission method and device, computer equipment and storage medium |
| CN114422823A (en) * | 2021-12-27 | 2022-04-29 | 炫彩互动网络科技有限公司 | Improved cloud game stream stable transmission method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110557677A (en) | Video transmission method and device | |
| EP2810180B1 (en) | Multi-path data transfer using network coding | |
| WO2018058803A1 (en) | Network-based real-time video transmission method and device | |
| WO2017157303A1 (en) | Anti-packet loss method, device and system for use in real-time communication | |
| US10652580B2 (en) | Video data processing method and apparatus | |
| US9794311B2 (en) | Transport accelerator implementing extended transmission control functionality | |
| US8306062B1 (en) | Method and apparatus of adaptive large receive offload | |
| EP2493105A1 (en) | Method and system for recovering lost media data packets | |
| EP3468132A1 (en) | Method and device for transmitting speech data | |
| WO2015066836A1 (en) | Video service data transmission method, data receiving device and data transmitting device | |
| US8621311B2 (en) | Data transfer device, method, and program | |
| CN112350803B (en) | Data packet transmission method, device and system, electronic equipment and storage medium | |
| US20150270930A1 (en) | Transport accelerator implementing selective utilization of redundant encoded content data functionality | |
| WO2022083371A1 (en) | Data transmission method and device | |
| CN113300817B (en) | Data transmission method and device | |
| US8948252B2 (en) | Moving picture transmission apparatus, moving picture transmission system, moving picture transmission method, and program | |
| CN116318545A (en) | Video data transmission method, device, equipment and storage medium | |
| Peisa et al. | Analytical model for TCP file transfers over UMTS | |
| US9307441B1 (en) | Systems and methods of transferring information to a wireless device | |
| CN114401208B (en) | Data transmission method and device, electronic equipment and storage medium | |
| CN114039702B (en) | Data transmission method, device, equipment and medium | |
| JP2011211616A (en) | Moving picture transmission apparatus, moving picture transmission system, moving picture transmission method, and program | |
| West et al. | TCP enhancements for heterogeneous networks | |
| JP2016019198A (en) | Communication apparatus, control method for communication apparatus, and program | |
| CN112751730B (en) | Statistical method, evaluation method, device, equipment and medium for packet loss prevention introduced delay |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Room 001, Floor 03, Floor 3, No. 33, Xiaoying West Road, Haidian District, Beijing 100085 Applicant after: Beijing Jinshan Shiyou Interactive Entertainment Technology Co.,Ltd. Address before: 2f04, No. 33, Xiaoying West Road, Haidian District, Beijing 100085 Applicant before: Beijing Xishanju Interactive Entertainment Technology Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20221102 Address after: 519000 Room 102, 202, 302 and 402, No. 325, Qiandao Ring Road, Tangjiawan Town, high tech Zone, Zhuhai City, Guangdong Province, Room 102 and 202, No. 327 and Room 302, No. 329 Applicant after: Zhuhai Jinshan Digital Network Technology Co.,Ltd. Address before: Room 001, Floor 03, Floor 3, No. 33, Xiaoying West Road, Haidian District, Beijing 100085 Applicant before: Beijing Jinshan Shiyou Interactive Entertainment Technology Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191210 |
|
| RJ01 | Rejection of invention patent application after publication |