WO2023273763A1

WO2023273763A1 - Video data transmission method and device

Info

Publication number: WO2023273763A1
Application number: PCT/CN2022/096020
Authority: WO
Inventors: 聂振兴; 张硕; 姚松平; 钟文国; 骆亮亮; 王皓; 郭兴民
Original assignee: 华为技术有限公司
Priority date: 2021-06-29
Filing date: 2022-05-30
Publication date: 2023-01-05
Also published as: CN115550683A

Abstract

An embodiment of the present application provides a video data transmission method and device, which relate to the technical field of streaming media transmission, and can ensure to a certain extent smooth transmission of important information in the video data in a transmission process. The method comprises: detecting the transmission quality of N links; determining, according to the transmission quality of the N links, video frames to be transmitted which respectively correspond to the N links; when the transmission quality of a k-th link meets a preset condition, the video frames to be transmitted which correspond to the k-th link are all video frames of the first video data; when the transmission quality of the k-th link does not meet the preset condition, the video frames to be transmitted which correspond to the k-th link are part of the video frames of the first video data, the video frames to be transmitted comprising all key data of the first video data; and transmitting on the N links the video frames to be transmitted which respectively correspond to the N links.

Description

Method and device for transmitting video data

This application claims the priority of a Chinese patent application filed with the State Intellectual Property Office on June 29, 2021, with application number 202110729347.X and application name "A Video Data Transmission Method and Device", the entire content of which is incorporated by reference incorporated in this application.

technical field

The embodiments of the present application relate to the technical field of streaming media transmission, and in particular, to a video data transmission method and device.

Background technique

The application of video in real life is more and more extensive, such as video call, webcast, and near-field projection. The transmission of video data is an important factor affecting the playback quality of video data.

At present, in the process of video data transmission, the video sending end transmits data packets of video data on two links respectively, so as to realize redundant transmission of video data, for example, transmit all video data on the first link data packets, all data packets of video data are also transmitted on the second link. Among the above two links, one link can be the main link, and the other link can be the auxiliary link. The data packets transmitted on the above two links are mutual backup. When the data packets of one link are lost , the video receiving end can obtain the lost data packets from the data packets transmitted on another link, so as to successfully complete the decoding of the video data.

However, when the bandwidth of at least one link in the above-mentioned two links for transmitting video data is limited (that is: two transmission links are strong and the other is weak), the link with limited bandwidth will not transmit video data. Part of the video data may be lost when the bandwidth is not limited. Therefore, the two links can no longer serve as backups for each other. In this case, if other factors (for example, The receiving end restarts) also loses part of the video data, which is the same as the video data lost by the bandwidth-limited link, and the video data lost by the two links includes important information of the video data, then it will cause video When the receiving end plays the received video data, problems such as freeze, blurred screen and high delay occur.

Contents of the invention

Embodiments of the present application provide a method and device for transmitting video data, which can guarantee the normal transmission of important information in the video data during the transmission process of the video data to a certain extent.

In order to achieve the above purpose, the embodiment of the present application adopts the following technical solutions:

In the first aspect, the embodiment of the present application provides a video data transmission method, the method includes: detecting the transmission quality of N links; wherein, the N links are used to transmit the first video data, and the N links are For links in different networks, the above-mentioned first video data includes a plurality of video frames, and N is a positive integer greater than or equal to 2; according to the transmission quality of the above-mentioned N links, determine the video to be transmitted corresponding to the N links respectively frame; wherein, when the transmission quality of the kth link meets the preset condition, the video frames to be transmitted corresponding to the kth link are all video frames of the above-mentioned first video data; the kth link When the transmission quality of the link does not meet the preset conditions, the video frame to be transmitted corresponding to the kth link is a partial video frame in all of the above-mentioned first video data; wherein, the value of k is 1, 2, ... ... any value in N, the video frame to be transmitted includes all key data of the first video data; the video frames to be transmitted respectively corresponding to the N links are transmitted in the above N links.

In the video data transmission method provided in the embodiment of the present application, the video sending end detects the transmission quality of the N links used to transmit the first video data, and determines the respective corresponding links of the N links according to the transmission quality of the N links. The video frames to be transmitted, wherein, when the transmission quality of the kth link meets the preset conditions, all the video frames in the first video data are used as the video frames to be transmitted of the kth link, and the kth link When the transmission quality of the link does not meet the preset condition, the partial video frame in the first video data is used as the video frame to be transmitted of the kth link; and, the video frame to be transmitted determined above includes the first video frame All key data in the data. In this way, when the transmission quality of the kth link is poor, this part of video frames in the first video data transmitted on the kth link includes all the key data in the first video data, thus to a certain extent It can ensure the normal transmission of the key data in the first video data, and further solve the problems such as stuttering, blurred screen and high delay when the video receiving end plays the received video data.

In a possible implementation manner, the detection of the transmission quality of the kth link specifically includes: obtaining feedback information of the kth link, where the feedback information includes transmission delay and/or retransmission times; according to the above kth link The feedback information of the kth link is used to determine the transmission quality of the kth link.

When the above feedback information is transmission time delay, if the transmission time delay of the kth link is less than the preset time delay, the transmission quality of the kth link meets the preset condition; if the transmission time of the kth link If the delay is greater than or equal to the preset delay, the transmission quality of the k-th link does not meet the above preset condition.

When the above feedback information is the number of retransmissions, if the number of retransmissions of the kth link is less than the preset number of times, the transmission quality of the kth link meets the above preset conditions; if the retransmission of the kth link If the number of times is greater than or equal to the preset number of times, then the transmission quality of the kth link does not meet the above preset condition.

When the above feedback information is the transmission delay and the number of retransmissions, if the transmission delay of the kth link is less than the preset time delay, and the number of retransmissions of the kth link is less than the preset number of times, then the kth link The transmission quality of the link meets the preset condition; if the transmission delay of the k link is greater than or equal to the preset threshold, and/or the number of retransmissions of the k link is greater than or equal to the preset number, then the The transmission quality of the kth link does not meet the preset condition.

In a possible implementation manner, when the transmission quality of the kth link does not meet the preset condition, determining the video frame to be transmitted corresponding to the kth link specifically includes: according to the kth link corresponding According to the preset proportion of the video frame of the first video data, according to the order of priority of the video frame of the first video data from high to low, select some video frames in the video frame of the above-mentioned first video data as the video frame to be transmitted corresponding to the kth link .

The above preset ratios satisfy:

Wherein, n represents preset ratio, and Num ₁ represents the quantity of the non-key video frame in the video frame of above-mentioned first video data, and Num ₂ represents the total quantity of the video frame of above-mentioned first video data; Wherein, this non-key video frame Refers to video frames that include non-critical data.

In the video data transmission method provided in the embodiment of the present application, the video sending end detects the transmission quality of the kth link used to transmit the first video data, and determines the kth link according to the transmission quality of the kth link Corresponding video frames to be transmitted, wherein, when the transmission quality of the kth link satisfies the preset condition, all the video frames in the first video data are used as the video frames to be transmitted of the kth link, and in the kth link When the transmission quality of the k links does not meet the preset condition, the partial video frames in the first video data are used as the video frames to be transmitted of the k link; and, the video frames to be transmitted determined above include the first video frame All key data in a video data. In this way, when the transmission quality of the kth link is poor, this part of video frames in the first video data transmitted on the kth link includes all the key data in the first video data, thus to a certain extent It can ensure the normal transmission of the key data in the first video data, and further solve the problems such as stuttering, blurred screen and high delay when the video receiving end plays the received video data.

In a possible implementation manner, after the video frames to be transmitted respectively corresponding to the N links are transmitted in the N links, the video data transmission method provided in the embodiment of the present application further includes: detecting the kth link The transmission quality of the link; according to the transmission quality of the k link, adjust the preset ratio corresponding to the k link.

In a possible implementation manner, the above-mentioned adjustment of the preset ratio corresponding to the k-th link according to the transmission quality of the k-th link specifically includes: when the transmission quality of the above-mentioned k-th link meets the preset condition In this case, the preset ratio is increased; when the transmission quality of the kth link does not meet the preset condition, the preset ratio is decreased.

In the video data transmission method provided in the embodiment of the present application, when the transmission quality of the k links meets the preset condition, the video sending end increases the preset ratio; that is to say, the transmission quality of the k link is lower than OK, the video sender increases the preset ratio so that the video sender can send more video frames in the video data to the video receiver, so that the video receiver can play high-quality videos based on more video frames in the video data video screen.

In a possible implementation manner, the method for transmitting video data provided in the embodiment of the present application further includes: according to the preset ratio corresponding to the adjusted k-th link, determining whether the second video data is in the k-th link The video frame to be transmitted corresponding to the link; the video frame to be transmitted corresponding to the kth link is transmitted in the kth link.

In a possible implementation manner, the video data transmission method provided by the embodiment of the present application further includes: performing frequency-domain layered coding on the first video data to obtain video frames of the first video data, and the first video data The video frame of the data includes at least one base frame and extended frame of the key frame, and the base frame and extended frame of multiple forward frames; the non-key video frame includes the extended frame of the key frame, the base frame of the forward frame and the forward frame The key data of the above-mentioned first video data is the basic frame of the key frame.

In a possible implementation manner, the video data transmission method provided in the embodiment of the present application further includes: the priority of the video frame of the first video data is the basic frame of the key frame, the forward frame in order from high to low The base frame of the key frame, the extended frame of the key frame, and the extended frame of the forward frame.

In a possible implementation manner, the video data transmission method provided in the embodiment of the present application further includes: the i-th video frame among the video frames of the first video data corresponding to the first link is the first video frame The i-th video frame in the video frame of the above-mentioned first video data corresponding to the second link is an extended frame of the above-mentioned first video frame; the above-mentioned first link and the above-mentioned second link are N chains Two different links in the road.

To sum up, the arrangement order of multiple video frames of the first video data transmitted on different links is: in the video frame queues corresponding to two different links, the video frames at the same position (or the same number or index) are the same The basic frame and extended frame of a video frame, the video sending end transmits the video frame in parallel on the two links according to the sequence of the video frame, and can quickly send the basic frame and extended frame of the video frame to the video receiver For the video receiving end, the basic frame and the extended frame of the video frame can also be quickly aggregated to obtain a decoded video frame.

In a possible implementation manner, the method for transmitting video data provided by the embodiment of the present application further includes: determining a set of retransmitted video frames, where the set of retransmitted video frames is the retransmitted corresponding to each of the above N links. The intersection of the video frames; in the jth link, transmit the video frames in the retransmission video frame set in the jth link; wherein, the transmission quality of the jth link is higher than the transmission quality of other links, The other link is a link except the j-th link among the above N links.

In a possible implementation manner, the transmission of the video frames in the above-mentioned retransmitted video frame set in the jth link includes: according to the priority order of the video frames in the above-mentioned retransmitted video frame set from high to low , transmit the video frames in the above retransmitted video frame set in the jth link.

In a possible implementation, the transmission of the video frames in the set of retransmitted video frames in the jth link by means of queue jumping transmission includes: inserting the set of retransmitted video frames into the jth link corresponding to The head of the queue of video frames to be transmitted.

In a possible implementation manner, before transmitting the video frames to be transmitted corresponding to the N links in the above-mentioned N links, the method further includes: encrypting the video frames to be transmitted respectively corresponding to the N links.

In the embodiment of this application, on the one hand, the video sending end transmits the video frames in the retransmission video frame set by jumping in the queue, and the video frames in the retransmission video frame set are transmitted preferentially, so that the video receiving end can obtain the lost video frames as soon as possible. Video frame, improve the speed of video frame aggregation at the video receiving end, and realize fast aggregation of video frames; The priority of the frame is transmitted from high to low in order to retransmit the video frames in the video frame set. Since the video frame with higher priority contains more important data information, retransmitting the video frame according to the priority can guarantee to a certain extent Important information in the first video data is not lost, so as to ensure the playback quality of the video data.

In the second aspect, the video data transmission device provided by the embodiment of the present application includes: a detection module, a determination module, and a transmission module; the detection module is used to detect the transmission quality of N links; wherein, the N links are used for transmission The first video data, the N links are links in different networks, the above-mentioned first video data includes a plurality of video frames, and N is a positive integer greater than or equal to 2; the determination module is used to determine according to the N links The transmission quality is to determine the video frames to be transmitted corresponding to the N links respectively; wherein, when the transmission quality of the kth link meets the preset condition, the video frame to be transmitted corresponding to the kth link is the above-mentioned All video frames of a video data; in the case that the transmission quality of the kth link does not meet the preset condition, the video frame to be transmitted corresponding to the kth link is a partial video of all the above-mentioned first video data frame; wherein, the value of k is 1, 2, ... any value in N, and the above-mentioned video frame to be transmitted includes all key data of the first video data; the transmission module is used for transmission in the above-mentioned N links The video frames to be transmitted respectively correspond to the N links.

In a possible implementation manner, the video data transmission device provided in the embodiment of the present application further includes: an acquisition module; the acquisition module is configured to acquire feedback information of the kth link, where the feedback information includes transmission delay and/or The number of retransmissions.

The determination module is configured to determine the transmission quality of the kth link according to the feedback information of the kth link.

When the feedback information is transmission delay, if the transmission delay of the kth link is less than the preset delay, the transmission quality of the kth link meets the preset condition; if the transmission time of the kth link If the delay is greater than or equal to the preset delay, the transmission quality of the k-th link does not meet the preset condition.

When the feedback information is the number of retransmissions, if the number of retransmissions of the kth link is less than the preset number of times, the transmission quality of the kth link meets the preset condition; if the number of retransmissions of the kth link is greater than or equal to the preset number of times, then the transmission quality of the kth link does not meet the preset condition.

When the feedback information is the transmission delay and the number of retransmissions, if the transmission delay of the kth link is less than the preset time delay, and the number of retransmissions of the kth link is less than the preset number of times, then the kth link The transmission quality of the link meets the preset condition; if the transmission delay of the k link is greater than or equal to the preset threshold, and/or the number of retransmissions of the k link is greater than or equal to the preset number, then the The transmission quality of the kth link does not meet the preset condition.

In a possible implementation manner, the video data transmission device provided in the embodiment of the present application includes: a control module; In order of priority from high to low, some video frames in the video frames of the first video data are selected as video frames to be transmitted corresponding to the kth link.

The above preset ratios satisfy:

Wherein, n represents preset ratio, and Num ₁ represents the quantity of the non-key video frame in the video frame of above-mentioned first video data, and Num ₂ represents the total quantity of the video frame of this first video data; Wherein, non-key video frame is Refers to video frames that include non-critical data.

In a possible implementation manner, the video data transmission device provided in the embodiment of the present application includes: an adjustment module; a detection module, configured to detect the transmission quality of the kth link; an adjustment module, configured to The transmission quality of the kth link is adjusted to a preset ratio corresponding to the kth link.

In a possible implementation manner, in the video data transmission device provided in the embodiment of the present application, the adjustment module is specifically configured to increase the preset ratio when the transmission quality of the kth link meets the preset condition; If the transmission quality of the kth link does not meet the preset condition, reduce the preset ratio.

In a possible implementation manner, the determination module is further configured to determine the video frame to be transmitted corresponding to the K link of the second video data according to the adjusted preset ratio corresponding to the K link; the above transmission The module is further configured to transmit the video frame to be transmitted corresponding to the kth link in the above kth link.

In a possible implementation manner, the video data transmission device provided in the embodiment of the present application includes: an encoding module; the encoding module is configured to perform frequency-domain layered encoding on the first video data to obtain video frames of the first video data , the video frame of the first video data includes at least one basic frame and extended frame of a key frame, and a plurality of basic frames and extended frames of forward frames; the above-mentioned non-key video frames include extended frames of key frames, extended frames of forward frames The basic frame and the extended frame of the forward frame, the key data of the above-mentioned first video data is the basic frame of the key frame.

In a possible implementation, in the video data transmission device provided in the embodiment of the present application, the priorities of the video frames of the first video data from low to high are the extended frame of the forward frame, the extended frame of the key frame, and the previous frame. The base frame of the direction frame and the base frame of the key frame.

In a possible implementation, in the video data transmission device provided in the embodiment of the present application, the i-th video frame among the video frames of the first video data corresponding to the first link is the basic frame of the first video frame, and the i-th video frame is the basic frame of the first video frame, and the i-th video frame is The i-th video frame in the video frame of the first video data corresponding to the two links is an extended frame of the first video frame; the first link and the second link are two different ones of the above N links link.

In a possible implementation manner, the video data transmission device provided by the embodiment of the present application further includes: a queue jumping module; the above-mentioned determining module is also used to determine a set of retransmitted video frames, and the set of retransmitted video frames is the above-mentioned N chain The intersection of the video frames to be retransmitted corresponding to the respective roads; the queue jumping module is used to transmit the video frames in the above retransmitted video frame set in the jth link in the queue jumping transmission mode; wherein, the jth link The transmission quality of is higher than the transmission quality of other links, and the other links are links other than the above j-th link among the above N links.

In a possible implementation, in the video data transmission device provided in the embodiment of the present application, the transmission module is further configured to retransmit the video frames in the order of priority from high to low in the video frame set, and in the j The video frames in the retransmitted video frame set are transmitted in the link.

In a possible implementation manner, the above-mentioned queue-jumping module is specifically configured to insert the set of retransmitted video frames into the head of the queue of video frames to be transmitted corresponding to the above-mentioned jth link.

In a possible implementation manner, the video data transmission device provided in the embodiment of the present application further includes an encryption module; the encryption module is configured to encrypt video frames to be transmitted respectively corresponding to the N links.

In the third aspect, the video data transmission device provided by the embodiment of the present application includes a memory and a processor, the memory is coupled to the processor; the memory is used to store computer program codes, and the computer program codes include computer instructions; when the computer instructions are When the processor is executed, the video data transmission device is made to execute the method described in any one of the first aspect and possible implementations thereof.

In the fourth aspect, a computer storage medium provided by an embodiment of the present application includes computer instructions, and when the computer instructions are run on the video data transmission device, the video data transmission device is made to execute the first aspect and its possible implementations. any one of the methods described.

In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on a computer, causes the computer to execute the method described in any one of the first aspect and possible implementations thereof.

It should be understood that the beneficial effects achieved by the technical solutions of the second aspect to the fifth aspect of the embodiment of the present application and the corresponding possible implementation manners can refer to the technical effects of the above-mentioned first aspect and the corresponding possible implementation manners.

Description of drawings

FIG. 1 is a schematic diagram of a screen projection method provided in an embodiment of the present application;

FIG. 2 is a schematic diagram of a scene using distributed cameras provided by the embodiment of the present application;

FIG. 3 is a second schematic diagram of a scene using distributed cameras provided by the embodiment of the present application;

FIG. 4 is a first schematic diagram of a video data transmission process provided by an embodiment of the present application;

FIG. 5 is a first schematic diagram of a coding method of video data provided by an embodiment of the present application;

FIG. 6 is a first schematic diagram of a video data transmission method provided by an embodiment of the present application;

FIG. 7 is a first schematic diagram of a multi-path transmission architecture provided by an embodiment of the present application;

FIG. 8 is a first structural schematic diagram of a video data transmission device provided by an embodiment of the present application;

FIG. 9 is a first schematic diagram of a video data transmission method provided by an embodiment of the present application;

FIG. 10 is a second schematic diagram of a video data transmission method provided by an embodiment of the present application;

FIG. 11 is a schematic diagram 1 of sorting a plurality of video frames provided by the embodiment of the present application;

FIG. 12 is a third schematic diagram of a video data transmission method provided by an embodiment of the present application;

FIG. 13 is a fourth schematic diagram of a video data transmission method provided by an embodiment of the present application;

FIG. 14 is a fifth schematic diagram of a video data transmission method provided by an embodiment of the present application;

FIG. 15 is a schematic structural diagram of a video data transmission device provided by the embodiment of the present application provided by the embodiment of the present application;

FIG. 16 is a second schematic structural diagram of a video data transmission device provided by the embodiment of the present application.

detailed description

The term "and/or" in this article is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist simultaneously, and there exists alone B these three situations.

The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order of objects. For example, the first link and the second link are used to distinguish links in different networks, rather than describing a specific sequence of links.

In the embodiments of the present application, words such as "exemplary" or "for example" are used as examples, illustrations or illustrations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present application shall not be interpreted as being more preferred or more advantageous than other embodiments or design schemes. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete manner.

In the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more. For example, a plurality of video frames refers to two or more video frames.

With the development of technology, streaming media (such as video) is more and more widely used in real life. In daily life, video data transmission is involved in near-field projection scenarios and distributed camera scenarios.

In the near-field projection scenario, the projection methods can include: mirror coordination, heterogeneous collaboration, and reverse control, etc.

Mirroring coordination: It refers to the real-time synchronous transmission of the video screen on the projecting side to the projected side so that it can be played synchronously on the projected side.

Exemplarily, referring to (a) in FIG. 1 , the large screen 101 is the projected side, and the mobile phone 102 is the projected side. When making a video call, the mobile phone 102 transmits the video image (specifically, the video data corresponding to the video image) during the video call to the large screen 101, and the large screen 101 can synchronously play the video image of the mobile phone 102.

Heterogeneous collaboration: refers to the synchronous transmission of a task running in the background on the projection side to the projected side, so that it can be played synchronously on the projected side. At this time, the projected side can perform other tasks.

Exemplarily, referring to (b) in FIG. 1 , the large screen 103 is the projected side, and the mobile phone 104 is the projected side. The screen projection relationship between the mobile phone 104 and the large screen 103 is heterogeneous collaboration. Display the chat interface (the chat interface displayed by the mobile phone running the social application) and the video playback interface (the interface displayed by the mobile phone running the video playback application) in a split-screen mode, and the mobile phone 104 can transmit the video picture displayed on the split screen 2 to the large screen 103 , the large screen 103 synchronously plays the video screen displayed on the mobile phone 104. After the screen projection is successful, the user can perform corresponding operations on the mobile phone 104 to put the video playback application in the background, so that the mobile phone 104 displays the chat interface in full screen.

Reverse control: refers to the transmission of the screen displayed on the projection side to the projected side, so that the screen is displayed in real time on the projected side, and then the user performs related operations on the projected side, so that the projected side can In response to the operation, a corresponding task is performed.

Referring to (c) in FIG. 1, the computer 105 is the side to be screened, and the mobile phone 106 is the side to be screened. The video picture displayed on the computer is synchronously transmitted to the computer 105, and the video picture is displayed on the display screen of the computer 105. Then, the user can operate the displayed content on the computer 105, and the mobile phone 106 can respond to the user's operation on the computer 105. For example, a user opens a certain video software in the mobile phone 106, and the mobile phone 106 displays an interface including multiple video resources. resource interface, and furthermore, when the user opens a certain video resource among multiple video resources on the computer 105, the mobile phone 106 can respond to the opening operation of the video resource, and the mobile phone 106 displays the video picture corresponding to the video resource, and at the same time, the mobile phone 106 The displayed video frame is also synchronously transmitted to the computer 105, so that the computer 105 displays the video frame.

To sum up, in the screen projection scenario described above, the screen projection side is the sending end of the video data, and the screen projection side is the receiving end of the video data.

In the distributed camera scenario, the use of distributed cameras can realize video calls, remote control of mobile phones to take pictures, mobile live broadcasts, and cross-device dual-view recording, etc.

(a) in Fig. 2 is a schematic diagram of a scene using a distributed camera for a video call, wherein the camera on the large screen 201 and the mobile phone 202 form a distributed camera. The camera on the screen 201 establishes a communication connection. Since the pixel of the camera on the large screen 201 is higher than the pixel of the self-contained camera of the mobile phone 202, when using the mobile phone 202 to carry out a video call, the camera on the large screen 201 is used as a video call. The camera of the video picture, specifically, collects the video picture through the camera on the large screen 201, and transmits the video picture synchronously to the mobile phone 202, and the mobile phone 202 displays the video picture in real time.

In the above video call scenario, the large screen 201 is the sending end of the video data, and the mobile phone 202 is the receiving end of the video data.

(b) in FIG. 2 is a schematic diagram of a scene where a distributed camera is used to remotely control a mobile phone to take pictures, wherein the smart watch 203 and the mobile phone 204 form a distributed camera; The watch 203 establishes a communication connection, and the camera of the mobile phone 204 transmits the collected preview image to the smart watch 203, and the preview image is displayed synchronously on the smart watch 203, and then the user can operate on the smart watch 203 to control the mobile phone 204 to take pictures. For example, if the user thinks that the picture in the preview interface currently displayed by the smart watch 203 is better, the user can select the "photograph" button on the smart watch 203, so that the mobile phone 204 responds to the "photograph" operation and collects images.

In the above scene where the mobile phone is remotely controlled to take pictures, the mobile phone 204 is the video data sending end, and the smart watch 203 is the video data receiving end.

(a) in FIG. 3 is a schematic diagram of a scene using a distributed camera for live broadcasting on a mobile phone, wherein the rear camera of the mobile phone 301 and the mobile phone 302 form a distributed camera, and the pixels of the rear camera of the mobile phone 301 are higher than those of the mobile phone 302 In the live broadcast scene, after the mobile phone 302 establishes a communication connection with the mobile phone 301, the live broadcast APP is run on the mobile phone 302, and the rear camera of the mobile phone 301 collects live images (such as scenery or anchor), and then the mobile phone 301 will collect The live broadcast picture is transmitted to the mobile phone 302 in real time, and the mobile phone 302 plays the live broadcast picture synchronously.

In the above scenario of mobile live broadcast, the mobile phone 301 is the sending end of the video data, and the mobile phone 302 is the receiving end of the video data.

(b) in Figure 3 is a schematic diagram of a scene where distributed cameras are used to perform cross-device dual-view recording, wherein mobile phone 303, mobile phone 305 and camera 304 form a distributed camera, and the mobile phone 303 and camera 304 are respectively established with mobile phone 305 Communication connection, after starting dual-view video recording, mobile phone 303 transmits the close-range picture of the target scene collected by the camera of this mobile phone 303 to mobile phone 305, and at the same time, camera 304 transmits the long-range picture of the target scene collected to mobile phone 305, and mobile phone 305 receives After the pictures sent by the mobile phone 303 and the camera 304, a close-range picture and a long-range picture are displayed on the display screen of the mobile phone 305 in a split-screen mode to realize double-view video recording.

In the above scene of cross-device dual-view video recording, the mobile phone 303 and the camera 304 are both video data sending ends, and the mobile phone 305 is the video data receiving end.

It can be understood that, in this embodiment of the present application, video data is a video stream, including multiple video frames, and each video frame is an image, therefore, a video frame may also be called an image frame.

Combining the above-mentioned multiple scenarios involving video data transmission, refer to the schematic diagram of the video data transmission process shown in Figure 4. First, the sending end encodes (preprocesses) the collected video data (including multiple video frames). The encoding method is time-domain encoding and frequency-domain encoding; secondly, the sending end encrypts the encoded video data, and sends the encrypted video data to the receiving end. When the receiving end receives the encrypted video data, first, reversely decrypt the encrypted video data according to the encryption method of the sending end to obtain the decrypted video data; then, reversely decode the video data according to the encoding method of the sending end to obtain The decoded video data, finally, the receiving end renders the decoded video data before playing.

Common video data coding methods may include time-domain coding and frequency-domain coding.

In the embodiment of the present application, the forward search frame is referred to as the forward frame for short, and the following embodiments will not be described again.

Time-domain encoding: the captured video frame can be encoded by time-domain encoding to obtain the small forward frame (ie: small P frame) and the large forward frame (large P frame) of the video frame, as shown in Figure 5 (a ), wherein, when the receiving end decodes a certain large P frame, it needs to rely on the decoding result of the previous large P frame of the large P frame and the corresponding small P frame of the large P frame.

Frequency domain encoding: according to the frequency domain distribution of video data, the collected video frames are encoded into basic frames and extended frames, as shown in (b) in Figure 5; the basic frame of a video frame contains the low frequency data of the video frame , the extended frame of a video frame includes the high-frequency data of the video frame, the size (data volume) of the basic frame and the extended frame is different, specifically, the size ratio of the basic frame and the extended frame is between 1:4 and 1:8 In one case, the sum of the size of the base frame and the extended frame of a video frame may be 1.2 times that of the video frame.

At present, multiple links can be used to transmit (i.e. multi-path transmission) video data, specifically as shown in Figure 6, the video sender transmits all data packets of the video data to be transmitted on the first link (i.e. data packets 1 to Data packet 8), also transmit all data packets of the video data to be transmitted on the second link, wherein, the data packets transmitted on the first link and the second link are mutual backups, when a data packet of a link occurs When it is lost, the video receiving end can obtain the lost data packet from the data packet transmitted by another link, so as to successfully complete the decoding of the video data.

However, when the bandwidth of the second link in the above two links is limited, the second link will lose a part of the video data packets when transmitting the data packets of the video data to be transmitted, if the first link is transmitting the video data packets to be transmitted In the process of data packets of video data, due to other factors (for example, the receiving end restarts), some video data packets are also lost, and the second link loses this part of video data packets and the first link loses this part of video data There are intersections of packets, and the video data packets in the intersection part may contain important information in the video data to be transmitted, which will cause the receiving end to fail to decode the video data to be transmitted normally, and will cause the video receiving end to play the received video Problems such as stuttering, blurred screen, and high delay occur when data is uploaded.

The above two multi-path transmission methods are implemented by the data flow module in the multi-path transmission architecture. As shown in Figure 7, the transmission process of the data packet is specifically: after the acquisition module at the sending end obtains the data packet sequence, it communicates with the receiving end through the communication module. Communication connection, then, the data stream module transmits the data packet sequence obtained by the above-mentioned acquisition module to the receiving end in the form of character stream or byte stream, wherein, the method for transmitting the data packet sequence from the sending end to the receiving end can be the above-mentioned multipath Any one of the transfer methods.

In the multi-path transmission method in the embodiment of the present application, the transmission of the same video data on different links is carried out in the manner of parallel transmission on multiple links, which will not be described in detail later.

Based on the problem that the important information in the video data cannot be guaranteed to be transmitted smoothly during the video data transmission process in the prior art, the embodiment of the present application provides a video data transmission method and device, a video transmission device or a video data sending end detection The transmission quality of N links; the N links are used to transmit video data, and the N links are links in different networks; the video data includes video frames of the first video data, and N is greater than or equal to 2 is a positive integer; the video transmission device determines video frames to be transmitted respectively corresponding to the N links according to the transmission quality of the N links; wherein, when the transmission quality of the kth link meets the preset condition, The video frame to be transmitted corresponding to the kth link is the video frame of the first video data; if the transmission quality of the kth link does not meet the preset condition, the kth link corresponding to The transmission video frame is a part of the video frame of the above-mentioned first video data; wherein, the value of k is any one of 1, 2, ... N, and the above-mentioned determined video frame to be transmitted includes the first video data All key data in the video frame; the video transmission device transmits the video frames to be transmitted respectively corresponding to the N links in the N links. Through the technical solutions provided by the embodiments of the present application, the smooth transmission of key data (ie, important information) in video data can be guaranteed to a certain extent.

Exemplary, the video transmission device in the embodiment of the present application can be a tablet computer, a mobile phone, a desktop, a laptop, a handheld computer, a notebook computer, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, and For cellular phones, personal digital assistants (personal digital assistant, PDA) and other devices, the embodiment of the present application does not impose special restrictions on the specific form of the video transmission device.

The execution subject of the video data transmission method provided by the embodiment of the present application may be a device supporting video data transmission (the following embodiments are referred to as the video transmission device for short), please refer to FIG. As an example, the video transmission device provided in the embodiment of the present application is introduced. Wherein, the mobile phone 800 shown in FIG. 8 is only an example of a video transmission device, and the mobile phone 800 may have more or fewer components than those shown in the figure, and two or more components may be combined, or Different component configurations are possible. The various components shown in Figure 8 may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and/or application specific integrated circuits.

As shown in Figure 8, the mobile phone 800 may include a processor 810, an external memory interface 820, an internal memory 821, a universal serial bus (universal serial bus, USB) interface 830, a charging management module 840, a power management module 841, a battery 842, Antenna 1, antenna 2, radio frequency module 850, communication module 860, audio module 870, speaker 870A, receiver 870B, microphone 870C, earphone jack 870D, sensor module 880, button 890, motor 891, indicator 892, camera 893, display 894, and a subscriber identification module (subscriber identification module, SIM) card interface 895, etc.

Wherein, the above-mentioned sensor module 880 may include sensors such as pressure sensor, gyroscope sensor, air pressure sensor, magnetic sensor, acceleration sensor, distance sensor, proximity light sensor, fingerprint sensor, temperature sensor, touch sensor, ambient light sensor and bone conduction sensor.

Processor 810 may include one or more processing units. For example, the processor 810 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a memory , video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.

The aforementioned controller may be the nerve center and command center of the mobile phone 800 . The controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.

A memory may also be provided in the processor 810 for storing instructions and data. In some embodiments, the memory in the processor 810 is a cache memory, which can store instructions or data that the processor 810 has just used or recycled. If the processor 810 needs to use the instruction or data again, it can be directly recalled from the memory. Repeated access is avoided, and the waiting time of the processor 810 is reduced, thereby improving the efficiency of the system.

In some embodiments, processor 810 may include an interface. The interface may include an integrated circuit (Inter-Integrated Circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit Sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input/output (general-purpose input/output, GPIO) interface, SIM interface, and/or USB interface, etc.

The I2C interface is a bidirectional synchronous serial bus, including a serial data line (serial data line, SDL) and a serial clock line (derail clock line, SCL). In some embodiments, processor 810 may include multiple sets of I2C buses. The processor 810 can be respectively coupled to the touch sensor 280K, the charger, the flashlight, the camera 893 and so on through different I2C bus interfaces. For example, the processor 810 can be coupled to the touch sensor 280K through the I2C interface, so that the processor 810 and the touch sensor 280K can communicate through the I2C bus interface to realize the touch function of the mobile phone 800 .

The I2S interface can be used for audio communication. In some embodiments, processor 810 may include multiple sets of I2S buses. The processor 810 may be coupled to the audio module 870 through an I2S bus to implement communication between the processor 810 and the audio module 870 . In some embodiments, the audio module 870 can transmit audio signals to the communication module 860 through the I2S interface, so as to realize the function of answering calls through the Bluetooth headset.

The PCM interface can also be used for audio communication, sampling, quantizing and encoding the analog signal. In some embodiments, the audio module 870 and the communication module 860 may be coupled through a PCM bus interface. In some embodiments, the audio module 870 can also transmit audio signals to the communication module 860 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication, and the sampling rates of the two interfaces are different.

The MIPI interface can be used to connect the processor 810 with peripheral devices such as a display screen 894 and a camera 893 . MIPI interface includes camera serial interface (camera serial interface, CSI), display serial interface (display serial interface, DSI), etc. In some embodiments, the processor 810 communicates with the camera 893 through the CSI interface to realize the shooting function of the mobile phone 800 . The processor 810 communicates with the display screen 894 through the DSI interface to realize the display function of the mobile phone 800 .

The interface connection relationship between modules shown in the embodiment of the present application is only a schematic illustration, and does not constitute a structural limitation of the mobile phone 800 . The mobile phone 800 may adopt different interface connection modes in the embodiment of the present application, or a combination of multiple interface connection modes.

The charging management module 840 is configured to receive charging input from the charger. Wherein, the charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 840 can receive the charging input of the wired charger through the USB interface 830 . In some wireless charging embodiments, the charging management module 840 can receive wireless charging input through the wireless charging coil of the mobile phone 800 . While the charging management module 840 is charging the battery 842 , it can also supply power to the mobile phone 800 through the power management module 841 .

The power management module 841 is used for connecting the battery 842 , the charging management module 840 and the processor 810 . The power management module 841 receives the input of the battery 842 and/or the charging management module 840, and supplies power for the processor 810, the internal memory 821, the external memory interface 820, the display screen 894, the camera 893, and the communication module 860, etc. The power management module 841 can also be used to monitor parameters such as battery capacity, battery cycle times, and battery health status (leakage, impedance). In some embodiments, the power management module 841 can also be disposed in the processor 810 . In some embodiments, the power management module 841 and the charging management module 840 can also be set in the same device.

The wireless communication function of the mobile phone 800 can be realized through the antenna 1, the antenna 2, the radio frequency module 850, the communication module 860, a modem, and a baseband processor.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in handset 800 can be used to cover single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: Cellular antennas can be multiplexed as WLAN diversity antennas. In some embodiments, an antenna may be used in conjunction with a tuning switch.

The radio frequency module 850 can provide a communication processing module applied on the mobile phone 800 including wireless communication solutions such as 2G/3G/4G/5G. The radio frequency module 850 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like. The radio frequency module 850 receives electromagnetic waves from the antenna 1, filters and amplifies the received electromagnetic waves, and transmits them to the modem for demodulation.

The communication module 860 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wi-Fi) network), bluetooth (blue yooth, BT), global navigation satellite system, etc. applied on the mobile phone 800. (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. For example, in the embodiment of the present application, the mobile phone 800 can access the Wi-Fi network through the wireless communication module 860 .

The wireless communication module 860 may be one or more devices integrating at least one communication processing module. The wireless communication module 860 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 810 . The wireless communication module 860 can also receive the signal to be sent from the processor 810, frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 to radiate out.

The mobile phone 800 realizes the display function through the GPU, the display screen 894, and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 894 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 810 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 894 is used to display images, videos and the like. The display screen 894 includes a display panel. For example, in this embodiment of the present application, the display screen 894 may be used to display the application interface of the above-mentioned first application, such as a device sharing interface, a device search interface, and a QR code scanning interface.

The mobile phone 800 can realize the shooting function through ISP, camera 893, video codec, GPU, display screen and application processor.

Camera 893 is used to capture still images or video. The object generates an optical image through the lens and projects it to the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other image signals. In some embodiments, the mobile phone 800 may include 1 or N cameras 893, where N is a positive integer greater than 1.

Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the mobile phone 800 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.

Video codecs are used to compress or decompress digital video. The handset 800 may support one or more video codecs. In this way, the mobile phone 800 can play or record videos in multiple encoding formats. In this application, the mobile phone 800 uses a frequency domain encoder to encode video data.

The external memory interface 820 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile phone 800. The external memory card communicates with the processor 810 through the external memory interface 820 to realize the data storage function. Such as saving music, video and other files in the external memory card.

Internal memory 821 may be used to store computer-executable program code, including instructions. The processor 810 executes various functional applications and data processing of the mobile phone 800 by executing instructions stored in the internal memory 821 . For example, in the embodiment of the present application, the processor 810 may execute instructions stored in the internal memory 821, and the internal memory 821 may include a program storage area and a data storage area.

The mobile phone 800 can implement audio functions through an audio module 870, a speaker 870A, a receiver 870B, a microphone 870C, an earphone interface 870D, and an application processor. Such as music playback, recording, etc.

The audio module 870 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 870 may also be used to encode and decode audio signals. In some embodiments, the audio module 870 may be set in the processor 810 , or some functional modules of the audio module 870 may be set in the processor 810 .

Speaker 870A, also called "horn", is used to convert audio electrical signals into sound signals.

Receiver 870B, also called "earpiece", is used to convert audio electrical signals into sound signals.

Microphone 870C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals.

The earphone interface 870D is used to connect wired earphones. The earphone interface 870D may be a USB interface 830, or a 3.5mm open mobile terminal platform (open mobile terminal platform, OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

The keys 890 include a power key, a volume key and the like. Key 890 may be a mechanical key. It can also be a touch button. The mobile phone 800 receives the input of the key 890 and generates a key signal input related to the user setting and function control of the mobile phone 800 .

The motor 891 can generate a vibrating prompt. The motor 891 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback.

The indicator 292 can be an indicator light, which can be used to indicate the charging status, the change of the battery capacity, and also can be used to indicate messages, missed calls, notifications and so on.

SIM card interface 895 is used for connecting SIM. The SIM card can be connected and separated from the mobile phone 800 by inserting it into the SIM card interface 895 or pulling it out from the SIM card interface 895 . The mobile phone 800 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. SIM card interface 895 can support Nano SIM card, Micro SIM card, SIM card, etc.

The methods in the following embodiments can all be implemented in a video transmission device having the above-mentioned hardware structure. The video transmission device is a video sending end. In the following embodiments, the above-mentioned video sending end is the mobile phone 800 shown in FIG. 8 as an example. The method of the application example will be described.

An embodiment of the present application provides a video data transmission method, which is applied to a video sending end. As shown in FIG. 9 , the video data transmission method may include S910-S930.

S910. The video sending end detects the transmission quality of the N links used to transmit the first video data.

In the embodiment of the present application, when the video sending end sends video data to the video receiving end, the video sending end can respectively send video data to the video receiving end through N links, where N is a positive integer greater than or equal to 2.

Based on the description of the above embodiment, it can be seen that the first video data sent by the video sending end to the video receiving end includes a plurality of video frames, and the first video data originates from a video segment in the video to be transmitted. For example, a video segment may be A group of pictures (group of pictures, GOP) or multiple groups of pictures, and a video segment may also be video segments divided according to other methods, which are not specifically limited in this embodiment of the present application.

Exemplarily, a GOP is composed of 1 key frame (for example, I frame) and 29 forward search frames (P frames).

It should be noted that the above N links are links in different networks, that is, the N links belong to different networks. For example, the value of N is 2, and the above two links used to transmit video data are link 1 and link 2 respectively, where link 1 can be a link in WIFI network 1, and link 2 can be WIFI Link in network 2.

In this embodiment of the application, the factors affecting the transmission quality of the link may include the transmission delay of the link, the number of retransmissions, etc., and the video sending end may determine the transmission quality of the link according to at least one of the above-mentioned influencing factors, specifically Details are described in the following Examples.

S920. The video sending end determines, according to the transmission quality of the N links, video frames to be transmitted respectively corresponding to the N links.

In this embodiment of the present application, one of the N links is used for description. For example, the k-th link is used as an example to describe the above S920, where the value of k is 1, 2, ... N any value in . It should be understood that the video frame to be transmitted corresponding to a link refers to the video frame to be transmitted on the link.

Specifically, the video sending end judges whether the transmission quality of the kth link meets the preset condition, and if the transmission quality of the kth link meets the preset condition, determines the video to be transmitted corresponding to the kth link Frames are all video frames of the first video data; in the case that the transmission quality of the kth link does not meet the preset condition (that is: the transmission quality of the kth link is poor), determine the kth chain The video frame to be transmitted corresponding to the road is a part of the video frame of the first video data, wherein, the video frame to be transmitted includes all key data in the first video data (key data can be understood as important data in the first video data or data indispensable for decoding the first video data).

In the embodiment of the present application, after determining the video frames to be transmitted respectively corresponding to the N links, the video sending end encrypts the video frames to be transmitted respectively corresponding to the N links, wherein the encryption method may be Hash algorithm encryption, It may also be a symmetric encryption algorithm, and the specific encryption algorithm used is not limited in this application.

Exemplarily, assuming that the first video data transmitted on two links (link 1 and link 2 respectively), the first video data includes 10 video frames, the video sender first judges the link 1 Whether the transmission quality meets the preset conditions, if the transmission quality of link 1 meets the preset conditions, then the above 10 video frames are the video frames to be transmitted in link 1; if the link 1 does not meet the preset conditions, then the above 10 video frames Part of the video frames in the video frames are determined as the video frames to be transmitted on the link 1, for example: 6 video frames in the above 10 video frames are determined as the video frames to be transmitted on the link 1. Similarly, the video transmission device also judges whether the transmission quality of link 2 satisfies the preset condition, and if the transmission quality of link 2 meets the preset condition, the above-mentioned 10 video frames are determined as the video frames to be transmitted of link 2; If link 2 does not meet the preset conditions, determine some of the above 10 video frames as video frames to be transmitted in link 2, for example: determine 8 of the above 10 video frames as link 2 Video frames to be transmitted on channel 2.

In the embodiment of the present application, when the transmission quality of the link does not meet the preset conditions, the video sending end transmits some video frames in the first video data on the link, which can reduce the transmission pressure of the link , to improve the transmission efficiency of the link.

S930. The video sending end transmits the to-be-transmitted video frames respectively corresponding to the N links on the N links.

In the embodiment of the present application, the video sending end transmits the corresponding video frames to be transmitted on N different links in parallel.

With reference to the example in S920, the video frames to be transmitted in link 1 are 6 video frames of the first video data, and the video frames to be transmitted in link 2 are 8 video frames of the first video data, so, the video sending end The above-mentioned 6 video frames are transmitted on the link 1 and the above-mentioned 8 video frames are transmitted on the link 2 in parallel.

In the video data transmission method provided in the embodiment of the present application, the video sending end detects the transmission quality of the N links used to transmit the first video data, and determines the respective corresponding links of the N links according to the transmission quality of the N links. The video frames to be transmitted, wherein, when the transmission quality of the kth link meets the preset conditions, all the video frames in the first video data are used as the video frames to be transmitted of the kth link, and the kth link When the transmission quality of the link does not meet the preset condition, the partial video frame in the first video data is used as the video frame to be transmitted of the kth link; and, the video frame to be transmitted determined above includes the first video frame All key data in the data. In this way, when the transmission quality of the kth link is poor, this part of video frames in the first video data transmitted on the kth link includes all the key data in the first video data, thus to a certain extent It can ensure the smooth transmission of the key data in the first video data, and further solve the problems such as stuttering, blurred screen and high delay when the video receiving end plays the received video data.

Optionally, referring to FIG. 9 , as shown in FIG. 10 , before S910 , the video data transmission method provided in the embodiment of the present application further includes S1010 .

S1010. The video sending end performs frequency-domain layered coding on the first video data to obtain multiple video frames of the first video data.

Optionally, the plurality of video frames of the above-mentioned first video data are video frames obtained after the video sending end encodes the first video data in the collected video stream. It can be seen from the description of the above-mentioned embodiments that the first video data ( unencoded) includes at least one key frame and a number of forward frames.

In an implementation manner, the first video data may be encoded using a frequency-domain layered encoding method to obtain multiple video frames of the first video data. Specifically, for any key frame in the first video data , the basic frame of the key frame and the extended frame of the key frame are obtained after frequency-domain layered coding. Similarly, for any forward frame in the first video data, the forward frame is obtained after frequency-domain layered coding The basic frame of the frame and the extended frame of the forward frame, so that the multiple video frames obtained after the encoding of the first video data include at least a basic frame and an extended frame of a key frame, and a plurality of basic frames and extended frames of the forward frame Extended frame.

For a detailed description of the method for frequency-domain layered coding, reference may be made to relevant descriptions in the foregoing embodiments, and details are not repeated here.

In the embodiment of the present application, the multiple video frames of the first video data transmitted on each of the N links can be arranged in a certain order, and the arrangement order of the multiple video frames corresponding to each of the N links is It may be the same, or it may be different.

Optionally, in an implementation manner, taking the first link and the second link among the N links as an example, the arrangement order of the multiple video frames corresponding to the first link is the same as that of the multiple video frames corresponding to the second link. The order of arrangement of the video frames is different. Specifically, among the multiple video frames of the first video data, the i-th video frame corresponding to the first link is the basic frame of the first video frame, and the i-th video frame corresponding to the second link is the basic frame of the first video frame. The i video frames are extended frames of the first video frame; wherein, the first link and the second link are two different links among the above N links. Optionally, the first video frame may be a key frame or a forward frame.

Exemplarily, FIG. 11 is a schematic diagram of the ordering of multiple video frames on the first link and multiple video frames on the second link. In FIG. 11 , the first video frame on the first link is The basic frame of a key frame (which can be called the first key frame), the first video frame on the second link is the extended frame of the first key frame, and for example, the fourth video frame on the first link is An extended frame of a forward frame (may be referred to as a first forward frame), and the fourth video frame on the second link is a basic frame of the first forward frame.

Based on the description of the above-mentioned embodiments, during the parallel transmission of video frames to be transmitted by the video sending end on multiple links, some factors may occur (for example: the receiving end restarts or the receiving end does not consume the message sent by the sending end in time, cause the message to time out) cause one or more video frames in the video frame to be transmitted to be lost on the above-mentioned multiple links at the same time, then the video receiving end cannot obtain the lost video frame from multiple links, in order to solve this For this kind of problem, the video sending end needs to start the retransmission mechanism, and retransmit the above-mentioned one or more lost video frames to the video receiving end.

Optionally, referring to FIG. 10 , as shown in FIG. 12 , in the case that one or more video frames are simultaneously lost on multiple links, the video data transmission method provided in this embodiment of the present application further includes S1210-S1220.

S1210. The video sending end determines a retransmission video frame set.

The foregoing set of retransmitted video frames is an intersection of video frames to be retransmitted corresponding to each of the N links.

Exemplarily, assuming that the value of N is 2, the set of video frames to be retransmitted in link 1 (that is, video frames lost on link 1) includes the basic frame of P1 frame, the extended frame of P3 frame, the set of P4 frame Extended frames of frames; the set of video frames to be retransmitted in link 2 (that is, lost video frames on link 2) includes the basic frame of P1 frame, the extended frame of P2 frame, and the extended frame of P4 frame. Take the intersection of the set of video frames to be retransmitted in link 1 and the set of video frames to be retransmitted in link 2, and use the intersection as a set of retransmitted video frames, the set of retransmitted video frames includes: the basic frame of P1 frame and extension frames of P4 frames.

S1220. The video sending end transmits the video frames in the retransmission video frame set in the jth link in a queue-jumping transmission manner.

The transmission quality of the jth link above is higher than the transmission quality of other links in the above N links except the jth link, and the jth link is the link with the highest transmission quality among the N links .

The upload queue cutting transmission method is to insert the video frames in the retransmitted video frame set into the video frame queue being transmitted by the current link, so as to preferentially transmit the video frames in the retransmitted video frame set.

Optionally, in this embodiment of the application, the video sending end may insert the video frame in the retransmitted video frame set into the head of the video frame queue being transmitted by the current link, that is, the video sending end temporarily stops transmitting the current video frame queue For the video frames in the retransmission video frame set, the video frames in the retransmission video frame set are transmitted preferentially. After the transmission of the video frames in the retransmission video frame set is completed, the video sending end continues to transmit the video frames in the above video frame queue.

Optionally, the above-mentioned method for the video sending end to transmit the video frames in the retransmission video frame set in the jth link specifically includes: the video sending end ranks the priority of the video frames in the retransmission video frame set from high to high In the lower order, the video frames in the retransmitted video frame set are transmitted in the jth link.

In the embodiment of the present application, the priorities of the above video frames from low to high are: the extended frame of the forward frame, the extended frame of the key frame, the basic frame of the forward frame, and the basic frame of the key frame, wherein the higher the priority High means the video data in that video frame is more important.

Based on the above example, the retransmission video frame set determined by the video sender includes the basic frame of the P1 frame and the extended frame of the P4 frame, which are used to transmit the video frames to be transmitted in the two links (such as link 1 and link 2) , the transmission quality of link 1 is better than that of link 2. For link 1, assume that the video frames included in the current video frame queue are the basic frame of the P20 frame, the extended frame of the P20 frame, the basic frame of the P21 frame, the extended frame of the P21 frame... 100 of the P frame For basic frames and extended frames of P100 frames, the video sending end suspends the transmission of the video frames in the current video frame queue (that is, suspends the transmission of the basic frames of P20 frames, etc.), and the video sending end retransmits the video frames according to the priority of the video frame set The basic frame of the P1 frame and the extended frame of the P4 frame are transmitted sequentially from high to low (the priority of the basic frame of the P1 frame is higher than that of the extended frame of the P4 frame).

It should be noted that, in the embodiment of the present application, the video sending end transmits video data on each link in a similar manner. The following takes the kth link as an example to introduce the video data transmission method in detail, as shown in FIG. As shown in 13, the method for transmitting video data on the kth link includes: S1301-S1308.

S1301. The video sending end performs frequency-domain layered coding on the first video data to obtain multiple video frames of the first video data.

It should be understood that, for the method of performing frequency-domain layered coding on the first video data by the above-mentioned video sending end to obtain multiple video frames of the first video data, refer to the relevant description of S1010 in the above-mentioned embodiment, and details are not repeated here.

S1302. The video sending end acquires feedback information of the kth link.

In this embodiment of the present application, the link feedback information may include transmission delay and/or retransmission times. Optionally, the transmission delay may be an average transmission delay of video frames, a sum of transmission delays of a predetermined number of video frames, or any one of transmission delays of a current video frame.

S1303. The video sending end determines the transmission quality of the k-th link according to the feedback information of the k-th link.

In the embodiment of the present application, the video sending end determines whether the transmission quality of the k-th link satisfies a preset condition according to the feedback information of the k-th link. specific:

When the feedback information of the kth link above is the transmission delay, if the transmission delay of the kth link is less than the preset delay, the transmission quality of the kth link meets the preset condition; if the kth link If the transmission delays of the k links are greater than or equal to the preset delays, then the transmission quality of the k-th link does not meet the preset condition.

When the feedback information of the kth link is the number of retransmissions, if the number of retransmissions of the kth link is less than the preset number of times, the transmission quality of the kth link meets the preset condition; if the kth link If the number of retransmissions of the kth link is greater than or equal to the preset number of times, then the transmission quality of the kth link does not meet the preset condition.

When the feedback information of the kth link is the transmission delay and the number of retransmissions, if the transmission delay of the kth link is less than the preset delay, and the retransmission times of the kth link is less than the preset If the number of times is set, the transmission quality of the kth link meets the preset conditions; if the transmission delay of the kth link is greater than or equal to the preset threshold, and/or the number of retransmissions of the kth link is greater than or is equal to the preset number of times, then the transmission quality of the kth link does not meet the preset condition.

It should be understood that in this embodiment of the present application, if the transmission quality of the kth link meets the preset condition, it means that the transmission quality of the kth link is better, and if the transmission quality of the kth link does not meet the preset condition, it means that the kth link The transmission quality of the link is poor.

S1304. The video sending end determines the video frame to be transmitted corresponding to the k th link according to the transmission quality of the k th link.

Referring to the description of S920, the video frames to be transmitted corresponding to the kth link may be all video frames of the first video data, or may be some video frames of the first video data. Specifically, S1304 includes S1304a or S1304b.

S1304a. In a case where the transmission quality of the kth link satisfies a preset condition, the video frames to be transmitted corresponding to the kth link are all video frames of the first video data.

S1304b. In the case that the transmission quality of the kth link does not meet the preset condition, the video sending end, according to the preset proportion corresponding to the kth link, according to the priority of the video frame of the first video data from high to low In the order of , select some video frames in the video frames of the first video data as the video frames to be transmitted corresponding to the kth link.

The above preset ratios satisfy:

Wherein, n represents preset ratio, and Num ₁ represents the quantity of the non-key video frame in the video frame of the first video data, and Num ₂ represents the total quantity of the video frame of the first video data; Wherein, non-key video frame refers to comprising Video frames for non-critical data.

Optionally, the non-key video frames in the first video data include extended frames of the key frame, basic frames of the forward frame, and extended frames of the forward frame. When the transmission quality of a certain link is poor, the video sending end may choose to discard non-key video frames in the video frames of the first video data, and the non-key video frames have little impact on video quality.

It should be understood that since the basic frame of the key frame contains important information in the key frame, therefore, in the embodiment of the present application, it is stipulated that the basic frame of the key frame must be transmitted in the link as the video frame to be transmitted (that is, the basis of the key frame frame is a non-discardable video frame). The priorities of the video frames of the first video data are, from high to low, as follows: the basic frame of the key frame, the basic frame of the forward frame, the extended frame of the key frame, and the extended frame of the forward frame.

Exemplarily, assuming that the video frames of the above-mentioned first video data include 100 video frames, and the preset ratio is 40% (that is, when the link quality is poor, 60 video frames need to be discarded), the 100 video frames include 48 extended frames of forward frames, 2 extended frames of key frames, 48 base frames of forward frames, and 2 base frames of key frames. When the transmission quality of the kth link does not meet the preset conditions, the video sending end selects 38 basic frames of the forward frame and 2 The basic frame of the key frame of is used as the video frame to be transmitted corresponding to the kth link.

S1305. The video sending end transmits the video frame to be transmitted corresponding to the k-th link on the k-th link.

In the video data transmission method provided in the embodiment of the present application, the video sending end detects the transmission quality of the kth link used to transmit the first video data, and determines the kth link according to the transmission quality of the kth link Corresponding video frames to be transmitted, wherein, when the transmission quality of the kth link satisfies the preset condition, all the video frames in the first video data are used as the video frames to be transmitted of the kth link, and in the kth link When the transmission quality of the k links does not meet the preset condition, the partial video frames in the first video data are used as the video frames to be transmitted of the k link; and, the video frames to be transmitted determined above include the first video frame All key data in a video data. In this way, when the transmission quality of the kth link is poor, this part of video frames in the first video data transmitted on the kth link includes all the key data in the first video data, thus to a certain extent It can ensure the smooth transmission of the key data in the first video data, and further solve the problems such as stuttering, blurred screen and high delay when the video receiving end plays the received video data.

S1306. The video sending end detects the transmission quality of the kth link.

It should be noted that the above S1306 is executed after the video sending end completes the transmission of the to-be-transmitted video frame corresponding to the k-th link on the k-th link.

In the embodiment of the present application, for a detailed description of the method for detecting the transmission quality of the k-th link at the video sending end, reference may be made to the relevant description of the foregoing embodiments, and details are not repeated here.

S1307. The video sending end adjusts a preset ratio corresponding to the k-th link according to the transmission quality of the k-th link.

In the embodiment of the present application, the above-mentioned video sending end will use the adjusted preset ratio for subsequent video data transmission according to the transmission quality of the k-th link, for example, the above-mentioned adjusted preset ratio is used to transmit the second The video data, wherein the second video data and the first video data come from two adjacent video segments of the same video to be transmitted.

It should be understood that in the embodiment of the present application, the transmission quality of the k-th link is continuously detected according to the above method, and the preset ratio is dynamically adjusted according to the transmission quality of the k-th link, which can guarantee to a certain extent the transmission quality of the video to be transmitted The smooth transmission of the frame, and can improve the utilization rate of the kth link.

Referring to FIG. 13, as shown in FIG. 14, the above S1307 includes S1401-S1403.

S1401. The video sending end judges whether the transmission quality of the k-th link satisfies a preset condition.

In this embodiment of the present application, for a detailed description of the method for the video sending end to determine whether the transmission quality of the k-th link meets the preset condition, reference may be made to the relevant description of S1303 above, which will not be repeated here.

S1402. When the transmission quality of the kth link satisfies a preset condition, the video sending end increases a preset ratio.

When the transmission quality of the k links meets the preset conditions, the video sender increases the preset ratio; that is to say, the transmission quality of the k link is better, and the video sender increases the preset ratio so that The video sending end sends more video frames in the video data to the video receiving end, thereby improving the playback quality of the video data decoded by the video receiving end.

S1403. When the transmission quality of the k-th link does not meet the preset condition, the video sending end reduces the preset ratio.

When the transmission quality of the kth link does not meet the preset conditions, the video sender reduces the preset ratio, that is, when the transmission quality of the kth link is poor, the video frame to be transmitted decreases The proportion of non-key video frames, so that the video sender can reduce the number of video data frames sent to the video receiver, thereby reducing the transmission pressure of the kth link and improving the transmission efficiency of the kth link.

S1308. The video sending end selects part of the videos in the video frames of the second video data according to the preset ratio corresponding to the adjusted k-th link and in the order of priority of the video frames of the second video data from high to low frame as the video frame to be transmitted corresponding to the kth link.

In the embodiment of the present application, after the video sending end determines the video frame to be transmitted corresponding to the kth link, the video sending end transmits the video frame to be transmitted corresponding to the kth link on the kth link.

Correspondingly, the embodiment of the present application provides a video data transmission device, the video data transmission device is used to execute each step in the above video data transmission method, and the embodiment of the present application can carry out the video data transmission device according to the above method example For the division of functional modules, for example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. The division of modules in the embodiment of this application is schematic, and is only a logical function division, and there may be other division methods in actual implementation.

In the case of dividing each functional module corresponding to each function, FIG. 15 shows a possible structural diagram of the video data transmission device involved in the above embodiment. As shown in FIG. 15 , the video data transmission device includes a detection module 1501 , a determination module 1502 and a transmission module 1503 .

The detection module 1501 is configured to detect the transmission quality of the N links, for example, execute S910 in the above method embodiment.

The determination module 1502 is configured to determine video frames to be transmitted respectively corresponding to the N links according to the transmission quality of the N links, for example, execute S920 in the above method embodiment.

The transmission module 1503 is configured to transmit the video frames to be transmitted respectively corresponding to the N links on the N links, for example, execute S930 in the above method embodiment.

Optionally, the video data transmission device provided in the embodiment of the present application further includes: an encoding module 1507; the encoding module 1507 is configured to perform frequency-domain layered encoding on the first video data to obtain multiple video frames of the first video data, for example Execute S1010 and S1301 in the above method embodiment.

Optionally, the determination module 1502 is also configured to determine a retransmission video frame set, for example, execute S1210 in the above method embodiment.

Optionally, the video data transmission device provided in the embodiment of the present application further includes: a queue jumping module 1508; the queue jumping module 1508 is used to transmit the video frames in the retransmission video frame set in the jth link in a queue jumping transmission mode, for example Execute S1220 in the above method embodiment.

Optionally, the video data transmission device provided in the embodiment of the present application further includes: an acquisition module 1504; the acquisition module 1504 is configured to acquire the feedback information of the kth link, for example, execute S1302 in the above method embodiment.

Optionally, the determination module 1502 is further configured to determine the transmission quality of the kth link according to the feedback information of the kth link, for example, execute S1303 in the above method embodiment.

Optionally, the determination module 1502 is further configured to determine the video frame to be transmitted corresponding to the kth link according to the transmission quality of the kth link. For example, execute S1304 in the above method embodiment.

Optionally, the video data transmission device provided in the embodiment of the present application further includes: a control module 1505; the control module 1505 is used to send the video according to the kth link when the transmission quality of the kth link does not meet the preset condition. According to the preset ratio corresponding to the kth link, according to the order of the priority of the video frame of the first video data from high to low, select some video frames in the video frame of the first video data as the link corresponding to the kth link The video frame to be transmitted; for example, execute S1304b in the above method embodiment.

Optionally, the above-mentioned transmission module 1503 is further configured to transmit the video frame to be transmitted corresponding to the k-th link on the k-th link; for example, execute S1305 in the above-mentioned method embodiment.

Optionally, the detection module 1501 is further configured to detect the transmission quality of the kth link, for example, execute S1306 in the above method embodiment.

Optionally, the video data transmission device provided in the embodiment of the present application further includes: an adjustment module 1506; the adjustment module 1506 is used for the video sending end to adjust the preset ratio corresponding to the k-th link according to the transmission quality of the k-th link , for example, execute S1307, S1402, and S1403 in the above method embodiment.

Optionally, the above-mentioned control module 1505 is further configured to select the link of the second video data according to the preset ratio corresponding to the adjusted k-th link, and according to the priority order of the video frames of the second video data from high to low. Some video frames in the video frames are used as video frames to be transmitted corresponding to the kth link; for example, S1308 in the above method embodiment is executed.

Each module of the above-mentioned video data transmission device can also be used to perform other actions in the above-mentioned method embodiment, and all relevant content of each step involved in the above-mentioned method embodiment can be referred to the function description of the corresponding functional module, and will not be repeated here. .

In the case of using an integrated unit, a schematic structural diagram of the video data transmission device provided in the embodiment of the present application is shown in FIG. 16 . In FIG. 16 , the video data transmission device includes: a processing module 1601 and a communication module 1602 . The processing module 1601 is used to control and manage the actions of the video data transmission device, for example, execute the steps performed by the detection module 1501, the determination module 1502, the acquisition module 1504, the control module 1505, the encoding module 1507 and the queue insertion module 1508, and/or Other processes for performing the techniques described herein. The communication module 1602 is used to support the interaction between the video data transmission apparatus and other devices, for example, to perform the steps performed by the above transmission module 1503, and/or to perform other processes of the technology described herein. As shown in FIG. 16 , the video data transmission device may further include a storage module 1603 for storing program codes of the video data transmission device, video frames to be transmitted, and the like.

Wherein, the processing module 1601 may be a processor or a controller, such as the processor 810 in FIG. 8 . The communication module 1602 may be a transceiver, an RF circuit, or a communication interface, etc., such as the mobile communication module 850 and/or the wireless communication module 860 in FIG. 8 . The storage module 1603 may be a memory, such as the internal memory 821 in FIG. 8 .

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using a software program, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on the computer, all or part of the processes or functions according to the embodiments of the present application will be generated. The computer can be a general purpose computer, special purpose computer, computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transferred from a website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device including a server, a data center, and the like integrated with one or more available media. The available medium may be a magnetic medium (for example, a floppy disk, a magnetic disk, a magnetic tape), an optical medium (for example, a digital video disc (digital video disc, DVD)), or a semiconductor medium (for example, a solid state drive (solid state drives, SSD)), etc. .

Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and brevity of the description, only the division of the above-mentioned functional modules is used as an example for illustration. In practical applications, the above-mentioned functions can be allocated according to needs It is completed by different functional modules, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. For the specific working process of the above-described system, device, and unit, reference may be made to the corresponding process in the foregoing method embodiments, and details are not repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be Incorporation may either be integrated into another system, or some features may be omitted, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: flash memory, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk, and other various media capable of storing program codes.

The above is only a specific implementation of the application, but the protection scope of the application is not limited thereto, and any changes or replacements within the technical scope disclosed in the application should be covered within the protection scope of the application . Therefore, the protection scope of the present application should be determined by the protection scope of the claims.

Claims

A method for transmitting video data, characterized in that it comprises:

Detecting the transmission quality of N links; wherein, the N links are used to transmit the first video data, the N links are links in different networks, and the first video data includes a plurality of video frames , N is a positive integer greater than or equal to 2;

According to the transmission quality of the N links, determine the video frames to be transmitted respectively corresponding to the N links; wherein, when the transmission quality of the k-th link meets the preset condition, the k-th link The video frames to be transmitted corresponding to the links are all video frames of the first video data; when the transmission quality of the kth link does not meet the preset condition, the to-be-transmitted video frames corresponding to the kth link The transmission video frame is a partial video frame in all of the first video data; wherein, the value of k is any one of 1, 2, ... N, and the video frame to be transmitted includes the first video data All key data of

The video frames to be transmitted respectively corresponding to the N links are transmitted in the N links.
The method according to claim 1, wherein detecting the transmission quality of the kth link comprises:

Acquire feedback information of the kth link, where the feedback information includes transmission delay and/or retransmission times;

Determine the transmission quality of the kth link according to the feedback information of the kth link;

When the feedback information is transmission delay, if the transmission delay of the kth link is less than a preset delay, the transmission quality of the kth link meets the preset condition; if the The transmission delay of the kth link is greater than or equal to the preset delay, then the transmission quality of the kth link does not meet the preset condition;

When the feedback information is the number of retransmissions, if the number of retransmissions of the kth link is less than a preset number of times, the transmission quality of the kth link meets the preset condition; if the kth link The number of retransmissions of the k link is greater than or equal to the preset number of times, then the transmission quality of the kth link does not meet the preset condition;

When the feedback information is the transmission delay and the number of retransmissions, if the transmission delay of the k-th link is less than a preset time delay, and the number of retransmissions of the k-th link is less than a preset number of times, Then the transmission quality of the kth link meets the preset condition; if the transmission delay of the kth link is greater than or equal to a preset threshold, and/or the retransmission of the kth link If the number of times is greater than or equal to the preset number of times, then the transmission quality of the kth link does not meet the preset condition.
The method according to claim 1 or 2, wherein when the transmission quality of the k-th link does not meet a preset condition, determining the video frame to be transmitted corresponding to the k-th link, include:

According to the preset ratio corresponding to the k-th link, according to the order of priority of the video frames of the first video data from high to low, select some video frames in the video frames of the first video data as The video frame to be transmitted corresponding to the kth link;

The preset ratio satisfies:
Wherein, n represents the preset ratio, Num 1 represents the number of non-key video frames in the video frames of the first video data, and Num 2 represents the total number of video frames of the first video data; wherein, The non-key video frame refers to a video frame including non-key data.
The method according to claim 3, wherein after transmitting the video frames to be transmitted respectively corresponding to the N links in the N links, the method further comprises:

Detecting the transmission quality of the kth link;

Adjusting the preset ratio corresponding to the kth link according to the transmission quality of the kth link.
The method according to claim 4, wherein, according to the transmission quality of the k link, adjusting the preset ratio corresponding to the k link includes:

When the transmission quality of the kth link meets the preset condition, increase the preset ratio;

If the transmission quality of the kth link does not meet the preset condition, reduce the preset ratio.
The method according to claim 4 or 5, characterized in that the method further comprises:

Determine the video frames to be transmitted corresponding to the K links of the second video data according to the adjusted preset ratios corresponding to the K links;

The video frame to be transmitted corresponding to the kth link is transmitted in the kth link.
The method according to any one of claims 3-6, wherein the method further comprises:

performing frequency-domain layered coding on the first video data to obtain a video frame of the first video data, the video frame of the first video data includes at least one basic frame and an extended frame of a key frame, and a plurality of forward frames The basic frame and the extended frame; the non-key video frame includes the extended frame of the key frame, the basic frame of the forward frame and the extended frame of the forward frame, and the key data of the first video data is the basis of the key frame frame.
The method according to claim 7, characterized in that,

The priority of the video frame of the first video data from high to low is the basic frame of the key frame, the basic frame of the forward frame, the extended frame of the key frame, and the extended frame of the forward frame frame.
The method according to any one of claims 1-7, characterized in that,

The i-th video frame in the video frame of the first video data corresponding to the first link is the basic frame of the first video frame, and the i-th video frame in the video frame of the first video data corresponding to the second link The video frames are extended frames of the first video frame; the first link and the second link are two different links among the N links.
The method according to any one of claims 1-8, further comprising:

Determine a retransmission video frame set, the retransmission video frame set is the intersection of the video frames to be retransmitted respectively corresponding to the N links;

Transmitting the video frames in the set of retransmitted video frames in the j-th link in a queue-jumping transmission mode; wherein, the transmission quality of the j-th link is higher than that of other links, and the other chains A path is a link except the j-th link among the N links.
The method according to claim 10, wherein transmitting the video frames in the set of retransmitted video frames in the jth link comprises:

The video frames in the set of retransmitted video frames are transmitted in the j-th link in descending order of priorities of the video frames.
The method according to any one of claims 1-11, wherein the video frames in the set of retransmitted video frames are transmitted in the jth link in a queue-jumping transmission mode, including:

Inserting the set of retransmitted video frames into the head of the queue of video frames to be transmitted corresponding to the jth link.
The method according to any one of claims 1-12, wherein before transmitting the video frames to be transmitted respectively corresponding to the N links in the N links, the method further comprises:

Encrypting video frames to be transmitted respectively corresponding to the N links.
A video data transmission device, characterized in that it includes a memory and a processor, the memory is coupled to the processor; the memory is used to store computer program codes, and the computer program codes include computer instructions; when the computer When the instructions are executed by the processor, the video data transmission device is made to execute the method according to any one of claims 1 to 13.
A computer storage medium, characterized in that it includes computer instructions, and when the computer instructions are run on the video data transmission device, the video data transmission device executes the method according to any one of claims 1 to 13 .
A computer program product, characterized in that, when the computer program product is run on a computer, the computer is made to execute the method according to any one of claims 1 to 13.