CN111107297A - Video transmission method, device, resource server and storage medium - Google Patents

Video transmission method, device, resource server and storage medium Download PDF

Info

Publication number
CN111107297A
CN111107297A CN201911400133.7A CN201911400133A CN111107297A CN 111107297 A CN111107297 A CN 111107297A CN 201911400133 A CN201911400133 A CN 201911400133A CN 111107297 A CN111107297 A CN 111107297A
Authority
CN
China
Prior art keywords
video
frame
transmitted
video frame
extraction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911400133.7A
Other languages
Chinese (zh)
Other versions
CN111107297B (en
Inventor
邓勇
朱敏
方文豪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Bigo Technology Pte Ltd
Original Assignee
Guangzhou Baiguoyuan Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Baiguoyuan Network Technology Co Ltd filed Critical Guangzhou Baiguoyuan Network Technology Co Ltd
Priority to CN201911400133.7A priority Critical patent/CN111107297B/en
Publication of CN111107297A publication Critical patent/CN111107297A/en
Application granted granted Critical
Publication of CN111107297B publication Critical patent/CN111107297B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
    • H04N21/64784—Data processing by the network
    • H04N21/64792—Controlling the complexity of the content stream, e.g. by dropping packets

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Security & Cryptography (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Telephonic Communication Services (AREA)

Abstract

The embodiment of the invention discloses a video transmission method, a video transmission device, a resource server and a storage medium. Wherein, the method comprises the following steps: if the network bandwidth of the current session party is smaller than the initial transmission total code rate of the video to be transmitted, determining the frame extraction proportion of different video frame types when the residual transmission total code rate after frame extraction under the corresponding video frame type of the video to be transmitted is most matched with the network bandwidth according to the frame extraction sequence of different video frame types in the video to be transmitted; and transmitting the video to be transmitted to the current conversation party according to the frame extraction proportion under different video frame types. The technical scheme provided by the embodiment of the invention enables the residual actual transmission total code rate of the video to be transmitted after frame extraction to be most matched with the network bandwidth of the current conversation party, avoids the situation that the playing is blocked due to the fact that the decoding fails because the key frame is lost due to insufficient bandwidth, and can reduce the transmission code rate of the video to be transmitted in a frame extraction mode without reducing the resolution of the video picture, thereby ensuring the definition of video playing to be unchanged.

Description

Video transmission method, device, resource server and storage medium
Technical Field
The embodiment of the invention relates to the field of video processing, in particular to a video transmission method, a video transmission device, a resource server and a storage medium.
Background
With the development of network streaming media technology, multi-user video application scenes such as multi-user video conferences or multi-user video chatting and the like are more and more common, so that communication among different users becomes more and more convenient, but due to the fact that a network state has a plurality of instabilities, if a media server issues a video packet uploaded by another user to a certain user, and the network downlink bandwidth of the user is low, the video packet is prone to generating a random packet loss condition in the issuing process, so that the fluency during video playing is low, and playing is prone to being blocked.
At present, in a multi-user video application scenario, if a network downlink bandwidth of a user is not enough to support complete transmission of a video packet at an adopted video code rate, only an intra-frame coded frame (I frame) subjected to video coding is screened out from the video packet to be transmitted, the screened I frame is issued to the user, and an inter-frame predictive coded frame (P frame) subjected to video coding is directly discarded.
Disclosure of Invention
The embodiment of the invention provides a video transmission method, a video transmission device, a resource server and a storage medium, which can avoid the situation of random packet loss or bandwidth waste in the video transmission process and ensure the definition of video playing.
In a first aspect, an embodiment of the present invention provides a video transmission method, where the method includes:
if the network bandwidth of the current session party is smaller than the initial total transmission code rate of the video to be transmitted, determining the frame extraction proportion of different video frame types when the residual total transmission code rate after frame extraction under the corresponding video frame type of the video to be transmitted is most matched with the network bandwidth according to the frame extraction sequence of different video frame types in the video to be transmitted;
and transmitting the video to be transmitted to the current conversation party according to the frame extraction proportion under different video frame types.
In a second aspect, an embodiment of the present invention provides a video transmission apparatus, including:
the frame extraction ratio determining module is used for determining the frame extraction ratio under different video frame types when the residual transmission total code rate after frame extraction under the corresponding video frame type of the video to be transmitted is most matched with the network bandwidth according to the frame extraction sequence of different video frame types in the video to be transmitted if the network bandwidth of the current session party is smaller than the initial transmission total code rate of the video to be transmitted;
and the video transmission module is used for transmitting the video to be transmitted to the current conversation party according to the frame extraction proportion under different video frame types.
In a third aspect, an embodiment of the present invention provides a resource server, where the resource server includes:
one or more processors;
storage means for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the video transmission method of any embodiment of the present invention.
In a fourth aspect, an embodiment of the present invention provides a video transmission system, where the system includes: in the resource server, the current session end and other session ends in communication with the resource server in the third aspect of the present invention, the other session ends upload corresponding videos to the resource server, and the resource server generates a video to be transmitted of the current session end according to the videos uploaded by the other session ends and transmits the video to be transmitted to the current session end according to the frame extraction ratio of the video to be transmitted under different video frame types.
In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements a video transmission method according to any embodiment of the present invention.
When the network bandwidth of the current session party is less than the initial transmission total code rate of the video to be transmitted, the video transmission method, the device, the resource server and the storage medium provided by the embodiment of the invention can perform frame extraction under the corresponding video frame types of the video to be transmitted in sequence according to the frame extraction sequence of the different video frame types preset in the video to be transmitted to reduce the initial transmission total code rate of the video to be transmitted, determine the frame extraction proportion selected under the different video frame types when the remaining transmission total code rate after frame extraction under the corresponding video frame types is most matched with the network bandwidth at the moment, further extract redundant video frames under the different video frame types of the video to be transmitted according to the frame extraction proportion under the different video frame types, and transmit the video to be transmitted after frame extraction to the current session party so that the actual transmission total code rate left after frame extraction of the video to be transmitted is most matched with the network bandwidth of the current session party, therefore, the situation that playing is blocked due to the fact that key frame packet loss is caused by insufficient bandwidth, decoding failure is caused is avoided, the situation that a large amount of random packet loss or bandwidth waste occurs in the video transmission process is reduced, the smoothness of video playing is improved, meanwhile, the total initial transmission code rate of the video to be transmitted is reduced through a frame extraction mode, the information content contained in the video frame does not need to be reduced, and the definition of video playing can be guaranteed to be unchanged.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings:
fig. 1 is a schematic diagram of a video transmission system according to an embodiment of the present invention;
fig. 2A is a flowchart of a video transmission method according to an embodiment of the present invention;
fig. 2B is a schematic diagram of a video transmission process according to an embodiment of the present invention;
fig. 3A is a flowchart of a video transmission method according to a second embodiment of the present invention;
fig. 3B is a schematic diagram of a video transmission process according to a second embodiment of the present invention;
fig. 4A is a flowchart of a video transmission method according to a third embodiment of the present invention;
fig. 4B is a schematic diagram of a video transmission process according to a third embodiment of the present invention;
fig. 5 is a schematic structural diagram of a video transmission apparatus according to a fourth embodiment of the present invention;
fig. 6 is a schematic structural diagram of a resource server according to a fifth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures. In addition, the embodiments and features of the embodiments in the present invention may be combined with each other without conflict.
The embodiment mainly aims at the phenomenon that in an application scene of a multi-person video session, the network bandwidth of a current session party is not matched with the actual transmission code rate of a video to be transmitted, so that bandwidth waste or video packet loss exists, and provides a downlink frame extraction strategy in the multi-person video session scene, firstly, frames are extracted in sequence under different video frame types of the video to be transmitted, and then when the actual transmission code rate remaining after the sequential frame extraction is most matched with the network bandwidth of the current session party, the frame extraction proportion selected under different video frame types is determined, then, under different video frame types of the video to be transmitted, the corresponding frame extraction proportion is adopted for frame extraction, and the video to be transmitted after the frame extraction is transmitted to the current session party, so that the actual transmission total code rate remaining after the frame extraction of the video to be transmitted is most matched with the network bandwidth of the current session party, and the situation that a large amount of random packet loss or bandwidth waste occurs in the video transmission process is avoided, the fluency of video playing is ensured as much as possible on the premise of ensuring that the definition is not changed.
Fig. 1 is a schematic diagram of a video transmission system according to an embodiment of the present invention. In particular, referring to fig. 1, the video transmission system may include a resource server 10, and a current session side 20 and other session sides 30 in communication with the resource server 10.
Wherein, the resource server 10 is a file server for forwarding video data of different session parties in an application scene of a multi-person video session, at this time, when any session party in the application scene of the multi-person video session is used as a current session party and video data of other session parties under the multi-person video session needs to be played, the resource server 10 synthesizes uploaded videos of each other session party to obtain a video to be transmitted of the current session party, when the network bandwidth of the current session party is smaller than the initial transmission total code rate of the video to be transmitted, firstly, according to the frame extraction sequence of different video frame types, frames are sequentially extracted under different video frame types of the video to be transmitted, and then, the frame extraction proportion selected under different video frame types when the actual transmission total code rate left after the sequential frame extraction is most matched with the network bandwidth of the current session party is determined, and then, frames are extracted under different video frame types of the video to be transmitted by adopting corresponding frame extraction proportions, and the video to be transmitted after frame extraction is sent to the current session party. The current session end 20 and the other session ends 30 are respectively user terminals where the current session end and the other session ends are located in an application scene of a multi-person video session, and various types of video communication software for supporting a user to participate in the multi-person video session are configured on the user terminals, the current session end 20 and the other session ends 30 in the multi-person video session upload corresponding video data to the resource server 10 in real time in the video session process, the resource server 10 performs frame extraction on the video data uploaded by the other session ends 30 according to the frame extraction proportion required by the current session end 20 under different video frame types, and transmits the video data of the other session ends 30 after corresponding frame extraction to the current session end 20.
Specifically, the resource server 10 receives the uploaded videos of each other session end 30 in real time, generates a video to be transmitted under the current session end 20, determines an initial total transmission rate when the video to be transmitted is not processed, and simultaneously detects the network bandwidth of the current session end 20, and when the network bandwidth is smaller than the initial total transmission rate of the video to be transmitted, it is described that the network bandwidth of the current session end does not support the transmission of all video frames of the video to be transmitted under the initial total transmission rate, and a phenomenon of video packet loss is easy to occur, so that in the embodiment of the present invention, a mode of performing frame extraction in the video to be transmitted can be adopted, the actual total transmission rate of the video to be transmitted is reduced, so that the actual total transmission rate of the video to be transmitted can be most matched with the network bandwidth of the current session end, and at this time, the resource server 10 can perform frame extraction sequences set in advance for different video frame types of the video to be transmitted, sequentially performing frame extraction under the corresponding video frame types of the video to be transmitted, judging whether the remaining total transmission code rate of the video to be transmitted after frame extraction is matched with the network bandwidth of the current session party, further determining the frame extraction proportion selected under different video frame types when the remaining total transmission code rate after frame extraction is most matched with the network bandwidth, further extracting redundant video frames under different video frame types of the video to be transmitted according to the frame extraction proportion under different video frame types, and transmitting the video to be transmitted after frame extraction to the current session party, so that the actual total transmission code rate remaining after frame extraction of the video to be transmitted is most matched with the network bandwidth of the current session party, thereby avoiding the situations of random packet loss or bandwidth waste in the video transmission process, improving the smoothness of video playing, and simultaneously reducing the initial total transmission code rate of the video to be transmitted in a frame extraction mode, the information content in the video frame does not need to be reduced, and the definition of video playing is ensured.
The specific frame extraction operation of the resource server 10 for the video to be transmitted under different video frame types in the embodiment of the present invention can be explained in detail in the following video transmission method, and is not specifically described here. It should be noted that the number of other session ends 30 in the embodiment of the present invention may be determined by the number of users participating in a video session in the video session of the current session party, which is not limited herein.
Example one
Fig. 2A is a flowchart of a video transmission method according to an embodiment of the present invention, which can be applied to any application scenario of a multi-user video session. The video transmission method provided by this embodiment may be performed by a video transmission apparatus provided by the embodiment of the present invention, which may be implemented by software and/or hardware, and integrated in a resource server that performs the method, where the resource server may be a file server for forwarding video session data between different users.
Specifically, referring to fig. 2A, the method may include the steps of:
s210, if the network bandwidth of the current session party is smaller than the initial total transmission code rate of the video to be transmitted, determining the frame extraction proportion of different video frame types when the residual total transmission code rate after frame extraction under the corresponding video frame type of the video to be transmitted is most matched with the network bandwidth according to the frame extraction sequence of different video frame types in the video to be transmitted.
In an application scenario of a multi-person video session, a current session party may be any session party participating in the multi-person video session, and at this time, for the current session party, a resource server may receive videos uploaded by other session parties participating in the multi-person video session in real time during a video session, and at this time, videos uploaded by other session parties need to be correspondingly issued to the current session party, so that a to-be-transmitted video issued to the current session party in this embodiment is composed of uploaded videos of other session parties in a video session where the current session party is located; meanwhile, since the video to be transmitted is usually encoded, at this time, according to different selected encoding types, video frames under a plurality of different video frame types exist in the video to be transmitted, for example, the video to be transmitted may include three video frame types of I frame, P frame and B frame, and if the current video frame is a B frame, the subsequent playing needs to decode the current video frame according to the adjacent frames before and after the current video frame, at this time, if the adjacent frames before and after the current video frame are extracted, the B frame cannot be decoded, and if the next video frame of the B frame is a P frame, at this time, the P frame cannot be decoded if the B frame is extracted, therefore, if there is a B frame in the video to be transmitted, the video to be transmitted after the frame extraction fails to be decoded by adopting a frame extraction manner, so that the different video frame types of the video to be transmitted in this embodiment only include an I frame and a P frame, that is, it is required that the video to be transmitted for the current session party in the video session room in which multiple persons participate by using the video transmission method of this embodiment is encoded only by using I frames and P frames. It should be noted that, because there is a random packet loss during streaming media transmission, most of streaming media transmission uses I frame and P frame transmission, and B frame coding is not usually used.
In addition, because the different video frame types of the video to be transmitted have different encoding modes, the video frames required for decoding are different, for example, an I frame can be successfully decoded according to itself, and a P frame needs to be decoded according to all video frames before the frame, if the I frame before the P frame is decimated, the P frame will fail to be decoded, so that in order to ensure that the video to be transmitted after frame decimation can be successfully decoded, a corresponding frame decimation sequence can be set in the different video frame types of the video to be transmitted, for example, frame decimation of the I frame after the P frame is set.
In this embodiment, the resource server will continuously receive the uploaded videos, which are collected by each other session party in the video session of the current session party within the preset time interval, at the preset time interval, in order to ensure the real-time performance of the video session, the preset time interval may be set to a shorter time interval, for example, the other session parties upload videos of the current time period to the resource server every 1s, at this time, the resource server may sequentially integrate each video frame in the uploaded videos of each other session party within the current preset time interval according to the uploading time of each video frame in the uploaded videos of each other session party, as shown in fig. 2B, so as to obtain the to-be-transmitted video for the current session party, at this time, according to the information amount contained in each video frame in the to-be-transmitted video and the preset time interval, calculating the initial total transmission code rate of a video to be transmitted at a current preset time interval, and detecting the network bandwidth of a current conversation party in order to ensure the accuracy of video transmission due to the real-time change of the network bandwidth; at this time, if the network bandwidth of the current session party is less than the initial total transmission rate of the video to be transmitted, it is indicated that the network bandwidth of the current session party does not support the complete transmission of the video to be transmitted under the initial total transmission rate, therefore, in this embodiment, according to the frame extraction sequence preset for different video frame types in the video to be transmitted, the corresponding initial frame extraction proportion is sequentially selected under each video frame type of the video to be transmitted to perform frame extraction on each video frame under the video frame type, and the remaining total transmission rate after frame extraction under each video frame type of the video to be transmitted is simultaneously calculated, so as to determine the matching degree of the remaining total transmission rate and the network bandwidth of the current session party, and when the remaining total transmission rate is most matched with the network bandwidth, the initial frame extraction proportion selected under different video frame types of the video to be transmitted is used as the final frame extraction proportion under different video frame types in this embodiment, therefore, when the video is transmitted in the following actual mode, all the video frames of the video frame types in the video to be transmitted are sequentially extracted according to the frame extraction proportion of the video frame types, and the stable transmission of the video to be transmitted is guaranteed.
For example, in this embodiment, for the initial frame-extracting ratio selected by the video to be transmitted in each video frame type, an enumeration manner may be adopted, the corresponding initial frame-extracting ratios in different video frame types may be continuously adjusted according to the difference between the network bandwidth and the actual total transmission code rate after frame extraction, and a binary search algorithm may be further adopted to continuously update until an optimal frame-extracting ratio is obtained, which is not limited in this embodiment.
For example, the initial frame extraction ratios of the video to be transmitted in different video frame types are respectively set according to the network state of the current session party, the frame extraction is sequentially performed according to the frame extraction sequence of the different video frame types in the video to be transmitted by adopting the initial frame extraction ratios in the corresponding video frame types, the remaining transmission total bit rate after the frame extraction of the video to be transmitted is obtained, the initial frame extraction ratios in the different video frame types are adjusted according to the matching difference degree of the remaining transmission total bit rate and the network bandwidth of the current session party, the new remaining transmission total bit rate is continuously obtained, and the latest initial frame extraction ratio in the different video frame types is used as the frame extraction ratio in the different video frame types in the embodiment until the latest matching difference degree is lower than the preset difference threshold.
Meanwhile, in order to ensure the accuracy of the total initial transmission rate of the current session end, the embodiment may determine the total initial transmission rate through a smoothing algorithm, for example, calculate the total initial transmission rate of the video to be transmitted in the corresponding preset time interval once every 4ms, and perform smoothing processing on the total initial transmission rate in the historical preset time interval at the current preset time interval to obtain the total initial transmission rate of the video to be transmitted in the current preset time interval.
In addition, the network bandwidth of the current session party has an initial transmission code rate greater than or equal to the video to be transmitted, and this embodiment may further include: and if the network bandwidth of the current conversation party is more than or equal to the initial transmission code rate of the video to be transmitted, setting the frame extraction proportion of the video to be transmitted under each video frame type as a non-frame extraction value.
Specifically, the network bandwidth of the current session party has an initial transmission code rate that is greater than or equal to the video to be transmitted, which indicates that the network bandwidth is large enough to completely support the complete transmission of the video to be transmitted at the initial transmission total code rate, and frame extraction is not required, so that the frame extraction ratio of the video to be transmitted in each video frame type is set to a corresponding non-frame extraction value, for example, the frame extraction ratio in each video frame type is 0, so that when the video to be transmitted is subsequently transmitted to the current session party, frame extraction is not required in each video frame type, and all the video is transmitted.
And S220, transmitting the video to be transmitted to the current conversation party according to the frame extraction proportion under different video frame types.
Specifically, after frame extraction ratios of videos to be transmitted under different video frame types are determined, the number of video frames which need to be extracted under each video frame type in the videos to be transmitted and received within a current preset time interval can be determined, and then the number of video frames which are allowed to be normally transmitted in the videos to be transmitted is determined; meanwhile, because the video frames between the P frames need to be referred to when the P frames in the video to be transmitted are decoded, in order to ensure normal decoding of the video frames after video transmission, in this embodiment, when each video frame in the video to be transmitted is transmitted, each video frame may be transmitted to the current session party according to the uploading time sequence of each video frame, at this time, the video frame type of the video frame currently being transmitted is determined, and the number of video frames that have been successfully transmitted to the current session party currently under the video frame type is determined, when the number of video frames currently being transmitted reaches the number of video frames allowed to be normally transmitted under the video frame type, any video frame under the video frame type is no longer transmitted, so that the video frame of the video to be transmitted under the video frame type and corresponding to the frame extraction ratio can remain and is not transmitted, that in this embodiment, the first video frame that needs to be transmitted can be determined in all the video frames under the video frame type according to the frame extraction ratio under the video frame type The video frame to be frame-extracted is completely transmitted under the video frame type before the video frame, and the video frame under the video frame type after the video frame are completely extracted, so that the successful decoding of each video frame after transmission is ensured; the steps are adopted for sequentially transmitting the video frames according to each video frame type of the video to be transmitted, so that the residual total transmission code rate of the video to be transmitted after frame extraction is carried out according to the corresponding frame extraction proportion under each video frame type can be most matched with the network bandwidth detected by the current conversation party at the current moment, the condition of random packet loss or bandwidth waste in the video transmission process is avoided, and the smoothness of video playing is improved.
According to the technical scheme provided by the embodiment, when the network bandwidth of the current session party is smaller than the initial transmission total bit rate of the video to be transmitted, the frame extraction sequence of different video frame types preset in the video to be transmitted can be sequentially carried out under the corresponding video frame types of the video to be transmitted so as to reduce the initial transmission total bit rate of the video to be transmitted, the frame extraction proportion selected under different video frame types when the residual transmission total bit rate after the frame extraction under the corresponding video frame types is most matched with the network bandwidth is determined at the moment, then redundant video frames are extracted under different video frame types of the video to be transmitted according to the frame extraction proportion under different video frame types, and the video to be transmitted after the frame extraction is transmitted to the current session party, so that the actual transmission total bit rate left after the frame extraction of the video to be transmitted is most matched with the network bandwidth of the current session party, and therefore, the generation of key frames due to insufficient bandwidth is avoided, the situation that playing is blocked due to decoding failure is caused, the situation that random packet loss or bandwidth waste occurs in the video transmission process is reduced, the smoothness of video playing is improved, meanwhile, the total initial transmission code rate of the video to be transmitted is reduced in a frame extraction mode, the information content in the video frame does not need to be reduced, and the definition of video playing can be guaranteed to be unchanged.
Example two
Fig. 3A is a flowchart of a video transmission method according to a second embodiment of the present invention, and fig. 3B is a schematic diagram of a video transmission process according to the second embodiment of the present invention. The present embodiment is optimized based on the above embodiments, and the present embodiment mainly explains in detail a specific determination process of a frame extraction ratio of a video to be transmitted under different video frame types.
Optionally, as shown in fig. 3A, this embodiment may include the following steps:
s310, if the network bandwidth of the current conversation party is smaller than the initial transmission total code rate of the video to be transmitted, determining the target video frame type of the last stage of frames to be extracted matched with the network bandwidth according to the frame extraction sequence of different video frame types in the video to be transmitted and the initial transmission code rate under different video frame types.
Specifically, since each video frame in the video to be transmitted has different video frame types after being correspondingly encoded, the total initial transmission code rate of the video to be transmitted is obtained by combining the initial transmission code rates of the video to be transmitted under different video frame types; the initial transmission code rates of the different video frame types can be determined according to the information content of the video to be transmitted received in the current preset time interval in each video frame type and the preset time interval, or the initial transmission code rates of the video to be transmitted in the historical preset time interval in the corresponding video frame type can be smoothed by adopting a smoothing algorithm to obtain the initial transmission code rate of the video to be transmitted in the current preset time interval in the video frame type.
Optionally, if the network bandwidth of the current session party is smaller than the total initial transmission rate of the video to be transmitted, first, according to the preset frame extraction sequence of different video frame types and the initial transmission rate of each video frame type, the video frame type without frame extraction supported by the network bandwidth of the current session party in the video transmission process is judged, that is, the target video frame type of the last stage to be extracted and matched with the network bandwidth of the current session party is determined according to the corresponding frame extraction sequence under all the video frame types.
Illustratively, if the network bandwidth of the current session party is BW, the different video frame types of the video to be transmitted include I-frame and P-frame, and the initial transmission code rate of the video to be transmitted under the I-frame is aIThe initial transmission code rate under P frame is aPAnd the I frame is decimated after the P frame, if aI<BW<aI+aPIf the network bandwidth does not support all transmission of P frames in the video to be transmitted and supports all transmission of I frames, the P frames need to be extracted, and the I frames do not need to be extracted, the target video frame type of the last-stage frame to be extracted matched with the network bandwidth is determined to be the P frame; if BW<aIAnd if the type of the last-stage target video frame to be subjected to frame extraction matched with the network bandwidth is determined to be the I frame, the last-stage target video frame to be subjected to frame extraction is determined to be the P frame.
S320, setting the frame extraction proportion of the video frame type behind the target video frame type as a non-frame extraction value, and setting the frame extraction proportion of the video frame type in front of the target video frame type as a full frame extraction value.
Optionally, because a video frame in a video frame type with a preceding frame extraction sequence may need to refer to a certain video frame in a video frame type with a subsequent frame extraction sequence for decoding, in order to ensure successful decoding of a video frame after transmission, it needs to be ensured that video frames in a video frame type with a preceding frame extraction sequence must be completely extracted during frame extraction, and then video frames in a video frame type with a subsequent frame extraction sequence are extracted, therefore, in this embodiment, after a target video frame type of a last stage to-be-extracted frame matched with the network bandwidth is determined, a frame extraction ratio in a video frame type after the target video frame type is set as a non-frame extraction value directly according to frame extraction sequences of different video frame types, that is, all video frames are extracted in the video frame type, so that a video frame in the video frame type does not need to be transmitted to a current session party; and meanwhile, the frame extraction proportion of the video frame type before the target video frame type is set as a full frame extraction value, namely, no video frame is extracted under the video frame type, so that all video frames under the video frame type are transmitted to the current conversation party.
Illustratively, if different video frame types of the video to be transmitted include an I frame and a P frame, and the target video frame type is a P frame, the frame extraction ratio under the I frame is set to 0, and then the specific frame extraction ratio under the P frame is determined.
S330, determining the frame extraction proportion of the target video frame type when the total target transmission code rate of the video to be transmitted under the target video frame type is most matched with the network bandwidth after the video to be transmitted under the target video frame type is continuously extracted after all video frames under the video frame type before the target video frame type are eliminated.
Specifically, when determining the frame extraction ratio of the video to be transmitted in the target video frame type, firstly, all video frames in the video frame type before the target video frame type need to be excluded from the video to be transmitted so as to accurately simulate the actual video transmission, thereby determining the remaining total transmission code rate after exclusion, continuously selecting the corresponding initial frame extraction ratio to perform frame extraction in the target video frame type according to the matching condition between the remaining total transmission code rate after exclusion and the network bandwidth of the current session party, calculating the remaining total target transmission code rate after frame extraction in the target video frame type, further judging the matching degree between the total target transmission code rate and the network bandwidth of the current session party, and when the total target transmission code rate after frame extraction in the target video frame type after exclusion of all video frames before the target video frame type is most matched with the network bandwidth, the initial frame extraction ratio selected for the target video frame type is used as the final frame extraction ratio in the target video frame type in this embodiment.
For example, in this embodiment, a binary search algorithm may be used to continuously optimize and calculate the frame-extracting ratio under the target video frame type, the maximum frame-extracting ratio max _ pro under the target video frame type is set to 100% at the beginning, the minimum frame-extracting ratio min _ pro is set to 0, at this time, the initial frame-extracting ratio is determined to be (max _ pro + min _ pro)/2, and then it is determined that all video frames under the video frame type before the target video frame type are excluded and the matching condition between the remaining target total transmission bitrate after frame-extracting and the network bandwidth is performed according to the initial frame-extracting ratio of (max _ pro + min _ pro)/2 under the target video frame type, if the network bandwidth is greater than the target total transmission bitrate, which indicates that the initial frame-extracting ratio (max _ pro + min _ pro)/2 is too large, the maximum frame-extracting ratio max _ pro is modified to be (max _ pro + min _ pro)/2, and the minimum frame extraction proportion min _ pro is still 0, the corresponding initial frame extraction proportion is continuously updated according to the new maximum frame extraction proportion or the minimum frame extraction proportion, the matching condition of the remaining target transmission total code rate and the network bandwidth after frame extraction is judged again, and the steps are sequentially circulated until the difference value between the final maximum frame extraction proportion and the minimum frame extraction proportion can reach the specified precision, at the moment, the remaining target transmission total code rate and the network bandwidth are enabled to reach the maximum matching, and the current initial frame extraction proportion is taken as the frame extraction proportion finally under the target video frame type.
And S340, transmitting the video to be transmitted to the current conversation party according to the frame extraction proportion under different video frame types.
According to the technical scheme provided by the embodiment, when the network bandwidth of the current session party is smaller than the initial transmission total bit rate of the video to be transmitted, the initial transmission total bit rate of the video to be transmitted can be reduced by sequentially performing frame extraction under the corresponding video frame types of the video to be transmitted according to the frame extraction sequence of the different video frame types preset in the video to be transmitted, at the moment, the video frame types which are closest to the frame extraction sequence and correspond to the video to be transmitted when the residual transmission total bit rate is most matched with the network bandwidth are completely extracted, the video frame types which are closest to the frame extraction sequence are completely reserved, so that redundant video frames are extracted under the different video frame types of the video to be transmitted, the video to be transmitted after frame extraction is ensured and transmitted to the current session party, the video frames after transmission can be successfully decoded, and the residual actual transmission total bit rate of the video to be transmitted after frame extraction is most matched with the network bandwidth of the current session party, therefore, the situation that playing is blocked due to the fact that key frame packet loss is caused by insufficient bandwidth, decoding failure is caused is avoided, the situation that random packet loss or bandwidth waste occurs in the video transmission process is reduced, the smoothness of video playing is improved, meanwhile, the total initial transmission code rate of the video to be transmitted is reduced through a frame extraction mode, the information content contained in the video frame does not need to be reduced, and the definition of video playing can be guaranteed to be unchanged.
EXAMPLE III
Fig. 4A is a flowchart of a video transmission method according to a third embodiment of the present invention, and fig. 4B is a schematic diagram of a video transmission process according to the third embodiment of the present invention. The present embodiment is optimized based on the above embodiments, and the present embodiment mainly explains in detail a specific calculation process of a frame extraction ratio of a video to be transmitted under different video frame types.
Optionally, as shown in fig. 4A, this embodiment may include the following steps:
s410, if the network bandwidth of the current conversation party is smaller than the initial transmission total code rate of the video to be transmitted, determining the frame extraction subsequence of each video frame type towards other conversation parties according to the initial transmission sub-code rate corresponding to each other conversation parties under different video frame types.
Specifically, the video to be transmitted for the current session party is composed of the uploaded videos of each other session party in the video session where the current session party is located, at this time, the video to be transmitted can include video frames in the uploaded videos of each other session party in the video frame type under each video frame type, at this time, the initial transmission bitrate of the video to be transmitted in each video frame type is obtained by combining the initial transmission bitrate of each other session party in the video frame type, at this time, the initial transmission bitrate of the other session parties in different video frame types can be determined by the information amount contained in the uploaded videos of the other session parties in the video frame type and the preset time interval, and the calculation can be performed by adopting a smoothing algorithm.
Illustratively, the resource server may continuously receive, at a preset time interval, the uploaded videos acquired by each other session party in the video session where the current session party is located within the preset time interval, sequentially integrate the video frames in each uploaded video according to the uploading time of each video frame in the uploaded videos of each other session party, obtain the to-be-transmitted video for the current session party, and simultaneously record the source, the video frame type, the amount of information included, and the like of each integrated video frame, as shown in fig. 4B, so as to calculate the initial transmission sub-code rate corresponding to each other session party in different video frame types in the subsequent process.
Optionally, if the network bandwidth of the current session party is less than the total initial transmission rate of the video to be transmitted, in this embodiment, all video frames in the video frame type are sequentially extracted in the corresponding video frame type according to the frame extraction sequence preset for different video frame types in the video to be transmitted, at this time, since video frames in the video frame type of each other session party exist in different video frame types, in order to ensure complete playing of videos of a plurality of other session parties as much as possible, in this embodiment, a frame extraction sub-sequence facing each other session party is also determined in each video frame type according to the initial transmission sub-code rate corresponding to each other session party in different video frame types, for example, video frames in the video frame type of other session parties with higher initial transmission sub-code rate in the corresponding video frame type are extracted first, and extracting the video frames of other conversation parties with lower initial transmission subcode rate under the video frame type.
At this time, if the video session of the current session party a includes B, C and D, the network bandwidth of the current session party is BW, the different video frame types of the video to be transmitted include I frame and P frame, the initial transmission sub-code rate of other conversation parties B under the I frame is bi, the initial transmission sub-code rate under the P frame is bp, the initial transmission sub-code rate of other conversation parties C under the I frame is ci, the initial transmission sub-code rate under the P frame is cp, the initial transmission sub-code rate of other session parties D under the I frame is di, the initial transmission sub-code rate under the P frame is dp, the initial transmission code rate of the video to be transmitted under the I frame is bi + ci + di, and determining the frame extraction sub-sequence from the I frame and the P frame to other conversation parties according to the initial transmission sub-code rates corresponding to other conversation parties under the I frame and the P frame.
And S420, determining the frame extraction sub-proportion corresponding to each other conversation party under different video frame types when the residual transmission total code rate of the video to be transmitted after frame extraction facing to each other conversation party under the corresponding video frame type is most matched with the network bandwidth according to the frame extraction sequence of different video frame types and the frame extraction sub-sequence under each video frame type.
Specifically, this embodiment may perform frame extraction on each video frame type in the video to be transmitted in sequence according to the frame extraction sequence of different video frame types, at this time, when performing frame extraction on each video frame type, because each other session party has a corresponding video frame under the corresponding video frame type, in order to ensure that the current session party plays videos of most other session parties completely as much as possible at this time, this embodiment may perform frame extraction in each video frame transmitted by each other session party under the video frame type according to the frame extraction sub-sequence to each other session party under each video frame type in sequence according to the initial frame extraction sub-proportion corresponding to each other session party under the video frame type, and calculate the remaining transmission total bit rate after performing frame extraction to each other session party under each video frame type of the video to be transmitted, and further determine the matching degree of the remaining transmission total bit rate with the network bandwidth of the current session party, and when the remaining transmission total code rate is most matched with the network bandwidth, the initial frame extraction sub-proportion of the video to be transmitted, which is selected by each other conversation party under different video frame types, is used as the frame extraction sub-proportion corresponding to each other conversation party under different video frame types in the embodiment.
For example, in this embodiment, determining a frame extraction sub-ratio corresponding to each other session party in different video frame types when the remaining total transmission code rate of the video to be transmitted after frame extraction for each other session party in the corresponding video frame type is most matched with the network bandwidth may specifically include: calculating the initial transmission code rate under each video frame type according to the initial transmission sub-code rate corresponding to each other conversation party under different video frame types, and determining the target video frame type of the last stage to-be-extracted frame matched with the network bandwidth; setting the frame extraction sub-proportion corresponding to each other conversation party under the video frame type behind the target video frame type as a non-frame extraction value, and setting the frame extraction sub-proportion corresponding to each other conversation party under the video frame type before the target video frame type as a full frame extraction value; and determining the frame extraction sub-proportion corresponding to each other conversation party under the target video frame type when the total target transmission code rate of the video to be transmitted, which is subjected to frame extraction towards each other conversation party under the target video frame type, is most matched with the network bandwidth after all video frames corresponding to each other conversation party under the video frame type before the target video frame type are eliminated.
Specifically, if the network bandwidth of the current session party is smaller than the total initial transmission rate of the video to be transmitted, the initial transmission sub-code rates corresponding to other session parties under each video frame type are combined to obtain the initial transmission code rate under the video frame type, and then the video frame type without frame extraction supported by the network bandwidth of the current session party in the video transmission process is judged according to the preset frame extraction sequence of different video frame types and the initial transmission code rate under each video frame type, that is, the target video frame type of the last stage to be subjected to frame extraction matched with the network bandwidth of the current session party is determined according to the corresponding frame extraction sequence under all the video frame types. Illustratively, if the video session in which the current session party a is located includes B, C and D other session parties, the network bandwidth of the current session party is BW, the different video frame types of the video to be transmitted include an I frame and a P frame, the initial transmission sub-bit rate of the other session party B under the I frame is bi, the initial transmission sub-bit rate of the P frame is bp, the initial transmission sub-bit rate of the other session party C under the I frame is ci, the initial transmission sub-bit rate of the P frame is cp, the initial transmission sub-bit rate of the other session party D under the I frame is di, the initial transmission sub-bit rate of the P frame is dp, the initial transmission bit rate of the video to be transmitted under the I frame is bi + ci + di, and the initial transmission bit rate of the P frame is bp + cp + dp.
Further, after the target video frame type of the last stage to-be-extracted frame is determined, frame extracting sub-ratios corresponding to other conversation parties under the video frame type behind the target video frame type are all set as non-frame extracting values, and frame extracting sub-ratios corresponding to other conversation parties under the video frame type before the target video frame type are all set as full frame extracting values; simultaneously, all video frames corresponding to other conversation parties under the video frame type before the target video frame type are excluded from the video to be transmitted so as to accurately simulate actual video transmission, thereby determining the remaining total transmission code rate after exclusion, sequentially selecting corresponding initial frame extraction sub-proportion from the video frames uploaded by other conversation parties under the video frame type according to the frame extraction sub-sequence from the target video frame type to other conversation parties, calculating the remaining target total transmission code rate after frame extraction from the target video frame type to other conversation parties, further judging the matching degree of the target total transmission code rate and the network bandwidth of the current conversation party, and excluding all video frames from the video frame type before the target video frame type to other conversation parties and performing frame extraction from the target video frame type to other conversation parties When the bandwidths are most matched, the initial frame extraction sub-proportion selected for each other conversation party under the target video frame type is used as the frame extraction sub-proportion corresponding to each other conversation party under the target video frame type finally in the embodiment; at this time, the frame extraction sub-proportion corresponding to each other conversation party under the target video frame type in this embodiment may be determined by a binary search algorithm.
And S430, adding the video frames left after frame extraction in the uploaded videos of other session parties according to the frame extraction sub-proportion corresponding to the other session parties under different video frame types to the downlink sending windows of the other session parties.
Optionally, after determining the frame extraction sub-proportion corresponding to each other session party under different video frame types of the video to be transmitted, the number of video frames extracted to each other session party under each video frame type may be determined according to the frame extraction sub-proportion corresponding to each other session party under different video frame types, and then the number of video frames allowed to be normally transmitted to each other session party under each video frame type is determined, at this time, in order to ensure normal decoding of each video frame after video transmission, in this embodiment, it may be determined according to the uploading time sequence of each video frame whether the video frame needs to be transmitted to the current session party, at this time, the session party source and the video frame type of the current video frame to be transmitted are determined, and the number of video frames which have been successfully transmitted to the current session party to the other session parties under the video frame type is determined, when the number of the video frames currently transmitted by the other session parties does not reach the number of the video frames allowed to be normally transmitted by the other session parties under the video frame type, the current video frame is added into the downlink transmission window of the other session parties, namely, each video frame under different video frame types left after the frame extraction of the uploaded video of the other session parties is carried out according to the frame extraction sub-proportion of each other session party under each video type is added into the downlink transmission window corresponding to the other session parties, so as to be transmitted to the current session party later.
And S440, transmitting the video frame in the downlink sending window to the current conversation party at regular time.
Optionally, in this embodiment, the video frames in the downlink sending windows of the other session parties may be periodically sent to the current session party, so as to ensure the matching degree between the network bandwidth and the transmission code rate.
In the technical scheme provided by this embodiment, when the network bandwidth of the current session party is less than the initial transmission total bit rate of the video to be transmitted, frames can be sequentially extracted from the uploaded videos of each other session party under the corresponding video frame type of the video to be transmitted according to the frame extraction sequence of the different video frame types preset in the video to be transmitted and the frame extraction sub-sequence of each video frame type to each other session party to reduce the initial transmission total bit rate of the video to be transmitted, at this time, the frame extraction sub-ratio of each other session party under different video frame types when the remaining transmission total bit rate of the video to be transmitted after frame extraction towards each other session party under the corresponding video frame type is most matched with the network bandwidth is determined, and then redundant video frames are extracted from the videos transmitted under different video frame types to each other session party under different video frame types according to the frame extraction sub-ratio of each other session party under different video frame types, and the video to be transmitted after frame extraction is transmitted to the current session party, so that the residual actual transmission total code rate of the video to be transmitted after frame extraction is most matched with the network bandwidth of the current session party, thereby avoiding the situation that the playing is blocked due to the decoding failure caused by the key frame packet loss caused by insufficient bandwidth, reducing the situation of random packet loss or bandwidth waste in the video transmission process, improving the smoothness of video playing, simultaneously reducing the initial transmission total code rate of the video to be transmitted in a frame extraction mode without reducing the information content contained in the video frame, and ensuring the definition of video playing to be unchanged.
Example four
Fig. 5 is a schematic structural diagram of a video transmission apparatus according to a fourth embodiment of the present invention, specifically, as shown in fig. 5, the apparatus may include:
a frame extraction ratio determining module 510, configured to determine, according to a frame extraction sequence of different video frame types in a video to be transmitted, a frame extraction ratio in different video frame types when a remaining total transmission code rate after frame extraction in a video frame type corresponding to the video to be transmitted is most matched with a network bandwidth if the network bandwidth of a current session party is smaller than an initial total transmission code rate of the video to be transmitted;
and the video transmission module 520 is configured to transmit a video to be transmitted to the current session party according to the frame extraction ratio under different video frame types.
According to the technical scheme provided by the embodiment, when the network bandwidth of the current session party is smaller than the initial transmission total bit rate of the video to be transmitted, the frame extraction sequence of different video frame types preset in the video to be transmitted can be sequentially carried out under the corresponding video frame types of the video to be transmitted so as to reduce the initial transmission total bit rate of the video to be transmitted, the frame extraction proportion selected under different video frame types when the residual transmission total bit rate after the frame extraction under the corresponding video frame types is most matched with the network bandwidth is determined at the moment, then redundant video frames are extracted under different video frame types of the video to be transmitted according to the frame extraction proportion under different video frame types, and the video to be transmitted after the frame extraction is transmitted to the current session party, so that the actual transmission total bit rate left after the frame extraction of the video to be transmitted is most matched with the network bandwidth of the current session party, and therefore, the generation of key frames due to insufficient bandwidth is avoided, the situation that playing is blocked due to decoding failure is caused, the situation that random packet loss or bandwidth waste occurs in the video transmission process is reduced, the smoothness of video playing is improved, meanwhile, the total initial transmission code rate of the video to be transmitted is reduced in a frame extraction mode, the information content in the video frame does not need to be reduced, and the definition of video playing can be guaranteed to be unchanged.
Further, the total initial transmission code rate of the video to be transmitted can be obtained by combining the initial transmission code rates of the video to be transmitted under different video frame types.
Further, the frame extraction ratio determining module 510 may be specifically configured to:
determining a target video frame type of a last stage to-be-extracted frame matched with a network bandwidth according to the frame extraction sequence of different video frame types in a video to be transmitted and the initial transmission code rate under different video frame types;
setting the frame extraction proportion of the video frame type behind the target video frame type as a non-frame extraction value, and setting the frame extraction proportion of the video frame type in front of the target video frame type as a full frame extraction value;
and determining the frame extraction proportion of the target video frame type when the total target transmission code rate of the video to be transmitted after the video to be transmitted is continuously extracted under the target video frame type after all video frames under the video frame type before the target video frame type are excluded from the video frame type are most matched with the network bandwidth.
Further, the video to be transmitted may be composed of uploaded videos of other session parties in a video session where the current session party is located.
Further, the frame decimation ratio determining module 510 may include:
the sub-sequence determining unit is used for determining the frame extraction sub-sequence of each video frame type towards other conversation parties according to the initial transmission sub-code rate corresponding to each other conversation party under different video frame types;
and the sub-ratio determining unit is used for determining the frame extraction sub-ratio corresponding to each other conversation party under different video frame types when the residual transmission total code rate of the video to be transmitted after frame extraction facing to each other conversation party under the corresponding video frame type is most matched with the network bandwidth according to the frame extraction sequence of different video frame types and the frame extraction sub-sequence under each video frame type.
Further, the sub-ratio determining unit may be specifically configured to:
calculating the initial transmission code rate under each video frame type according to the initial transmission sub-code rate corresponding to each other conversation party under different video frame types, and determining the target video frame type of the last stage to-be-extracted frame matched with the network bandwidth;
setting the frame extraction sub-proportion corresponding to each other conversation party under the video frame type behind the target video frame type as a non-frame extraction value, and setting the frame extraction sub-proportion corresponding to each other conversation party under the video frame type before the target video frame type as a full frame extraction value;
and determining the frame extraction sub-proportion corresponding to each other conversation party under the target video frame type when the total target transmission code rate of the video to be transmitted, which is subjected to frame extraction towards each other conversation party under the target video frame type, is most matched with the network bandwidth after all video frames corresponding to each other conversation party under the video frame type before the target video frame type are eliminated.
Further, the frame extraction sub-proportion corresponding to each other conversation party under the target video frame type can be determined through a binary search algorithm.
Further, the video transmission module 520 may be specifically configured to:
adding the video frames left after frame extraction in the uploaded videos of other session parties according to the frame extraction sub-proportion corresponding to the other session parties under different video frame types to downlink sending windows of the other session parties;
and transmitting the video frame in the downlink sending window to the current conversation party at fixed time.
Further, the total initial transmission code rate of the video to be transmitted may be determined by a smoothing algorithm.
Further, the frame-extraction ratio determining module 510 may be further configured to:
and if the network bandwidth of the current conversation party is more than or equal to the initial transmission code rate of the video to be transmitted, setting the frame extraction proportion of the video to be transmitted under each video frame type as a non-frame extraction value.
The video transmission device provided by the embodiment can be applied to the video transmission method provided by any embodiment, and has corresponding functions and beneficial effects.
EXAMPLE five
Fig. 6 is a schematic structural diagram of a resource server according to a fifth embodiment of the present invention, as shown in fig. 6, the resource server includes a processor 60, a storage device 61, and a communication device 62; the number of the processors 60 in the resource server may be one or more, and one processor 60 is taken as an example in fig. 6; the processor 60, the storage device 61 and the communication device 62 in the resource server may be connected by a bus or other means, and the connection by the bus is taken as an example in fig. 6.
The storage device 61 is a computer-readable storage medium, and can be used for storing software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the video transmission method provided in the embodiment of the present invention. The processor 60 executes various functional applications of the resource server and data processing by running software programs, instructions, and modules stored in the storage device 61, that is, implements the above-described video transmission method.
The storage device 61 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the storage device 61 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the storage 61 may further include memory located remotely from the processor 60, which may be connected to the device over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The communication means 62 may be used to implement a network connection or a mobile data connection of the respective session side with the resource server.
The resource server provided by this embodiment can be used to execute the video transmission method provided by any of the above embodiments, and has corresponding functions and advantages.
EXAMPLE six
An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, can implement the video transmission method in any of the embodiments. The method specifically comprises the following steps:
if the network bandwidth of the current session party is smaller than the initial transmission total code rate of the video to be transmitted, determining the frame extraction proportion of different video frame types when the residual transmission total code rate after frame extraction under the corresponding video frame type of the video to be transmitted is most matched with the network bandwidth according to the frame extraction sequence of different video frame types in the video to be transmitted;
and transmitting the video to be transmitted to the current conversation party according to the frame extraction proportion under different video frame types.
Of course, the storage medium provided by the embodiment of the present invention contains computer-executable instructions, and the computer-executable instructions are not limited to the operations of the method described above, and may also perform related operations in the video transmission method provided by any embodiment of the present invention.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
It should be noted that, in the embodiment of the video transmission apparatus, the units and modules included in the embodiment are merely divided according to the functional logic, but are not limited to the above division as long as the corresponding functions can be implemented.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (14)

1. A video transmission method, comprising:
if the network bandwidth of the current session party is smaller than the initial total transmission code rate of the video to be transmitted, determining the frame extraction proportion of different video frame types when the residual total transmission code rate after frame extraction under the corresponding video frame type of the video to be transmitted is most matched with the network bandwidth according to the frame extraction sequence of different video frame types in the video to be transmitted;
and transmitting the video to be transmitted to the current conversation party according to the frame extraction proportion under different video frame types.
2. The method of claim 1, wherein the total initial transmission code rate of the video to be transmitted is obtained by combining initial transmission code rates of the video to be transmitted under different video frame types.
3. The method of claim 2, wherein determining the frame-decimation proportion for different video frame types when the remaining total transmission code rate after frame-decimation for the corresponding video frame type of the video to be transmitted is best matched with the network bandwidth comprises:
determining a target video frame type of the last stage to-be-extracted frame matched with the network bandwidth according to the frame extraction sequence of different video frame types in the to-be-transmitted video and the initial transmission code rate under different video frame types;
setting the frame extraction proportion of the video frame type behind the target video frame type as a non-frame extraction value, and setting the frame extraction proportion of the video frame type in front of the target video frame type as a full frame extraction value;
and determining the frame extraction proportion of the target video frame type when the total target transmission code rate of the video to be transmitted after the video to be transmitted is continuously extracted under the target video frame type after all video frames under the video frame type before the target video frame type are excluded from the video frame type are most matched with the network bandwidth.
4. The method according to claim 1, wherein the video to be transmitted is composed of uploaded videos of other session parties in a video session room where the current session party is located.
5. The method of claim 4, wherein determining the frame-decimation proportion for different video frame types when the remaining total transmission code rate after frame-decimation for the corresponding video frame type of the video to be transmitted is best matched with the network bandwidth comprises:
determining a frame extraction subsequence of each video frame type to other conversation parties according to the initial transmission sub-code rate corresponding to each other conversation parties under different video frame types;
and determining the frame extraction sub-proportion corresponding to each other conversation party under different video frame types when the residual transmission total code rate of the video to be transmitted after frame extraction towards each other conversation party under the corresponding video frame type is most matched with the network bandwidth according to the frame extraction sequence of different video frame types and the frame extraction sub-sequence under each video frame type.
6. The method of claim 5, wherein determining the frame extraction sub-proportion corresponding to each of the other conversation parties in different video frame types when the remaining total transmission code rate of the video to be transmitted after frame extraction for each of the other conversation parties in the corresponding video frame type is most matched with the network bandwidth comprises:
calculating the initial transmission code rate under each video frame type according to the initial transmission sub-code rate corresponding to each other conversation party under different video frame types, and determining the target video frame type of the last stage to-be-extracted frame matched with the network bandwidth;
setting the frame extraction sub-proportion corresponding to each other conversation party under the video frame type behind the target video frame type as a non-frame extraction value, and setting the frame extraction sub-proportion corresponding to each other conversation party under the video frame type before the target video frame type as a full frame extraction value;
and determining the frame extraction sub-proportion corresponding to each other conversation party under the target video frame type when the target transmission total code rate of the video to be transmitted after frame extraction to each other conversation party under the target video frame type is matched with the network bandwidth most closely after all video frames corresponding to each other conversation party under the video frame type before the target video frame type are excluded.
7. The method of claim 6, wherein the decimated sub-proportion for each of the other parties to the target video frame type is determined by a binary search algorithm.
8. The method according to claim 6, wherein said transmitting the video to be transmitted to the current session party according to the frame-extracting proportion under different video frame types comprises:
adding the residual video frames in the uploaded videos of the other conversation parties after frame extraction according to the frame extraction sub-proportion corresponding to the other conversation parties under different video frame types to downlink sending windows of the other conversation parties;
and transmitting the video frame in the downlink sending window to the current conversation party at regular time.
9. The method according to any of claims 1-8, wherein the initial total bitrate of transmission of the video to be transmitted is determined by a smoothing algorithm.
10. The method according to any one of claims 1-8, further comprising:
and if the network bandwidth of the current conversation party is more than or equal to the initial transmission code rate of the video to be transmitted, setting the frame extraction proportion of the video to be transmitted under each video frame type as a non-frame extraction value.
11. A video transmission apparatus, comprising:
the frame extraction ratio determining module is used for determining the frame extraction ratio under different video frame types when the residual transmission total code rate after frame extraction under the corresponding video frame type of the video to be transmitted is most matched with the network bandwidth according to the frame extraction sequence of different video frame types in the video to be transmitted if the network bandwidth of the current session party is smaller than the initial transmission total code rate of the video to be transmitted;
and the video transmission module is used for transmitting the video to be transmitted to the current conversation party according to the frame extraction proportion under different video frame types.
12. A resource server, characterized in that the resource server comprises:
one or more processors;
storage means for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the video transmission method of any of claims 1-10.
13. A video transmission system, comprising the resource server of claim 12, and a current session end and other session ends that communicate with the resource server, where the other session ends upload corresponding videos to the resource server, and the resource server generates a video to be transmitted of the current session end according to the videos uploaded by the other session ends, and transmits the video to be transmitted to the current session end according to frame extraction ratios of the video to be transmitted under different video frame types.
14. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out a video transmission method according to any one of claims 1 to 10.
CN201911400133.7A 2019-12-30 2019-12-30 Video transmission method, device, resource server and storage medium Active CN111107297B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911400133.7A CN111107297B (en) 2019-12-30 2019-12-30 Video transmission method, device, resource server and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911400133.7A CN111107297B (en) 2019-12-30 2019-12-30 Video transmission method, device, resource server and storage medium

Publications (2)

Publication Number Publication Date
CN111107297A true CN111107297A (en) 2020-05-05
CN111107297B CN111107297B (en) 2021-06-01

Family

ID=70425472

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911400133.7A Active CN111107297B (en) 2019-12-30 2019-12-30 Video transmission method, device, resource server and storage medium

Country Status (1)

Country Link
CN (1) CN111107297B (en)

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112055174A (en) * 2020-08-27 2020-12-08 深圳英飞拓智能技术有限公司 Video transmission method and device and computer readable storage medium
CN112203043A (en) * 2020-09-14 2021-01-08 中标慧安信息技术股份有限公司 Video transmission method and system
CN112312229A (en) * 2020-10-27 2021-02-02 唐桥科技(杭州)有限公司 Video transmission method and device, electronic equipment and storage medium
CN112399268A (en) * 2020-11-12 2021-02-23 唐桥科技(杭州)有限公司 Real-time streaming media transmission method and device and electronic equipment
CN112468763A (en) * 2020-11-11 2021-03-09 北京融讯科创技术有限公司 Video transmission and display method, device, equipment and storage medium of conference television
CN112468764A (en) * 2021-01-28 2021-03-09 浙江华创视讯科技有限公司 Method, system, server and storage medium for streaming media adaptive transmission
CN112541390A (en) * 2020-10-30 2021-03-23 四川天翼网络服务有限公司 Frame-extracting dynamic scheduling method and system for violation analysis of examination video
CN113747102A (en) * 2021-08-31 2021-12-03 百果园技术(新加坡)有限公司 Video call processing method, device, equipment and storage medium
CN113905200A (en) * 2021-10-08 2022-01-07 山东亚华电子股份有限公司 Video processing method and device based on statistics
CN115134629A (en) * 2022-05-23 2022-09-30 阿里巴巴（中国）有限公司 Video transmission method, system, device and storage medium
CN116033170A (en) * 2023-03-28 2023-04-28 阿里巴巴（中国）有限公司 Video decoding method, video encoding/decoding system, and video decoding device
CN117135364A (en) * 2023-10-26 2023-11-28 深圳市宏辉智通科技有限公司 Video decoding method and system
WO2023246744A1 (en) * 2022-06-24 2023-12-28 华为技术有限公司 Communication method and apparatus
CN117714693A (en) * 2024-02-06 2024-03-15 成都科玛奇信息科技有限责任公司 Medical image data compression transmission method, system, equipment and medium
CN118042069A (en) * 2024-04-11 2024-05-14 深圳市慧明捷科技有限公司 Adaptive transmission method and system for video code stream of network conference

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105357592A (en) * 2015-10-26 2016-02-24 山东大学苏州研究院 Selective frame dropping method for streaming media adaptive transmission
CN105791260A (en) * 2015-11-30 2016-07-20 武汉斗鱼网络科技有限公司 Network self-adaptive stream media service quality control method and device
CN106792247A (en) * 2016-12-13 2017-05-31 飞狐信息技术（天津）有限公司 Reduce the live TV stream method for pushing and device, live broadcast system of time delay
CN108174239A (en) * 2017-12-04 2018-06-15 中国联合网络通信集团有限公司 A kind of video transmission method and equipment
CN109151612A (en) * 2017-06-27 2019-01-04 华为技术有限公司 A kind of video transmission method, equipment and system
CN109729439A (en) * 2019-01-11 2019-05-07 北京三体云联科技有限公司 Method for real-time video transmission
US10469857B2 (en) * 2016-09-26 2019-11-05 Samsung Display Co., Ltd. System and method for electronic data communication
EP2912813B1 (en) * 2012-10-23 2019-12-04 Telefonaktiebolaget LM Ericsson (publ) A method and apparatus for distributing a media content service

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2912813B1 (en) * 2012-10-23 2019-12-04 Telefonaktiebolaget LM Ericsson (publ) A method and apparatus for distributing a media content service
CN105357592A (en) * 2015-10-26 2016-02-24 山东大学苏州研究院 Selective frame dropping method for streaming media adaptive transmission
CN105791260A (en) * 2015-11-30 2016-07-20 武汉斗鱼网络科技有限公司 Network self-adaptive stream media service quality control method and device
US10469857B2 (en) * 2016-09-26 2019-11-05 Samsung Display Co., Ltd. System and method for electronic data communication
CN106792247A (en) * 2016-12-13 2017-05-31 飞狐信息技术（天津）有限公司 Reduce the live TV stream method for pushing and device, live broadcast system of time delay
CN109151612A (en) * 2017-06-27 2019-01-04 华为技术有限公司 A kind of video transmission method, equipment and system
CN108174239A (en) * 2017-12-04 2018-06-15 中国联合网络通信集团有限公司 A kind of video transmission method and equipment
CN109729439A (en) * 2019-01-11 2019-05-07 北京三体云联科技有限公司 Method for real-time video transmission

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112055174A (en) * 2020-08-27 2020-12-08 深圳英飞拓智能技术有限公司 Video transmission method and device and computer readable storage medium
CN112203043A (en) * 2020-09-14 2021-01-08 中标慧安信息技术股份有限公司 Video transmission method and system
CN112312229A (en) * 2020-10-27 2021-02-02 唐桥科技(杭州)有限公司 Video transmission method and device, electronic equipment and storage medium
CN112541390B (en) * 2020-10-30 2023-04-25 四川天翼网络股份有限公司 Frame extraction dynamic scheduling method and system for examination video violation analysis
CN112541390A (en) * 2020-10-30 2021-03-23 四川天翼网络服务有限公司 Frame-extracting dynamic scheduling method and system for violation analysis of examination video
CN112468763B (en) * 2020-11-11 2023-11-24 北京融讯科创技术有限公司 Video transmission and display method, device and equipment for conference television and storage medium
CN112468763A (en) * 2020-11-11 2021-03-09 北京融讯科创技术有限公司 Video transmission and display method, device, equipment and storage medium of conference television
CN112399268A (en) * 2020-11-12 2021-02-23 唐桥科技(杭州)有限公司 Real-time streaming media transmission method and device and electronic equipment
CN112468764B (en) * 2021-01-28 2021-05-04 浙江华创视讯科技有限公司 Method, system, server and storage medium for streaming media adaptive transmission
CN112468764A (en) * 2021-01-28 2021-03-09 浙江华创视讯科技有限公司 Method, system, server and storage medium for streaming media adaptive transmission
CN113747102A (en) * 2021-08-31 2021-12-03 百果园技术(新加坡)有限公司 Video call processing method, device, equipment and storage medium
WO2023029994A1 (en) * 2021-08-31 2023-03-09 百果园技术(新加坡)有限公司 Video call processing method and apparatus, device and storage medium
CN113747102B (en) * 2021-08-31 2024-04-26 百果园技术(新加坡)有限公司 Video call processing method, device, equipment and storage medium
CN113905200A (en) * 2021-10-08 2022-01-07 山东亚华电子股份有限公司 Video processing method and device based on statistics
CN113905200B (en) * 2021-10-08 2023-07-11 山东亚华电子股份有限公司 Video processing method and device based on statistics
CN115134629A (en) * 2022-05-23 2022-09-30 阿里巴巴（中国）有限公司 Video transmission method, system, device and storage medium
CN115134629B (en) * 2022-05-23 2023-10-31 阿里巴巴（中国）有限公司 Video transmission method, system, equipment and storage medium
WO2023246744A1 (en) * 2022-06-24 2023-12-28 华为技术有限公司 Communication method and apparatus
CN116033170A (en) * 2023-03-28 2023-04-28 阿里巴巴（中国）有限公司 Video decoding method, video encoding/decoding system, and video decoding device
CN117135364A (en) * 2023-10-26 2023-11-28 深圳市宏辉智通科技有限公司 Video decoding method and system
CN117135364B (en) * 2023-10-26 2024-02-02 深圳市宏辉智通科技有限公司 Video decoding method and system
CN117714693A (en) * 2024-02-06 2024-03-15 成都科玛奇信息科技有限责任公司 Medical image data compression transmission method, system, equipment and medium
CN118042069A (en) * 2024-04-11 2024-05-14 深圳市慧明捷科技有限公司 Adaptive transmission method and system for video code stream of network conference

Also Published As

Publication number Publication date
CN111107297B (en) 2021-06-01

Similar Documents

Publication Publication Date Title
CN111107297B (en) Video transmission method, device, resource server and storage medium
US10659847B2 (en) Frame dropping method for video frame and video sending apparatus
US20190198027A1 (en) Audio frame loss recovery method and apparatus
US11881945B2 (en) Reference picture selection and coding type decision processing based on scene contents
US8971415B2 (en) Video communication system, device and method based on feedback reference frames
US8605638B2 (en) Voice messaging method and mobile terminal supporting voice messaging in mobile messenger service
CN102648584B (en) Use the system of forward error correction inspection available bandwidth, method and medium
US9232244B2 (en) Efficient frame forwarding in large scale real-time screen content sharing meetings
CN110113610B (en) Data transmission method and device
CN112866746A (en) Multi-path streaming cloud game control method, device, equipment and storage medium
CN110708569B (en) Video processing method and device, electronic equipment and storage medium
CN102196249A (en) Monitoring data playback method, EC (Encoder) and video management server
CN111654724B (en) Low-bit-rate coding transmission method of video conference system
CN113573063B (en) Video encoding and decoding method and device
CN109525852B (en) Live video stream processing method, device and system and computer readable storage medium
CN109587488B (en) Long reference frame selection method based on rate distortion optimization and frame loss prediction
CN103929682A (en) Method and device for setting key frames in video live broadcast system
CN101192903B (en) Data frame coding and decoding control method
CN115460458B (en) Video frame loss method and device
US20230082211A1 (en) Methods and devices for controlling a transmission of a video stream
EP2654311B1 (en) Synchronization method and synchronization apparatus for multicast group quick access, and terminal
CN103621063A (en) Systems, methods, and media for controlling a presentation of data images in a video stream
CN111064960B (en) Video coding method, device, server and storage medium
US20130111302A1 (en) Method and apparatus for recovering burst data loss by duplicating block code symbols
CN101102212A (en) End-to-end information transmission method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20220530

Address after: 31a, 15 / F, building 30, maple mall, bangrang Road, Brazil, Singapore

Patentee after: Baiguoyuan Technology (Singapore) Co.,Ltd.

Address before: Floor 4, 5, 6, 13, 14, 15, 16, Jisheng business center, 278 Xingtai Road, Shiqiao street, Panyu District, Guangzhou, Guangdong 510000

Patentee before: GUANGZHOU BAIGUOYUAN NETWORK TECHNOLOGY Co.,Ltd.

TR01 Transfer of patent right