CN113242446B - Video frame caching method, video frame forwarding method, communication server and program product - Google Patents

Video frame caching method, video frame forwarding method, communication server and program product Download PDF

Info

Publication number
CN113242446B
CN113242446B CN202110482611.4A CN202110482611A CN113242446B CN 113242446 B CN113242446 B CN 113242446B CN 202110482611 A CN202110482611 A CN 202110482611A CN 113242446 B CN113242446 B CN 113242446B
Authority
CN
China
Prior art keywords
frame
video
terminal
video data
key frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110482611.4A
Other languages
Chinese (zh)
Other versions
CN113242446A (en
Inventor
张伟
黄书敏
吴灿锐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing ByteDance Network Technology Co Ltd
Original Assignee
Beijing ByteDance Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing ByteDance Network Technology Co Ltd filed Critical Beijing ByteDance Network Technology Co Ltd
Priority to CN202110482611.4A priority Critical patent/CN113242446B/en
Publication of CN113242446A publication Critical patent/CN113242446A/en
Application granted granted Critical
Publication of CN113242446B publication Critical patent/CN113242446B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • H04N21/23106—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion involving caching operations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback

Abstract

The communication server acquires a video frame to be processed uploaded by a first terminal; judging whether the timestamp of the existing key frame in the existing cached video data in the cache is earlier than the timestamp of the video frame to be processed; and if so, caching the video frame to be processed to obtain current cached video data comprising the latest key frame, wherein the current cached video data is used for providing a caching forwarding service of the video stream for the second terminal. By the method, after the second terminal and the communication server complete capability negotiation, the communication server can directly provide key frame issuing for the second terminal according to the current cache video data in the cache, so that the problem of communication quality reduction caused by large occupation of communication resources due to the fact that key frame requests needing to be sent to the first terminal for multiple times in the prior art are solved.

Description

Video frame caching method, video frame forwarding method, communication server and program product
Technical Field
The disclosed embodiments relate to the field of computers, and in particular, to a video frame caching method, a video frame forwarding method, a communication server, and a program product.
Background
With the development of communication technology, real-time video communication has been widely applied in different scenes such as video conferences and multi-person chatting.
In the existing real-time communication system, after the communication server completes capability negotiation with the second terminal, the video stream data from the first terminal is forwarded to the second terminal. In the process, the second terminal initiates a key frame request to the communication server, and the communication server forwards the request to the first terminal; the first terminal issues a corresponding key frame to the communication server according to the key frame request, and the communication server forwards the key frame to the second terminal; and the second terminal decodes and renders the video stream after obtaining the key frame.
However, when there are multiple second terminals, the first terminal needs to respond to the key frame issuing request multiple times and issue the key frame, and this process occupies the communication resource of the second terminal, which is likely to cause the problem of the quality degradation of the whole communication.
Disclosure of Invention
In view of the above problems, the embodiments of the present disclosure provide a video frame caching method, a video frame forwarding method, a communication server, and a program product.
In a first aspect, the present disclosure provides a method for buffering video frames, including:
acquiring a video frame to be processed uploaded by a first terminal;
judging whether the timestamp of the existing key frame in the existing cached video data in the cache is earlier than the timestamp of the video frame to be processed;
and if so, caching the video frame to be processed to obtain current cached video data comprising the latest key frame, wherein the current cached video data is used for providing a caching forwarding service of the video stream for the second terminal.
In a second aspect, the present disclosure provides a method for forwarding a video frame, including:
acquiring a subscription request initiated by a second terminal;
constructing a reference frame queue comprising the latest key frame according to the frame reference relation of each frame in the current cache video data in the cache;
forwarding the video frame to the second terminal according to the reference frame queue;
wherein, the current buffered video data in the buffer is obtained according to the buffering method of the video frame in the first aspect.
In a third aspect, the present disclosure provides a communication server, comprising:
the first transceiving unit is used for acquiring a video frame to be processed uploaded by a first terminal;
the cache unit is used for judging whether the timestamp of the existing key frame in the existing cache video data in the cache is earlier than the timestamp of the video frame to be processed; and if so, caching the video frame to be processed to obtain current cached video data comprising the latest key frame, wherein the current cached video data is used for providing a caching forwarding service of the video stream for the second terminal.
In a fourth aspect, the present disclosure provides a communication server comprising:
the second receiving and sending unit is used for acquiring a subscription request initiated by a second terminal;
the forwarding unit is used for constructing a reference frame queue according to the frame reference relation of each frame in the current cache video data in the cache; forwarding a video frame to the second terminal through the second transceiver unit according to the reference frame queue; wherein, the current buffered video data in the buffer is obtained according to the buffering method of the video frame in the first aspect.
In a fifth aspect, the present disclosure provides an electronic device, comprising:
at least one processor; and
a memory;
the memory stores computer execution instructions;
the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the method of the first or second aspect.
In a sixth aspect, the present disclosure also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, the method according to the first aspect or the second aspect is implemented.
In a seventh aspect, the present disclosure provides a computer program product comprising computer instructions which, when executed by a processor, implement the method according to the first or second aspect.
The embodiment of the disclosure provides a video frame caching method, a forwarding method, a communication server and a program product, wherein the communication server acquires a to-be-processed video frame uploaded by a first terminal; judging whether the timestamp of the existing key frame in the existing cached video data in the cache is earlier than the timestamp of the video frame to be processed; and if so, caching the video frame to be processed to obtain current cached video data comprising the latest key frame, wherein the current cached video data is used for providing a caching forwarding service of the video stream for the second terminal. By the method, after the second terminal and the communication server finish capability negotiation, the communication server can directly provide key frame issuing for the second terminal according to the current cache video data in the cache, so that the problem of communication quality reduction caused by large occupation of communication resources due to the fact that key frame requests needing to be sent to the first terminal for multiple times in the prior art is solved.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present disclosure, and for those skilled in the art, other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is a schematic flow chart of a conventional video frame forwarding process for real-time communication;
FIG. 2 is a schematic diagram of a network architecture upon which the present disclosure is based;
fig. 3 is a schematic flowchart of a method for buffering video frames according to an embodiment of the present disclosure;
fig. 4 is a schematic signaling interaction diagram of another video frame buffering method according to an embodiment of the present disclosure;
fig. 5 is a schematic flowchart of a method for forwarding a video frame according to an embodiment of the present disclosure;
fig. 6 is a schematic signaling interaction diagram of another video frame forwarding method according to an embodiment of the present disclosure;
fig. 7 is a block diagram of a communication server provided in an embodiment of the present disclosure;
fig. 8 is a block diagram of another communication server provided in the embodiments of the present disclosure;
fig. 9 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present disclosure.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are some, but not all embodiments of the present disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.
With the development of communication technology, real-time video communication has been widely applied in different scenes such as video conferences and multi-person chatting.
In the existing real-time communication system, after the communication server completes capability negotiation with the subscriber, video stream data from the distributor is forwarded to the subscriber.
In the transmission process of video stream data, since a large amount of repeated information is included in a video, in order to improve transmission efficiency, the video stream data is generally encoded to compress the large amount of repeated information, so that the data volume of the video stream data is reduced, and the transmission efficiency of the video stream is improved.
In the conventional video stream coding/decoding mechanism, a key frame (e.g., I frame) is an independent frame with all information, and a non-key frame (e.g., P frame) is a video frame that requires a key frame for decoding. In the process of coding and decoding video frames, non-key frames including P frames need to depend on their associated key frame I frames to realize coding and decoding of the non-key frames.
Based on the above encoding and decoding mechanism, in the process of transmitting the video stream data from the publishing terminal to the subscribing terminal through the communication server, when the first video frame in the video stream received by the subscribing terminal is a non-key frame, the subscribing terminal cannot decode the video stream until the key frame is received due to the lack of the key frame corresponding to the non-key frame.
For this reason, in the existing video frame forwarding process of real-time communication, the subscribing end generally initiates a key frame request to the publishing end, and timely acquires the key frame of the video stream, so as to facilitate decoding of the subscribing end.
Specifically, fig. 1 is a schematic flow chart of a video frame forwarding process of existing real-time communication, as shown in fig. 1, after a subscriber (a second terminal) completes capability negotiation with a communication server, the subscriber (the second terminal) may subscribe to a video stream of a publisher (a first terminal). At this time, the subscriber (the second terminal) first initiates a key frame request to the communication server, and the communication server forwards the key frame request to the distributor (the first terminal) after receiving the key frame request.
After receiving the key frame request, the publishing terminal (first terminal) performs corresponding encoding to obtain a key frame, and then the publishing terminal (first terminal) issues the corresponding key frame to the communication server and forwards the key frame to the subscribing terminal by the communication server; and the subscriber (second terminal) decodes and renders the acquired video stream after acquiring the key frame, and provides the subscriber (second terminal) with the playing of the video stream.
However, with the upgrading and development of networks, real-time communication is no longer limited to one-to-one communication, and in more application scenarios, a one-to-many communication mode is adopted in real-time communication. That is, a plurality of subscribers simultaneously subscribe to video stream data of the same publisher.
That is, in this scenario, the existing process publisher receives the key frame requests from multiple subscribers, and compiles and issues key frames multiple times. The process occupies the communication resources of the subscriber, which is likely to cause the problem of communication quality degradation.
To solve such a problem, according to an embodiment of the present disclosure, a to-be-processed video frame uploaded by a publishing terminal (i.e., a first terminal described below) may be obtained first through a communication server; then, judging whether the time stamp of the existing key frame in the existing cached video data in the cache is earlier than the time stamp of the video frame to be processed; and if so, caching the video frame to be processed to obtain current cached video data comprising the latest key frame, wherein the current cached video data is used for providing a caching and forwarding service of the video stream for the subscriber terminal.
By the method, after the capability negotiation between the subscriber and the communication server is completed, the communication server can directly provide key frame issuing for the subscriber according to the current cached video data in the cache, so that the problem of communication quality reduction caused by a large amount of occupied communication resources due to the fact that key frame requests which are initiated to the publishing terminal for multiple times in the prior art are solved.
For better describing the present application, a first terminal in the present disclosure is referred to as a terminal carrying the aforementioned publishing terminal, and a second terminal in the present disclosure is referred to as a terminal carrying the aforementioned subscribing terminal.
Referring to fig. 2, fig. 2 is a schematic diagram of a network architecture based on the present disclosure, and the network architecture shown in fig. 2 may specifically include a first terminal 1, a second terminal 2 and a communication server 3.
The first terminal 1 and the second terminal 2 may be supported by a user terminal, and specifically may be hardware devices that can be used for real-time communication, such as a user mobile phone, a desktop computer, an intelligent home device, and a tablet computer, and the first terminal 1 and the second terminal 2 may respectively be installed with clients for real-time communication, such as a video client, a chat client, and the like. The first terminal 1 is generally a distribution terminal and refers to a terminal that sends video stream data, and the second terminal 2 is generally a subscription terminal and refers to a terminal that receives video stream data.
The communication server 3 may be a stand-alone server or a server cluster disposed at the cloud end, and may be configured to provide support and service for the respective real-time communication for the clients of the first terminal 1 and the second terminal 2.
The video frame caching method and the video frame forwarding method provided by the present disclosure are both carried in the communication server 3 shown in fig. 2, and the communication server 3 is connected with the first terminal 1 and the second terminal 2 through a network, and a communication channel is constructed for the two terminals. Meanwhile, after the second terminal 2 and the communication server 3 finish capability negotiation, the communication server 3 can directly provide key frames for the second terminal 2 to issue according to the current cached video data in the cache through caching and forwarding the video frames by the communication server 3, so that the problem of communication quality reduction caused by the fact that a large amount of communication resources are occupied due to the fact that key frame requests need to be initiated to the first terminal 1 for many times in the prior art is solved.
Based on the foregoing network architecture, in a first aspect, referring to fig. 3, fig. 3 is a schematic flowchart of a video frame caching method according to an embodiment of the present disclosure.
The method for caching video frames provided by the embodiment of the disclosure comprises the following steps:
101, acquiring a video frame to be processed uploaded by a first terminal;
step 102, judging whether a timestamp of an existing key frame in existing cached video data in a cache is earlier than a timestamp of the video frame to be processed;
step 103, if the timestamp of the existing key frame in the relevant key frame in the cache is earlier than the timestamp of the video frame to be processed, caching the video frame to be processed to obtain current cache video data including the latest key frame, where the current cache video data is used for providing a cache forwarding service of a video stream for the second terminal.
It should be noted that the main body of the implementation of the video frame caching method provided in this example is the aforementioned communication server. Before performing step 101, the communication server will complete capability negotiation with the first terminal and the second terminal, respectively.
In the present disclosure, video stream data transmitted between the first terminal, the communication server, and the second terminal may be encapsulated based on a format of an RTP packet. Accordingly, each data frame in the video stream is generally composed of a plurality of RTP packets, and each RTP packet belonging to the same video frame will have the same timestamp and identification, so that the communication server or the second terminal receiving the RTP packets can identify the video frame to which the RTP packet belongs based on the timestamp and identification. In addition, since the RTP packets are packetized, the RTP packets may be transmitted out of order during transmission, that is, the RTP packets belonging to different video frames are transmitted at the same time, and the communication server or the second terminal receiving the RTP packets identifies the RTP packets to obtain the video frames.
For convenience of explanation, the scheme will not be described in units of RTP packets in this disclosure, but the description of the scheme is directly performed in units of video frames.
In the present embodiment, different from the existing real-time communication process, a buffer area for video frames is provided in the communication server, and the communication server buffers the video stream data uploaded by the first terminal based on the buffering method provided by the present disclosure in the buffer area, so that when the second terminal needs to be provided with forwarding of the video stream data, a data forwarding service can be provided for the second terminal based on the video frames buffered in the buffer area.
Specifically, the first terminal may continuously upload video stream data, and one of the video frames in the video stream data will be the video frame to be processed in the present disclosure. Similar to the foregoing description of the transmission of the RTP packet, for the video frames in the video stream data, the transmission order of each video frame may be out of order, that is, for one end of the video stream data, each video frame may be transmitted out of order based on the timestamp, and after the communication server or the second terminal receives a plurality of video frames, the communication server or the second terminal needs to sort the time frame orders based on the timestamps to obtain the video stream data.
That is, as described above, since the video stream data is continuously uploaded, some buffered video data is already stored in the buffer, and the buffered video data is obtained by the communication server performing buffer processing on the video frame before the current processing.
After the communication server obtains the video frames to be processed, a caching method is adopted to cache the video frames to be processed. Firstly, the communication server reads the timestamp of the existing key frame in the cached video data, and then the timestamp of the video frame to be processed of the communication server is compared with the timestamp of the existing key frame to judge whether the video frame to be processed is older than the existing key frame.
If the timestamp of the current video frame to be processed is earlier than the existing key frame, it indicates that the latest key frame in the video data may have been cached in the cached video data of the communication server, and at this time, the communication server may directly end this caching process, and select the next video frame to be processed from the video stream data uploaded by the first terminal and perform the next caching process.
If the timestamp of the current video frame to be processed is later than the existing key frame, the communication server can cache the video frame to be processed first, at this time, the existing cached video data and the video frame to be processed form the current cached video data, and the latest key frame is certainly included in the current cached video data.
When the communication server receives a subscription request initiated by the second terminal, the communication server can directly issue the latest key frame for the second terminal based on the current cached video data at the moment, so that the second terminal can decode and render the video stream data according to the latest key frame. The delivery method may refer to any one of the embodiments in the second aspect, or an existing delivery method, and this embodiment does not limit or give details to this part.
Fig. 4 is a schematic signaling interaction diagram of another video frame buffering method according to an embodiment of the present disclosure.
Referring to fig. 4, in order to reduce the buffering pressure of the communication server, when buffering the video frames, it is further required to ensure that the video frames in the buffer are valid, that is, after the buffering of the video frames to be processed is completed, a timeout determination may also be performed.
In an optional embodiment, after the communication server buffers the video frames to be processed, the method further includes: performing timeout judgment on the latest key frame in the current cache video data; and if the latest key frame is overtime, clearing the current cache video data in the cache, and initiating a key frame request to the first terminal.
Specifically, after the communication server completes the buffering of the video frames to be processed, the timeout determination regarding the arrival time may also be made for the latest key frame therein. When the communication server determines that the latest key frame is overtime, the current cached video data cannot provide forwarding of an effective key frame for the second terminal, at this time, the current cached video data no longer has the caching value, the communication server clears the current cached video data in the cache, and executes the process in the prior art, that is, the communication server initiates a key frame request to the first terminal, receives a new key frame obtained and forwards the new key frame to the second terminal. Of course, after receiving the new key frame, the communication server may also buffer the new key frame for subsequent use.
In an alternative embodiment, in order to save the cache space, when the latest key frame in the currently cached video data is determined to be overtime, the communication server may first determine whether the video frame to be processed is a key frame.
That is, if the current cached video data includes a video frame to be processed, it is indicated that the timestamp of the video frame to be processed is earlier than the timestamp of the existing key frame in the original cache. In this case, the pending video frame is the latest key frame and it has not timed out. At this time, the communication server may clear the video frame with the timestamp earlier than the to-be-processed video frame in the currently cached video data, so as to save the caching space.
Correspondingly, if the video frame to be processed in the current cached video data is not a key frame, it is indicated that the latest key frame is an existing key frame in the existing cached video data before the current caching process.
At this time, the communication server judges the time-out according to the time stamp of the existing key frame in the existing cache video data, namely calculates the time difference between the time stamp of the existing key frame and the arrival time of the time frame to be processed, if the time difference is greater than a preset time threshold, the existing key frame is already time-out, namely the latest key frame is already time-out; if the time difference is not greater than the preset time threshold, it indicates that the existing key frame is not overtime, i.e. the latest key frame is not overtime.
In other optional embodiments, in order to further reduce the caching pressure, the communication server performs timeout determination on the latest key frame in the current cached video data, and if the latest key frame is not overtime, the communication server further determines whether the frame number of the current cached video data is greater than a preset frame number threshold; and if so, clearing the current cache video data in the cache, and initiating a key frame request to the first terminal.
The threshold value of the preset frame number can be set to 100, and the value can be determined based on the experience of those skilled in the art.
The video frame caching method provided by the embodiment of the disclosure can enable the communication server to directly provide the second terminal with the key frame for issuing according to the current cached video data in the cache after the second terminal and the communication server complete capability negotiation, so as to solve the problem that communication quality is reduced due to the fact that a large amount of communication resources are occupied because the key frame requests which are initiated to the first terminal for multiple times in the prior art are required.
Based on the foregoing caching method, in a second aspect, referring to fig. 5, fig. 5 is a schematic flowchart of a method for forwarding a video frame according to an embodiment of the present disclosure.
The method for forwarding the video frame provided by the embodiment of the disclosure comprises the following steps:
step 201, obtaining a subscription request initiated by a second terminal;
step 202, constructing a reference frame queue including the latest key frame according to the frame reference relation of each frame in the current cached video data in the cache;
step 203, forwarding the video frame to the second terminal according to the reference frame queue;
the current cached video data in the cache is obtained based on the video frame caching method described in any one of the first aspect, that is, the specific obtaining manner of the current cached video data referred to in this embodiment may refer to any one of the foregoing first aspect, and details of this portion will not be repeated in this embodiment. It should be noted that the main execution body of the video frame forwarding method provided in this example is the aforementioned communication server. Before executing step 201, the communication server will complete capability negotiation with the first terminal and the second terminal respectively, and perform buffering on the to-be-processed video frame delivered by the publishing frame as described above.
In this embodiment, different from the issuing of the video stream in the existing real-time communication process, since the cached video data composed of the latest key frame is cached in the cache area in the communication server, when the cached video data is forwarded to the second terminal, the video frames in the current cached video can be screened again, so as to ensure that the issued video frames are all valid frames.
Specifically, the communication server may obtain a subscription request initiated by the second terminal, and then, based on the subscription request, the communication server may read the current cached video data in the cache, and forward the video frame by using the video frame forwarding method provided by the present disclosure.
For the communication server, firstly, a reference frame queue of key frames is constructed based on the current cache video data in the cache. As previously described, a typical video frame is divided into a key frame I frame and a non-key frame P frame, which for each non-key frame P frame needs to be decoded and rendered with reference to its nearest key frame I frame.
Thus, at the time of forwarding, the communication server may first build a reference frame queue including the latest key frame based on such frame reference relationships. And then sending the reference frame queue to the second terminal for decoding by using the key frame therein.
Fig. 6 is a schematic signaling interaction diagram of another video frame forwarding method according to an embodiment of the present disclosure.
With reference to fig. 6, further, before the reference frame queue is constructed, it is to be avoided that the key frame is overtime, that is, before the communication server constructs the reference frame queue including the latest key frame according to the frame reference relationship of each frame in the current cached video data in the cache, the communication server is further configured to determine whether the first frame of the current cached video data in the cache is the key frame; and if so, executing the step of constructing a reference frame queue comprising the latest key frame according to the frame reference relation of each frame in the current cached video data in the cache. If not, a key frame request is sent to the first terminal.
Specifically, similar to the manner described in the foregoing caching method, since the caching of the video frames is performed in real time, and the sorting of the video frames in the currently cached video data is performed based on the caching order, if the first frame in the currently cached video data is not a key frame, the latest key frame in the currently cached video data may have timed out.
In this case, it has not taken any sense to send the latest key frame to the second terminal that may have timed out. That is to say, when the communication server finds that the first frame of the currently cached video data in the cache is not a key frame, it may directly adopt the existing communication flow, that is, initiate a key frame request to the first terminal, so as to forward the received new key frame to the second terminal for decoding and rendering.
And when the communication server finds that the first frame of the current cached video data in the cache is a key frame, the step of constructing a reference frame queue comprising the latest key frame according to the frame reference relationship of each frame in the current cached video data in the cache can be executed.
Further, in order to improve the efficiency of constructing the reference frame queue, considering timeliness, the communication server may sort the current cached video data according to the timestamp, and select the key frame and the video frame having a reference relationship with the key frame according to the frame reference relationship, so as to obtain the reference frame queue.
For example, if the current cached video data is sorted from old to new according to the time stamp, the sequence of I-P-P-SP-P can be obtained. Where the SP is a super P-frame that can reference the I-frame directly, the P-frame needs to reference the previous frame. Then the available reference frame queue is I-SP-P based on the frame reference relationship. Namely, the communication server sends the I-SP-P to the second terminal.
In order to further ensure the validity of the transmitted key frames in the reference frame queue to avoid the occurrence of the condition of issuing the key frames which are overtime to the second terminal, in other optional embodiments, when the communication server forwards the video frames to the second terminal according to the reference frame queue, the communication server further judges whether the reference frame queue includes the key frames which are not overtime; if yes, forwarding the video frame to the second terminal; otherwise, a key frame request is initiated to the first terminal.
When the communication server judges whether the reference frame queue comprises the key frame which is not overtime, the communication server judges whether the reference frame queue comprises the I frame or the SP frame which is not overtime.
The timeout determination may be specifically determined based on the arrival time of the frame, that is, a time difference between the arrival time of the I frame or the SP frame in the reference frame queue and the arrival time of the latest video frame is calculated, and if the time difference is greater than a preset time threshold, it indicates that the I frame or the SP frame is timeout, otherwise, the I frame or the SP frame is not timeout.
For the timeout determination, when any one of the video frames in the I frame or the SP frame in the reference frame queue is not timeout, the communication server will issue the reference frame queue to the second terminal; when the I frame or the SP frame in the reference frame queue is overtime, the communication server can directly send a key frame request to the first terminal.
In other optional embodiments, the communication server is further provided with a function of sending down speed control, so as to further ensure the communication quality of the second terminal.
Specifically, the communication server may obtain bandwidth information of the second terminal, and determine a code rate for forwarding the video frame according to the bandwidth information; and then forwarding the video frame to the second terminal according to the code rate. At this time, the code rate of the communication server issuing the cached video data should be smaller than the determined code rate of the forwarded video frame.
In this process, the obtaining of the bandwidth information of the second terminal may be implemented by a bandwidth detection module or a bandwidth detection algorithm, which is generally executed and completed when the second terminal establishes communication with the communication server.
Of course, when the second terminal just establishes communication with the communication server and the communication server does not obtain the bandwidth information of the second terminal, the communication server may forward the video frame to the second terminal according to the preset default code rate. At this time, the code rate of the communication server issuing the cached video data should be smaller than the default code rate.
By the processing mode, the cached video data can be issued according to a certain code rate, so that smooth and smooth communication is ensured.
According to the method for forwarding the video frames, after a subscription request initiated by a second terminal is obtained, a communication server constructs a reference frame queue comprising the latest key frames according to the frame reference relation of each frame in the current cached video data in a cache; and forwarding the video frame to the second terminal according to the reference frame queue. By the method, after the second terminal and the communication server finish capability negotiation, the communication server can directly provide key frame issuing for the second terminal according to the current cache video data in the cache, so that the problem of communication quality reduction caused by large occupation of communication resources due to the fact that key frame requests needing to be sent to the first terminal for multiple times in the prior art is solved.
In a third aspect, corresponding to the method for buffering video frames in the foregoing embodiment, fig. 7 is a block diagram of a communication server provided in the embodiment of the present disclosure. For ease of illustration, only portions that are relevant to embodiments of the present disclosure are shown. Referring to fig. 7, the communication server includes:
the first transceiving unit 10 is configured to acquire a to-be-processed video frame uploaded by a first terminal;
a cache unit 20, where the cache unit 20 is configured to determine whether a timestamp of an existing key frame in existing cached video data in a cache is earlier than a timestamp of the video frame to be processed; and if so, caching the video frame to be processed to obtain current cached video data comprising the latest key frame, wherein the current cached video data is used for providing a caching and forwarding service of the video stream for the second terminal.
Optionally, after the buffering the to-be-processed video frame, the buffering unit 20 is further configured to:
performing timeout judgment on the latest key frame in the current cache video data; if the latest key frame is overtime, clearing the current cache video data in the cache, and initiating a key frame request to the first terminal through the first transceiving unit 10.
Optionally, if the latest key frame is not timed out, the buffering unit 20 is further configured to:
judging whether the frame number of the current cache video data is greater than a preset frame number threshold value or not; if yes, clearing the current cache video data in the cache, and initiating a key frame request to the first terminal through the first transceiver unit 10.
Optionally, the cache unit 20 performs timeout determination on the latest key frame in the current cached video data, and is specifically configured to:
judging whether a video frame to be processed in the current cache video data is a key frame or not; if yes, the latest key frame in the current cache video data is not overtime, and the video frame with the timestamp being earlier than the video frame to be processed in the current cache video data is eliminated.
Optionally, if the video frame to be processed in the current cached video data is not a key frame, the cache unit 20 performs timeout determination on the latest key frame in the current cached video data according to a timestamp of the latest key frame in the existing cached video data.
Optionally, after initiating the key frame request to the first terminal, the first transceiver unit 10 is further configured to: and receiving a new key frame returned by the first terminal, and caching the new key frame through the caching unit 20.
The embodiment of the disclosure provides a communication server, which can directly provide a key frame for a second terminal according to current cached video data in a cache after the second terminal and the communication server complete capability negotiation, so as to solve the problem that communication quality is reduced due to the fact that communication resources are greatly occupied because a key frame request needs to be initiated to a first terminal for multiple times in the prior art.
In a fourth aspect, corresponding to the video frame forwarding method in the foregoing embodiment, fig. 8 is a block diagram of another communication server provided in the embodiment of the present disclosure. For ease of illustration, only portions that are relevant to embodiments of the present disclosure are shown. Referring to fig. 8, the communication server includes:
a second transceiver unit 30, configured to obtain a subscription request initiated by a second terminal;
the forwarding unit 40 is configured to construct a reference frame queue according to a frame reference relationship of each frame in the current cached video data in the cache; forwarding video frames to the second terminal through the second transceiving unit 30 according to the reference frame queue; wherein, the current buffered video data in the buffer is obtained by the method for buffering video frames according to any one of the first aspect.
Optionally, before the forwarding unit 40 constructs a reference frame queue including the latest key frame according to the frame reference relationship of each frame in the current cached video data in the cache, the forwarding unit is further configured to:
judging whether the first frame of the current cached video data in the cache is a key frame; if not, a key frame request is initiated to the first terminal through the second transceiver unit 30; and if so, executing the step of constructing a reference frame queue comprising the latest key frame according to the frame reference relation of each frame in the current cached video data in the cache.
Optionally, the forwarding unit 40 constructs a reference frame queue including the latest key frame according to the frame reference relationship of each frame in the current cached video data in the cache, and is specifically configured to:
and sequencing all video frames in the current cache video data according to the time stamps, and selecting the key frames and the video frames having reference relations with the key frames according to the frame reference relations to obtain the reference frame queue.
Optionally, the forwarding unit 40 is further configured to determine whether the reference frame queue includes a key frame that is not timed out;
if yes, the second transceiver 30 forwards the video frame to the second terminal; otherwise, the second transceiver unit 30 initiates a key frame request to the first terminal.
Optionally, when determining whether the reference frame queue includes a key frame that is not timed out, the forwarding unit 40 is specifically configured to:
and judging whether the reference frame queue comprises an I frame or an SP frame which is not overtime.
Optionally, the second transceiver unit 30 is specifically configured to:
acquiring bandwidth information of the second terminal, and determining a code rate for forwarding the video frame according to the bandwidth information; and forwarding the video frame to the second terminal according to the code rate.
Optionally, the second transceiver unit 30 is further configured to forward the video frame to the second terminal according to a preset default code rate before acquiring the bandwidth information of the second terminal and determining the code rate for forwarding the video frame according to the bandwidth information.
After the communication server provided by the embodiment of the disclosure acquires the subscription request initiated by the second terminal, a reference frame queue including the latest key frame is constructed according to the frame reference relationship of each frame in the current cache video data in the cache; and forwarding the video frame to the second terminal through the transceiving unit according to the reference frame queue. By the method, after the second terminal and the communication server complete capability negotiation, the communication server can directly provide key frame issuing for the second terminal according to the current cache video data in the cache, so that the problem of communication quality reduction caused by large occupation of communication resources due to the fact that key frame requests needing to be sent to the first terminal for multiple times in the prior art are solved.
The electronic device provided in this embodiment may be configured to implement the technical solutions of the method embodiments, and the implementation principles and technical effects are similar, which are not described herein again.
Referring to fig. 9, a schematic structural diagram of an electronic device 900 suitable for implementing the embodiment of the present disclosure is shown, where the electronic device 900 may be a terminal device or a media library. Among them, the terminal Device may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a Digital broadcast receiver, a Personal Digital Assistant (PDA), a tablet computer (PAD), a Portable Multimedia Player (PMP), a car terminal (e.g., car navigation terminal), etc., and a fixed terminal such as a Digital TV, a desktop computer, etc. The electronic device shown in fig. 9 is only an example, and should not bring any limitation to the functions and the use range of the embodiment of the present disclosure.
As shown in fig. 9, the electronic device 900 may include a song recommending apparatus (e.g., a central processing unit, a graphic processor, etc.) 901, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 902 or a program loaded from a storage device 908 into a Random Access Memory (RAM) 903. In the RAM 903, various programs and data necessary for the operation of the electronic apparatus 900 are also stored. The recommendation device 901, ROM 902, and RAM 903 of songs are connected to each other through a bus 904. An input/output (I/O) interface 905 is also connected to bus 904.
Generally, the following devices may be connected to the I/O interface 905: input devices 906 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 907 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 908 including, for example, magnetic tape, hard disk, etc.; and a communication device 909. The communication means 909 may allow the electronic apparatus 900 to communicate with other apparatuses wirelessly or by wire to exchange data. While fig. 9 illustrates an electronic device 900 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may be alternatively implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device 909, or installed from the storage device 908, or installed from the ROM 902. The above-described functions defined in the method of the embodiment of the present disclosure are performed when the computer program is executed by the recommendation apparatus 901 for songs.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or media library. In the latter scenario, the remote computer may be connected to the user's computer through any type of Network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The computer program product provided by this embodiment includes computer instructions, and the computer instructions are executed by the processor to perform the song recommendation method according to any of the foregoing items, which have similar implementation principles and technical effects, and are not described herein again.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of a unit does not in some cases constitute a limitation of the unit itself, for example, the first obtaining unit may also be described as a "unit obtaining at least two internet protocol addresses".
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), complex Programmable Logic Devices (CPLDs), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The following are some embodiments of the disclosure.
In a first aspect, according to one or more embodiments of the present disclosure, a method for buffering video frames includes:
acquiring a video frame to be processed uploaded by a first terminal;
judging whether the timestamp of the existing key frame in the existing cached video data in the cache is earlier than the timestamp of the video frame to be processed;
and if so, caching the video frame to be processed to obtain current cached video data comprising the latest key frame, wherein the current cached video data is used for providing a caching forwarding service of the video stream for the second terminal.
In an optional embodiment, after the buffering the to-be-processed video frame, the method further includes:
performing timeout judgment on the latest key frame in the current cache video data;
and if the latest key frame is overtime, clearing the current cache video data in the cache, and initiating a key frame request to the first terminal.
In an optional embodiment, if the latest key frame is not timed out, the method further includes:
judging whether the frame number of the current cache video data is greater than a preset frame number threshold value or not;
and if so, clearing the current cache video data in the cache, and initiating a key frame request to the first terminal.
In an optional embodiment, the time-out determination on the latest key frame in the currently cached video data includes:
judging whether the video frame to be processed in the current cache video data is a key frame or not;
if so, the latest key frame in the current cache video data is not overtime, and the video frame with the timestamp earlier than the video frame to be processed in the current cache video data is eliminated.
In an optional embodiment, if the video frame to be processed in the currently cached video data is not a key frame, the timeout determination of the latest key frame in the currently cached video data in the caching method is performed according to a timestamp of the latest key frame in existing cached video data.
In an optional embodiment, after initiating the key frame request to the first terminal, the method further includes:
and receiving a new key frame returned by the first terminal, and caching the new key frame.
In a second aspect, according to one or more embodiments of the present disclosure, a method for forwarding a video frame includes:
acquiring a subscription request initiated by a second terminal;
constructing a reference frame queue comprising the latest key frame according to the frame reference relation of each frame in the current cache video data in the cache;
forwarding the video frame to the second terminal according to the reference frame queue;
wherein the current buffered video data in the buffer is obtained by the method for buffering video frames according to any one of claims 1-6.
In an optional embodiment, before the constructing a reference frame queue including a latest key frame according to a frame reference relationship of each frame in the current cached video data in the cache, the method further includes:
judging whether the first frame of the current cached video data in the cache is a key frame;
if not, initiating a key frame request to the first terminal;
and if so, executing the step of constructing a reference frame queue comprising the latest key frame according to the frame reference relation of each frame in the current cached video data in the cache.
In an optional embodiment, the constructing a reference frame queue including a latest key frame according to a frame reference relationship of each frame in the current cached video data in the cache includes:
and sequencing all video frames in the current cache video data according to the time stamps, and selecting the key frames and the video frames having reference relations with the key frames according to the frame reference relations to obtain the reference frame queue.
In an optional embodiment, the forwarding, according to the reference frame queue, the video frame to the second terminal through the transceiver unit includes:
judging whether the reference frame queue comprises a key frame which is not overtime;
if yes, forwarding the video frame to the second terminal; otherwise, a key frame request is initiated to the first terminal.
In an optional embodiment, the determining whether the reference frame queue includes a key frame that has not timed out includes:
and judging whether the reference frame queue comprises an I frame or an SP frame which is not overtime.
In an optional embodiment, the forwarding the video frame to the second terminal includes:
acquiring bandwidth information of the second terminal, and determining a code rate for forwarding a video frame according to the bandwidth information;
and forwarding the video frame to the second terminal according to the code rate.
In an optional embodiment, before the obtaining the bandwidth information of the second terminal and determining the code rate for forwarding the video frame according to the bandwidth information, the method further includes:
and forwarding the video frame to the second terminal according to a preset default code rate.
In a third aspect, in accordance with one or more embodiments of the present disclosure, a communication server comprises:
the first transceiving unit is used for acquiring a video frame to be processed uploaded by a first terminal;
the cache unit is used for judging whether the timestamp of the existing key frame in the existing cache video data in the cache is earlier than the timestamp of the video frame to be processed; and if so, caching the video frame to be processed to obtain current cached video data comprising the latest key frame, wherein the current cached video data is used for providing a caching forwarding service of the video stream for the second terminal.
Optionally, after the buffering the to-be-processed video frame, the buffering unit is further configured to:
performing timeout judgment on the latest key frame in the current cache video data; and if the latest key frame is overtime, clearing the current cache video data in the cache, and initiating a key frame request to the first terminal through the transceiving unit.
Optionally, if the latest key frame is not timed out, the buffering unit is further configured to:
judging whether the frame number of the current cache video data is greater than a preset frame number threshold value or not; and if so, clearing the current cache video data in the cache, and initiating a key frame request to the first terminal through the transceiving unit.
Optionally, the cache unit performs timeout determination on the latest key frame in the current cached video data, and is specifically configured to:
judging whether a video frame to be processed in the current cache video data is a key frame or not; if so, the latest key frame in the current cache video data is not overtime, and the video frame with the timestamp earlier than the video frame to be processed in the current cache video data is eliminated.
Optionally, if the video frame to be processed in the current cached video data is not a key frame, the cache unit performs timeout determination on the latest key frame in the current cached video data according to a timestamp of the latest key frame in the existing cached video data.
Optionally, after initiating the key frame request to the first terminal, the first transceiver unit is further configured to: and receiving a new key frame returned by the first terminal, and caching the new key frame through the caching unit.
In a fourth aspect, according to one or more embodiments of the present disclosure, a communication server comprises:
the second receiving and sending unit is used for acquiring a subscription request initiated by a second terminal;
the forwarding unit is used for constructing a reference frame queue according to the frame reference relation of each frame in the current cached video data in the cache; forwarding a video frame to the second terminal through the second transceiver unit according to the reference frame queue; wherein, the currently buffered video data in the buffer is obtained by the method for buffering video frames according to any one of the first aspect.
Optionally, before the forwarding unit constructs a reference frame queue including the latest key frame according to the frame reference relationship of each frame in the current cached video data in the cache, the forwarding unit is further configured to:
judging whether the first frame of the current cached video data in the cache is a key frame; if not, a key frame request is sent to the first terminal through the second transceiver unit; and if so, executing the step of constructing a reference frame queue comprising the latest key frame according to the frame reference relation of each frame in the current cache video data in the cache.
Optionally, the forwarding unit constructs a reference frame queue including a latest key frame according to a frame reference relationship of each frame in the current cached video data in the cache, and is specifically configured to:
and sequencing all video frames in the current cache video data according to the time stamps, and selecting the key frames and the video frames having reference relations with the key frames according to the frame reference relations to obtain the reference frame queue.
Optionally, the forwarding unit is further configured to determine whether the reference frame queue includes a key frame that is not timed out;
if yes, the second transceiver unit forwards the video frame to the second terminal; otherwise, the second transceiving unit initiates a key frame request to the first terminal.
Optionally, when determining whether the reference frame queue includes a key frame that is not timed out, the forwarding unit is specifically configured to:
and judging whether the reference frame queue comprises an I frame or an SP frame which is not overtime.
Optionally, the second transceiver unit is specifically configured to:
acquiring bandwidth information of the second terminal, and determining a code rate for forwarding a video frame according to the bandwidth information; and forwarding the video frame to the second terminal according to the code rate.
Optionally, before the second transceiver unit obtains the bandwidth information of the second terminal and determines the code rate for forwarding the video frame according to the bandwidth information, the second transceiver unit is further configured to forward the video frame to the second terminal according to a preset default code rate.
In a fifth aspect, in accordance with one or more embodiments of the present disclosure, an electronic device comprises: at least one processor and memory;
the memory stores computer execution instructions;
the at least one processor executing the memory-stored computer-executable instructions causes the at least one processor to perform the method of any one of the preceding claims.
In a sixth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium has stored therein computer-executable instructions that, when executed by a processor, implement a method as in any one of the preceding.
Seventh aspect, according to one or more embodiments of the present disclosure, a computer program product comprising computer instructions for execution by a processor of a method as described in any of the previous claims.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.
Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims (16)

1. A method for buffering video frames, comprising:
acquiring a video frame to be processed uploaded by a first terminal;
judging whether the timestamp of the existing key frame in the existing cached video data in the cache is earlier than the timestamp of the video frame to be processed;
if so, caching the video frame to be processed to obtain current cached video data comprising the latest key frame, wherein the current cached video data is used for providing a caching forwarding service of a video stream for a second terminal;
performing timeout judgment on the latest key frame in the current cache video data;
and if the latest key frame is overtime, clearing the current cache video data in the cache, and initiating a key frame request to the first terminal.
2. The method of claim 1, wherein if the latest key frame is not timed out, the method further comprises:
judging whether the frame number of the current cache video data is greater than a preset frame number threshold value or not;
and if so, clearing the current cache video data in the cache and initiating a key frame request to the first terminal.
3. The buffering method according to claim 1, wherein the time-out determination of the latest key frame in the currently buffered video data comprises:
judging whether a video frame to be processed in the current cache video data is a key frame or not;
if so, the latest key frame in the current cache video data is not overtime, and the video frame with the timestamp earlier than the video frame to be processed in the current cache video data is eliminated.
4. The caching method according to claim 3, wherein if the video frame to be processed in the currently cached video data is not a key frame, the timeout determination for the latest key frame in the currently cached video data is performed according to a timestamp of the latest key frame in the existing cached video data.
5. The caching method according to any one of claims 2 to 4, wherein after initiating the key frame request to the first terminal, further comprising:
and receiving a new key frame returned by the first terminal, and caching the new key frame.
6. A method for forwarding video frames is characterized by comprising the following steps:
acquiring a subscription request initiated by a second terminal;
constructing a reference frame queue comprising the latest key frame according to the frame reference relation of each frame in the current cache video data in the cache;
forwarding the video frame to the second terminal according to the reference frame queue;
wherein the current buffered video data in the buffer is obtained by the method for buffering video frames according to any one of claims 1-5.
7. The forwarding method according to claim 6, wherein before constructing the reference frame queue including the latest key frame according to the frame reference relationship of each frame in the currently buffered video data in the buffer, the method further comprises:
judging whether the first frame of the current cached video data in the cache is a key frame;
if not, initiating a key frame request to the first terminal;
and if so, executing the step of constructing a reference frame queue comprising the latest key frame according to the frame reference relation of each frame in the current cache video data in the cache.
8. The forwarding method according to claim 6, wherein the constructing a reference frame queue including the latest key frame according to the frame reference relationship of each frame in the currently buffered video data in the buffer comprises:
and sequencing all video frames in the current cache video data according to the time stamps, and selecting a key frame and a video frame having a reference relation with the key frame according to the frame reference relation to obtain the reference frame queue.
9. The forwarding method according to claim 6, wherein forwarding video frames to the second terminal according to the reference frame queue comprises:
judging whether the reference frame queue comprises a key frame which is not overtime;
if yes, forwarding the video frame to the second terminal; otherwise, a key frame request is initiated to the first terminal.
10. The forwarding method of claim 9, wherein the determining whether the reference frame queue includes a key frame that has not timed out comprises:
and judging whether the reference frame queue comprises an I frame or an SP frame which is not overtime.
11. The forwarding method according to any one of claims 6 to 10, wherein the forwarding video frames to the second terminal comprises:
acquiring bandwidth information of the second terminal, and determining a code rate for forwarding a video frame according to the bandwidth information;
and forwarding the video frame to the second terminal according to the code rate.
12. The forwarding method according to claim 11, wherein before obtaining the bandwidth information of the second terminal and determining the code rate for forwarding the video frame according to the bandwidth information, the method further comprises:
and forwarding the video frame to the second terminal according to a preset default code rate.
13. A communications server, comprising:
the first transceiving unit is used for acquiring a video frame to be processed uploaded by a first terminal;
the cache unit is used for judging whether the timestamp of the existing key frame in the existing cache video data in the cache is earlier than the timestamp of the video frame to be processed; if so, caching the video frame to be processed to obtain current cached video data comprising the latest key frame, wherein the current cached video data is used for providing a caching forwarding service of a video stream for a second terminal;
the cache unit is further configured to perform timeout determination on a latest key frame in the currently cached video data after the caching of the video frame to be processed is performed; and if the latest key frame is overtime, clearing the current cache video data in the cache, and initiating a key frame request to the first terminal.
14. A communications server, comprising:
the second receiving and sending unit is used for acquiring a subscription request initiated by a second terminal;
the forwarding unit is used for constructing a reference frame queue according to the frame reference relation of each frame in the current cache video data in the cache; forwarding video frames to the second terminal through the second transceiving unit according to the reference frame queue; wherein the current buffered video data in the buffer is obtained by the method for buffering video frames according to any one of claims 1-5.
15. An electronic device, comprising:
at least one processor; and
a memory;
the memory stores computer-executable instructions;
the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the method of any one of claims 1-11.
16. A computer-readable storage medium having stored thereon computer-executable instructions which, when executed by a processor, implement the method of any one of claims 1-11.
CN202110482611.4A 2021-04-30 2021-04-30 Video frame caching method, video frame forwarding method, communication server and program product Active CN113242446B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110482611.4A CN113242446B (en) 2021-04-30 2021-04-30 Video frame caching method, video frame forwarding method, communication server and program product

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110482611.4A CN113242446B (en) 2021-04-30 2021-04-30 Video frame caching method, video frame forwarding method, communication server and program product

Publications (2)

Publication Number Publication Date
CN113242446A CN113242446A (en) 2021-08-10
CN113242446B true CN113242446B (en) 2023-02-28

Family

ID=77131852

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110482611.4A Active CN113242446B (en) 2021-04-30 2021-04-30 Video frame caching method, video frame forwarding method, communication server and program product

Country Status (1)

Country Link
CN (1) CN113242446B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113965714B (en) * 2021-09-10 2023-06-23 北京百度网讯科技有限公司 Video stream processing method and device, electronic equipment and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102857730A (en) * 2012-08-23 2013-01-02 苏州阔地网络科技有限公司 Method and system for caching frame data
CN106162235A (en) * 2016-08-17 2016-11-23 北京百度网讯科技有限公司 Method and apparatus for Switch Video stream
CN109302574A (en) * 2017-07-25 2019-02-01 华为技术有限公司 A kind of method and apparatus handling video flowing
CN110798734A (en) * 2018-08-02 2020-02-14 广州视源电子科技股份有限公司 Video frame caching and forwarding method and device and computer equipment

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8693553B2 (en) * 2007-12-28 2014-04-08 Nokia Corporation Methods, apparatuses, and computer program products for adaptive synchronized decoding of digital video
CN101854533B (en) * 2010-06-10 2012-05-23 华为技术有限公司 Frequency channel switching method, device and system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102857730A (en) * 2012-08-23 2013-01-02 苏州阔地网络科技有限公司 Method and system for caching frame data
CN106162235A (en) * 2016-08-17 2016-11-23 北京百度网讯科技有限公司 Method and apparatus for Switch Video stream
CN109302574A (en) * 2017-07-25 2019-02-01 华为技术有限公司 A kind of method and apparatus handling video flowing
CN110798734A (en) * 2018-08-02 2020-02-14 广州视源电子科技股份有限公司 Video frame caching and forwarding method and device and computer equipment

Also Published As

Publication number Publication date
CN113242446A (en) 2021-08-10

Similar Documents

Publication Publication Date Title
WO2022095795A1 (en) Communication method and apparatus, computer readable medium, and electronic device
CN111147606B (en) Data transmission method, device, terminal and storage medium
CN106998485B (en) Video live broadcasting method and device
CN110290398B (en) Video issuing method and device, storage medium and electronic equipment
CN112312137A (en) Video transmission method and device, electronic equipment and storage medium
CN112954354B (en) Video transcoding method, device, equipment and medium
CN112312229A (en) Video transmission method and device, electronic equipment and storage medium
CN111225209A (en) Video data plug flow method, device, terminal and storage medium
CN111711784B (en) Conference control method and device, readable medium and electronic equipment
US20240121455A1 (en) Method, apparatus, electronic device and storage medium for video bitrate switching
CN113891132A (en) Audio and video synchronization monitoring method and device, electronic equipment and storage medium
CN110545472B (en) Video data processing method and device, electronic equipment and computer readable medium
CN113242446B (en) Video frame caching method, video frame forwarding method, communication server and program product
CN113794942B (en) Method, apparatus, system, device and medium for switching view angle of free view angle video
CN113364767A (en) Streaming media data display method and device, electronic equipment and storage medium
CN113259729B (en) Data switching method, server, system and storage medium
CN113542856A (en) Reverse playing method, device, equipment and computer readable medium for online video
CN113037751A (en) Method and system for creating audio and video receiving stream
CN112929704A (en) Data transmission method, device, electronic equipment and storage medium
CN112153322B (en) Data distribution method, device, equipment and storage medium
CN108200481B (en) RTP-PS stream processing method, device, equipment and storage medium
CN113727183B (en) Live push method, apparatus, device, storage medium and computer program product
CN113965779A (en) Cloud game data transmission method, device and system and electronic equipment
CN113364672B (en) Method, device, equipment and computer readable medium for determining media gateway information
CN112887742B (en) Live stream processing method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant