CN114449282B - Video transmission control method and device, equipment and storage medium - Google Patents

Video transmission control method and device, equipment and storage medium Download PDF

Info

Publication number
CN114449282B
CN114449282B CN202210109588.9A CN202210109588A CN114449282B CN 114449282 B CN114449282 B CN 114449282B CN 202210109588 A CN202210109588 A CN 202210109588A CN 114449282 B CN114449282 B CN 114449282B
Authority
CN
China
Prior art keywords
code rate
video frame
rate value
receiving end
predicted
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210109588.9A
Other languages
Chinese (zh)
Other versions
CN114449282A (en
Inventor
黄巍
查毅勇
韩云博
吴刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202210109588.9A priority Critical patent/CN114449282B/en
Publication of CN114449282A publication Critical patent/CN114449282A/en
Application granted granted Critical
Publication of CN114449282B publication Critical patent/CN114449282B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/154—Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/164—Feedback from the receiver or from the transmission channel
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Abstract

The embodiment of the application discloses a video transmission control method, a video transmission control device, video transmission control equipment and a computer readable storage medium. The method comprises the following steps: acquiring state data of a transmitted video frame in the process of transmitting the video from a transmitting end to a receiving end, wherein the state data are used for describing the transmitting and receiving states of the transmitted video frame; predicting a code rate value adapted to the network state of the receiving end according to the state data; the sending end is controlled to send a video frame responding to the operation instruction of the receiving end according to the code rate value, so that the receiving end obtains a video frame picture responding to the operation instruction in real time; and iteratively executing the prediction of the code rate value and controlling the sending end to send a video frame responding to the operation instruction of the receiving end according to the predicted code rate value through the continuously acquired state data until stopping video transmission. By adopting the technical scheme of the embodiment of the application, the client can not be blocked.

Description

Video transmission control method and device, equipment and storage medium
The application is a divisional application with application number 201911370468.9 and the name of video transmission control method, device, equipment and storage medium, which is filed on 12 months and 26 days in 2019.
Technical Field
The present disclosure relates to the field of video transmission technologies, and in particular, to a video transmission control method and apparatus, a device, and a computer readable storage medium.
Background
With the development of network technology, cloud gaming is widely sought after. In a cloud game running mode, the game runs on a cloud server side, and rendered game pictures are compressed and then transmitted to a client side through a network. The client does not need to have any high-end processor and display card, and can smoothly play games only by having a video decoding function and a certain network bandwidth.
However, since the cloud server needs to transmit each frame of game picture to the client in real time, the running of the cloud game depends on a large network bandwidth and is very sensitive to network delay. In practical applications, the client network is usually a wireless network with network fluctuation characteristics, and if the bandwidth of the network fluctuation decreases, the client may cause a game screen to clip or an operation lag response.
In order to ensure the smoothness of cloud game operation, in the prior art, a client network is monitored in real time, and if the network delay is monitored to be increased, the resolution of the cloud game is immediately reduced, so that the game picture is prevented from being blocked.
However, the above-mentioned technology is always lagged in response to network changes, and when the bandwidth is reduced to a certain extent, the resolution of the cloud game starts to be reduced to remedy after the game picture is jammed, so that the problem that the game picture is jammed still exists in the prior art.
Disclosure of Invention
In order to solve the technical problem that in the prior art, the game picture is blocked, embodiments of the present application provide a video transmission control method, a device, and a computer readable storage medium, which are based on video transmission control performed in the embodiments of the present application, and can effectively prevent the game picture from being blocked due to network fluctuation in the cloud game operation.
The technical scheme adopted by the embodiment of the application is as follows:
a video transmission control method, comprising: acquiring state data of a transmitted video frame in the process of transmitting the video from a transmitting end to a receiving end, wherein the state data are used for describing the transmitting and receiving states of the transmitted video frame; predicting a code rate value adapted to the network state of the receiving end according to the state data; the sending end is controlled to send a video frame responding to the operation instruction of the receiving end according to the code rate value, so that the receiving end obtains a video frame picture responding to the operation instruction in real time; and iteratively executing the prediction of the code rate value and controlling the sending end to send a video frame responding to the operation instruction of the receiving end according to the predicted code rate value through the continuously acquired state data until stopping video transmission.
A video transmission control apparatus comprising: the system comprises a state data acquisition module, a state data transmission module and a state data transmission module, wherein the state data acquisition module is used for acquiring state data of a transmitted video frame in the process of transmitting a video from a transmitting end to a receiving end, and the state data is used for describing the transmitting and receiving states of the transmitted video frame; the code rate value prediction module is used for predicting a code rate value adapted to the network state of the receiving end according to the state data; the video frame transmission control module is used for controlling the transmitting end to transmit a video frame responding to the operation instruction of the receiving end according to the code rate value, so that the receiving end can acquire a video frame picture responding to the operation instruction in real time; and the iteration execution module is used for iteratively executing the prediction of the code rate value and controlling the sending end to send the video frame responding to the operation instruction of the receiving end according to the predicted code rate value through the continuously acquired state data until the video transmission is stopped.
A video transmission control device comprising a processor and a memory having stored thereon computer readable instructions which, when executed by the processor, implement a video transmission control method as described above.
A computer-readable storage medium having stored thereon computer-readable instructions which, when executed by a processor of a computer, cause the computer to perform a video transmission control method as described above.
In the technical scheme, the code rate value prediction adapted to the network state of the receiving end is performed based on the continuously acquired state data of the transmitted video frames in the process of transmitting the video from the transmitting end to the receiving end, and the transmitting end is controlled to transmit the video frames responding to the operation instruction of the receiving end to the receiving end according to the predicted code rate value, so that the code rate values of the video frames transmitted between the transmitting end and the receiving end are adapted to the real-time network state of the receiving end in the whole video transmission process, and therefore the receiving end cannot have the clamping condition of video frame pictures.
In a cloud game operation scene, a cloud server side is used as a sending end of video transmission, a client side is used as a receiving end of video transmission, and the code rate value of a game picture transmitted between the cloud server side and the client side is also suitable for the real-time network state of the client side, so that the client side can not be blocked.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application. It is apparent that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art. In the drawings:
FIG. 1 is a schematic illustration of an implementation environment to which the present application relates;
fig. 2 is a flowchart illustrating a video transmission control method according to an exemplary embodiment;
FIG. 3 is a flow chart of step 120 in one embodiment of the embodiment shown in FIG. 2;
FIG. 4 is a schematic diagram of a code rate prediction model, according to an example embodiment;
FIG. 5 is a flow chart of step 120 in another embodiment of the embodiment shown in FIG. 2;
FIG. 6 is a flow chart of step 210 in one embodiment of the embodiment shown in FIG. 5;
FIG. 7 is a schematic diagram of a cloud gaming operating system, according to an example embodiment;
FIG. 8 is a schematic diagram of a cloud gaming operating system shown according to another exemplary embodiment;
fig. 9 is a block diagram of a video transmission control apparatus according to an exemplary embodiment;
fig. 10 is a schematic structural view of a video transmission control apparatus according to an exemplary embodiment.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application as detailed in the accompanying claims.
The block diagrams depicted in the figures are merely functional entities and do not necessarily correspond to physically separate entities. That is, the functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
The flow diagrams depicted in the figures are exemplary only, and do not necessarily include all of the elements and operations/steps, nor must they be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations.
Reference to "a plurality" in this application means two or more than two. "and/or" describes an association relationship of an association object, meaning that there may be three relationships, e.g., a and/or B may represent: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.
Referring to fig. 1, fig. 1 is a schematic diagram of an implementation environment according to the present invention, which may be a video transmission system, where the video transmission system includes a transmitting end 100 and a receiving end 200, and the transmitting end 100 and the receiving end 200 communicate through a network 300.
The transmitting end 100 is configured to transmit a video frame, for example, a video frame corresponding to a game frame, to the receiving end 200, and after the receiving end 200 receives the video frame, display a corresponding video frame, and the user may trigger an operation instruction in the displayed video frame.
The operation instruction generated by the receiving end 200 is transmitted to the transmitting end 100 in real time, so that the transmitting end 100 generates a video frame in response to the operation instruction according to the operation instruction, and simultaneously transmits the generated video frame to the receiving end 200, so that the receiving end 200 acquires a video frame in response to the operation instruction in real time.
Thus, in the video transmission, the transmitting end 100 will continuously transmit video frames to the receiving end 200, where the video frames include video frames generated by the transmitting end 100 in response to the operation instruction transmitted by the receiving end 200, and the receiving end 200 will also continuously display corresponding video frames according to the received video frames, and acquire the operation instruction triggered in the video frames, so as to transmit the acquired operation instruction to the transmitting end in real time.
In this implementation environment, the transmitting end 100 may be any electronic device that generates and transmits a video frame, for example, may be a device such as a server, the receiving end 200 may be any electronic device that receives a video frame and displays a video frame, for example, may be a device such as a smart phone, a tablet computer, or a computer, and the network 300 may be a wireless network or a wired network, which is not limited in this application.
If the network 300 has a fluctuation condition of bandwidth reduction, the receiving end 200 cannot acquire and display the video frame sent by the sending end 100 in real time, and cannot acquire the video frame picture responding to the operation instruction in real time for the operation instruction triggered in the receiving end 200, so that the situation that the video frame picture display performed in the receiving end 200 is blocked or the operation instruction is delayed and responded is caused.
In order to solve a series of problems caused by network fluctuation, embodiments of the present application provide a video transmission control method, and further provide a video transmission control device, a video transmission control apparatus, and a computer readable storage medium, by collecting historical network data during video transmission, and predicting, in real time, a code rate value adapted to a network state of a receiving end according to the collected historical network data, so as to adjust, in real time, a sending code rate of a video frame according to the predicted code rate value, so that the sending code rate of the video frame is always matched with the network state of the receiving end, and problems such as video frame picture blocking and operation command hysteresis response of the receiving end do not occur.
Referring to fig. 2, fig. 2 is a flowchart illustrating a video transmission control method according to an exemplary embodiment, the method at least includes the following steps:
Step 110, in the process of transmitting video from the transmitting end to the receiving end, status data of the transmitted video frame is obtained, where the status data is used to describe the transmitting and receiving status of the transmitted video frame.
Firstly, in the video transmission process from the transmitting end to the receiving end, the transmitting end packages and encapsulates the video frames into video frames to be transmitted frame by frame, and the receiving end displays the corresponding video frames of the received video frames.
In the process, if the receiving end acquires the operation instruction triggered in the displayed video frame picture, the operation instruction is sent to the sending end in real time, the sending end generates a video frame responding to the operation instruction by responding to the operation instruction of the receiving end, and the generated video frame is transmitted to the receiving end, so that the receiving end obtains the video frame picture responding to the operation instruction in real time.
Under the condition that the network state is stable and the bandwidth is large, unbuffered video acquisition can be realized between the sending end and the receiving end, the video frame generation and sending process of the sending end for responding to the operation instruction of the receiving end and the video frame picture acquisition process of the receiving end for responding to the operation instruction of the receiving end are real-time, and the receiving end can smoothly display the video frame picture.
Under the condition that the network state of the receiving end has larger fluctuation, for example, the network state has larger delay, the receiving end can have the problems of video frame picture blocking, operation instruction delayed response and the like.
In order to avoid these problems, in this embodiment, in the process of transmitting video from a transmitting end to a receiving end, status data of a transmitted video frame needs to be acquired.
The state data of the transmitted video frame is related data for describing the transmission and reception states of the transmitted video frame, and these data can reflect the historical network state of the receiving end. For example, the status data of the transmitted video frame may be at least one of data size and code rate value of the video frame transmitted by the transmitting end, time interval between receiving adjacent video frames by the receiving end, difference between download time and display time of the video frame in the receiving end, play frame rate of the receiving end, and the like, which are not limited herein.
Step 120, predicting the code rate value adapted to the network state of the receiving end according to the state data.
The code rate value adapted to the network state of the receiving end means that in the network state, the transmitting end transmits the video frame to the receiving end according to the code rate value, so that the receiving end does not have the problem of blocking and the like in the process of receiving the video frame and displaying the corresponding video frame picture.
As described above, the state data of the transmitted video frame can reflect the historical network state of the receiving end, so that the network state of the receiving end in a future period of time can be effectively predicted according to the historical network state of the receiving end, and the code rate value suitable for the network state can be determined according to the predicted network state.
Note that, the prediction of the code rate value may be specifically performed by the transmitting end or may be specifically performed by the receiving end, which is not limited in this embodiment.
And 130, the control transmitting end transmits the video frame responding to the operation instruction of the receiving end according to the predicted code rate value, so that the receiving end obtains the video frame picture responding to the operation instruction in real time.
As described above, the video frame in response to the operation instruction of the receiving end may be generated after the sending end obtains the operation instruction sent by the receiving end, where the video frame generated by the sending end needs to be transmitted to the receiving end in real time, so as to ensure that the receiving end does not have the problems of delayed response of the operation instruction.
The control transmitting end transmitting the video frame responding to the operation instruction of the receiving end according to the predicted code rate value means that the control transmitting end recodes the generated video frame according to the predicted code rate value after generating the video frame responding to the operation instruction of the receiving end, and then transmits the recoded video frame to the receiving end.
Because the code rate value predicted in step 120 is adapted to the network state of the receiving end, by controlling the transmitting end to transmit the video frame responding to the operation instruction of the receiving end to the receiving end according to the code rate value obtained by prediction, the receiving end can obtain the video frame picture responding to the operation instruction in real time.
It should be noted that, if the prediction process of the code rate value is performed by the transmitting end, the transmitting end encodes the video frame according to the code rate value predicted by itself after generating the video frame in response to the operation instruction of the receiving end.
If the code rate value prediction is executed by the receiving end, the sending end needs to be controlled to acquire a code rate request initiated by the receiving end, and the code rate request carries the code rate value which is predicted by the receiving end and is adapted to the network state of the receiving end, so that the sending end encodes the video frame according to the code rate value in the code rate request after generating the video frame responding to the operation instruction of the receiving end.
And 140, iteratively executing the prediction of the code rate value and controlling the transmitting end to transmit the video frame responding to the operation instruction of the receiving end according to the predicted code rate value through the continuously acquired state data until the video transmission is stopped.
In the process of video transmission from the transmitting end to the receiving end, the state data of the transmitted video frames are continuously acquired, so that the change condition of the network state of the receiving end is dynamically acquired.
By iteratively executing code rate value prediction adapted to the network state of the receiving end, the network fluctuation condition is known in advance before the network state of the receiving end fluctuates, and the self-adaptive adjustment of the video frame sending code rate is carried out according to the network fluctuation condition, so that the process of the receiving end for obtaining the video frame responding to the operation instruction is smooth even if the network state of the receiving end fluctuates greatly in the whole video transmission process, and the problems of video frame picture blocking, operation instruction hysteresis response and the like do not occur.
In addition, the adaptive adjustment of the video frame transmission rate according to the embodiment does not simply reduce the resolution of the video frame to be transmitted in the transmitting end, for example, when the delay of the receiving end network is predicted, the resolution of the video frame to be transmitted is reduced from 1080P to 720P, when the recovery of the receiving end network is predicted, the resolution is restored, but the optimal code rate value which can be adapted to the current network condition of the receiving end is predicted by the state data of the transmitted video frame, instead of only changing within a few fixed code rate ranges.
Therefore, in the whole video transmission process, the transmission code rate of the video frames is continuously adjustable in real time, the video frame transmission performed by the transmitting end can be accurately adapted to the change of the network state of the receiving end, and the receiving end can be ensured to obtain high-quality video pictures to the greatest extent.
For the video picture acquisition by the user through the receiving end, the code rate value adjustment by the transmitting end can enable the visual transition of the user to be smoother, the situation that the picture in the visual field is suddenly and obviously reduced is avoided, and therefore the user experience is good.
FIG. 3 is a flow chart of step 120 in one embodiment of the embodiment shown in FIG. 2. As shown in fig. 3, in an exemplary embodiment, predicting a code rate value adapted to a network state of a receiving end according to state data may include the steps of:
step 121, according to the obtained state data, predicting the network state of the receiving end receiving the video frame in the next prediction time period;
step 122, determining a code rate value adapted to the network state corresponding to the next predicted time period, where the code rate value is used to control the transmitting end to transmit the video frame in the next predicted time period.
Firstly, it should be noted that, in this embodiment, the code rate value adapted to the network state of the receiving end is dynamically predicted according to a preset prediction time period, where the time lengths corresponding to the preset prediction time period may be the same or different, and the present application is not limited.
And predicting a code rate value which is matched with the network state corresponding to the next prediction time period of the receiving end according to the acquired state data of the transmitted video frame in the current prediction time period to be subjected to code rate value prediction, and controlling the transmitting end to transmit the video frame according to the code rate value in the next prediction time period by the corresponding code rate value obtained through prediction.
It should be noted that, the video frame transmission performed by the transmitting end in the current prediction period is also performed based on the code rate value predicted in the previous prediction period.
As described above, the state data of the transmitted video frame can reflect the historical network state change condition of the receiving end, so that in the current time period, according to the obtained state data of the transmitted video frame, the receiving end can be predicted to receive the network state of the video frame in the next prediction time period, thereby determining the adaptive code rate value according to the predicted code rate value.
In one embodiment, the code rate value prediction adapted to the network state of the receiving end can be performed according to a pre-constructed code rate prediction model. And continuously inputting the continuously acquired state data into the code rate prediction model, so that the code rate prediction model continuously predicts the code rate value corresponding to the next prediction period according to the preset prediction period.
An exemplary schematic structure of a code rate prediction model may be shown in fig. 4, where the model is a reinforced network learning model, and includes a plurality of neural networks, based on the neural networks, the code rate prediction model predicts a network state that a receiving end receives a video frame in a next prediction time period according to input state data, and correspondingly outputs a code rate value adapted to the network state corresponding to the next prediction time period.
The neural network included in the code rate prediction model may be a neural network such as RNN (Recurrent Neural Network, cyclic neural network), CNN (Convolutiona lNeura lNetworks, convolutional neural network), or the like, and the CNN may specifically be a network structure of 1 layer or 2 layers, which is not limited in this place.
As an exemplary embodiment, as also shown in fig. 4, the rate prediction model contains a decision network, which can be understood to be a neural network. After other neural networks perform preprocessing such as feature extraction and the like on input state data, a decision network performs self-adaptive code rate decision according to the corresponding features of the state data, and a corresponding code rate decision result is output.
The code rate decision result may include selection probabilities corresponding to different code rate values, and the code rate prediction model may determine a code rate value corresponding to a maximum selection probability output by the decision network as a code rate value adapted to a network state corresponding to a next prediction time period by the receiving end. Therefore, based on the method provided by the embodiment, the prediction of the code rate value corresponding to each prediction time period is dynamically performed according to the continuously acquired state data and the preset prediction time period, and the video frame sending code rate in each prediction time period is dynamically adjusted, so that the video transmission between the sending end and the receiving end is orderly and adaptively performed, and the problem caused by the network state fluctuation of the receiving end is effectively avoided.
Fig. 5 is a flow chart of step 120 in another embodiment of the embodiment shown in fig. 2. As shown in fig. 5, in an exemplary embodiment, before step 121, predicting, according to the state data, a code rate value adapted to the state of the network of the receiving end may further include the following steps:
step 210, calculating a reward value corresponding to the code rate value predicted in the previous prediction time period according to the state data, wherein the reward value is used for representing the effect of predicting the code rate value in the previous prediction time period.
Firstly, it should be noted that, the prize value is used to represent the predicting effect of the code rate value adapted to the network state of the receiving end, the higher the prize value, the better the predicting effect of the code rate value, and the higher the picture quality of the video frame picture displayed by the receiving end.
For the current prediction time period, the transmitting end in the current prediction time period transmits the video frame according to the code rate value predicted in the previous prediction time period, so that the effect of predicting the code rate value in the previous prediction time period can be reflected by the state data of the transmitted video frame in the current prediction time period.
And, the state data of the transmitted video frames acquired in the current prediction period includes not only the state data of the video frames transmitted in the historical prediction period, but also the state data of the video frames transmitted according to the corresponding code rate values in the current prediction period.
Therefore, according to the acquired state data, the rewarding value corresponding to the code rate value predicted in the previous predicted time period can be calculated, and the effect of predicting the code rate value in the previous predicted time period can be determined according to the obtained rewarding value.
And 220, adjusting a prediction parameter for predicting the code rate value according to the reward value, wherein the adjusted prediction parameter is used for predicting the code rate value which is suitable for the network state of the receiving end in the next prediction time period.
The prediction parameters refer to relevant parameters involved in the process of performing code rate value prediction adapted to the network state of the receiving end according to the state data, for example, for a code rate value prediction model, the prediction parameters may be relevant weight parameters in a neural network contained in the model.
The code rate value of video frame transmission in the current prediction time period is predicted according to the corresponding prediction parameters and state parameters in the previous prediction time period, so that on the basis of determining the prediction effect corresponding to the previous prediction time period according to the reward value, the prediction parameters for code rate value prediction in the previous prediction time period are adjusted according to the reward value, and when the code rate value is predicted next time, a better code rate value prediction effect can be obtained through the prediction parameters obtained by adjustment.
As also shown in fig. 4, in an exemplary embodiment, the code rate value prediction model may include a judgment network for performing the descriptions of steps 210 and 220. The evaluation network is also understood to be a neural network, among other things.
And after the code rate value prediction model performs primary code rate value prediction, and applies the predicted code rate value to the video frame of the next prediction time period for transmission, acquiring a corresponding rewarding value through a judging network, and feeding back the rewarding value to the judging network, so that the judging network indicates the code rate value prediction model to adjust the code rate value prediction weight in the model according to the rewarding value, for example, indicates to adjust the weight parameter in a decision network.
Therefore, according to the rewarding value of each execution code rate value prediction, the embodiment dynamically adjusts the relevant prediction parameters in the next code rate value prediction, so that the code rate value prediction effect carried out each time is further optimized, the accuracy of the code rate value prediction is continuously improved, and the stability of the video picture quality in the receiving end is further ensured.
FIG. 6 is a flow chart of step 210 in one embodiment of the embodiment shown in FIG. 5. As shown in fig. 6, calculating the prize value corresponding to the code rate value predicted in the previous predicted time period according to the status data may include the following steps:
Step 211, according to the state data, obtaining the blocking time of the video frame transmitted in the prediction time period, and obtaining the code rate value of the video frame transmitted by the transmitting end in the previous prediction time period.
The blocking time of the video frame is the time required for the receiving end to initiate an operation instruction to the sending end until the video frame responding to the operation instruction is received after the receiving end acquires the operation instruction triggered in the video frame picture.
Therefore, according to the state data of the video frame sent in the time period, the first time of sending a certain operation instruction in the receiving period can be determined, the second time of responding to the operation instruction and obtained by the receiving terminal is determined, and the click time is obtained by calculating the time difference between the second time and the first time.
And according to the state data of the video frame sent in the previous prediction time period, acquiring the code rate value of the video frame sent by the sending end in the previous prediction time period.
And step 212, calculating the difference between the code rate value of the video frame transmitted by the transmitting end in the previous prediction time period and the code rate value predicted in the previous prediction time period to obtain the fluctuation code rate.
The code rate value predicted in the previous prediction time period is the code rate value of the video frame sent by the sending end in the current prediction time period, and can be correspondingly obtained through the state data of the video frame sent in the current prediction time period.
From this, the fluctuating code rate is substantially the code rate difference of the video frame transmitted by the transmitting end in the previous predicted time period and the current predicted time period, and reflects the condition that the transmitting end performs code rate switching in the current predicted time period.
And step 213, calculating a reward value corresponding to the code rate value predicted in the previous prediction time period according to the code rate value, the cartoon time and the fluctuation code rate predicted in the previous prediction time period.
As mentioned above, the code rate value predicted in the previous prediction time period is the code rate value of the video frame transmitted in the current prediction time.
In this embodiment, the magnitude of the prize value is mainly affected by the sending rate of the video frame in the current prediction time, the blocking time of the video frame transmitted in the current prediction time period, and the rate switching in the current prediction time period.
In one embodiment, the prize value is calculated from a prize function, which is as follows:
QoE＝q(R n )-μT n -γ|q(R n+1 )-q(R n )|
wherein "QoE" represents a prize value, "n" represents a sequence number of each predicted time period, "q ()" represents a preset nonlinear function for performing nonlinear operation on a code rate value, "R", and n "code rate value representing video frame transmitted in nth predicted period," T n "indicates the click time in the nth predicted period," |q (R n+1 )-q(R n ) The | "indicates the fluctuation of the code rate in the nth predicted period," μ "indicates the weight of the impact of the katen time on the prize value, and" γ "indicates the weight of the fluctuation of the code rate on the prize value.
In this embodiment, the reward function is designed according to the quality of experience standard of the user, when the network delay of the receiving end is low, the influence of the picture quality of the receiving end on the user experience is dominant, and the influence degree of the picture quality of the receiving end tends to be saturated after the picture quality reaches a certain degree; when the network delay of the receiving end is higher, obvious blocking or operation hysteresis occurs at the receiving end, and the network delay of the receiving end is taken as a main factor affecting the user experience.
It is worth mentioning that smoothness factors, namely the influence of code rate fluctuation on the user experience instruction, are added in the user experience quality standard reflected by the reward function, so that the code rate switching performed by the receiving end is guaranteed not to influence the user experience.
In one embodiment, during the current prediction period, before the step of performing the prize value calculation, the weight of the time in the prize value calculation is adjusted, i.e. the weight μ in the prize value calculation formula is adjusted, according to the time of the corresponding time of the current prediction period.
If the stuck time corresponding to the current predicted time period is smaller than the preset time threshold, the weight of the stuck time in the calculation of the prize value in the last predicted time period is reduced by a set value, otherwise, the weight is increased by the set value.
Illustratively, the weight adjustment rule is as follows:
Figure BDA0003493809230000111
that is, before each calculation of the prize value, the prize function is updated according to the corresponding catton time of the current prediction time period, and the prize value calculated according to the updated prize function is used for adjusting the prediction parameter in the next prediction of the code rate value, so as to dynamically perform adaptive adjustment of the code rate value, further improve the adaptive degree of the code rate value of the video frame sent in each prediction time period and the network state of the receiving end in each prediction time period in the whole video transmission process, and ensure that the receiving end can obtain a high-quality video picture.
In addition, according to the above-mentioned dynamic adaptive adjustment process of code rate value, the present embodiment adjusts code rate value with the goal of obtaining high-quality video pictures by the receiving end, and fully considers video viewing experience of the user based on the receiving end in the actual application scene. For example, when the network delay of the receiving end is high, obvious catton occurs in the receiving end or the response speed for the user operation is slow, and by the method provided by the embodiment, the problems can be avoided, so that the user experience is not affected.
The video transmission method of the present application will be described below in an exemplary application scenario.
The exemplary application scenario is specifically a cloud game running scenario, in which a cloud game runs in a cloud server, the cloud server is used as a transmitting end to transmit video frames corresponding to a rendered game video to a client through a code stream transmission channel, and the client decodes the received video frames to obtain corresponding game pictures.
In the cloud game running process, a user performs game operation in a game picture displayed by a client, so that the client generates a user operation instruction, the user operation instruction is uploaded to a cloud server in real time, the cloud server renders a game video responding to the user operation instruction according to the user operation instruction, and then the rendered game video is sent to the client in a video frame mode.
Therefore, the client does not need to have any high-end processor and display card, and can smoothly play games only by having a video decoding function and a certain network bandwidth.
At present, cloud games at a high-definition PC (Persona lComputer ) end require about 30 megabandwidth to ensure normal game experience of a user, however, in an actual game environment, a client network is often a wireless network, exhibits fluctuation characteristics, is easy to occur under the condition of bandwidth reduction, causes problems of game clamping or user operation hysteresis response and the like of the client, and quite damages the user experience.
Based on the video transmission control method provided by the application, the related problems caused by client network fluctuation in cloud game operation can be well solved.
Specifically, during the cloud game running process, status data of the sent video frames, such as the data size and the code rate value of the video frames sent by the sending end, the time interval between the receiving end and the adjacent video frames, the difference between the downloading time and the display time of the video frames in the receiving end, the play frame rate of the receiving end, and the like, are continuously obtained, and are used for predicting the code rate value adapted to the network status of the client.
According to the continuously acquired state data, code rate value prediction adaptive to the network state of the client is iterated, and the transmitting end is controlled to transmit corresponding video frames according to the predicted code rate value, so that the client can acquire video frame pictures responding to operation instructions in real time, and cloud game operation in the client can be smoothly performed.
Therefore, state data capable of reflecting the historical network state of the client is collected in the cloud game running process, the impending change of the client network is predicted in real time according to the collected state data, and meanwhile, the code rate value suitable for the state of the client network is obtained by timely responding to the predicted network change.
The cloud server is controlled to send the subsequent video frames according to the predicted code rate value, so that the code rate value of the video frame sent by the cloud server is always matched with the network state of the client, and therefore the problems of game picture blocking, delayed response of a user operation instruction and the like of the client are avoided, and the user experience is not affected.
For example, during cloud game operation, the data interaction process between the cloud server and the client may be as shown in fig. 7 and 8. Wherein fig. 7 corresponds to a video transmission control method being specifically performed by the cloud server, and fig. 8 corresponds to a video transmission control method being specifically performed by the client.
It should be further noted that, in order to facilitate understanding of the data interaction process between the cloud server and the client, the video transmission control method provided in the present application may be loaded into the rate adaptation module shown in fig. 7 and 8. In the running process of the cloud game, the code rate self-adaptive module executes the video transmission control method described in the foregoing embodiment.
As shown in fig. 7, the code rate adaptive module is configured in a cloud server, a game program loaded by the cloud server generates a cloud game rendering video according to a user operation instruction, the cloud game rendering video is input to an encoder, the code rate adaptive module predicts a code rate value adapted to a client network state according to collected state data, the predicted code rate value is sent to the encoder, the encoder encodes the game rendering video according to the received code rate value to obtain video frames, and then the video frames are sent to the client frame by frame. The client decodes the received video frame to obtain a game picture, a user triggers a user operation instruction in the game picture, the user operation instruction is packaged together with the state information of the client, and the user operation instruction is fed back to the cloud server through an uplink dedicated channel. And the feedback instruction processing module of the cloud server analyzes and splits the feedback information, sends an operation instruction into a game program, and inputs the state information of the client to the code rate self-adaptive module.
As shown in fig. 8, the rate adaptation module is configured in a cloud server, the cloud server generates cloud game rendering videos according to user operation instructions by the loaded game program, and inputs the generated cloud game rendering videos to an encoder, and the encoder encodes the game rendering videos according to a rate request of a client and transmits video frames to the client frame by frame. The client decodes the received video frames to obtain game pictures, a user triggers a user operation instruction in the game pictures, the code rate self-adaptive module predicts a code rate value requested to the cloud server according to the collected state data, the code rate value is adapted to the network state of the client, the user operation instruction and code rate request information are packaged together, and the code rate self-adaptive module feeds back the code rate value to the cloud server through an uplink dedicated channel. And the feedback instruction processing module of the cloud server analyzes and splits the feedback information, sends an operation instruction into a game program, and inputs code rate request information into the encoder.
Therefore, the cloud server continuously generates game videos according to the user operation instructions sent by the client, and the code rate self-adaptive module also continuously predicts code rate values, so that the game videos transmitted to the client by the cloud server are always adapted to the network state of the client, and therefore the problems of game picture blocking, user operation instruction delayed response and the like do not occur in the client.
Compared with the prior art, the method and the device respond after monitoring that the client network fluctuates, the method and the device respond timely before the client network changes, so that the click-through rate of video pictures in the client can be reduced by 50%.
In addition, in the code rate value prediction process, the prediction accuracy of the code rate value can be continuously improved along with the video transmission by dynamically adjusting relevant parameters in the code rate value prediction process, so that the prediction effect is improved to a great extent.
Therefore, based on accurate estimation and pre-estimation of the code rate value, the method and the device can avoid reserving larger bandwidth for ensuring that the client does not get stuck. As mentioned above, the current cloud game system generally needs a bandwidth of more than 30 megabytes to resist the possible larger network fluctuation, and by adopting the technical scheme of the application, the stable running of the cloud game can be ensured only by using a bandwidth of 15 megabytes, so that the user game experience is improved, and meanwhile, the bandwidth overhead of the cloud server is saved.
Fig. 9 is a block diagram illustrating a video transmission control apparatus according to an exemplary embodiment.
As shown in fig. 9, in an exemplary embodiment, the video transmission control apparatus includes a status data acquisition module 310, a code rate value prediction module 320, a video frame transmission control module 330, and an iteration execution module 340.
The status data obtaining module 310 is configured to obtain status data of a transmitted video frame during a process of transmitting video from a transmitting end to a receiving end, where the status data is used to describe a transmitting and receiving status of the transmitted video frame.
The code rate value prediction module 320 is configured to predict a code rate value adapted to a network state of the receiving end according to the state data.
The video frame transmission control module 330 is configured to control the transmitting end to transmit a video frame in response to the operation instruction of the receiving end according to the code rate value, so that the receiving end obtains the video frame in response to the operation instruction in real time.
The iteration execution module 340 is configured to iteratively execute prediction of the code rate value and control the transmitting end to transmit a video frame in response to the operation instruction of the receiving end according to the predicted code rate value, through the continuously acquired state data, until stopping video transmission.
In another exemplary embodiment, the code rate value prediction module 320 includes a network prediction unit and a code rate value determination unit.
The network prediction unit is used for predicting the network state of the receiving end receiving the video frame in the next prediction time period according to the acquired state data.
The code rate value determining unit is used for determining a code rate value which is matched with the network state corresponding to the next prediction time period, and the code rate value is used for controlling the transmitting end to transmit the video frame in the next prediction time period.
In another exemplary embodiment, the code rate value prediction module 320 further includes a prize value calculating unit and a prediction parameter adjusting unit.
The rewarding value calculating unit is used for calculating rewarding values corresponding to code rate values predicted in the previous prediction time period according to the state data, and the rewarding values are used for representing the effect of code rate value prediction in the previous prediction time period.
The prediction parameter adjusting unit is used for adjusting a prediction parameter for predicting the code rate value according to the reward value, and the adjusted prediction parameter is used for predicting the code rate value which is adapted to the network state of the receiving end in the next prediction time period.
In another exemplary embodiment, the prize value calculating unit includes an information obtaining subunit, a fluctuating code rate obtaining subunit, and an information calculating subunit.
The information acquisition subunit is used for acquiring the cartoon time of the video frame transmitted in the prediction time period according to the state data, and acquiring the code rate value of the video frame transmitted by the transmitting end in the previous prediction time period.
The fluctuation code rate obtaining subunit is used for calculating the difference between the code rate value of the video frame sent by the sending end in the previous prediction period and the code rate value predicted in the previous prediction period to obtain the fluctuation code rate.
The information calculating subunit is used for calculating the rewarding value according to the code rate value, the cartoon time and the fluctuation code rate predicted in the last prediction time period.
In another exemplary embodiment, the information acquisition subunit includes a time information acquisition subunit and a katen time acquisition subunit.
The time information acquisition subunit is used for determining a first time when the receiving end transmits the operation instruction according to the state data of the video frame transmitted in the prediction time period, and determining a second time when the receiving end acquires the video frame responding to the operation instruction.
The clamping time acquisition subunit is used for calculating the time difference between the second time and the first time to obtain the clamping time.
In another exemplary embodiment, the prize value calculating unit further includes a weight adjustment subunit, configured to adjust a weight of the click time in the prize value calculation according to the click time corresponding to the predicted time period, so as to calculate the prize value corresponding to the previous predicted time period according to the adjusted weight.
In another exemplary embodiment, the weight adjustment subunit includes a weight acquisition subunit and a weight determination subunit.
The weight acquisition subunit is used for acquiring the weight in the calculation of the reward value in the last prediction time period.
The weight judging subunit is used for reducing the weight by a set value under the condition that the katon time corresponding to the predicted time period is smaller than a preset time threshold, otherwise, increasing the weight by the set value.
In another exemplary embodiment, the video frame transmission control module 330 includes a video frame generation unit and a video frame encoding unit.
The video frame generating unit is used for controlling the transmitting end to generate a video frame responding to the operation instruction according to the received operation instruction of the receiving end.
The video frame coding unit is used for recoding the generated video frame according to the predicted code rate value and transmitting the recoded video frame to the receiving end.
In another exemplary embodiment, the video frame encoding unit is configured to control the transmitting end to recode the generated video frame according to the self-predicted code rate value, and transmit the recoded video frame to the receiving end.
In another exemplary embodiment, the video frame transmission control module 330 further includes a code rate request obtaining unit, configured to control the sending end to obtain a code rate request initiated by the receiving end, where the code rate request carries a code rate value adapted to a network state of the receiving end.
In another exemplary embodiment, the status data acquired by the status data acquisition module 310 includes at least one of a data size and a code rate value of a video frame transmitted by the transmitting end, a time interval between receiving adjacent video frames by the receiving end, a difference between a download time and a display time of the video frame in the receiving end, and a play frame rate of the receiving end.
It should be noted that, the apparatus provided in the foregoing embodiments and the method provided in the foregoing embodiments belong to the same concept, and the specific manner in which each module and unit perform the operation has been described in detail in the method embodiments, which is not repeated herein.
Another aspect of the present application also provides a video transmission control apparatus, including a processor and a memory, wherein the memory has stored thereon computer readable instructions that, when executed by the processor, implement a video transmission control method as described above.
Fig. 10 is a schematic structural view of a video transmission control apparatus according to an exemplary embodiment.
It should be noted that the video transmission control apparatus is just one example adapted to the present invention, and is not to be construed as providing any limitation to the scope of use of the present invention. Nor should the video transmission control device be construed as necessarily relying on or necessarily having one or more of the components of the exemplary video transmission control device shown in fig. 10.
As shown in fig. 10, in an exemplary embodiment, the video transmission control apparatus includes a processing component 401, a memory 402, a power supply component 403, a multimedia component 404, an audio component 405, a sensor component 407, and a communication component 408. The above components are not required, and the video transmission control device may add other components or reduce some components according to its own functional requirements, which is not limited in this embodiment.
The processing component 401 generally controls overall operation of the video transmission control device, such as operations associated with display, data communication, and log data processing, among others. The processing component 401 may include one or more processors 409 to execute instructions to perform all or part of the steps of the operations described above. Further, the processing component 401 may include one or more modules to facilitate interactions between the processing component 401 and other components. For example, the processing component 401 may include a multimedia module to facilitate interaction between the multimedia component 404 and the processing component 401.
The memory 402 is configured to store various types of data to support operation at the video transmission control device, examples of which include instructions for any application or method operating on the video transmission control device. The memory 402 has stored therein one or more modules configured to be executed by the one or more processors 409 to perform all or part of the steps of the video transmission control method described in the above embodiments.
The power supply component 403 supplies power to various components of the video transmission control apparatus. The power supply components 403 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for video transmission control devices.
The multimedia component 404 includes a screen between the video transmission control device and the user that provides an output interface. In some embodiments, the screen may include TP (touch panel) and LCD (Liquid Crystal Display ). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation.
The audio component 405 is configured to output and/or input an audio signal. For example, the audio component 405 includes a microphone configured to receive external audio signals when the video transmission control device is in an operational mode, such as a call mode, a recording mode, and a speech recognition mode. In some embodiments, the audio component 405 also includes a speaker for outputting audio signals.
The sensor assembly 407 includes one or more sensors for providing status assessment of various aspects for the video transmission control device. For example, the sensor assembly 407 may detect an on/off state of the video transmission control apparatus, and may also detect a temperature change of the video transmission control apparatus.
The communication component 408 is configured to facilitate communication between the video transmission control device and other devices in a wired or wireless manner. The video transmission control device may access a Wireless network based on a communication standard, such as Wi-Fi (Wireless-Fidelity).
It will be appreciated that the configuration shown in fig. 10 is merely illustrative, and that the video transmission control apparatus may include more or less components than those shown in fig. 10, or have different components than those shown in fig. 10. Each of the components shown in fig. 10 may be implemented in hardware, software, or a combination thereof.
Another aspect of the present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a video transmission control method as described above.
The computer-readable storage medium may be contained in the video transmission control apparatus described in the above embodiment or may exist alone without being incorporated in the video transmission control apparatus.
The foregoing is merely a preferred exemplary embodiment of the present application and is not intended to limit the embodiments of the present application, and those skilled in the art may make various changes and modifications according to the main concept and spirit of the present application, so that the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (12)

1. A video transmission control method, comprising:
acquiring state data of a transmitted video frame in the process of transmitting the video from a transmitting end to a receiving end, wherein the state data are used for describing the transmitting and receiving states of the transmitted video frame;
predicting a code rate value adapted to the network state of the receiving end according to the state data;
the sending end is controlled to send a video frame responding to the operation instruction of the receiving end according to the code rate value, so that the receiving end obtains a video frame picture responding to the operation instruction in real time;
iteratively executing the prediction of the code rate value and controlling the sending end to send a video frame responding to the operation instruction of the receiving end according to the predicted code rate value through the continuously acquired state data until stopping video transmission;
after primary code rate value prediction is carried out, and the predicted code rate value is applied to video frames of the next prediction time period to be sent, a reward value corresponding to the code rate value predicted in the previous prediction time period is calculated according to the acquired state data, and a prediction parameter for carrying out code rate value prediction is adjusted according to the reward value, wherein the adjusted prediction parameter is used for predicting the code rate value adapted to the network state of the receiving end in the next prediction time period;
The prize value is calculated according to the following formula:
QoE＝q(R n )-μT n -γ|q(R n+1 )-q(R n )|
wherein "QoE" represents a prize value, "n" represents a sequence number of each predicted time period, "q ()" represents a preset nonlinear function for performing nonlinear operation on a code rate value, "R", and n "code rate value representing video frame transmitted in nth predicted period," T n "indicates the click time in the nth predicted period," |q (R n+1 )-q(R n ) The "μ" represents the code rate fluctuation in the nth predicted period, and the "μ" represents the cardThe pause time affects the weight of the prize value, and "gamma" indicates the weight of the fluctuating code rate affecting the prize value.
2. The method of claim 1, wherein calculating the prize value corresponding to the code rate value predicted for the last predicted time period based on the acquired status data comprises:
according to the state data, obtaining the blocking time of the video frame transmitted in the prediction time period, and obtaining the code rate value of the video frame transmitted by the transmitting end in the previous prediction time period;
calculating the difference between the code rate value of the video frame sent by the sending end in the previous prediction time period and the code rate value predicted in the previous prediction time period to obtain a fluctuation code rate;
and calculating the rewarding value according to the code rate value predicted in the last predicted time period, the click time and the fluctuation code rate.
3. The method according to claim 2, wherein the obtaining, based on the status data, a click time of the video frame transmitted in the predicted period of time includes:
determining a first time when the receiving end transmits an operation instruction and a second time when the receiving end acquires the video frame responding to the operation instruction according to the state data of the video frame transmitted in the prediction time period;
and calculating the time difference between the second time and the first time to obtain the clamping time.
4. The method of claim 2, wherein prior to calculating the prize value based on the predicted code rate value for the last predicted period of time, the click-through time, and the fluctuating code rate, the method further comprises:
and adjusting the weight of the stuck time in the prize value calculation according to the stuck time corresponding to the predicted time period, so as to calculate the prize value corresponding to the previous predicted time period according to the adjusted weight.
5. The method of claim 4, wherein adjusting the weight of the click time in the prize value calculation based on the click time corresponding to the predicted time period comprises:
Acquiring weight in the calculation of the reward value of the previous prediction time period;
if the corresponding katon time of the predicted time period is smaller than a preset time threshold, the weight is reduced by a set value;
otherwise, the weight is increased by the set value.
6. The method according to claim 1, wherein said controlling the transmitting end to transmit the video frame in response to the receiving end operation instruction according to the code rate value comprises:
the sending end is controlled to generate a video frame responding to the operation instruction according to the received operation instruction of the receiving end;
and recoding the generated video frame according to the predicted code rate value, and transmitting the recoded video frame to the receiving end.
7. The method of claim 6, wherein the predicting of the code rate value is performed by the sender; the recoding of the generated video frame according to the predicted code rate value and the transmission of the recoded video frame to the receiving end comprise the following steps:
and controlling the sending end to recode the generated video frame according to the code rate value predicted by the sending end, and sending the recoded video frame to the receiving end.
8. The method of claim 6, wherein the predicting of the code rate value is performed by the receiving end; before recoding the generated video frame according to the predicted code rate value and transmitting the video frame obtained by recoding to the receiving end, the method further comprises:
and controlling the sending end to acquire a code rate request initiated by the receiving end, wherein the code rate request carries a code rate value adapted to the network state of the receiving end.
9. The method of claim 1, wherein the status data comprises at least one of a data size and a code rate value of a video frame transmitted by the transmitting end, a time interval between receiving adjacent video frames by the receiving end, a difference between a download time and a display time of a video frame in the receiving end, and a play frame rate of the receiving end.
10. A video transmission control apparatus, comprising:
the system comprises a state data acquisition module, a state data transmission module and a state data transmission module, wherein the state data acquisition module is used for acquiring state data of a transmitted video frame in the process of transmitting a video from a transmitting end to a receiving end, and the state data is used for describing the transmitting and receiving states of the transmitted video frame;
the code rate value prediction module is used for predicting a code rate value adapted to the network state of the receiving end according to the state data;
The video frame transmission control module is used for controlling the transmitting end to transmit a video frame responding to the operation instruction of the receiving end according to the code rate value, so that the receiving end obtains a video frame picture responding to the operation instruction in real time;
the iteration execution module is used for iteratively executing the prediction of the code rate value and controlling the sending end to send a video frame responding to the operation instruction of the receiving end according to the predicted code rate value through the continuously acquired state data until the video transmission is stopped;
after primary code rate value prediction is carried out, and the predicted code rate value is applied to video frames of the next prediction time period to be sent, a reward value corresponding to the code rate value predicted in the previous prediction time period is calculated according to the acquired state data, and a prediction parameter for carrying out code rate value prediction is adjusted according to the reward value, wherein the adjusted prediction parameter is used for predicting the code rate value adapted to the network state of the receiving end in the next prediction time period;
the prize value is calculated according to the following formula:
QoE＝q(R n )-μT n -γ|q(R n+1 )-q(R n )|
wherein "QoE" represents a prize value, "n" represents a sequence number of each predicted time period, "q ()" represents a preset nonlinear function for performing nonlinear operation on a code rate value, "R", and n "code rate value representing video frame transmitted in nth predicted period," T n "indicates the click time in the nth predicted period," |q (R n+1 )-q(R n ) The | "indicates the fluctuation of the code rate in the nth predicted period," μ "indicates the weight of the impact of the katen time on the prize value, and" γ "indicates the weight of the fluctuation of the code rate on the prize value.
11. A video transmission control apparatus, characterized by comprising:
a memory storing computer readable instructions;
a processor reading computer readable instructions stored in a memory to perform the method of any one of claims 1-9.
12. A computer readable storage medium having stored thereon computer readable instructions which, when executed by a processor of a computer, cause the computer to perform the method of any of claims 1-9.
CN202210109588.9A 2019-12-26 2019-12-26 Video transmission control method and device, equipment and storage medium Active CN114449282B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210109588.9A CN114449282B (en) 2019-12-26 2019-12-26 Video transmission control method and device, equipment and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210109588.9A CN114449282B (en) 2019-12-26 2019-12-26 Video transmission control method and device, equipment and storage medium
CN201911370468.9A CN111107440B (en) 2019-12-26 2019-12-26 Video transmission control method and device, equipment and storage medium

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN201911370468.9A Division CN111107440B (en) 2019-12-26 2019-12-26 Video transmission control method and device, equipment and storage medium

Publications (2)

Publication Number Publication Date
CN114449282A CN114449282A (en) 2022-05-06
CN114449282B true CN114449282B (en) 2023-05-23

Family

ID=70424682

Family Applications (2)

Application Number Title Priority Date Filing Date
CN201911370468.9A Active CN111107440B (en) 2019-12-26 2019-12-26 Video transmission control method and device, equipment and storage medium
CN202210109588.9A Active CN114449282B (en) 2019-12-26 2019-12-26 Video transmission control method and device, equipment and storage medium

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN201911370468.9A Active CN111107440B (en) 2019-12-26 2019-12-26 Video transmission control method and device, equipment and storage medium

Country Status (1)

Country Link
CN (2) CN111107440B (en)

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111617466B (en) * 2020-05-12 2023-04-28 咪咕文化科技有限公司 Method and device for determining coding format and method for realizing cloud game
CN113824992B (en) * 2020-06-19 2023-04-14 京东方科技集团股份有限公司 Terminal and uploading method, server and monitoring method, medium, equipment and system
CN113518250B (en) * 2020-08-07 2022-08-02 腾讯科技（深圳）有限公司 Multimedia data processing method, device, equipment and readable storage medium
EP4199526A4 (en) * 2020-09-14 2023-08-23 Huawei Technologies Co., Ltd. Communication method and apparatus
CN112291495B (en) * 2020-10-16 2021-08-03 厦门大学 Wireless video low-delay anti-interference transmission method based on reinforcement learning
CN112383791B (en) * 2020-11-12 2023-07-21 咪咕视讯科技有限公司 Media data processing method and device, electronic equipment and storage medium
CN113115060B (en) * 2021-04-07 2022-10-25 中国工商银行股份有限公司 Video transmission method, device and system
CN113207017B (en) * 2021-07-07 2021-09-17 北京蔚领时代科技有限公司 Streaming media data transmission system for cloud game
CN113489745B (en) * 2021-07-29 2024-04-05 百果园技术(新加坡)有限公司 Video data transmission method, device, equipment and storage medium
CN114222169B (en) * 2021-12-14 2023-10-20 咪咕互动娱乐有限公司 Video streaming method, communication device and computer readable storage medium
CN114245197B (en) * 2022-02-24 2022-05-17 广州市保伦电子有限公司 Audio and video interaction low-delay transmission method and background server
CN115002513B (en) * 2022-05-25 2023-10-20 咪咕文化科技有限公司 Audio and video scheduling method and device, electronic equipment and computer readable storage medium
CN117336521A (en) * 2022-06-27 2024-01-02 腾讯科技（深圳）有限公司 Control method and device, equipment and medium for video stream transmission
CN116055790B (en) * 2022-07-29 2024-03-19 荣耀终端有限公司 Video playing method and system and electronic equipment
CN117138352B (en) * 2023-10-30 2024-01-30 腾讯科技（深圳）有限公司 Application program running method, device, equipment and storage medium

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8840475B2 (en) * 2002-12-10 2014-09-23 Ol2, Inc. Method for user session transitioning among streaming interactive video servers
EP2343901B1 (en) * 2010-01-08 2017-11-29 BlackBerry Limited Method and device for video encoding using predicted residuals
CN106027312A (en) * 2016-06-28 2016-10-12 青岛海信宽带多媒体技术有限公司 Network bandwidth prediction method and network bandwidth prediction device
CN107613330B (en) * 2017-10-20 2019-09-20 清华大学 A kind of video code rate adaptive approach
CN108079578B (en) * 2017-10-26 2020-12-01 咪咕互动娱乐有限公司 Code rate adjusting method and device based on cloud game and storage medium
WO2019100204A1 (en) * 2017-11-21 2019-05-31 深圳市大疆创新科技有限公司 Video processing method, device, aerial vehicle, system, and storage medium
CN108063961B (en) * 2017-12-22 2020-07-31 深圳市云网拜特科技有限公司 Self-adaptive code rate video transmission method and system based on reinforcement learning
CN109982118B (en) * 2019-03-27 2021-05-14 北京奇艺世纪科技有限公司 Video code rate self-adaptive adjustment method and device and electronic equipment
CN110198444B (en) * 2019-04-16 2021-07-09 浙江大华技术股份有限公司 Video frame encoding method, video frame encoding apparatus, and device having storage function
CN110290402B (en) * 2019-07-31 2021-11-05 腾讯科技（深圳）有限公司 Video code rate adjusting method, device, server and storage medium

Also Published As

Publication number Publication date
CN111107440B (en) 2022-02-25
CN111107440A (en) 2020-05-05
CN114449282A (en) 2022-05-06

Similar Documents

Publication Publication Date Title
CN114449282B (en) Video transmission control method and device, equipment and storage medium
US9549152B1 (en) Application content delivery to multiple computing environments using existing video conferencing solutions
US11831566B2 (en) Method and apparatus for transmitting scene image of virtual scene, computer device, and computer-readable storage medium
US9503491B2 (en) Playback stall avoidance in adaptive media streaming
US8903897B2 (en) System and method for providing interactive content to non-native application environments
EP3543859B1 (en) Automatically selecting and distributing recordings of applications executing on computer devices over networked environments
CN112995776B (en) Method, device, equipment and storage medium for determining screen capture frame rate of shared screen content
EP2272237B1 (en) Method of transmitting data in a communication system
US20200090324A1 (en) Method and Apparatus for Determining Experience Quality of VR Multimedia
WO2019117942A1 (en) Adaptive display brightness adjustment
US20190172181A1 (en) Systems and methods for frame time smoothing based on modified animation advancement and use of post render queues
KR20220031001A (en) Reinforcement Learning in Real-Time Communication
CN111078172A (en) Display fluency adjusting method and device, electronic equipment and storage medium
US11575958B2 (en) Progressive increase in multimedia streaming quality
US11074394B2 (en) Adaptive content delivery
CN113422751B (en) Streaming media processing method and device based on online reinforcement learning and electronic equipment
US8972563B2 (en) Updating changes to caches
US10506071B2 (en) Method and apparatus for playing multimedia contents in a communication
CN110192181B (en) Streaming application environment with recovery of lost or delayed input events
KR20230028250A (en) Reinforcement learning-based rate control
WO2020123211A1 (en) Handling timestamp inaccuracies for streaming network protocols
KR20190032100A (en) Method and apparatus for providing advertisement image
CN113628311A (en) Image rendering method, image rendering device, electronic device, and storage medium
KR102158856B1 (en) Remote computing system and remote computing method
EP4270969A1 (en) Rebuffering reduction in adaptive bit-rate video streaming

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant