CN115278289B - Cloud application rendering video frame processing method and device - Google Patents
Cloud application rendering video frame processing method and device Download PDFInfo
- Publication number
- CN115278289B CN115278289B CN202211178006.9A CN202211178006A CN115278289B CN 115278289 B CN115278289 B CN 115278289B CN 202211178006 A CN202211178006 A CN 202211178006A CN 115278289 B CN115278289 B CN 115278289B
- Authority
- CN
- China
- Prior art keywords
- frame
- frame rate
- cloud application
- video
- terminal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000009877 rendering Methods 0.000 title claims description 42
- 238000003672 processing method Methods 0.000 title claims description 14
- 238000000034 method Methods 0.000 claims abstract description 54
- 230000008569 process Effects 0.000 claims abstract description 19
- 238000012545 processing Methods 0.000 claims abstract description 18
- 230000001502 supplementing effect Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 238000012423 maintenance Methods 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006266 hibernation Effects 0.000 description 2
- 208000003028 Stuttering Diseases 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000006866 deterioration Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000007958 sleep Effects 0.000 description 1
- 230000000087 stabilizing effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234381—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The application provides a method and a device for processing video frames rendered by cloud application, wherein the method comprises the following steps: determining the maximum frame rate supported by the terminal, and when the actual operating frame rate of the cloud application is less than or equal to the maximum frame rate supported by the terminal and is positioned in one of at least one preset frame rate maintaining interval, adjusting the video stream frame rate input by the encoding unit or the video stream frame rate output by the encoding unit to be a frame rate maintaining value corresponding to the frame rate maintaining interval; the method comprises the steps of coding rendered video frames of the cloud application through a coding unit, and sending a coding result to a terminal, wherein the rendered video frames are video frames obtained by pictures generated in the running process of the cloud application rendered by a system.
Description
Technical Field
The invention relates to the field of cloud application, in particular to a method and a device for processing video frames rendered by cloud application, electronic equipment and a storage medium.
Background
With the development of the technology of moving the mobile terminal application to the cloud terminal operation, the application operation mode with low performance consumption to the user terminal is more and more accepted by mobile internet companies, platforms and users. For the cloud platform, the key technology is that a user can see an instant running picture applied to the cloud, and delay and blockage are avoided as far as possible, so that the use experience of the user on the cloud application is guaranteed, and the experience is infinitely close to the experience of local running of the terminal equipment. However, as the complexity of the current application increases, the rendering frame rate may be unstable, resulting in problems of stutter, delay, etc., and in addition, the long tail effect generated by network transmission may cause severe deterioration of the terminal experience, so that it is necessary to stabilize the video push stream frame rate and frame interval in the cloud system.
In addition, in practical applications, the frame rate of the cloud application may jitter due to system defects and resource shortage, and may fluctuate in the loading process, so the cloud application is not always operated at a fixed frame rate, and in this case, if the push stream frame rate is fixed, bandwidth and cloud system encoder resources may be wasted.
Disclosure of Invention
Therefore, the embodiment of the application provides a cloud application rendering video frame processing method and device, an electronic device and a storage medium, which can stabilize the frame rate of a cloud system real-time image video stream, and can adapt to the frame rate change of cloud application, so that a terminal player can obtain experience of local application running, and bandwidth and cloud system encoder resources are saved.
In a first aspect, an embodiment of the present application provides a method for processing a video frame rendered by a cloud application, including:
determining a maximum frame rate supported by a terminal, and when the actual operating frame rate of the cloud application is less than or equal to the maximum frame rate supported by the terminal and is located in one of preset frame rate maintaining intervals, adjusting the frame rate of a video stream input by an encoding unit or the frame rate of an output video stream to a frame rate maintaining value corresponding to the frame rate maintaining interval;
and coding a rendered video frame of the cloud application through the coding unit, and sending a coding result to the terminal, wherein the rendered video frame is a video frame obtained by rendering a picture generated in the running process of the cloud application by a system.
In a second aspect, an embodiment of the present application further provides a device for processing a video frame rendered by a cloud application, including:
the adjusting unit is used for determining the maximum frame rate supported by the terminal, and when the actual operating frame rate of the cloud application is less than or equal to the maximum frame rate supported by the terminal and is positioned in one of at least one preset frame rate maintaining interval, adjusting the frame rate of the video stream input by the encoding unit or the frame rate of the output video stream to a frame rate maintaining value corresponding to the frame rate maintaining interval;
and the encoding unit is used for encoding a rendered video frame of the cloud application and sending an encoding result to the terminal, wherein the rendered video frame is a video frame obtained by rendering a picture generated in the running process of the cloud application by a system.
In a third aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, performs the steps of the method for processing video frames rendered by a cloud application according to the first aspect.
In a fourth aspect, an embodiment of the present application further provides an electronic device, including: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, when the electronic device runs, the processor and the storage medium communicate with each other through the bus, and the processor executes the machine-readable instructions to execute the steps of the method for processing the video frame rendered by the cloud application according to the first aspect.
To sum up, according to the cloud application video frame rendering processing method and apparatus, the electronic device, and the storage medium provided in the embodiments of the present application, a maximum frame rate supported by a terminal is determined, and when an actual operating frame rate of a cloud application is less than or equal to the maximum frame rate supported by the terminal and the actual operating frame rate of the cloud application is within one frame rate maintaining interval of at least one preset frame rate maintaining interval, a video stream frame rate input by a coding unit or an output video stream frame rate is adjusted to a frame rate maintaining value corresponding to the frame rate maintaining interval; and then, the rendered video frames of the cloud application are coded by the coding unit, and the coding result is sent to the terminal, so that the frame rate of the real-time picture video stream of the cloud system can be stabilized, the frame rate change of the cloud application can be adapted, a terminal player can obtain the experience of the application in local operation, and the bandwidth and the cloud system encoder resources are saved.
Drawings
Fig. 1 is a schematic flowchart of an embodiment of a processing method for rendering a video frame by a cloud application according to the present application;
fig. 2 is a schematic structural diagram of an embodiment of a device for processing a video frame rendered by a cloud application according to the present application;
fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for illustrative and descriptive purposes only and are not used to limit the scope of protection of the present application. Additionally, it should be understood that the schematic drawings are not necessarily drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be performed out of order, and steps without logical context may be performed in reverse order or simultaneously. In addition, one skilled in the art, under the guidance of the present disclosure, may add one or more other operations to the flowchart, or may remove one or more operations from the flowchart.
In addition, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that in the embodiments of the present application, the term "comprising" is used to indicate the presence of the features stated hereinafter, but does not exclude the addition of further features.
Referring to fig. 1, a schematic flow chart of a processing method for rendering a video frame by a cloud application according to an embodiment of the present disclosure is shown, where the method is applied to a cloud server, and may include:
s10, determining a maximum frame rate supported by a terminal, and adjusting a video stream frame rate input by an encoding unit or an output video stream frame rate to a frame rate maintaining value corresponding to a frame rate maintaining interval when an actual operating frame rate of a cloud application is less than or equal to the maximum frame rate supported by the terminal and the actual operating frame rate of the cloud application is within one of preset at least one frame rate maintaining interval;
in this embodiment, it should be noted that when the terminal applies for the remote cloud application, the cloud server may obtain the terminal device model and the system version sent by the terminal, and determine the maximum frame rate value supportable by the terminal according to the terminal device model and the system version, specifically, the cloud server maintains a corresponding relationship between the terminal device model and the system version and the maximum frame rate value supportable by the terminal, and after the cloud server obtains the terminal device model and the system version of a certain terminal, the cloud server may query the mapping relationship to obtain the corresponding maximum frame rate value supportable by the terminal. When the actual operation frame rate of the cloud application is obtained, an average value or a maximum frame rate value of the operation frame rates of the cloud application in a sliding window with a specified size may be used as the actual operation frame rate of the cloud application during the sliding window, and the sliding window is moved by one sampling frame rate each time.
It can be understood that, for each type of cloud application, an actual possible operating frame rate of the type of cloud application may be divided into at least one frame rate interval according to an operating frame rate provided by the type of cloud application, so as to obtain at least one frame rate maintaining interval, where each frame rate maintaining interval corresponds to one frame rate maintaining value. And when the cloud server determines that the actual operating frame rate of the cloud application is less than or equal to the maximum frame rate supported by the terminal and the actual operating frame rate of the cloud application is within a certain frame rate maintaining interval, adjusting the frame rate of the video stream input by the encoding unit or the frame rate of the video stream output by the encoding unit to be a frame rate maintaining value corresponding to the frame rate maintaining interval. When the operating frame rate of the cloud application is decreased or increased from one frame rate maintaining interval to another frame rate maintaining interval (for example, when the end user changes the operating frame rate of the cloud application or the cloud application loads resources), the cloud server may adjust the frame rate of the video stream input by the encoding unit or the frame rate of the video stream output by the encoding unit to the frame rate maintaining value corresponding to the another frame rate maintaining interval, so as to match the current actual operating experience of the cloud application.
S11, coding the rendered video frame of the cloud application through the coding unit, and sending a coding result to the terminal, wherein the rendered video frame is a video frame obtained by rendering a picture generated in the running process of the cloud application by a system.
In this embodiment, it should be noted that, after adjusting the frame rate of the video stream input by the encoding unit or the frame rate of the video stream output by the encoding unit to the frame rate maintaining value corresponding to the frame rate maintaining interval, the frame rate and the frame interval of the video stream output by the encoding unit after encoding the video frame may be stabilized, that is, the encoding unit has an asymmetric input/output frame rate function, that is, the input frame rate is not large, and after the output frame rate is specified, the video frame with the specified frame rate can be maintained and output through dynamic frame complementing and frame dropping, and the video frame interval is ensured to be uniform, where the frame rate of the video stream output by the encoding unit after encoding the video frame is the frame rate maintaining value.
In this embodiment, the cloud server can calculate the frame rate closest to the actual experience of the cloud application at present according to the maximum frame rate supported by the terminal and by combining the actual operating frame rate of the cloud application, and can also consider the performance limit of the terminal device.
The method for processing the video frames rendered by the cloud application, provided by the embodiment of the application, includes the steps of determining a maximum frame rate supported by a terminal, and adjusting a video stream frame rate input by a coding unit or an output video stream frame rate to a frame rate maintenance value corresponding to a frame rate maintenance interval when an actual operating frame rate of the cloud application is smaller than or equal to the maximum frame rate supported by the terminal and the actual operating frame rate of the cloud application is located in one of preset at least one frame rate maintenance interval; and then, the rendered video frame of the cloud application is coded through the coding unit, and the coding result is sent to the terminal.
On the basis of the foregoing method embodiments, the video stream input by the encoding unit or the video stream output by the encoding unit may stabilize the frame rate according to the following method:
sleeping for a first time length, and waiting for a second time length after the sleeping is finished if the dislocation frame is judged to be absent;
and if the situation that no new rendering video frame exists is known when the waiting is finished, supplementing one frame by using the latest rendering video frame.
In this embodiment, it should be noted that, after the end of the hibernation, if it is determined that there is an error frame, a frame is complemented by using the latest rendered video frame. The case where a faulty frame exists includes: during the hibernation process, the system renders a new video frame. The second time period may be set as desired, such as 2ms, 3ms, etc. And if the existence of a new rendering video frame is known in the waiting process, acquiring the latest rendering video frame as an effective video frame. When the method is applied to the Android platform, whether a new rendered video frame exists can be determined by judging whether an available frame notification of the Android virtual screen is received, so that if the available frame notification of the Android virtual screen is not received in the waiting process, it is determined that the new rendered video frame does not exist at the end of waiting, and if not, it is determined that the new rendered video frame exists in the waiting process.
In this embodiment, the video frames generated by the cloud application during operation are arranged twice, so that the original irregular frame interval becomes regular, when the frame interval becomes larger, the previous frame is supplemented at a proper position, and when the frame interval becomes smaller, the corresponding video frame is discarded, so that the frame rate and the frame interval are stable, and the problem that the image is suddenly changed during operation and watched by a terminal user is avoided.
Based on the foregoing embodiment of the method, the first time length = frame interval + first frame time-system current time, where the frame interval is determined according to a frame rate, and the first frame time represents a rendering time of the first frame and indicates multiplication.
In this embodiment, it should be noted that the frame interval may be 1000/frame rate ms. The frame number can be taken from 1, and when the frame number is equal to 1, the first frame time = the current time of the system; when the number of frames is not equal to 1, the first frame time = the rendering time of the first frame. When the frame number is equal to 1, firstly sleeping for a frame interval (namely, a first time length), then judging whether an error frame exists, if no error frame exists, waiting for a second time length, if the first frame exists in the waiting process, encoding the first frame, and sending an encoding result to the terminal, or if the first frame does not exist in the waiting process, acquiring the first frame from the system, encoding the first frame, and sending the encoding result to the terminal. When the frame number is equal to 1, the logic for determining whether there is a frame with an error can be skipped, and the logic for waiting for the second duration can be directly executed. When the frame number is equal to 2, firstly sleeping frame interval x 2+ first frame time-system current time, then judging whether a dislocation frame exists, if the dislocation frame exists, encoding the latest video frame (namely the latest video frame rendered by the system during sleeping, such as a third frame), and sending the encoding result to the terminal, or if the dislocation frame does not exist, waiting for a second time, acquiring that a new video frame (namely the new video frame rendered by the system during waiting, such as the second frame) exists during waiting, encoding the latest video frame (namely the second frame) in the new video frame, and sending the encoding result to the terminal, or acquiring that the new video frame does not exist during waiting, encoding the first frame, and sending the encoding result to the terminal. When the frame number is equal to 3, firstly sleeping the frame interval x 3+ the first frame time-the current time of the system, then judging whether a dislocation frame exists, if the dislocation frame exists, encoding the latest video frame rendered by the system during sleeping, and sending the encoding result to the terminal, or if the dislocation frame does not exist, waiting for a second time length, acquiring that a new video frame exists in the waiting process, encoding the latest video frame in the new video frame, and sending the encoding result to the terminal, or acquiring that the new video frame does not exist when the waiting is finished, encoding the video frame sent when the frame number is equal to 2, and sending the encoding result to the terminal. The processing procedure when the frame number is greater than 3 is the same as the processing procedure when the frame number is equal to 3, and is not described herein again.
On the basis of the foregoing method embodiment, the determining to know that there is no error frame may include:
judging whether the rendering time of the latest rendering video frame is consistent with the rendering time of the sent last rendering video frame;
if the frame number is consistent with the frame number, determining that no error frame exists, otherwise, determining that an error frame exists.
In this embodiment, it should be noted that, in the process of sleeping after the last frame of picture is sent, the system renders a new video frame, which is a case where an error frame exists, so that it may be determined whether an error frame exists by determining whether the latest rendered video frame and the last rendered video frame that has been sent are the same video frame, specifically, it may be determined whether the rendering time of the latest rendered video frame and the rendering time of the last rendered video frame that has been sent are the same, if the rendering times are the same, it is indicated that the latest rendered video frame is the last rendered video frame that has been sent, it is indicated that an error frame does not exist, otherwise, it is indicated that the latest rendered video frame and the last rendered video frame that has been sent are not the same video frame, and it is indicated that an error frame exists.
On the basis of the foregoing method embodiment, the video stream input by the encoding unit or the video stream output by the encoding unit may stabilize the frame rate according to the following method:
firstly, the input video stream frame rate or the output video stream frame rate is subjected to frame supplementing to the maximum frame rate which can be supported by the cloud application, and then frame loss processing is carried out on the video frame queue after frame supplementing.
In this embodiment, it should be noted that when the actual operating frame rate of the cloud application is lower than the frame rate maintaining value, frame supplementing is required, and when a frame is specifically supplemented, a frame may be just supplemented and immediately sent into another frame, so that frame interval instability is caused. Specifically, a video stream with a frame rate of the maximum frame rate (in ms) supportable by the cloud application may be interpolated at time intervals of 1000/cloud application, and then a video stream with a frame rate of the frame rate maintained at the frame rate may be obtained by uniformly frame-selecting.
Based on the foregoing method embodiments, the frame rate maintaining value may be an upper limit of the frame rate maintaining interval.
On the basis of the foregoing method embodiment, the method may further include:
when the actual operating frame rate of the cloud application is less than or equal to the maximum frame rate supported by the terminal and the actual operating frame rate of the cloud application is within a preset following operating interval, adjusting the video stream frame rate input by the encoding unit or the video stream frame rate output by the encoding unit to the actual operating frame rate of the cloud application.
In this embodiment, it should be noted that the following operation interval and the at least one frame rate maintaining interval form an actual possible operation frame rate of the cloud application, specifically, the following operation interval range may be [0, N1], the at least one frame rate maintaining interval may include [ N1+1, N2], [ N2+1, N3], where N1, N2, and N3 are positive integers, N1 may be set as needed, for example, set to 10FPS (Frames Per Second), and N2 and N3 are cloud application operation frame rates that can be set by a user (that is, operation frame rates provided by the cloud application).
In addition, it should be noted that, if the actual operating frame rate of the cloud application is greater than the maximum frame rate supported by the terminal, the video stream input by the encoding unit or the video stream output by the encoding unit needs to be frame dropped at a time interval corresponding to the maximum frame rate supported by the terminal, and a specific frame dropping method is consistent with the method for stabilizing the output frame rate by the encoding unit using the sleep policy in the foregoing embodiment, and is not described here again. By the method, under the condition that the terminal is an old model or a TV box and cannot perfectly play high-resolution and high-frame-rate videos, the cloud server can output the low-frame-rate videos to guarantee the experience of a low-end model when the actual running frame rate of the cloud application is high.
Referring to fig. 2, a schematic structural diagram of a processing apparatus for rendering a video frame by a cloud application according to an embodiment of the present application is shown, including:
an adjusting unit 20, configured to determine a maximum frame rate supported by a terminal, and adjust a frame rate of a video stream input by an encoding unit or a frame rate of an output video stream to a frame rate maintaining value corresponding to a frame rate maintaining interval when an actual operating frame rate of a cloud application is less than or equal to the maximum frame rate supported by the terminal and the actual operating frame rate of the cloud application is within one frame rate maintaining interval of at least one preset frame rate maintaining interval;
the encoding unit 21 is configured to encode a rendered video frame of the cloud application, and send an encoding result to the terminal, where the rendered video frame is a video frame obtained by rendering a picture generated in an operation process of the cloud application by a system.
In the cloud application rendering video frame processing apparatus provided in the embodiment of the present application, by means of the adjusting unit 20, a maximum frame rate supported by a terminal is determined first, and when an actual operating frame rate of a cloud application is less than or equal to the maximum frame rate supported by the terminal and the actual operating frame rate of the cloud application is located in one frame rate maintaining interval of at least one preset frame rate maintaining interval, a video stream frame rate input by the encoding unit 21 or an output video stream frame rate is adjusted to a frame rate maintaining value corresponding to the frame rate maintaining interval; by means of the encoding unit 21, the rendered video frames of the cloud application are encoded, and the encoding result is sent to the terminal, so that the frame rate of the real-time picture video stream of the cloud system can be stabilized, the frame rate change of the cloud application can be adapted, a terminal player can obtain experience of the application in local operation, and bandwidth and cloud system encoder resources are saved.
The implementation process of the cloud application rendering video frame processing device provided by the embodiment of the present application is consistent with that of the cloud application rendering video frame processing method provided by the embodiment of the present application, and the achieved effect is also the same as that of the cloud application rendering video frame processing method provided by the embodiment of the present application, and is not repeated here.
As shown in fig. 3, an electronic device provided in an embodiment of the present application includes: a processor 30, a memory 31 and a bus 32, wherein the memory 31 stores machine-readable instructions executable by the processor 30, when the electronic device is running, the processor 30 communicates with the memory 31 via the bus 32, and the processor 30 executes the machine-readable instructions to perform the steps of the method for processing video frames by rendering a cloud application as described above.
Specifically, the memory 31 and the processor 30 can be general-purpose memory and processor, and are not limited in particular, and when the processor 30 runs a computer program stored in the memory 31, the cloud application rendered video frame processing method can be executed.
Corresponding to the cloud application video frame rendering processing method, an embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to perform the steps of the cloud application video frame rendering processing method.
It can be clearly understood by those skilled in the art that, for convenience and simplicity of description, the specific working process of the system and the apparatus described above may refer to the corresponding process in the method embodiment, and is not described in detail in this application. In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice, and for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or modules through some communication interfaces, and may be in an electrical, mechanical or other form.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A processing method for rendering video frames by a cloud application is characterized by comprising the following steps:
determining a maximum frame rate supported by a terminal, and when an actual operating frame rate of a cloud application is less than or equal to the maximum frame rate supported by the terminal and the actual operating frame rate of the cloud application is within one frame rate maintaining interval of at least one preset frame rate maintaining interval, adjusting a video stream frame rate input by an encoding unit or an output video stream frame rate to a frame rate maintaining value corresponding to the frame rate maintaining interval, wherein the frame rate maintaining values corresponding to different frame rate maintaining intervals are different;
and encoding the rendered video frame of the cloud application through the encoding unit, and sending an encoding result to the terminal, wherein the rendered video frame is a video frame obtained by rendering a picture generated in the running process of the cloud application by a system.
2. The method of claim 1, wherein the video stream input by the encoding unit or the video stream output by the encoding unit is stabilized in frame rate according to the following method:
sleeping for a first time length, and waiting for a second time length if judging that no error frame exists after the sleeping is finished;
and if the situation that no new rendering video frame exists is known when the waiting is finished, supplementing one frame by using the latest rendering video frame.
3. The method of claim 2, wherein the first time length = frame interval frame number + first frame time-system current time, wherein the frame interval is determined according to a frame rate, and the first frame time represents a rendering time of the first frame, and represents multiplication.
4. The method of claim 2 or 3, wherein the determining to know that there is no erroneous frame comprises:
judging whether the rendering time of the latest rendering video frame is consistent with the rendering time of the sent last rendering video frame;
if the frame number is consistent with the frame number, determining that no error frame exists, otherwise, determining that the error frame exists.
5. The method of claim 1, wherein the video stream input by the encoding unit or the video stream output by the encoding unit is stabilized in frame rate according to the following method:
firstly, the input video stream frame rate or the output video stream frame rate is subjected to frame supplementing to the maximum frame rate which can be supported by the cloud application, and then frame loss processing is carried out on the video frame queue after frame supplementing.
6. The method of claim 1, wherein the frame rate maintenance value is an upper limit of the frame rate maintenance interval.
7. The method of claim 1, further comprising:
when the actual operating frame rate of the cloud application is less than or equal to the maximum frame rate supported by the terminal and the actual operating frame rate of the cloud application is within a preset following operating interval, adjusting the video stream frame rate input by the encoding unit or the video stream frame rate output by the encoding unit to the actual operating frame rate of the cloud application.
8. A cloud application rendered video frame processing apparatus, comprising:
the adjusting unit is used for determining the maximum frame rate supported by the terminal, and when the actual operating frame rate of the cloud application is less than or equal to the maximum frame rate supported by the terminal and is located in one of preset at least one frame rate maintaining interval, adjusting the frame rate of the video stream input by the encoding unit or the frame rate of the output video stream to a frame rate maintaining value corresponding to the frame rate maintaining interval, wherein the frame rate maintaining values corresponding to different frame rate maintaining intervals are different;
and the encoding unit is used for encoding the rendered video frame of the cloud application and sending the encoding result to the terminal, wherein the rendered video frame is a video frame obtained by rendering a picture generated in the running process of the cloud application by a system.
9. A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, performs the steps of the cloud application rendered video frame processing method according to any of claims 1 to 7.
10. An electronic device, comprising: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor and the storage medium communicating via the bus when the electronic device is running, the processor executing the machine-readable instructions to perform the steps of the cloud application rendered video frame processing method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211178006.9A CN115278289B (en) | 2022-09-27 | 2022-09-27 | Cloud application rendering video frame processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211178006.9A CN115278289B (en) | 2022-09-27 | 2022-09-27 | Cloud application rendering video frame processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115278289A CN115278289A (en) | 2022-11-01 |
| CN115278289B true CN115278289B (en) | 2023-01-20 |
Family
ID=83756442
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211178006.9A Active CN115278289B (en) | 2022-09-27 | 2022-09-27 | Cloud application rendering video frame processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115278289B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115671726B (en) * | 2022-12-29 | 2023-03-28 | 腾讯科技（深圳）有限公司 | Game data rendering method, device, equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113395512A (en) * | 2021-05-27 | 2021-09-14 | 北京达佳互联信息技术有限公司 | Stuck detection method and device, stuck detection server and storage medium |
| CN114913471A (en) * | 2022-07-18 | 2022-08-16 | 深圳比特微电子科技有限公司 | Image processing method and device and readable storage medium |
| WO2022194140A1 (en) * | 2021-03-18 | 2022-09-22 | 上海达龙信息科技有限公司 | Remote video transmitting method and transmitting apparatus, storage medium, and electronic device |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011192229A (en) * | 2010-03-17 | 2011-09-29 | Hitachi Consumer Electronics Co Ltd | Server device and information processing method |
| WO2020062062A1 (en) * | 2018-09-28 | 2020-04-02 | 深圳市大疆软件科技有限公司 | Cluster rendering method, device and system |
| CN110881136B (en) * | 2019-11-14 | 2021-11-02 | 腾讯科技（深圳）有限公司 | Video frame rate control method and device, computer equipment and storage medium |
| CN111124824A (en) * | 2019-12-18 | 2020-05-08 | 北京智明星通科技股份有限公司 | Mobile phone game frame rate recommendation method and device and mobile terminal |
| CN111298427A (en) * | 2020-02-04 | 2020-06-19 | 南京大学 | Method for reducing picture jitter in virtual reality cloud game system |
| CN111866533B (en) * | 2020-07-13 | 2022-03-08 | 网宿科技股份有限公司 | Live broadcast transcoding method and device |
| CN111882626B (en) * | 2020-08-06 | 2023-07-14 | 腾讯科技（深圳）有限公司 | Image processing method, device, server and medium |
| CN113542757B (en) * | 2021-07-20 | 2024-04-02 | Oppo广东移动通信有限公司 | Image transmission method and device for cloud application, server and storage medium |
-
2022
- 2022-09-27 CN CN202211178006.9A patent/CN115278289B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022194140A1 (en) * | 2021-03-18 | 2022-09-22 | 上海达龙信息科技有限公司 | Remote video transmitting method and transmitting apparatus, storage medium, and electronic device |
| CN113395512A (en) * | 2021-05-27 | 2021-09-14 | 北京达佳互联信息技术有限公司 | Stuck detection method and device, stuck detection server and storage medium |
| CN114913471A (en) * | 2022-07-18 | 2022-08-16 | 深圳比特微电子科技有限公司 | Image processing method and device and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115278289A (en) | 2022-11-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111882626B (en) | Image processing method, device, server and medium | |
| EP4192015A1 (en) | Video encoding method, video decoding method, apparatus, electronic device, storage medium, and computer program product | |
| US10110650B2 (en) | Client side stream switching | |
| US9167007B2 (en) | Stream complexity mapping | |
| EP2300928B1 (en) | Client side stream switching | |
| US9213521B2 (en) | Control method of information processing apparatus and information processing apparatus | |
| EP3410302B1 (en) | Graphic instruction data processing method, apparatus | |
| CN110913245A (en) | Method and device for controlling video transcoding code rate | |
| CN109729353B (en) | Video coding method, device, system and medium | |
| US20230050250A1 (en) | Method and apparatus for encoding video, and storage medium | |
| CN115278289B (en) | Cloud application rendering video frame processing method and device | |
| CN112929704B (en) | Data transmission method, device, electronic equipment and storage medium | |
| CN111093094A (en) | Video transcoding method, device and system, electronic equipment and readable storage medium | |
| CN111617466B (en) | Method and device for determining coding format and method for realizing cloud game | |
| CN112422963A (en) | Cloud computer system, encoding method, device, equipment and storage medium | |
| WO2023174254A1 (en) | Video posting method and apparatus, and device and storage medium | |
| CN111970565A (en) | Video data processing method and device, electronic equipment and storage medium | |
| CN113973224B (en) | Media information transmission method, computing device and storage medium | |
| CN113490055A (en) | Data processing method and device | |
| CN114245175A (en) | Video transcoding method and device, electronic equipment and storage medium | |
| CN111836092A (en) | Data processing method and device of virtual desktop and related components | |
| CN116866658A (en) | Video data processing method, device, equipment and medium | |
| CN115209189B (en) | Video stream transmission method, system, server and storage medium | |
| CN115767149A (en) | Video data transmission method and device | |
| CN104185084A (en) | Graphics server and method for managing streaming parameters |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20231206 Address after: Room 672, 6th Floor, Building A3A4, Zhong'an Chuanggu Science and Technology Park, No. 900 Wangjiang West Road, High tech Zone, China (Anhui) Pilot Free Trade Zone, Hefei City, Anhui Province, 230031 Patentee after: Anhui Haima Cloud Technology Co.,Ltd. Address before: 301700 room 2d25, Building 29, No.89 Heyuan Road, Jingjin science and Technology Valley Industrial Park, Wuqing District, Tianjin Patentee before: HAIMAYUN (TIANJIN) INFORMATION TECHNOLOGY CO.,LTD. |