CN115278289A - Cloud application rendering video frame processing method and device - Google Patents
Cloud application rendering video frame processing method and device Download PDFInfo
- Publication number
- CN115278289A CN115278289A CN202211178006.9A CN202211178006A CN115278289A CN 115278289 A CN115278289 A CN 115278289A CN 202211178006 A CN202211178006 A CN 202211178006A CN 115278289 A CN115278289 A CN 115278289A
- Authority
- CN
- China
- Prior art keywords
- frame
- frame rate
- cloud application
- video
- terminal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000009877 rendering Methods 0.000 title claims description 38
- 238000003672 processing method Methods 0.000 title claims description 14
- 238000000034 method Methods 0.000 claims abstract description 53
- 230000008569 process Effects 0.000 claims abstract description 19
- 238000012545 processing Methods 0.000 claims abstract description 18
- 230000001502 supplementing effect Effects 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 6
- 238000012423 maintenance Methods 0.000 claims 2
- 230000008859 change Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006266 hibernation Effects 0.000 description 2
- 208000003028 Stuttering Diseases 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000006866 deterioration Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000007958 sleep Effects 0.000 description 1
- 230000000087 stabilizing effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234381—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The application provides a method and a device for processing video frames rendered by cloud application, wherein the method comprises the following steps: determining a maximum frame rate supported by a terminal, and when the actual operating frame rate of the cloud application is less than or equal to the maximum frame rate supported by the terminal and is within one frame rate maintaining interval of at least one preset frame rate maintaining interval, adjusting the frame rate of the video stream input by the encoding unit or the frame rate of the video stream output by the encoding unit to a frame rate maintaining value corresponding to the frame rate maintaining interval; the method comprises the steps of coding rendered video frames of the cloud application through a coding unit, and sending a coding result to a terminal, wherein the rendered video frames are video frames obtained by pictures generated in the running process of the cloud application rendered by a system.
Description
Technical Field
The invention relates to the field of cloud application, in particular to a method and a device for processing video frames rendered by cloud application, electronic equipment and a storage medium.
Background
With the development of the technology of moving the mobile terminal application to the cloud terminal operation, the application operation mode with low performance consumption to the user terminal is more and more accepted by mobile internet companies, platforms and users. For the cloud platform, the key technology is that a user can see an instant running picture applied to the cloud, and delay and blockage are avoided as far as possible, so that the use experience of the user on the cloud application is guaranteed, and the experience is infinitely close to the experience of local running of the terminal equipment. However, as the complexity of the current application increases, the rendering frame rate may be unstable, resulting in problems of stutter, delay, etc., and in addition, the long tail effect generated by network transmission may cause severe deterioration of the terminal experience, so that it is necessary to stabilize the video push stream frame rate and frame interval in the cloud system.
In addition, in practical applications, the frame rate of the cloud application may fluctuate during the loading process due to system defects and resource shortage, and therefore the cloud application is not always operated at a fixed frame rate, and in this case, if the push stream frame rate is fixed, bandwidth and cloud system encoder resources are wasted.
Disclosure of Invention
Therefore, the embodiment of the application provides a cloud application rendering video frame processing method and device, an electronic device and a storage medium, which can stabilize the frame rate of a cloud system real-time image video stream, and can adapt to the frame rate change of cloud application, so that a terminal player can obtain experience of local application running, and bandwidth and cloud system encoder resources are saved.
In a first aspect, an embodiment of the present application provides a method for processing a video frame rendered by a cloud application, including:
determining a maximum frame rate supported by a terminal, and adjusting a video stream frame rate input by an encoding unit or an output video stream frame rate to a frame rate maintaining value corresponding to a frame rate maintaining interval when an actual operating frame rate of a cloud application is less than or equal to the maximum frame rate supported by the terminal and the actual operating frame rate of the cloud application is within one of preset at least one frame rate maintaining interval;
and coding a rendered video frame of the cloud application through the coding unit, and sending a coding result to the terminal, wherein the rendered video frame is a video frame obtained by rendering a picture generated in the running process of the cloud application by a system.
In a second aspect, an embodiment of the present application further provides a device for processing a video frame rendered by a cloud application, including:
the adjusting unit is used for determining the maximum frame rate supported by the terminal, and when the actual operating frame rate of the cloud application is less than or equal to the maximum frame rate supported by the terminal and is positioned in one of at least one preset frame rate maintaining interval, adjusting the frame rate of the video stream input by the encoding unit or the frame rate of the output video stream to a frame rate maintaining value corresponding to the frame rate maintaining interval;
and the encoding unit is used for encoding the rendered video frame of the cloud application and sending the encoding result to the terminal, wherein the rendered video frame is a video frame obtained by rendering a picture generated in the running process of the cloud application by a system.
In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method for processing a video frame rendered by a cloud application according to the first aspect is performed.
In a fourth aspect, an embodiment of the present application further provides an electronic device, including: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, when the electronic device runs, the processor and the storage medium communicate with each other through the bus, and the processor executes the machine-readable instructions to execute the steps of the method for processing the video frame rendered by the cloud application according to the first aspect.
To sum up, according to the cloud application video frame rendering processing method and apparatus, the electronic device, and the storage medium provided in the embodiments of the present application, a maximum frame rate supported by a terminal is determined, and when an actual operating frame rate of a cloud application is less than or equal to the maximum frame rate supported by the terminal and the actual operating frame rate of the cloud application is within one frame rate maintaining interval of at least one preset frame rate maintaining interval, a video stream frame rate input by a coding unit or an output video stream frame rate is adjusted to a frame rate maintaining value corresponding to the frame rate maintaining interval; and then, the rendered video frames of the cloud application are coded by the coding unit, and the coding result is sent to the terminal, so that the frame rate of the real-time picture video stream of the cloud system can be stabilized, the frame rate change of the cloud application can be adapted, a terminal player can obtain the experience of the application in local operation, and the bandwidth and the cloud system encoder resources are saved.
Drawings
Fig. 1 is a schematic flowchart of an embodiment of a processing method for rendering a video frame by a cloud application according to the present application;
fig. 2 is a schematic structural diagram of an embodiment of a device for processing a video frame rendered by a cloud application according to the present disclosure;
fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for illustrative and descriptive purposes only and are not used to limit the scope of protection of the present application. Further, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be performed out of order, and steps without logical context may be performed in reverse order or simultaneously. One skilled in the art, under the guidance of this application, may add one or more other operations to, or remove one or more operations from, the flowchart.
In addition, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that in the embodiments of the present application, the term "comprising" is used to indicate the presence of the features stated hereinafter, but does not exclude the addition of further features.
Referring to fig. 1, a schematic flow chart of a processing method for rendering a video frame by a cloud application according to an embodiment of the present disclosure is shown, where the method is applied to a cloud server, and may include:
s10, determining a maximum frame rate supported by a terminal, and adjusting a video stream frame rate input by an encoding unit or an output video stream frame rate to a frame rate maintaining value corresponding to a frame rate maintaining interval when an actual operating frame rate of a cloud application is less than or equal to the maximum frame rate supported by the terminal and the actual operating frame rate of the cloud application is within one of preset at least one frame rate maintaining interval;
in this embodiment, it should be noted that, when the terminal applies for the remote cloud application, the cloud server may obtain the terminal device model and the system version sent by the terminal, and determine the maximum frame rate value supportable by the terminal according to the terminal device model and the system version, specifically, the cloud server maintains a corresponding relationship between the terminal device model and the system version and the maximum frame rate value supportable by the terminal, and after the cloud server obtains the terminal device model and the system version of a certain terminal, the mapping relationship may be queried to obtain the corresponding maximum frame rate value supportable by the terminal. When the actual operating frame rate of the cloud application is obtained, an average value or a maximum frame rate value of the operating frame rates of the cloud application in a sliding window with a specified size may be used as the actual operating frame rate of the cloud application during the sliding window, and the sliding window moves by one sampling frame rate each time.
It can be understood that, for each type of cloud application, an actual possible operating frame rate of the type of cloud application may be divided into at least one frame rate interval according to an operating frame rate provided by the type of cloud application, so as to obtain at least one frame rate maintaining interval, where each frame rate maintaining interval corresponds to one frame rate maintaining value. And when the cloud server determines that the actual operating frame rate of the cloud application is less than or equal to the maximum frame rate supported by the terminal and the actual operating frame rate of the cloud application is within a certain frame rate maintaining interval, adjusting the frame rate of the video stream input by the encoding unit or the frame rate of the video stream output by the encoding unit to be a frame rate maintaining value corresponding to the frame rate maintaining interval. When the operating frame rate of the cloud application is decreased or increased from one frame rate maintaining interval to another frame rate maintaining interval (for example, when the end user changes the operating frame rate of the cloud application or the cloud application loads resources), the cloud server may adjust the frame rate of the video stream input by the encoding unit or the frame rate of the video stream output by the encoding unit to the frame rate maintaining value corresponding to the another frame rate maintaining interval, so as to match the current actual operating experience of the cloud application.
S11, coding the rendered video frame of the cloud application through the coding unit, and sending a coding result to the terminal, wherein the rendered video frame is a video frame obtained by rendering a picture generated in the running process of the cloud application by a system.
In this embodiment, it should be noted that, after adjusting the frame rate of the video stream input by the encoding unit or the frame rate of the video stream output by the encoding unit to the frame rate maintaining value corresponding to the frame rate maintaining interval, the frame rate and the frame interval of the video stream output by the encoding unit after encoding the video frame may be stabilized, that is, the encoding unit has an asymmetric input/output frame rate function, that is, the input frame rate is not large, and after the output frame rate is specified, the video frame with the specified frame rate can be maintained and output by dynamically supplementing or dropping frames, and the video frame interval is ensured to be uniform, where the frame rate of the video stream output by the encoding unit after encoding the video frame is the frame rate maintaining value.
In this embodiment, the cloud server can calculate the frame rate closest to the actual experience of the cloud application at present by combining the actual operating frame rate of the cloud application according to the maximum frame rate supported by the terminal, and can also take into account the performance limit of the terminal device.
The method for processing the video frames rendered by the cloud application comprises the steps of determining a maximum frame rate supported by a terminal, and when an actual operating frame rate of the cloud application is smaller than or equal to the maximum frame rate supported by the terminal and is located in one of at least one preset frame rate maintaining interval, adjusting a video stream frame rate input by a coding unit or an output video stream frame rate to a frame rate maintaining value corresponding to the frame rate maintaining interval; and then, the rendered video frames of the cloud application are coded by the coding unit, and the coding result is sent to the terminal, so that the frame rate of the real-time picture video stream of the cloud system can be stabilized, the frame rate change of the cloud application can be adapted, a terminal player can obtain the experience of the application in local operation, and the bandwidth and the cloud system encoder resources are saved.
On the basis of the foregoing method embodiment, the video stream input by the encoding unit or the video stream output by the encoding unit may stabilize the frame rate according to the following method:
sleeping for a first time length, and waiting for a second time length if judging that no error frame exists after the sleeping is finished;
and if the situation that no new rendering video frame exists is known when the waiting is finished, supplementing one frame by using the latest rendering video frame.
In this embodiment, it should be noted that, after the end of the hibernation, if it is determined that there is an error frame, a frame is complemented by using the latest rendered video frame. The case where a wrong frame exists includes: during hibernation, the system renders a new video frame. The second time period may be set as desired, such as 2ms, 3ms, etc. And if the existence of a new rendering video frame is known in the waiting process, acquiring the latest rendering video frame as an effective video frame. When the method is applied to the Android platform, whether a new rendered video frame exists can be determined by judging whether an available frame notification of the Android virtual screen is received, so that if the available frame notification of the Android virtual screen is not received in the waiting process, it is determined that the new rendered video frame does not exist at the end of waiting, and if not, it is determined that the new rendered video frame exists in the waiting process.
In this embodiment, the video frames generated by the cloud application during operation are arranged twice, so that the original irregular frame interval becomes regular, when the frame interval becomes larger, the previous frame is supplemented at a proper position, and when the frame interval becomes smaller, the corresponding video frame is discarded, so that the frame rate and the frame interval are stable, and the problem that the image is suddenly changed during operation and watched by a terminal user is avoided.
Based on the foregoing method embodiment, the first time length = frame interval + first frame time-system current time, where the frame interval is determined according to a frame rate, and the first frame time represents a rendering time of the first frame and represents multiplication.
In this embodiment, it should be noted that the frame interval may be 1000/frame rate ms. The frame number can be taken from 1, and when the frame number is equal to 1, the first frame time = the current time of the system; when the number of frames is not equal to 1, the first frame time = the rendering time of the first frame. When the frame number is equal to 1, firstly sleeping for a frame interval (namely, a first time length), then judging whether an error frame exists, if no error frame exists, waiting for a second time length, if the first frame exists in the waiting process, encoding the first frame, and sending an encoding result to the terminal, or if the first frame does not exist in the waiting process, acquiring the first frame from the system, encoding the first frame, and sending the encoding result to the terminal. When the frame number is equal to 1, the logic for determining whether there is a frame with an error can be skipped, and the logic for waiting for the second duration can be directly executed. When the frame number is equal to 2, firstly sleeping the frame interval x 2+ the first frame time-the system current time, then judging whether a dislocation frame exists, if the dislocation frame exists, encoding the latest video frame (namely, the latest video frame rendered by the system during sleeping, such as a third frame), and sending the encoding result to the terminal, or if the dislocation frame does not exist, waiting for a second time length, acquiring that a new video frame exists in the waiting process (namely, the new video frame rendered by the system during waiting, such as the second frame), encoding the latest video frame (namely, the second frame) in the new video frame, and sending the encoding result to the terminal, or acquiring that no new video frame exists when waiting to finish, encoding the first frame, and sending the encoding result to the terminal. When the frame number is equal to 3, firstly sleeping the frame interval x 3+ the first frame time-the current time of the system, then judging whether a dislocation frame exists, if the dislocation frame exists, encoding the latest video frame rendered by the system during sleeping, and sending the encoding result to the terminal, or if the dislocation frame does not exist, waiting for a second time length, acquiring that a new video frame exists in the waiting process, encoding the latest video frame in the new video frame, and sending the encoding result to the terminal, or acquiring that the new video frame does not exist when the waiting is finished, encoding the video frame sent when the frame number is equal to 2, and sending the encoding result to the terminal. The processing procedure when the frame number is greater than 3 is the same as the processing procedure when the frame number is equal to 3, and is not described herein again.
On the basis of the foregoing method embodiment, the determining to know that there is no error frame may include:
judging whether the rendering time of the latest rendering video frame is consistent with the rendering time of the sent last rendering video frame;
if the frame number is consistent with the frame number, determining that no error frame exists, otherwise, determining that an error frame exists.
In this embodiment, it should be noted that, in the process of sleeping after sending the previous frame, the system renders a new video frame, which is the case where there is an error frame, so that it may be determined whether there is an error frame by determining whether the latest rendered video frame and the last rendered video frame that has been sent are the same video frame, specifically, it may be determined whether the rendering time of the latest rendered video frame and the rendering time of the last rendered video frame that has been sent are the same, if so, it is determined that the latest rendered video frame is the last rendered video frame that has been sent, it is determined that there is no error frame, otherwise, it is determined that the latest rendered video frame and the last rendered video frame that has been sent are not the same video frame, and it is determined that there is an error frame.
On the basis of the foregoing method embodiment, the video stream input by the encoding unit or the video stream output by the encoding unit may stabilize the frame rate according to the following method:
firstly, frame supplementing is carried out on the input video stream frame rate or the output video stream frame rate to the maximum frame rate which can be supported by the cloud application, and then frame dropping processing is carried out on the video frame queue after frame supplementing.
In this embodiment, it should be noted that when the actual operating frame rate of the cloud application is lower than the frame rate maintaining value, frame supplementing is required, and when a frame is specifically supplemented, a frame may be just supplemented and immediately sent into another frame, so that frame interval instability is caused. Specifically, frames may be inserted at a time interval of 1000/maximum frame rate (unit is ms) supportable by the cloud application into a video stream with a frame rate supportable by the cloud application, and then frames are uniformly picked to obtain a video stream with a frame rate maintaining value for the frame rate.
Based on the foregoing method embodiments, the frame rate maintaining value may be an upper limit of the frame rate maintaining interval.
On the basis of the foregoing method embodiment, the method may further include:
when the actual operating frame rate of the cloud application is less than or equal to the maximum frame rate supported by the terminal and the actual operating frame rate of the cloud application is within a preset following operating interval, adjusting the video stream frame rate input by the encoding unit or the video stream frame rate output by the encoding unit to the actual operating frame rate of the cloud application.
In this embodiment, it should be noted that the following operation interval and the at least one frame rate maintaining interval form an actual possible operation frame rate of the cloud application, specifically, the following operation interval range may be [0, N1], and the at least one frame rate maintaining interval may include [ N1+1, N2], [ N2+1, N3], where N1, N2, and N3 are positive integers, N1 may be set as needed, for example, set as 10FPS (Frames Per Second), and N2 and N3 are cloud application operation frame rates that can be set by a user (i.e., operation frame rates provided by the cloud application).
In addition, it should be noted that, if the actual operating frame rate of the cloud application is greater than the maximum frame rate supported by the terminal, the video stream input by the encoding unit or the video stream output by the encoding unit needs to be frame dropped at a time interval corresponding to the maximum frame rate supported by the terminal, and a specific frame dropping method is consistent with the method for stabilizing the output frame rate by the encoding unit using the sleep policy in the foregoing embodiment, and is not described here again. By the method, under the condition that the terminal is an old model or a TV box and cannot perfectly play high-resolution and high-frame-rate videos, the cloud server can output the low-frame-rate videos to guarantee the experience of a low-end model when the actual running frame rate of the cloud application is high.
Referring to fig. 2, a schematic structural diagram of a processing apparatus for rendering a video frame by a cloud application according to an embodiment of the present application is shown, including:
an adjusting unit 20, configured to determine a maximum frame rate supported by a terminal, and adjust a frame rate of a video stream input by an encoding unit or a frame rate of an output video stream to a frame rate maintaining value corresponding to a frame rate maintaining interval when an actual operating frame rate of a cloud application is less than or equal to the maximum frame rate supported by the terminal and the actual operating frame rate of the cloud application is within one frame rate maintaining interval of at least one preset frame rate maintaining interval;
the encoding unit 21 is configured to encode a rendered video frame of the cloud application, and send an encoding result to the terminal, where the rendered video frame is a video frame obtained by rendering a picture generated in an operation process of the cloud application by a system.
In the cloud application rendering video frame processing apparatus provided in the embodiment of the present application, by means of the adjusting unit 20, a maximum frame rate supported by a terminal is determined first, and when an actual operating frame rate of a cloud application is less than or equal to the maximum frame rate supported by the terminal and the actual operating frame rate of the cloud application is located in one frame rate maintaining interval of at least one preset frame rate maintaining interval, a video stream frame rate input by the encoding unit 21 or an output video stream frame rate is adjusted to a frame rate maintaining value corresponding to the frame rate maintaining interval; by means of the encoding unit 21, the rendered video frames of the cloud application are encoded, and the encoding result is sent to the terminal, and by means of the scheme, the frame rate of the real-time picture video stream of the cloud system can be stabilized, the frame rate change of the cloud application can be adapted to the frame rate change of the cloud application, so that a terminal player can obtain experience when the application runs locally, and bandwidth and cloud system encoder resources are saved.
The implementation process of the cloud application rendered video frame processing device provided by the embodiment of the application is consistent with that of the cloud application rendered video frame processing method provided by the embodiment of the application, and the achieved effect is also the same as that of the cloud application rendered video frame processing method provided by the embodiment of the application, and is not repeated here.
As shown in fig. 3, an electronic device provided in an embodiment of the present application includes: a processor 30, a memory 31 and a bus 32, wherein the memory 31 stores machine-readable instructions executable by the processor 30, when the electronic device is operated, the processor 30 communicates with the memory 31 through the bus 32, and the processor 30 executes the machine-readable instructions to execute the steps of the method for processing the video frame by rendering the cloud application.
Specifically, the memory 31 and the processor 30 can be general-purpose memory and processor, and are not limited in particular, and when the processor 30 runs a computer program stored in the memory 31, the cloud application rendered video frame processing method can be executed.
Corresponding to the cloud application video frame rendering processing method, an embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to perform the steps of the cloud application video frame rendering processing method.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to corresponding processes in the method embodiments, and are not described in detail in this application. In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical division, and there may be other divisions in actual implementation, and for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or modules through some communication interfaces, and may be in an electrical, mechanical or other form.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A processing method for rendering video frames by a cloud application is characterized by comprising the following steps:
determining a maximum frame rate supported by a terminal, and adjusting a video stream frame rate input by an encoding unit or an output video stream frame rate to a frame rate maintaining value corresponding to a frame rate maintaining interval when an actual operating frame rate of a cloud application is less than or equal to the maximum frame rate supported by the terminal and the actual operating frame rate of the cloud application is within one of preset at least one frame rate maintaining interval;
and encoding the rendered video frame of the cloud application through the encoding unit, and sending an encoding result to the terminal, wherein the rendered video frame is a video frame obtained by rendering a picture generated in the running process of the cloud application by a system.
2. The method of claim 1, wherein the video stream input by the encoding unit or the video stream output by the encoding unit is stabilized in frame rate according to the following method:
sleeping for a first time length, and waiting for a second time length if judging that no error frame exists after the sleeping is finished;
and if the situation that no new rendering video frame exists is known when the waiting is finished, supplementing one frame by using the latest rendering video frame.
3. The method of claim 2, wherein the first time length = frame interval + head frame time-system current time, wherein the frame interval is determined according to a frame rate, the head frame time representing a rendering time of a head frame, and the head frame time representing multiplication.
4. The method of claim 2 or 3, wherein the determining to know that there is no erroneous frame comprises:
judging whether the rendering time of the latest rendering video frame is consistent with the rendering time of the sent last rendering video frame;
if the frame number is consistent with the frame number, determining that no error frame exists, otherwise, determining that an error frame exists.
5. The method of claim 1, wherein the input video stream or the output video stream of the encoding unit stabilizes the frame rate as follows:
firstly, the input video stream frame rate or the output video stream frame rate is subjected to frame supplementing to the maximum frame rate which can be supported by the cloud application, and then frame loss processing is carried out on the video frame queue after frame supplementing.
6. The method of claim 1, wherein the frame rate maintenance value is an upper limit of the frame rate maintenance interval.
7. The method of claim 1, further comprising:
when the actual operating frame rate of the cloud application is less than or equal to the maximum frame rate supported by the terminal and the actual operating frame rate of the cloud application is within a preset following operating interval, adjusting the video stream frame rate input by the encoding unit or the video stream frame rate output by the encoding unit to the actual operating frame rate of the cloud application.
8. A cloud application rendered video frame processing apparatus, comprising:
the adjusting unit is used for determining the maximum frame rate supported by the terminal, and when the actual operating frame rate of the cloud application is less than or equal to the maximum frame rate supported by the terminal and is positioned in one of at least one preset frame rate maintaining interval, adjusting the frame rate of the video stream input by the encoding unit or the frame rate of the output video stream to a frame rate maintaining value corresponding to the frame rate maintaining interval;
and the encoding unit is used for encoding the rendered video frame of the cloud application and sending the encoding result to the terminal, wherein the rendered video frame is a video frame obtained by rendering a picture generated in the running process of the cloud application by a system.
9. A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, performs the steps of the cloud application rendered video frame processing method according to any one of claims 1 to 7.
10. An electronic device, comprising: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor and the storage medium communicating via the bus when the electronic device is running, the processor executing the machine-readable instructions to perform the steps of the cloud application rendered video frame processing method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211178006.9A CN115278289B (en) | 2022-09-27 | 2022-09-27 | Cloud application rendering video frame processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211178006.9A CN115278289B (en) | 2022-09-27 | 2022-09-27 | Cloud application rendering video frame processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115278289A true CN115278289A (en) | 2022-11-01 |
| CN115278289B CN115278289B (en) | 2023-01-20 |
Family
ID=83756442
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211178006.9A Active CN115278289B (en) | 2022-09-27 | 2022-09-27 | Cloud application rendering video frame processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115278289B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115671726A (en) * | 2022-12-29 | 2023-02-03 | 腾讯科技（深圳）有限公司 | Game data rendering method, device, equipment and storage medium |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011192229A (en) * | 2010-03-17 | 2011-09-29 | Hitachi Consumer Electronics Co Ltd | Server device and information processing method |
| CN110881136A (en) * | 2019-11-14 | 2020-03-13 | 腾讯科技（深圳）有限公司 | Video frame rate control method and device, computer equipment and storage medium |
| WO2020062062A1 (en) * | 2018-09-28 | 2020-04-02 | 深圳市大疆软件科技有限公司 | Cluster rendering method, device and system |
| CN111124824A (en) * | 2019-12-18 | 2020-05-08 | 北京智明星通科技股份有限公司 | Mobile phone game frame rate recommendation method and device and mobile terminal |
| CN111298427A (en) * | 2020-02-04 | 2020-06-19 | 南京大学 | Method for reducing picture jitter in virtual reality cloud game system |
| CN111866533A (en) * | 2020-07-13 | 2020-10-30 | 网宿科技股份有限公司 | Live broadcast transcoding method and device |
| CN113395512A (en) * | 2021-05-27 | 2021-09-14 | 北京达佳互联信息技术有限公司 | Stuck detection method and device, stuck detection server and storage medium |
| CN113542757A (en) * | 2021-07-20 | 2021-10-22 | Oppo广东移动通信有限公司 | Image transmission method and device for cloud application, server and storage medium |
| WO2022028089A1 (en) * | 2020-08-06 | 2022-02-10 | 腾讯科技（深圳）有限公司 | Image processing method and device, server, and medium |
| CN114913471A (en) * | 2022-07-18 | 2022-08-16 | 深圳比特微电子科技有限公司 | Image processing method and device and readable storage medium |
| WO2022194140A1 (en) * | 2021-03-18 | 2022-09-22 | 上海达龙信息科技有限公司 | Remote video transmitting method and transmitting apparatus, storage medium, and electronic device |
-
2022
- 2022-09-27 CN CN202211178006.9A patent/CN115278289B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011192229A (en) * | 2010-03-17 | 2011-09-29 | Hitachi Consumer Electronics Co Ltd | Server device and information processing method |
| WO2020062062A1 (en) * | 2018-09-28 | 2020-04-02 | 深圳市大疆软件科技有限公司 | Cluster rendering method, device and system |
| CN110881136A (en) * | 2019-11-14 | 2020-03-13 | 腾讯科技（深圳）有限公司 | Video frame rate control method and device, computer equipment and storage medium |
| CN111124824A (en) * | 2019-12-18 | 2020-05-08 | 北京智明星通科技股份有限公司 | Mobile phone game frame rate recommendation method and device and mobile terminal |
| CN111298427A (en) * | 2020-02-04 | 2020-06-19 | 南京大学 | Method for reducing picture jitter in virtual reality cloud game system |
| CN111866533A (en) * | 2020-07-13 | 2020-10-30 | 网宿科技股份有限公司 | Live broadcast transcoding method and device |
| WO2022028089A1 (en) * | 2020-08-06 | 2022-02-10 | 腾讯科技（深圳）有限公司 | Image processing method and device, server, and medium |
| WO2022194140A1 (en) * | 2021-03-18 | 2022-09-22 | 上海达龙信息科技有限公司 | Remote video transmitting method and transmitting apparatus, storage medium, and electronic device |
| CN113395512A (en) * | 2021-05-27 | 2021-09-14 | 北京达佳互联信息技术有限公司 | Stuck detection method and device, stuck detection server and storage medium |
| CN113542757A (en) * | 2021-07-20 | 2021-10-22 | Oppo广东移动通信有限公司 | Image transmission method and device for cloud application, server and storage medium |
| CN114913471A (en) * | 2022-07-18 | 2022-08-16 | 深圳比特微电子科技有限公司 | Image processing method and device and readable storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115671726A (en) * | 2022-12-29 | 2023-02-03 | 腾讯科技（深圳）有限公司 | Game data rendering method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115278289B (en) | 2023-01-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111882626B (en) | Image processing method, device, server and medium | |
| CN112104879B (en) | Video coding method and device, electronic equipment and storage medium | |
| US9213521B2 (en) | Control method of information processing apparatus and information processing apparatus | |
| EP3410302B1 (en) | Graphic instruction data processing method, apparatus | |
| CN112533059B (en) | Image rendering method and device, electronic equipment and storage medium | |
| CN109729353B (en) | Video coding method, device, system and medium | |
| CN113676730B (en) | Video coding method and device, electronic equipment and storage medium | |
| CN115278289B (en) | Cloud application rendering video frame processing method and device | |
| CN111093094A (en) | Video transcoding method, device and system, electronic equipment and readable storage medium | |
| CN112422963B (en) | Cloud computer system, encoding method, encoding device, cloud computer equipment and storage medium | |
| CN111617466B (en) | Method and device for determining coding format and method for realizing cloud game | |
| WO2023045551A1 (en) | Camera control method and system, electronic device and storage medium | |
| CN113490055A (en) | Data processing method and device | |
| CN112866746A (en) | Multi-path streaming cloud game control method, device, equipment and storage medium | |
| CN113973224B (en) | Media information transmission method, computing device and storage medium | |
| CN114245175A (en) | Video transcoding method and device, electronic equipment and storage medium | |
| CN116866658A (en) | Video data processing method, device, equipment and medium | |
| CN112929704A (en) | Data transmission method, device, electronic equipment and storage medium | |
| CN115514960A (en) | Video coding method and device, electronic equipment and storage medium | |
| CN115767149A (en) | Video data transmission method and device | |
| CN115225902A (en) | High-resolution VR cloud game solution method based on scatter coding and computer equipment | |
| CN104185084A (en) | Graphics server and method for managing streaming parameters | |
| CN115118921B (en) | Method and system for video screen-combining self-adaptive output in cloud conference | |
| EP4131952A1 (en) | Transmission device, transmission method, and program | |
| CN116996649B (en) | Screen projection method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20231206 Address after: Room 672, 6th Floor, Building A3A4, Zhong'an Chuanggu Science and Technology Park, No. 900 Wangjiang West Road, High tech Zone, China (Anhui) Pilot Free Trade Zone, Hefei City, Anhui Province, 230031 Patentee after: Anhui Haima Cloud Technology Co.,Ltd. Address before: 301700 room 2d25, Building 29, No.89 Heyuan Road, Jingjin science and Technology Valley Industrial Park, Wuqing District, Tianjin Patentee before: HAIMAYUN (TIANJIN) INFORMATION TECHNOLOGY CO.,LTD. |
|
| TR01 | Transfer of patent right |