CN113842635A - Method and system for improving fluency of cloud game - Google Patents

Method and system for improving fluency of cloud game Download PDF

Info

Publication number
CN113842635A
CN113842635A CN202111044508.8A CN202111044508A CN113842635A CN 113842635 A CN113842635 A CN 113842635A CN 202111044508 A CN202111044508 A CN 202111044508A CN 113842635 A CN113842635 A CN 113842635A
Authority
CN
China
Prior art keywords
resolution
game
picture
super
video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111044508.8A
Other languages
Chinese (zh)
Inventor
乔建苹
马文齐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Normal University
Original Assignee
Shandong Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Normal University filed Critical Shandong Normal University
Priority to CN202111044508.8A priority Critical patent/CN113842635A/en
Publication of CN113842635A publication Critical patent/CN113842635A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/30—Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
    • A63F13/31—Communication aspects specific to video games, e.g. between several handheld game devices at close range
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/30—Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
    • A63F13/35—Details of game servers
    • A63F13/352—Details of game servers involving special game server arrangements, e.g. regional servers connected to a national server or a plurality of servers managing partitions of the game world
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/70—Game security or game management aspects
    • A63F13/77—Game security or game management aspects involving data related to game devices or game servers, e.g. configuration data, software version or amount of memory
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/131—Protocols for games, networked simulations or virtual reality
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4781—Games
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/50—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by details of game servers
    • A63F2300/57—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by details of game servers details of game services offered to the player

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Business, Economics & Management (AREA)
  • Computer Security & Cryptography (AREA)
  • General Business, Economics & Management (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The invention provides a method for improving fluency of a cloud game, which belongs to the technical field of image processing and comprises the steps of receiving a game picture video stream rendered by a server end by using low resolution; decoding the video stream to obtain a video picture; and performing up-sampling on the decoded video picture to generate a game picture for the client to display. According to the method, the game picture resolution of the server is reduced through a super-resolution method, the rendering amount of the GPU of the server is reduced, and the pause phenomenon caused by the overweight load of the GPU is reduced; the super-resolution method is used for reducing the game picture resolution of the server, reducing the data volume needing to be transmitted to the client and reducing the phenomena of cloud game pause and delay caused by network transmission; the game image resolution of the server can be reduced, the GPU performance requirement of the server is reduced, and the low-performance GPU with low cost can be replaced when the game is rendered, so that the equipment cost of the cloud game server is reduced.

Description

Method and system for improving fluency of cloud game
Technical Field
The invention relates to the technical field of image processing, in particular to a method and a system for improving cloud game fluency.
Background
Cloud gaming is a cloud computing based gaming approach. And in the running mode of the cloud game, all games run at the server side, and the rendered game pictures are compressed and then transmitted to the user through the network. At the client, the user's game device does not require any high-end processor and graphics card, but only basic video decompression and display capabilities.
The image super-resolution reconstruction technique is a technique of restoring a corresponding HR (high resolution) image using an LR (low resolution) image to thereby improve the image resolution. The super-resolution reconstruction technology can be divided into two types, one is a non-deep learning method, namely, the image reconstruction is realized by a traditional super-resolution method, and the other is a deep learning method, namely, the image feature extraction, mapping and reconstruction are realized by a convolutional neural network.
Video is composed of successive images, and a video super-resolution reconstruction technique refers to a technique of generating a corresponding high-resolution video version from a corresponding low-resolution video version. The frame rate is the number of frames of a video or game screen per second of a refreshed image, and the measurement unit is FPS (number of frames per second displayed). The higher the frame rate, the smoother the video and game images will be. In order to ensure a good game experience of the user, the game frame rate is usually not lower than 30FPS, and the optimal play frame rate is 60FPS or 120 FPS.
The current cloud game implementation mode is as follows: and rendering the game picture at the server side, encoding the game picture into a video stream, transmitting the video stream to the client side, and decoding and displaying the video stream by the client side.
In order to achieve a high quality playing experience for the user, cloud games use a high frame rate and resolution. Generally, the frame rate of a picture in a cloud game is not lower than 30FPS, and the picture resolution is not lower than 1920 × 1080. The high frame rate and the high resolution generate a large amount of video stream data to be transmitted, which puts a very high requirement on the network transmission rate between the server and the client, and if the network rate between the server and the client does not meet the requirement, the cloud game is jammed and delayed, thereby seriously affecting the playing experience.
In addition, if the network conditions of the server and the client are unstable, the network transmission rate is suddenly and slowly changed, and the cloud game is also subjected to a pause phenomenon. In the process of rendering and encoding game pictures, a GPU (graphics processing unit) is usually used for encoding directly in order to improve encoding efficiency. However, since the GPU is also used for rendering the game screen, if the rendering amount of the game is too large, the GPU is overloaded, and the coding time of the GPU is prolonged, which results in a cloud game jam.
Disclosure of Invention
The invention aims to provide a method and a system for improving the fluency of a cloud game. To solve at least one technical problem in the background art.
In order to achieve the purpose, the invention adopts the following technical scheme:
in one aspect, the invention provides a method for improving fluency of a cloud game, comprising the following steps:
receiving a game picture video stream rendered by a server side by using low resolution; decoding the video stream to obtain a video picture; and performing up-sampling on the decoded video picture to generate an up-sampled super-resolution game picture, and displaying the super-resolution game picture on the client.
Preferably, the decoded video picture is upsampled using an interpolation upsampling algorithm to generate an upsampled game picture.
Preferably, the upsampled game picture is sharpened for display at the client.
Preferably, the upsampling magnification of the interpolation upsampling algorithm is a ratio of a resolution size of a picture to be displayed by the client to a size of a game picture rendered by the server.
Preferably, the interpolation upsampling algorithm is Lanczos interpolation, nearest neighbor interpolation, bilinear quadratic interpolation or bicubic interpolation.
Preferably, the decoded video picture is up-sampled by using a super-resolution model to generate a super-resolution game picture, and the super-resolution game picture is used for the client to display.
Preferably, the training of the super-resolution model comprises:
the same controller is used for controlling two groups of cloud game servers, and different resolutions are respectively used for rendering game pictures; the resolution size of the game picture rendered at the higher resolution is the resolution size of the game picture displayed by the client, and the resolution size of the game picture rendered at the lower resolution is the resolution size of the game picture rendered by the server;
encoding the higher-resolution rendered game pictures and the lower-resolution rendered game pictures into a high-resolution video stream and a low-resolution video stream respectively;
decoding the high resolution video stream and the low resolution video stream into a high resolution video and a low resolution video stream, respectively;
respectively converting the high-resolution video and the low-resolution frequency stream into a high-resolution image sequence and a low-resolution image sequence;
the high resolution image sequence and the low resolution image sequence form a super resolution data set;
cutting the super-resolution data set into overlapped image blocks with uniform size;
and determining a network structure and a loss function which need to be used, using the image blocks obtained by cutting, training and finally generating a super-resolution model.
In a second aspect, the present invention provides a system for improving fluency of a cloud game, including:
the receiving module is used for receiving the game picture video stream rendered by the server end by using low resolution; the decoding module is used for decoding the video stream to obtain a video picture; the sampling module is used for carrying out up-sampling on the decoded video picture to generate an up-sampling super-resolution game picture; and the display module is used for displaying the generated super-resolution game picture on the client.
In a third aspect, the present invention provides a non-transitory computer readable storage medium for storing computer instructions which, when executed by a processor, implement the method for improving fluency of a cloud game as described above.
In a fourth aspect, the present invention provides an electronic device comprising: a processor, a memory, and a computer program; when the electronic device runs, the processor executes the computer program stored in the memory, so that the electronic device executes instructions for implementing the method for improving the fluency of the cloud game.
The invention has the beneficial effects that: the super-resolution method is used for reducing the game picture resolution of the server side, reducing the rendering amount of a GPU of the server side and reducing the pause phenomenon caused by the overweight load of the GPU; the super-resolution method is used for reducing the game picture resolution of the server, reducing the data volume needing to be transmitted to the client and reducing the phenomena of cloud game pause and delay caused by network transmission; the resolution ratio of game pictures at the server end can be reduced, the GPU performance requirement at the server end is reduced, a high-performance GPU is not needed when some games are rendered, and a low-performance GPU which is cheaper can be replaced, so that the equipment cost of the cloud game server is reduced.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic flow chart of a method for improving fluency of a cloud game according to embodiment 1 of the present invention;
fig. 2 is a flowchart illustrating a method for improving fluency of a cloud game according to embodiment 3 of the present invention;
fig. 3 is a video recording screenshot of a game running at 1080P resolution according to embodiment 3 of the present invention;
fig. 4 is a video recording screenshot of a game running at 720P resolution according to embodiment 3 of the present invention;
fig. 5 is a schematic view of an up-sampled game screen according to embodiment 3 of the present invention;
fig. 6 is a schematic view of a game picture after sharpening an upsampled game picture according to embodiment 3 of the present invention;
fig. 7 is a schematic diagram illustrating details of a game screen finally displayed by a client according to embodiment 3 of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below by way of the drawings are illustrative only and are not to be construed as limiting the invention.
It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
For the purpose of facilitating an understanding of the present invention, the present invention will be further explained by way of specific embodiments with reference to the accompanying drawings, which are not intended to limit the present invention.
It should be understood by those skilled in the art that the drawings are merely schematic representations of embodiments and that the elements shown in the drawings are not necessarily required to practice the invention.
Example 1
This embodiment 1 provides a system for promoting cloud recreation fluency, and this system includes:
the receiving module is used for receiving the game picture video stream rendered by the server end by using low resolution; the decoding module is used for decoding the video stream to obtain a video picture; the sampling module is used for performing up-sampling on the decoded video picture to generate an up-sampled super-resolution game picture; and the display module is used for displaying the generated game picture on the client.
As shown in fig. 1, in embodiment 1, a method for improving cloud game fluency is implemented by using the system for improving cloud game fluency, where the method includes:
receiving a game picture video stream rendered by a server side by using a low resolution by using a receiving module of a client side; decoding the video stream by using a decoding module of the client to obtain a video picture; and the sampling module of the client is used for up-sampling the decoded video picture to generate an up-sampling super-resolution game picture, and the display module is used for displaying the up-sampling super-resolution game picture on the client.
In this embodiment 1, the sampling module performs upsampling on the decoded video picture by using an interpolation upsampling algorithm to generate an upsampled super-resolution game picture.
In this embodiment 1, the system further includes a sharpening module, configured to sharpen the generated up-sampled super-resolution game picture.
The up-sampling multiplying power of the interpolation up-sampling algorithm is the ratio of the resolution size of the picture required to be displayed by the client to the size of the game picture rendered by the server.
In specific application, the interpolation upsampling algorithm is Lanczos interpolation, nearest neighbor interpolation, bilinear quadratic interpolation or bicubic interpolation, and the like.
In this embodiment 1, using a Lanczos interpolation algorithm as an example, the smoothness of the cloud game is improved. The resolution size of a game picture required to be displayed by the simulated client side is 1920 × 1080(1080P), and the size of a game picture required to be displayed by the simulated server side is 1280 × 720 (720P).
The method comprises the steps of running a game at 1080P resolution and 720P resolution respectively and recording the video, converting the game video which is run and recorded at 720P resolution into a 720P image sequence, and calculating the up-sampling multiplying factor of interpolation up-sampling, wherein the up-sampling multiplying factor is 1080 ÷ 720 ═ 1.5. And (5) upsampling the 720P image sequence by using a Lanczos interpolation algorithm, wherein the upsampling multiplying power is 1.5, and generating the upsampled image sequence. The upsampled image sequence is spatially convolved with a convolution kernel of 3 × 3, generating a sharpened image sequence.
In this embodiment 1, the convolution kernel of 3 × 3 is:
[[0,-0.5,0],
[-0.5,3,-0.5],
[0,-0.5,0]]
and synthesizing the sharpened game image sequence into a video, wherein the video is a game picture displayed by the simulated client.
Example 2
This embodiment 2 provides a system for promoting cloud game fluency, this system includes:
the receiving module is used for receiving the game picture video stream rendered by the server end by using low resolution; the decoding module is used for decoding the video stream to obtain a video picture; the sampling module is used for performing up-sampling on the decoded video picture to generate an up-sampled super-resolution game picture; and the display module is used for displaying the generated game picture on the client.
In embodiment 2, the method for improving cloud game fluency is implemented by using the system for improving cloud game fluency, and includes:
receiving a game picture video stream rendered by a server side by using a low resolution by using a receiving module of a client side; decoding the video stream by using a decoding module of the client to obtain a video picture; and the sampling module of the client is used for up-sampling the decoded video picture to generate an up-sampling super-resolution game picture, and the display module is used for displaying the up-sampling super-resolution game picture on the client.
In this embodiment 2, a super-resolution model is used to up-sample a decoded video picture, so as to generate a super-resolution game picture, and the super-resolution game picture is used for a client to display.
In this embodiment 2, the training of the super-resolution model includes:
the same controller is used for controlling the two groups of cloud game servers, and different resolutions are respectively used for rendering game pictures; the resolution size of the game picture rendered at the higher resolution is the resolution size of the game picture displayed by the client, and the resolution size of the game picture rendered at the lower resolution is the resolution size of the game picture rendered by the server;
encoding the higher-resolution rendered game pictures and the lower-resolution rendered game pictures into a high-resolution video stream and a low-resolution video stream respectively;
decoding the high resolution video stream and the low resolution video stream into a high resolution video and a low resolution video stream, respectively;
respectively converting the high-resolution video and the low-resolution frequency stream into a high-resolution image sequence and a low-resolution image sequence;
the high resolution image sequence and the low resolution image sequence form a super resolution data set;
cutting the super-resolution data set into overlapped image blocks with uniform size;
and determining a network structure and a loss function which need to be used, using the image blocks obtained by cutting, training and finally generating a super-resolution model.
Example 3
The invention provides a method for improving cloud game fluency based on super-resolution, the general flow is shown in figure 2, wherein the super-resolution method is divided into a cloud game super-resolution method based on interpolation and deep learning.
In this embodiment 3, the method for super-resolution of cloud games based on interpolation is as follows:
determining the resolution size of a game picture required to be displayed by a client and the size of a game picture rendered by a server; the server side renders game pictures and transmits the video stream to the client side, and the client side decodes the video stream to obtain video pictures; the video picture decoded by the client is up-sampled by using an interpolation up-sampling algorithm to generate an up-sampled game picture; and sharpening the up-sampling game picture, and using the generated game picture for the client to display.
The cloud game super-resolution method based on deep learning comprises the steps that a server side renders game pictures by using low resolution and transmits video streams to a client side, and the client side decodes the video streams to obtain the video pictures; the method comprises the steps that a super-resolution generation model is used for carrying out up-sampling on a video picture decoded by a client side to generate a super-resolution game picture; and using the super-resolution game picture for client display.
In this embodiment 3, using the cloud game super-resolution method based on deep learning, the training of the super-resolution model includes:
the same controller is used for controlling the two groups of cloud game servers and carrying out game operation; the two groups of cloud game servers, one group of cloud game servers uses low-resolution rendering game pictures, and the other group of cloud game servers uses high-resolution rendering game pictures; the resolution size of the high-resolution game picture is the resolution size of the game picture displayed by the client side in the invention; the resolution size of the low-resolution game picture is the size of the game picture rendered by the server side in the invention; encoding the high and low resolution game pictures into high and low resolution video streams; decoding the high and low resolution video streams into high and low resolution video; converting the high-resolution video and the low-resolution video into a high-resolution image sequence and a low-resolution image sequence; the sequence of high and low resolution images constitutes a super resolution data set. Cutting the super-resolution data set into overlapped image blocks with uniform size; and determining a network structure and a loss function which need to be used, training, and finally generating a super-resolution model.
In this embodiment, the network structure that can be selected, such as SRCNN, RLSP, OVSR, etc., needs to satisfy: the speed is fast enough to reason about the low resolution video received by the client, at least up to 30 FPS. Alternative loss functions, such as L1 loss, L2 loss, Perceptial loss, GAN loss.
In this embodiment 3, the game demonstrated is "wilderness Dart Guest: and (4) the redemption 2' shows the effect of the cloud game super-resolution method based on interpolation in a simulation mode. The following are the specific steps in this example 3:
1. determining that the resolution size of a game picture required to be displayed by a simulated client is 1920 × 1080(1080P), and the size of a game picture rendered by a simulated server is 1280 × 720 (720P);
2. running the game at 1080P resolution and 720P resolution respectively and recording the video, wherein the recorded video screenshots are shown in FIGS. 3 and 4 respectively;
3. converting the 720P running and recorded game video into a 720P image sequence;
4. calculating the up-sampling multiplying power of interpolation up-sampling, wherein the up-sampling multiplying power is 1080 ÷ 720 ═ 1.5
5. And (5) upsampling the 720P image sequence by using a Lanczos interpolation algorithm, wherein the upsampling multiplying power is 1.5, and generating the upsampled image sequence. FIG. 5 is an example of an up-sampled image;
6. performing spatial convolution on the up-sampling image sequence by using a convolution kernel of 3 multiplied by 3 to generate a sharpened image sequence;
7. the 3 × 3 convolution kernel is:
[[0,-0.5,0],
[-0.5,3,-0.5],
[0,-0.5,0]]
8. and synthesizing the sharpened game image sequence into a video, wherein the video is a game picture displayed by the simulated client. Fig. 6 is a game screen displayed by the simulated client. FIG. 7 is a comparison diagram of game screen details, the left diagram is a 720P game detail, the middle diagram is a game detail after interpolation and over-scoring, and the right diagram is a 1080P game detail.
Example 4
Embodiment 4 of the present invention provides a non-transitory computer-readable storage medium, where the non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the method for improving fluency of a cloud game is implemented, where the method includes:
receiving a game picture video stream rendered by a server side by using low resolution; decoding the video stream to obtain a video picture; and performing up-sampling on the decoded video picture to generate a super-resolution up-sampling game picture, and displaying the super-resolution up-sampling game picture on the client.
Example 5
Embodiment 5 provides a computer program (product) including a computer program, when running on one or more processors, for implementing the method for improving fluency of a cloud game as described above, the method including:
receiving a game picture video stream rendered by a server side by using low resolution; decoding the video stream to obtain a video picture; and performing up-sampling on the decoded video picture to generate a super-resolution up-sampling game picture, and displaying the super-resolution up-sampling game picture on the client.
Example 6
An embodiment 6 of the present invention provides an electronic device, including: a processor, a memory, and a computer program; wherein, a processor is connected with a memory, a computer program is stored in the memory, when the electronic device runs, the processor executes the computer program stored in the memory, so as to make the electronic device execute instructions for implementing the method for improving cloud game fluency as described above, and the method comprises:
receiving a game picture video stream rendered by a server side by using low resolution; decoding the video stream to obtain a video picture; and performing up-sampling on the decoded video picture to generate a super-resolution up-sampling game picture, and displaying the super-resolution up-sampling game picture on the client.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Although the embodiments of the present invention have been described with reference to the accompanying drawings, it is not intended to limit the scope of the present invention, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive efforts based on the technical solutions disclosed in the present invention.

Claims (10)

1. A method for improving fluency of cloud games is characterized by comprising the following steps:
receiving a game picture video stream rendered by a server side by using low resolution; decoding the video stream to obtain a video picture; and performing up-sampling on the decoded video picture to generate a super-resolution up-sampling game picture, and displaying the super-resolution up-sampling game picture on the client.
2. The method of improving fluency of cloud games as recited in claim 1, wherein an interpolation upsampling algorithm is used to upsample the decoded video picture to generate an upsampled game picture.
3. The method for improving cloud game fluency according to claim 2, wherein the upsampled game picture is sharpened for client display.
4. The method for improving cloud game fluency according to claim 2, wherein the upsampling magnification of the interpolation upsampling algorithm is a ratio of a resolution size of a picture to be displayed by the client to a size of a game picture rendered by the server.
5. The method for improving cloud game fluency according to any of claims 2-4, wherein the interpolation upsampling algorithm is Lanczos interpolation, nearest neighbor interpolation, bilinear quadratic interpolation or bicubic interpolation.
6. The method for improving cloud game fluency as recited in claim 1, wherein a super-resolution model is used to up-sample the decoded video pictures, generating super-resolution game pictures, and using the super-resolution game pictures for client display.
7. The method for improving cloud game fluency according to claim 6, wherein the training of the super-resolution model comprises:
the same controller is used for controlling two groups of cloud game servers, and different resolutions are respectively used for rendering game pictures; the resolution size of the game picture rendered at the higher resolution is the resolution size of the game picture displayed by the client, and the resolution size of the game picture rendered at the lower resolution is the resolution size of the game picture rendered by the server;
encoding the higher-resolution rendered game pictures and the lower-resolution rendered game pictures into a high-resolution video stream and a low-resolution video stream respectively;
decoding the high resolution video stream and the low resolution video stream into a high resolution video and a low resolution video stream, respectively;
respectively converting the high-resolution video and the low-resolution frequency stream into a high-resolution image sequence and a low-resolution image sequence;
the high resolution image sequence and the low resolution image sequence form a super resolution data set;
cutting the super-resolution data set into overlapped image blocks with uniform size;
and determining a network structure and a loss function which need to be used, using the image blocks obtained by cutting, training and finally generating a super-resolution model.
8. A system for enhancing fluency of cloud games, comprising:
the receiving module is used for receiving the game picture video stream rendered by the server end by using low resolution; the decoding module is used for decoding the video stream to obtain a video picture; the sampling module is used for performing up-sampling on the decoded video picture to generate an up-sampled super-resolution game picture; and the display module is used for displaying the generated game picture on the client.
9. A non-transitory computer readable storage medium storing computer instructions which, when executed by a processor, implement the method of enhancing fluency of cloud gaming of any of claims 1-7.
10. An electronic device, comprising: a processor, a memory, and a computer program; wherein a processor is connected with the memory, a computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory, so that the electronic device executes the instructions for implementing the method for improving the fluency of the cloud game according to any one of claims 1-7.
CN202111044508.8A 2021-09-07 2021-09-07 Method and system for improving fluency of cloud game Pending CN113842635A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111044508.8A CN113842635A (en) 2021-09-07 2021-09-07 Method and system for improving fluency of cloud game

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111044508.8A CN113842635A (en) 2021-09-07 2021-09-07 Method and system for improving fluency of cloud game

Publications (1)

Publication Number Publication Date
CN113842635A true CN113842635A (en) 2021-12-28

Family

ID=78973329

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111044508.8A Pending CN113842635A (en) 2021-09-07 2021-09-07 Method and system for improving fluency of cloud game

Country Status (1)

Country Link
CN (1) CN113842635A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023045649A1 (en) * 2021-09-26 2023-03-30 腾讯科技（深圳）有限公司 Video frame playing method and apparatus, and device, storage medium and program product

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023045649A1 (en) * 2021-09-26 2023-03-30 腾讯科技（深圳）有限公司 Video frame playing method and apparatus, and device, storage medium and program product

Similar Documents

Publication Publication Date Title
US10594901B2 (en) Game engine application direct to video encoder rendering
CN109600666B (en) Video playing method, device, medium and electronic equipment in game scene
CN103597839B (en) Video-frequency compression method, video reconstruction method and system and encoder
US9075560B2 (en) Real-time compression with GPU/CPU
Fenney Texture compression using low-frequency signal modulation
US20180192026A1 (en) Method and System for Real-Time Rendering Displaying High Resolution Virtual Reality (VR) Video
AU2011317052B2 (en) Composite video streaming using stateless compression
WO2021236668A1 (en) Neural super-sampling for real-time rendering
US20230421796A1 (en) Methods and apparatus for foveated compression
CN103098466A (en) Image processing device, image processing method, data structure for video files, data compression device, data decoding device, data compression method, data decoding method and data structure for compressed video files
CN110827380B (en) Image rendering method and device, electronic equipment and computer readable medium
CN113946402A (en) Cloud mobile phone acceleration method, system, equipment and storage medium based on rendering separation
EP4030341A1 (en) Image recognition method, video playback method, related device, and medium
Zhang et al. Efficient volumetric video streaming through super resolution
CN102497545B (en) Content adaptive and art directable scalable video coding
CN111510739A (en) Video transmission method and device
CN113842635A (en) Method and system for improving fluency of cloud game
CN115225615B (en) Illusion engine pixel streaming method and device
CN115941965A (en) Cloud desktop coding method, reconstruction method, display method and display system
US20230054523A1 (en) Enhancing 360-degree video using convolutional neural network (cnn)-based filter
Lietsch et al. A CUDA-supported approach to remote rendering
CN108259779A (en) A kind of method that quick processing large format video image is realized using part breadth data
Chi et al. RA360SR: A real-time acceleration-adaptive 360-degree video super-resolution system
CN114710673A (en) Image up-sampling method for SUVC coding and decoding
CN114938461A (en) Video processing method, device and equipment and readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination