CN112717389A - Cloud game image screen-splash repairing method and device - Google Patents
Cloud game image screen-splash repairing method and device Download PDFInfo
- Publication number
- CN112717389A CN112717389A CN202011642578.9A CN202011642578A CN112717389A CN 112717389 A CN112717389 A CN 112717389A CN 202011642578 A CN202011642578 A CN 202011642578A CN 112717389 A CN112717389 A CN 112717389A
- Authority
- CN
- China
- Prior art keywords
- game
- picture
- screen
- frame
- splash
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 90
- 230000008439 repair process Effects 0.000 claims abstract description 64
- 230000008569 process Effects 0.000 claims abstract description 43
- 238000012549 training Methods 0.000 claims description 50
- 238000012545 processing Methods 0.000 claims description 17
- 238000012360 testing method Methods 0.000 claims description 15
- 238000013135 deep learning Methods 0.000 claims description 6
- 230000000694 effects Effects 0.000 abstract description 9
- 230000000007 visual effect Effects 0.000 abstract description 4
- 230000005540 biological transmission Effects 0.000 description 8
- 230000008859 change Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/50—Controlling the output signals based on the game progress
- A63F13/52—Controlling the output signals based on the game progress involving aspects of the displayed game scene
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/30—Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
- A63F13/31—Communication aspects specific to video games, e.g. between several handheld game devices at close range
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/30—Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
- A63F13/35—Details of game servers
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/60—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Hardware Design (AREA)
- Geometry (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses a method and a device for repairing a cloud game image screen, wherein the method comprises the following steps: receiving and caching a game picture restoration model sent by a server; receiving and caching a game coding picture sent by a server; when a screen-splash picture appears in the process of decoding the game coding picture, obtaining a repair frame of the screen-splash picture according to the game picture repair model and the game coding picture; the display restoration frame replaces the screen frame when the screen frame appears, the influence of the screen frame phenomenon on the display of the game picture is made up, the display effect of the game picture is enhanced, and the visual experience of a user is improved.
Description
Technical Field
The invention relates to the technical field of cloud servers, in particular to a method and a device for repairing a cloud game image screen.
Background
When the user conducts the cloud game, the running process of the game is conducted on the server. The server sends the game video pictures to the client, and the client receives the video frame data and then displays the video frame data.
However, in the game video transmission process, due to network fluctuation and other factors, the client receives video frame data and in the decoding and playing process, the problem of incomplete frame data is caused, and the screen splash phenomenon is generated. The screen-splash phenomenon means that images have obvious tearing or dislocation characteristics, and the screen-splash phenomenon may continuously affect multiple frames of images, and the screen-splash phenomenon seriously affects the game experience of users.
In the related art, the cloud game has a high requirement on the real-time performance of the video, so that a video buffer area cannot be set, the client receives and decodes one frame of data to display one frame of data, the incomplete frame of data is processed in a direct decoding and displaying mode, the screen-splash phenomenon is directly ignored in a video transmission scene with a high requirement on the real-time performance, and the screen-splash problem cannot be effectively solved.
Disclosure of Invention
In order to solve the technical problem, the invention provides a method and a device for repairing a flower screen of a cloud game image.
The invention provides a cloud game image screen-splash repairing method which is applied to a client side and comprises the following steps:
receiving and caching a game picture restoration model sent by a server;
receiving and caching a game coding picture sent by a server;
when a screen-splash picture appears in the process of decoding the game coding picture, obtaining a repair frame of the screen-splash picture according to the game picture repair model and the game coding picture;
and displaying the repair frame to replace the screen splash frame when the screen splash frame appears.
The method also has the following characteristics: when a splash screen picture appears in the process of decoding the game coding picture, obtaining a repair frame of the splash screen picture according to the game picture repair model and the game coding picture, wherein the repair frame comprises:
acquiring a screen frame when the screen frame appears;
inputting the splash screen frame and the game coding picture into the game picture restoration model;
and obtaining a repair frame with a screen splash picture.
The method also has the following characteristics: the method further comprises the following steps:
judging whether a next I frame is received or not from the moment when the screen splash picture appears;
if yes, decoding the game coding picture and displaying;
if not, obtaining a repair frame with a screen-splash picture according to the game picture repair model and the game coding picture.
The invention provides a cloud game image restoration and restoration method which is applied to a server side and comprises the following steps:
sending a game picture restoration model to the client based on the received request;
and sending the game coding picture to the client.
The method also has the following characteristics: the game picture restoration model includes a basic picture restoration model and a scene picture restoration model, and the sending of the game picture restoration model to the client based on the received request includes:
sending a basic picture restoration model to the client based on the received game running request; and the number of the first and second groups,
and sending the scene picture restoration model to the client based on the received game scene switching request.
The method also has the following characteristics: the method further comprises the following steps:
receiving training data, wherein the training data comprises game videos of various scene pictures;
and constructing a game picture restoration model based on deep learning according to the training data.
The method also has the following characteristics: the method further comprises the following steps:
inputting a sample picture video frame and a test video frame obtained after the sample picture video frame is subjected to screen-blooming processing to the game picture restoration model for training;
after the training is finished, the game picture restoration model outputs a test restoration frame;
comparing the test repair frame with the sample picture video frame to obtain a deviation parameter;
and adjusting the game picture restoration model according to the deviation parameter until the deviation parameter is smaller than a preset parameter.
The invention provides a cloud game image screen-splash repairing device which is applied to a client side, and comprises:
the receiving module is used for receiving the game picture restoration model and the game coding picture sent by the server;
the cache module is used for caching the game picture restoration model and the game coding picture;
the processing module is used for obtaining a repair frame of the screen splash picture according to the game picture repair model and the game coding picture when the screen splash picture appears in the process of decoding the game coding picture;
and the display module is used for displaying the repair frame to replace the screen splash frame when the screen splash frame appears.
The device also has the following characteristics: the processing module is specifically configured to:
acquiring a screen frame when the screen frame appears;
inputting the splash screen frame and the game coding picture into the game picture restoration model;
and obtaining a repair frame with a screen splash picture.
The device also has the following characteristics: the processing module is further configured to:
judging whether a next I frame is received or not from the moment when the screen splash picture appears;
if yes, decoding the game coding picture and displaying;
if not, obtaining a repair frame with a screen-splash picture according to the game picture repair model and the game coding picture.
The invention provides a cloud game image screen-splash repairing device which is applied to a server side, and the device comprises:
a receiving unit configured to receive a request;
the sending unit is used for sending the game picture restoration model to the client;
the sending unit is also used for sending the game coding picture to the client.
The device also has the following characteristics: the game picture restoration model includes a basic picture restoration model and a scene picture restoration model, and the receiving unit is specifically configured to:
receiving a game running request and a game scene switching request;
the sending unit is specifically configured to:
and sending the basic picture restoration model and the scene picture restoration model to a client.
The device also has the following characteristics: the device further comprises a training unit, wherein the training unit is specifically configured to:
receiving training data, wherein the training data comprises game videos of various scene pictures;
and constructing a game picture restoration model based on deep learning according to the training data.
The device also has the following characteristics: the training unit is further configured to:
inputting a sample picture video frame and a test video frame obtained after the sample picture video frame is subjected to screen-blooming processing to the game picture restoration model for training;
after the training is finished, the game picture restoration model outputs a test restoration frame;
comparing the test repair frame with the sample picture video frame to obtain a deviation parameter;
and adjusting the game picture restoration model according to the deviation parameter until the deviation parameter is smaller than a preset parameter.
According to the cloud game image screen-splash repairing method and device, when the screen-splash phenomenon is found in the running process of the cloud game, the repairing frame of the screen-splash picture is obtained according to the cached game coded picture and the game picture repairing model, the repairing frame is used for replacing the screen-splash frame when the screen-splash picture appears for displaying, the influence of the screen-splash phenomenon on the display of the game picture is made up, the display effect of the game picture is enhanced, and the visual experience of a user is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate an embodiment of the invention and, together with the description, serve to explain the invention and not to limit the invention. In the drawings:
FIG. 1 is a flow diagram illustrating a cloud game image screensaver method according to an exemplary embodiment;
FIG. 2 is a flow diagram illustrating a cloud game image screensaver method according to an example embodiment;
FIG. 3 is a flow diagram illustrating a cloud game image screensaver method according to an example embodiment;
FIG. 4 is a block diagram illustrating a cloud game image screensaver apparatus according to an exemplary embodiment;
fig. 5 is a block diagram illustrating a cloud game image screensaver apparatus according to an example embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.
When the user conducts the cloud game, the running process of the game is conducted on the server. The server sends the game video pictures to the client, and the client receives the video frame data and then displays the video frame data.
However, in the game video transmission process, due to network fluctuation and other factors, the client receives video frame data and in the decoding and playing process, the problem of incomplete frame data is caused, and the screen splash phenomenon is generated. The screen-splash phenomenon means that images have obvious tearing or dislocation characteristics, and the screen-splash phenomenon may continuously affect multiple frames of images, and the screen-splash phenomenon seriously affects the game experience of users.
In the related art, the cloud game has a high requirement on the real-time performance of the video, so that a video buffer area cannot be set, the client receives and decodes one frame of data to display one frame of data, the incomplete frame of data is processed in a direct decoding and displaying mode, the screen-splash phenomenon is directly ignored in a video transmission scene with a high requirement on the real-time performance, and the screen-splash problem cannot be effectively solved.
In order to solve the problems, the disclosure provides a method for repairing a screen of a cloud game image, when a screen-blooming phenomenon is found in the running process of a cloud game, according to a cached game coded picture and a game picture repairing model, a repairing frame of the screen-blooming picture is obtained, the repairing frame is used for replacing the screen-blooming frame when the screen-blooming picture appears for displaying, the influence of the screen-blooming picture on the display of the game picture is compensated, the display effect of the game picture is enhanced, and the visual experience of a user is improved.
According to an exemplary embodiment, as shown in fig. 1, the present disclosure provides a cloud game image screensaver method, and the method in this embodiment is applied to a client. The client can be a mobile terminal such as a mobile phone and a tablet personal computer, and the user can download the cloud game APP at the client to run the cloud game. The method comprises the following steps:
and S110, receiving and caching the game picture restoration model sent by the server.
Generally, a cloud game includes a basic game screen and a more precise scene screen determined according to a user's selection. And the network condition between the client and the server changes continuously, and the position of the network with poor condition can not be predicted. Therefore, in order to ensure the display effect of the client in the running process of the cloud game, the phenomenon of screen splash is avoided.
In the implementation process, in the game starting stage, a basic picture repairing model fed back by a server side based on a game running request is received and cached. When a user needs to switch game scenes along with the progress of a game, the server side sends a scene picture restoration model corresponding to the game scenes along with the change of the game scenes. And the client receives and caches the scene picture restoration model fed back by the server based on the game scene switching request.
Here, it should be noted that, in this step, the game screen restoration model received by the client from the server is stored in the server in advance. Different game picture restoration models are correspondingly stored for different games running on the server. And when the client runs any game, the server sends a game picture repairing model corresponding to the game running on the client according to the request sent by the client.
And S120, receiving and caching the game coding picture sent by the server.
After the game is run on the server, the client is only responsible for displaying the game picture, and the client needs to continuously receive the game picture from the server so as to ensure the smoothness of picture display in the game process of the user.
Because the game video frames are displayed in the display process of the game video pictures and the number of the video frames is large, the server side transmits the game video frames in a coding mode in the transmission process, namely, the game coding pictures are transmitted. In the process of coding and sending, on one hand, the safety in the sending process can be improved, and on the other hand, the sending speed can be increased. Here, the encoding method of the game encoded picture in this step may be encoding by a conventional method.
Since the server performs encoding processing during transmission, the client can perform display only after decoding the game encoded picture in this step. The encoding and decoding processes of the video frame can be performed by adopting the conventional technical means in the field, and the detailed description is omitted here.
In addition, in the above steps S110 and S120, the client stores the received game picture prediction model and the game video frame in a cache manner, and when the user quits the game on the client, the cached content is cleared, and the memory of the client is not occupied. Meanwhile, as the game progresses, the content cached in the early stage of the game can be covered by the content cached in the later stage of the game, the content of the client side is not occupied too much, the occupation of the content of the client side is reduced while the picture display effect is improved, and the game experience of a user is improved. Of course, if the storage space of the mobile phone of the user is large enough, all game coded pictures received in the game process can be stored, so that more cache resources can be available in the later repair process.
S130, when the screen-splash picture appears in the process of decoding the game coding picture, obtaining a repair frame of the screen-splash picture according to the game picture repair model and the game coding picture.
The client side can judge whether a screen-splash picture appears or not in the process of decoding the game coding picture sent by the server side, and if the screen-splash picture appears, the user can see the screen-splash when the screen-splash picture is displayed on the client side subsequently, so that the game experience of the user is influenced. In order to improve the display effect, the game experience is improved. When the screen splash phenomenon occurs, the screen splash picture is repaired according to the game picture repair model received from the server and the cached game coding picture, and a repaired display frame, namely a repaired frame, of the screen splash picture is obtained. In the repairing process, the game coded pictures with the preset frame number before the moment when the screen splash picture appears can be used for repairing, and all the received game coded pictures can also be used for repairing.
For video frames, in the H264 protocol, pictures are organized in groups (GOP, i.e. a sequence), a group being a segment of a coded data stream of pictures, starting with an I-frame and ending with the next I-frame. A group is a series of data streams generated by encoding a picture with a less significant difference in content. When the motion change is less, a group can be long, because the motion change is less, the content change of the image picture is small, so that an I frame can be coded, and then P frames and B frames are always coded. When the motion changes much, it is possible that a sequence is shorter, e.g. containing one I-frame and 3, 4P-frames.
Where the I-frame is the first frame of a group of frames, there is only one I-frame in a group. The I frame is an intra-frame coded frame, is an independent frame with all information, can be independently decoded without referring to other images, and can be simply understood as a static picture. If I is really lost during transmission, the most direct effect of the picture is picture sticking because the following frames cannot be decoded correctly and only wait for the next GOP.
A P frame is an inter-prediction coded frame that requires reference to its previous I or P frame for coding. P frames have no complete picture data, only data that is different from the picture of its previous reference frame. P-frames generally occupy fewer data bits than I-frames, but have the disadvantage that, due to their complex dependency on previous P-and I-reference frames, P-frames are very sensitive to transmission errors, so if a P-frame is lost, the picture will be mosaiced because the forward reference frame is erroneous and the data after the true motion change is filled in.
The B frame is a bidirectional predictive coding frame, that is, the B frame records the difference between the current frame and the previous and next frames. That is, to decode a B frame, not only the previous buffer picture but also the decoded picture are acquired, and the final picture is acquired by superimposing the previous and subsequent pictures on the data of the current frame. The compression rate of the B frame is high, but CPU resources are consumed during decoding. The B frame takes the former I frame or P frame and the latter P frame as reference frames, and the B frame is not a reference frame and can not cause the diffusion of decoding errors. B frames are rarely used in the live broadcast system, because the decoding consumes CPU, and the decoding of B frames needs to wait for the next P frame data, which causes decoding delay, and the live broadcast system has high delay requirement, so B frames are not generally used. But this is not the case for on-demand systems.
In the decoding process of the game coded picture in the embodiment, if the phenomenon of screen splash is found, one group of pictures sequence, namely GOP, is completely replaced by the repaired repair frame (including a plurality of I frames, P frames and B frames), and only one part of the repair frame is replaced, so that the phenomenon of screen splash is avoided in the display process.
In the step, the method for the client to determine whether the screen splash occurs is to determine whether the screen splash occurs according to the conditions of the I frame, the P frame and the B frame at the decoding position in the decoding process, for example, when the P frame is lost in the decoding process or the I frame is lost, the screen splash phenomenon can be determined. Here, it should be noted that, as for the method for determining the screen splash phenomenon, those skilled in the art know, and are not described again, except for the above-mentioned various methods, other problems that may occur may cause screen splash, and are not described again.
In performing this step, in decoding the game coded picture, a flower frame at the time of occurrence of the flower frame is first acquired. And then inputting the screen-blooming frame and the cached game coded picture into a game picture restoration model, wherein the game picture restoration model is obtained by repeatedly training a neural network, so that the game picture restoration model has better prediction accuracy, and can quickly determine and output the restored restoration frame when the screen-blooming picture appears.
And S140, displaying the repair frame to replace the screen splash frame when the screen splash frame appears.
When the splash screen is determined to appear in the decoding process, in order to avoid the splash screen from being displayed to the user, the repair frame repaired by the game screen repair model is used for replacing the splash screen frame to be displayed.
Different models can be used for prediction in different stages of game running, for example, in the initial stage of game running, when a user does not enter a game scene, a basic picture repairing model can be used for repairing a screenful picture appearing in the initial stage of game. When the user switches the scene according to the game, the scene picture restoration model can be used for restoring the flower screen picture.
Here, it should be noted that, because a plurality of game scenes may exist in a game under a normal condition, the method in this embodiment may store a plurality of scene picture prediction models according to the plurality of game scenes, when a client requests a certain game scene, the server sends a scene picture restoration model corresponding to the game scene to the client, and if the client does not request a certain game scene, the scene picture restoration model corresponding to the scene is not sent to the client, so that the unused restoration model is prevented from occupying a storage space of the client, and at the same time, traffic occupation is also reduced. According to the method in the embodiment, the scene picture restoration model is loaded according to the requirements of the client, so that the prediction precision is improved, the size of the model is reduced, and the user experience is improved.
For the cloud game, each game has its own style, and various game pictures consistent and consistent with the game style in the embodiment can be understood as basic pictures, and when the screen splash phenomenon occurs in the decoding process of the basic pictures, the basic pictures can be repaired by using a basic picture repairing model. The scene picture restoration model in the present embodiment may be understood as a game scene that does not occur frequently, the image picture display process may follow regularly, and the picture display content may be predicted, for example, a setting interface or the like that occurs during the game process. When the screen splash phenomenon occurs in the scene, the scene picture restoration model is used for restoring the image video frame to obtain the restored frame, so that the problem of picture loss is solved, and the user experience is improved.
In this embodiment, to further improve the display effect, it is determined whether the next I frame is received from the moment when the screen splash occurs, and if so, the received game coded picture is decoded and appears as yes. If the I frame is not received, it indicates that the screen is still in the splash screen state, and at this time, the above steps S130 and S140 are repeatedly performed until the next I frame is received.
According to the cloud game image screen-splash repairing method in the embodiment, the screen-splash phenomenon is not ignored any more, when the screen-splash phenomenon is found in the decoding process, the screen-splash frame in the screen-splash frame is repaired to obtain the repairing frame according to the cached game coded picture and the game picture repairing model obtained from the server side, the screen-splash phenomenon is avoided, and the visual display effect in the game process is better.
Meanwhile, the game picture restoration model in the embodiment is received from the server, and the client needs to use the cached game coded picture and the model to perform forward propagation calculation, so that the calculation speed is higher, and the deployment scene of the mobile terminal equipment can be met.
According to an exemplary embodiment, as shown in fig. 2, the present embodiment provides a method for repairing a cloud game image screensaver, which is applied to a server, for example, a server running a cloud game. The method in this embodiment comprises the steps of:
s210, based on the received request, the game picture restoration model is sent to the client.
In the step, the server side can send the game picture restoration model to the client side only when the server side receives the request sent by the client side, and the server side can not send the game picture restoration model to the client side when the server side does not receive the request sent by the client side, so that the storage resource and the processing resource of the client side are prevented from being occupied.
Here, it should be noted that the game screen repair model includes a basic screen repair model and a scene screen repair model, and the repair mode sent by the server to the client is different for different received requests.
In one example, the server receives a game running request from the client, and at this time, the client does not enter the scene switching process, and the server sends the basic picture restoration model to the client.
In another example, as the game progresses, the client sends a game scene switching request, and the server sends a scene screen restoration model corresponding to the game scene switching request sent by the client to the client.
In this embodiment, the server sends the repair model to the client according to the game running stage and the request sent by the client, and for the scene picture repair model that the client does not relate to, the server does not send the repair model to the client, thereby reducing the occupation of client resources.
The game picture restoration model involved in the step is trained in advance and is pre-stored in the server. When the game picture restoration model is trained, the server receives training data, wherein the training data comprises game videos of various scene pictures. And constructing a game picture restoration model based on deep learning according to the received training data.
The game is run on the server, so that the training data is convenient to obtain, and the cost is low. When training data is acquired, game videos of different game scene pictures are acquired as original data for training. Meanwhile, when the game picture restoration model is sent to the client terminal in the later period, the basic picture restoration model and the scene picture restoration model are distinguished, so that the training is also distinguished in the training process.
In order to further improve the repairing accuracy of the trained game picture repairing model, after the preliminary training of the game picture repairing model is completed, model rectification processing is required. In the process of correcting the deviation, inputting a sample picture video frame and a test video frame obtained by performing screen-blooming processing on the sample picture video frame to a game picture restoration model for training. And after the training is finished, the game picture restoration model outputs a test restoration frame. And comparing the test restoration frame with the sample picture video frame to obtain a deviation parameter, and adjusting the game picture restoration model according to the deviation parameter until the deviation parameter is smaller than a preset parameter. The preset parameters can be set according to the accuracy required by the game picture restoration model, and the smaller the numerical value of the preset parameters is, the more accurate the trained game picture restoration model is.
The training in this embodiment is unsupervised training (without data labeling), and the training data (i.e., the real video frame) is compared with the predicted frame, and the intrinsic parameters of the repair model to be built are adjusted according to the deviation value. Through the training, the accuracy of the restoration model is continuously adjusted, and finally the accuracy can be sent to the client side so that when the screen splash phenomenon occurs in the decoding process, the screen frame with the screen splash picture is restored by using the game picture restoration model.
And S220, sending the game coding picture to the client.
In the running process of the game, the client does not perform operation processing and the like in various games, and only displays video frames in the running process of the game. Therefore, the server needs to send the encoded video frames to the client.
In the embodiment, hierarchical training is adopted, that is, different scene picture restoration models and game-level basic picture restoration models are trained according to scenes, and then dynamic loading is performed in the process of requesting a game by a subsequent client, so that the prediction precision and the model size are balanced. Meanwhile, due to the adoption of unsupervised training, fine manual labeling is not needed, and the training cost is reduced.
According to an exemplary embodiment, as shown in fig. 3, the present implementation provides a cloud game image screen-splash repairing method, including the following steps:
s301, the client sends a game running request to the server.
S302, the server runs the game.
S303, the server side sends the basic picture restoration model to the client side.
S304, the server side encodes the game picture.
S305, the server sends the coded game picture to the client.
S306, the client sends a game scene switching request to the server.
And S307, the server side sends the scene picture restoration model to the client side.
S308, the client decodes the game coding picture.
S309, a screen splash picture appears in the decoding process of the client.
S310, the client side obtains a splash screen frame when the splash screen picture appears.
S311, the client inputs the screen frame and the cached game coding picture to the game picture restoration model.
S312, the client side obtains the repair frame with the screen splash picture.
And S313, displaying the repair frame to replace the screen splash frame when the screen splash frame appears.
S314, the client judges whether the next I frame is received.
And S315, if the next I frame is received, decoding and displaying the game coding picture.
And S316, if the next I frame is not received, continuously obtaining the repair frame with the screen splash picture and displaying the repair frame.
According to an exemplary embodiment, as shown in fig. 4, the embodiment discloses a cloud game image flower screen restoration device applied to a client for implementing the cloud game image flower screen restoration method shown in fig. 1. The cloud game image screenplay restoration device 100 in this embodiment includes a receiving module 110, a caching module 120, a processing module 130, and a display module 140.
The receiving module 110 is configured to receive the game picture restoration model and the game coded picture sent by the server. The buffer module 120 is used for buffering the game picture restoration model and the game coding picture. The processing module 130 is configured to, when a splash screen occurs in the process of decoding a game coded picture, obtain a repair frame in which the splash screen occurs according to the game picture repair model and the game coded picture, and determine whether a next I frame is received from the moment when the splash screen occurs; if yes, decoding the game coding picture and displaying; if not, obtaining a repair frame with a screen splash picture according to the game picture repair model and the game coding picture. The display module 140 is configured to display the repair frame to replace the flowered frame when the flowered picture appears.
According to an exemplary embodiment, as shown in fig. 5, the embodiment discloses a cloud game image screen-splash repairing device, which is applied to a server and is used for implementing the cloud game image screen-splash repairing method shown in fig. 2. The cloud game image flower screen restoration device 200 in this embodiment includes a receiving unit 210, a transmitting unit 220, and a training unit 230.
The receiving unit 210 is configured to receive requests, such as a game running request and a game scene switching request. The sending unit 220 is configured to send the game screen restoration model to the client; and is also used for sending the game coding pictures to the client. The training unit 230 is specifically configured to receive training data, where the training data includes game videos of various scene pictures, and construct a game picture restoration model based on deep learning according to the training data.
The above-described aspects may be implemented individually or in various combinations, and such variations are within the scope of the present invention.
It will be understood by those skilled in the art that all or part of the steps of the above methods may be implemented by instructing the relevant hardware through a program, and the program may be stored in a computer-readable storage medium, such as a read-only memory, a magnetic or optical disk, and the like. Alternatively, all or part of the steps of the foregoing embodiments may also be implemented by using one or more integrated circuits, and accordingly, each module/unit in the foregoing embodiments may be implemented in the form of hardware, and may also be implemented in the form of a software functional module. The present invention is not limited to any specific form of combination of hardware and software.
It is to be noted that, in this document, the terms "comprises", "comprising" or any other variation thereof are intended to cover a non-exclusive inclusion, so that an article or apparatus including a series of elements includes not only those elements but also other elements not explicitly listed or inherent to such article or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of additional like elements in the article or device comprising the element.
The above embodiments are merely to illustrate the technical solutions of the present invention and not to limit the present invention, and the present invention has been described in detail with reference to the preferred embodiments. It will be understood by those skilled in the art that various modifications and equivalent arrangements may be made without departing from the spirit and scope of the present invention and it should be understood that the present invention is to be covered by the appended claims.
Claims (14)
1. A cloud game image screen-splash repairing method is applied to a client side and is characterized by comprising the following steps:
receiving and caching a game picture restoration model sent by a server;
receiving and caching a game coding picture sent by a server;
when a screen-splash picture appears in the process of decoding the game coding picture, obtaining a repair frame of the screen-splash picture according to the game picture repair model and the game coding picture;
and displaying the repair frame to replace the screen splash frame when the screen splash frame appears.
2. The cloud game image screen splash repairing method according to claim 1, wherein when a screen splash appears in the process of decoding the game coded picture, obtaining a repairing frame in which the screen splash appears according to the game picture repairing model and the game coded picture, comprises:
acquiring a screen frame when the screen frame appears;
inputting the splash screen frame and the game coding picture into the game picture restoration model;
and obtaining a repair frame with a screen splash picture.
3. The cloud game image screenplay repair method of claim 1, wherein the method further comprises:
judging whether a next I frame is received or not from the moment when the screen splash picture appears;
if yes, decoding the game coding picture and displaying;
if not, obtaining a repair frame with a screen-splash picture according to the game picture repair model and the game coding picture.
4. A cloud game image screen-splash repairing method is applied to a server side and is characterized by comprising the following steps:
sending a game picture restoration model to the client based on the received request;
and sending the game coding picture to the client.
5. The cloud game image screenplay restoration method according to claim 4, wherein the game screen restoration model comprises a base screen restoration model and a scene screen restoration model, and wherein sending the game screen restoration model to the client based on the received request comprises:
sending a basic picture restoration model to the client based on the received game running request; and the number of the first and second groups,
and sending the scene picture restoration model to the client based on the received game scene switching request.
6. The cloud game image screenplay restoration method according to claim 4, wherein the method further comprises:
receiving training data, wherein the training data comprises game videos of various scene pictures;
and constructing a game picture restoration model based on deep learning according to the training data.
7. The cloud game image screenplay restoration method according to claim 4, wherein the method further comprises:
inputting a sample picture video frame and a test video frame obtained after the sample picture video frame is subjected to screen-blooming processing to the game picture restoration model for training;
after the training is finished, the game picture restoration model outputs a test restoration frame;
comparing the test repair frame with the sample picture video frame to obtain a deviation parameter;
and adjusting the game picture restoration model according to the deviation parameter until the deviation parameter is smaller than a preset parameter.
8. The utility model provides a cloud game image flower screen prosthetic devices, is applied to the client, its characterized in that, the device includes:
the receiving module is used for receiving the game picture restoration model and the game coding picture sent by the server;
the cache module is used for caching the game picture restoration model and the game coding picture;
the processing module is used for obtaining a repair frame of the screen splash picture according to the game picture repair model and the game coding picture when the screen splash picture appears in the process of decoding the game coding picture;
and the display module is used for displaying the repair frame to replace the screen splash frame when the screen splash frame appears.
9. The cloud game image screenplay repair device of claim 8, wherein the processing module is specifically configured to:
acquiring a screen frame when the screen frame appears;
inputting the splash screen frame and the game coding picture into the game picture restoration model;
and obtaining a repair frame with a screen splash picture.
10. The cloud game image screenplay repair apparatus of claim 8, wherein the processing module is further configured to:
judging whether a next I frame is received or not from the moment when the screen splash picture appears;
if yes, decoding the game coding picture and displaying;
if not, obtaining a repair frame with a screen-splash picture according to the game picture repair model and the game coding picture.
11. A cloud game image flower screen restoration device is applied to a server side and is characterized by comprising:
a receiving unit configured to receive a request;
the sending unit is used for sending the game picture restoration model to the client;
the sending unit is also used for sending the game coding picture to the client.
12. The cloud game image screenplay restoration device according to claim 11, wherein the game screen restoration model includes a base screen restoration model and a scene screen restoration model, and the receiving unit is specifically configured to:
receiving a game running request and a game scene switching request;
the sending unit is specifically configured to:
and sending the basic picture restoration model and the scene picture restoration model to a client.
13. The cloud game image screenplay repair device of claim 11, wherein the device further comprises a training unit, the training unit being specifically configured to:
receiving training data, wherein the training data comprises game videos of various scene pictures;
and constructing a game picture restoration model based on deep learning according to the training data.
14. The cloud game image screenplay repair device of claim 13, wherein the training unit is further configured to:
inputting a sample picture video frame and a test video frame obtained after the sample picture video frame is subjected to screen-blooming processing to the game picture restoration model for training;
after the training is finished, the game picture restoration model outputs a test restoration frame;
comparing the test repair frame with the sample picture video frame to obtain a deviation parameter;
and adjusting the game picture restoration model according to the deviation parameter until the deviation parameter is smaller than a preset parameter.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011642578.9A CN112717389A (en) | 2020-12-31 | 2020-12-31 | Cloud game image screen-splash repairing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011642578.9A CN112717389A (en) | 2020-12-31 | 2020-12-31 | Cloud game image screen-splash repairing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112717389A true CN112717389A (en) | 2021-04-30 |
Family
ID=75609166
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011642578.9A Pending CN112717389A (en) | 2020-12-31 | 2020-12-31 | Cloud game image screen-splash repairing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112717389A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114595021A (en) * | 2022-03-10 | 2022-06-07 | Oppo广东移动通信有限公司 | Method and device for repairing splash screen, electronic equipment, chip and storage medium |
| CN115187490A (en) * | 2022-09-07 | 2022-10-14 | 常州海图信息科技股份有限公司 | Video and graphics processing system and method |
| WO2024072797A1 (en) * | 2022-09-29 | 2024-04-04 | Nvidia Corporation | Improved frame selection for streaming applications |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108900831A (en) * | 2018-05-25 | 2018-11-27 | 烽火通信科技股份有限公司 | Flower screen event detecting method and its detection system |
| CN110225348A (en) * | 2019-06-24 | 2019-09-10 | 北京大米科技有限公司 | Restorative procedure, device, electronic equipment and the storage medium of video data |
| CN111681167A (en) * | 2020-06-03 | 2020-09-18 | 腾讯科技（深圳）有限公司 | Image quality adjusting method and device, storage medium and electronic equipment |
| CN112001854A (en) * | 2020-07-14 | 2020-11-27 | 浙江大华技术股份有限公司 | Method for repairing coded image and related system and device |
-
2020
- 2020-12-31 CN CN202011642578.9A patent/CN112717389A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108900831A (en) * | 2018-05-25 | 2018-11-27 | 烽火通信科技股份有限公司 | Flower screen event detecting method and its detection system |
| CN110225348A (en) * | 2019-06-24 | 2019-09-10 | 北京大米科技有限公司 | Restorative procedure, device, electronic equipment and the storage medium of video data |
| CN111681167A (en) * | 2020-06-03 | 2020-09-18 | 腾讯科技（深圳）有限公司 | Image quality adjusting method and device, storage medium and electronic equipment |
| CN112001854A (en) * | 2020-07-14 | 2020-11-27 | 浙江大华技术股份有限公司 | Method for repairing coded image and related system and device |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114595021A (en) * | 2022-03-10 | 2022-06-07 | Oppo广东移动通信有限公司 | Method and device for repairing splash screen, electronic equipment, chip and storage medium |
| CN114595021B (en) * | 2022-03-10 | 2023-12-12 | Oppo广东移动通信有限公司 | Method and device for repairing screen, electronic equipment, chip and storage medium |
| CN115187490A (en) * | 2022-09-07 | 2022-10-14 | 常州海图信息科技股份有限公司 | Video and graphics processing system and method |
| WO2024072797A1 (en) * | 2022-09-29 | 2024-04-04 | Nvidia Corporation | Improved frame selection for streaming applications |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112717389A (en) | Cloud game image screen-splash repairing method and device | |
| CN109618179B (en) | Rapid play starting method and device for ultra-high definition video live broadcast | |
| US20220232222A1 (en) | Video data processing method and apparatus, and storage medium | |
| CN112822503B (en) | Method, device and equipment for playing live video stream and storage medium | |
| CN110582012B (en) | Video switching method, video processing device and storage medium | |
| CN112533059B (en) | Image rendering method and device, electronic equipment and storage medium | |
| CN110636346A (en) | Code rate self-adaptive switching method and device, electronic equipment and storage medium | |
| US11694316B2 (en) | Method and apparatus for determining experience quality of VR multimedia | |
| US11451858B2 (en) | Method and system of processing information flow and method of displaying comment information | |
| CN109688465B (en) | Video enhancement control method and device and electronic equipment | |
| CN113395512B (en) | Stuck detection method and device, stuck detection server and storage medium | |
| CN104320701A (en) | Switching method and device for signal source | |
| CN112839241A (en) | Cloud game image frame loss compensation method and device | |
| CN110636372B (en) | Video decoding method, video playing device, electronic equipment and storage medium | |
| CN109640094B (en) | Video decoding method and device and electronic equipment | |
| CN107004018B (en) | Data processing method and device | |
| CN115225902A (en) | High-resolution VR cloud game solution method based on scatter coding and computer equipment | |
| CN116993572A (en) | Image processing method, device, electronic equipment and storage medium | |
| CN113709574B (en) | Video screenshot method and device, electronic equipment and computer readable storage medium | |
| CN111885417B (en) | VR video playing method, device, equipment and storage medium | |
| CN114268830A (en) | Cloud director synchronization method, device, equipment and storage medium | |
| JP7304419B2 (en) | Transmission device, transmission method and program | |
| CN113905272B (en) | Control method of set top box, electronic equipment and storage medium | |
| CN104703030B (en) | Video encoding method and image coding device | |
| CN114095763B (en) | Video list pushing method and play quality assessment method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |