CN109304031B

CN109304031B - Virtual cloud game platform based on heterogeneous intelligent terminal

Info

Publication number: CN109304031B
Application number: CN201811092449.XA
Authority: CN
Inventors: 邢建川; 韩保祯; 康亮; 丁志新; 刘佳玺; 孟令豪
Original assignee: University of Electronic Science and Technology of China
Current assignee: University of Electronic Science and Technology of China
Priority date: 2018-09-19
Filing date: 2018-09-19
Publication date: 2021-11-02
Anticipated expiration: 2038-09-19
Also published as: CN109304031A

Abstract

The invention discloses a virtual cloud game platform based on heterogeneous intelligent terminals, which is characterized in that a networking game which can only normally run on a PC (personal computer) originally is placed on a server and runs, a real-time interface of the game is sent to an intelligent terminal of a player, and the player operates the intelligent terminal of the player and sends operation information to the server to realize the control of the game. Therefore, the purpose of playing games on intelligent terminals such as smart phones and set-top boxes remotely is achieved, cloud games are made possible, the existing game playing mode is changed, and the game industry is expanded to another field.

Description

Virtual cloud game platform based on heterogeneous intelligent terminal

Technical Field

The invention belongs to the field of computers, and particularly relates to a virtual cloud game platform based on a heterogeneous intelligent terminal.

Background

With the continuous development of computer technology, various online games come out endlessly, so as to provide richer entertainment ways for users. The online game is an entertainment mode that people participate in online games through communication means such as internet or mobile communication. The online game is rapidly becoming the backbone industry of the network culture market due to its convenience and universality, and not only exceeds the traditional culture industry, but also grows at a very fast speed. Therefore, how to better provide the online game service to the player to flourish the network culture market is one of the important demands of the current information technology development.

The networking game not only provides a new economic growth point of the information age culture industry by unique advantages and means, but also enables manufacturers of electronic industries such as mobile phones, personal digital equipment and the like to obtain the added value of products by virtue of game functions by virtue of the attraction of the networking game to people, enables advertisements of media such as networks and the like to obtain the browsing amount for several times, enables websites to be profitable accordingly, and the promotion of related products of the electronic game drives a new service industry and provides a large number of employment posts for society.

The current online game is a large game based on a PC (personal computer) or a mobile phone game, the current game playing mode is monotonous, the android mobile phone can only play the android game, and the PC can only play the PC game, which is very unfavorable for the development of the whole game industry, for a fierce game, multiple versions are developed at the same time to meet the requirements of different intelligent devices, but at the same time, strict requirements are also provided for the hardware requirements of the intelligent devices, for example, the intelligent mobile phone can play a large-scale network game, certain requirements are provided for the memory, processor and the like of mobile phone hardware, otherwise, a user can not obtain good game experience, and meanwhile, if a client provides such a requirement, whether the game which is not suitable for being installed on the intelligent device of the user can be played on the mobile phone, although the same type of games suitable for a handheld terminal can exist, however, some players are keen on one of them, and the game adapted to the PC terminal also has certain requirements for hardware requirements of the PC, and besides these requirements, the players more want to sit to play games on the PC, and so on, which limits the development of the game industry, and although different versions of games can be developed, it will consume a lot of capital investment and time consumption.

Disclosure of Invention

The invention aims to: aiming at the problems, the aim of playing games on intelligent terminals such as smart phones and set-top boxes remotely is achieved, cloud games are possible, the existing game playing mode is changed, and the game industry is promoted to another field.

The invention discloses a virtual cloud game platform based on a heterogeneous intelligent terminal, which comprises a client, a login server, a database server, a game hall server, a game server and a streaming media server, wherein the login server is connected with the database server;

the client is an operation end for a user to play online games, and sends game operation control data to the game server (for example, data transmission of operation control information is realized through a remote operation hook); receiving game data from the streaming media server and storing the game data into a local data receiving buffer area; the client reads the game data from the data buffer area and carries out decoding processing, and carries out image rendering processing after carrying out audio and video synchronization processing on the decoded game data;

the login server is connected with the database server and used for performing login verification on the user on the accessed client and distributing a matched game hall server for the client passing the login verification according to the login address of the client; wherein, the matching mode is as follows: the load of the game hall is minimum and the connection time delay with the client is shortest; the database server stores login information (login account and password) of a user;

the game hall server is used for distributing the game server for the accessed client under the condition of meeting the preset load balance condition and returning the distribution information to the client; after a logging server distributes game hall servers, a client initiates a connection request to the distributed game hall servers to establish connection, then the game hall servers provide real server (game server) positions of various online games for users on the client under the condition of meeting load balance, a plurality of virtual machines (game running virtual servers) are established on the game server, and one virtual machine corresponds to one client;

the client initiates a game request to an allocated game server, and the game server allocates a virtual machine for the requested client to run the networking game software selected by the client; the game server sends the corresponding relation between the client and the virtual machine to the streaming media server in real time; meanwhile, the virtual machine on the game server runs the online game software in real time based on the operation control data from the client;

the streaming media server is used for collecting game data of the online game running on the game server in real time, and the collected game data comprises audio and video data; respectively encoding collected audio and video data in different threads by adopting a multi-thread mode, packaging the encoded video and audio data and transmitting the packaged data to corresponding clients through a real-time message transmission protocol; when the video data is coded, the scheme of coding I frames and P frames is directly adopted for the video data; during transmission, detecting the congestion state of the network in real time, and if the congestion exists, reducing the transmission rate of the code stream; and if the congestion disappears, increasing the transmission rate of the code stream.

In summary, due to the adoption of the technical scheme, the invention has the beneficial effects that:

the invention realizes a cloud game platform based on an intelligent terminal, and solves the technical problem that large-scale network games can be played on different intelligent devices, thereby liberating the intelligent devices of players, namely the players do not need to spend much money to buy a good device for playing a game; meanwhile, the development work of different versions in game development is solved, namely, the game development only needs to develop one version based on one intelligent terminal, so that the game development period and other various maintenance works are reduced, and the game development cost is reduced.

Drawings

FIG. 1 is a schematic diagram of the network architecture of the present invention;

FIG. 2 is a schematic diagram of a server architecture of the present invention;

fig. 3 is a schematic diagram of the overall topology of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the following embodiments and accompanying drawings.

The invention has the main idea that a virtualized cloud game platform is realized by utilizing a streaming media technology, a virtualization technology and a remote control related technology, large-scale networking games (suitable for being installed on a PC terminal) are placed on a virtual server, then the networking games are operated on the virtual server, a real-time interface of the networking games is sent to an intelligent terminal of a player, the player operates the intelligent terminal of the player and sends operation information to the virtual server to realize a game control function, so that the requirements of the player are met, and hardware equipment of the intelligent terminal of the player is greatly liberated. The method is roughly realized by the following steps: firstly, setting a virtualization server for serving a plurality of clients, for example, implementing a virtualization server by using a virtualization technology Xen; then, encoding and packaging the game picture on the server and transmitting data through a Real Time Messaging Protocol (RTMP), wherein in the invention, an open source library ffmpeg (open source computer program for recording, converting digital audio and video and converting the digital audio and video into stream) is preferably selected to realize encoding and packaging; meanwhile, a corresponding client is arranged and used for receiving data transmitted by the virtual server and performing image rendering, audio and video synchronization and other related work, in the invention, QT (cross-platform C + + graphical user interface application program development framework) is preferably selected for client research and development; in addition, the invention also carries out data transmission and other work on the control information by using the corresponding remote operation hook, and the optimization of a plurality of related technologies is included to realize the real-time effect of the invention.

The invention aims to solve the real-time requirement of the cloud game, which is a relatively emerging application direction at present, in the prior art, the streaming media technology used in a remote control and live broadcast system is solved, the real-time requirement is not high, generally 500ms delay is acceptable, but in the game industry, the delay can only give a relatively good game experience to a player under the condition of less than 100ms, so the invention reduces the transmission delay of the whole streaming media to be within the range of 100ms by optimizing the related technology, and the whole cloud game platform can be applied to the environment of the cloud game and can generate good effect. The specific optimization scheme of the invention comprises the following contents:

(1) firstly, for the selection of stream media transmission protocol, according to the research of related scheme and comparison of various Real-Time stream media transmission protocols, the invention adopts RTMP protocol as the transmission protocol of the invention, although RTSP (Real Time Streaming protocol) or RTP (Real-Time Transport protocol) protocol can also complete the transmission of stream media data and can also achieve good Real-Time, but the distribution condition of domestic current CDN (content distribution network) network and the supporting effect of RTMP protocol are better than other two protocols, and the transmission layer protocol of RTMP is realized by TCP, which is a reliable transmission protocol, so the occurrence of packet loss condition can be avoided by using RTMP protocol, in addition, a network self-adapting algorithm is added in the invention, by detecting network, if the packet loss phenomenon occurs, the transmission rate of code stream is reduced by parameter setting, and the compression ratio of the image is increased along with the compression ratio of the image by setting parameters so as to reduce the resolution ratio of the image, thus reducing the transmission quantity of data and avoiding the generation of congestion, thereby improving the transmission rate and the display rate of the image and reducing delay. The problems of jamming, screen splash and the like in the streaming media transmission process are solved through the several ways, and a smooth game environment is provided for the player.

(2) And (4) synchronizing the coding and decoding of the audio and video. If the programming is done serially, throughout the implementation of the scheme, the entire program will have many cycles, in addition, in the cycle of encoding and decoding, a long waiting time often appears, and the image encoding and decoding is a very time-consuming operation originally, therefore, in order to reduce the delay overhead of the whole cloud game platform in the same block, the cloud game platform of the invention firstly distinguishes the coding and decoding of the image from the coding and decoding of the audio in a multithreading mode, the operation is carried out in different threads, so that the coding and decoding operation of the audio and video can be carried out simultaneously, the waiting time is greatly reduced, in order to improve the encoding efficiency of images, a GPU (graphics processor) is used on a server, the image coding is accelerated, so that the delay of the block can be reduced, and the delay of coding one frame of image only needs about 15ms through running tests.

(3) Setting for data caching. The time delay problem of the whole cloud game platform can be influenced by the arrangement of the data buffer area of the sending terminal and the receiving buffer area of the client terminal, because if the data buffer area of the sending terminal is set to be too small, the data storage is incomplete or the data in the buffer area is full and cannot be stored, if the sending buffer is set too large, the problems of too long data sending waiting time or data packet sticking and the like can be caused, therefore, in order to solve the problems and improve the real-time performance of the whole cloud game platform, the method of the invention is to arrange no data cache region at the server side, after the audio and video data are coded, the data are transmitted in time by using an RTMP protocol, a corresponding data receiving buffer area is arranged on the client, and the client reads the data in the buffer area to perform corresponding operations such as decoding, audio and video synchronization and the like, so that the data transmission efficiency is improved to a certain extent.

(4) In the technology of image coding and decoding, the invention adopts relatively mature H.264, in the coding process, the realization mechanism of the bottom layer is the technology of combining intra-frame prediction and inter-frame prediction, and a plurality of images are divided into I, P and B frames, wherein the I frame is a main frame, also called intra-frame coding frame, which is an independent frame with all information, and can independently decode without referring to other images; the P frame is also called an inter-frame predictive coding frame, and represents the difference between the current frame picture and the previous frame (the previous frame may be an I frame or a P frame); the B frame is also called bidirectional prediction frame, that is, the B frame records the difference between the frame and the previous and subsequent frames, and the implementation method mainly adopts forward and backward prediction, but the implementation method can generate great coding delay, so in order to improve the real-time property, the invention abandons the B frame coding technology during coding and decoding, and directly adopts the scheme of I frame and P frame coding.

Referring to fig. 1, the virtual cloud game platform based on the heterogeneous intelligent terminal is divided into three-layer network architecture: a presentation layer, a business logic layer and a data access layer; the user realizes the operation of the player of the online game through the client terminal arranged by the invention, the UI (user interface) design of the client terminal and the image picture of the game played by the server terminal are mainly designed on the presentation layer, the subsystem development of the client terminal and each module of the server is mainly included on the service logic layer, and the data access layer is mainly used for storing the relevant data such as the player account number and the like.

Referring to fig. 2, the virtual server of the present invention is designed for a distributed system, when a client is connected, an account is logged in first, and at this time, the client is connected to a login server, which is connected to a database server and is responsible for tasks such as querying whether a login account password is correct, whether a registered account is valid, and the like, and the login server is also responsible for allocating a game hall server to serve the game hall server according to an address where the client logs in, so that the load of the game hall is minimized, and the connection delay with the client is minimized.

The game hall server is equivalent to a foreground machine, and the server is responsible for providing the positions of real servers where various games are located and providing various load balancing algorithms to ensure the load balancing of the subsequent game servers.

The game server is a machine for really running various games, is responsible for running various games, and transmits audio and video information in real time through the streaming media server. The game server adopts a virtual technology, allocates a virtual machine for each accessed client to run the selected game, and sends the corresponding relation between the client and the virtual machine to the streaming media server so that the streaming media server can transmit the corresponding game running data (pictures and sound effects) to the corresponding client in real time; meanwhile, the game server receives the operation control information sent by the client, and the virtual server completes various operations on the game based on the received operation control information.

In order to ensure the continuity of data transmission, the cloud game platform is provided with a streaming media server which is responsible for the transmission and the cache of special audio and video data.

The whole topological structure chart is shown in fig. 3, and mainly relates to development of a client and development and construction work related to a server, and the main method comprises the following steps:

(1) the method comprises the steps of building a virtualization server, wherein the building work of the virtualization server mainly uses a Xen virtualization technology, building a plurality of virtual servers on one physical machine, building a corresponding game platform on each virtual server, and installing corresponding game application software in the game platform.

(2) And (4) data acquisition and transmission of the game platform. The module is an important link in the development of the cloud game platform, is equivalent to an S module in a CS model developed by the invention, and is mainly responsible for intercepting game pictures and audio, then coding, compressing and packaging data of the game pictures and sound, and then transmitting the data by using a real-time transmission protocol RTMP. The module is mainly developed by adopting an ffmpeg open source library, then related technologies are optimized to improve the real-time performance, the related technologies of multithreading are utilized to improve the coding and decoding efficiency, and a real-time transmission protocol is adopted to transmit related data.

(3) And (5) developing a client program. The client program is developed by utilizing the QT to perform related technologies, and mainly comprises the steps of receiving data by utilizing a real-time transmission protocol, decoding, caching and synchronizing the data, and finally performing image rendering by utilizing the image rendering technology of the QT.

The invention is mainly how to reduce the delay to meet the requirement of the whole system, based on the relevant technology used in the invention, the work in the aspect of system optimization is carried out, the specific optimization comprises the selection of the coding and decoding modes in the coding and decoding technology, the optimization of relevant parameters in the coding and decoding technology, the setting of the aspects of data caching and the like, and the improvement of the coding and decoding efficiency by utilizing the technologies in the aspects of multithreading and the like.

Therefore, different terminals (such as an MAC (media access control) machine (apple computer), a tablet personal computer, a smart phone, a set top box and the like) provided with the client program can be connected to the cloud game platform, real-time game pictures and sound effects are obtained through the streaming media server, and the client sends operation control information of a user on the game to the corresponding game server so as to obtain good online game experience.

While the invention has been described with reference to specific embodiments, any feature disclosed in this specification may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise; all of the disclosed features, or all of the method or process steps, may be combined in any combination, except mutually exclusive features and/or steps.

Claims

1. The virtual cloud game platform based on the heterogeneous intelligent terminal is characterized by comprising a client, a login server, a database server, a game hall server, a game server and a streaming media server;

a virtualization server is realized by using a virtualization technology Xen, a plurality of virtual machines are established on the game server, and one virtual machine corresponds to one client; the realization process of the virtualization server is as follows: establishing a plurality of virtualization servers on one physical machine by using a virtualization technology Xen, establishing a corresponding game platform on each virtualization server, and installing corresponding game application software in each game platform;

the client is an operation end for the user to play the online game and sends game operation control data to the game server; receiving game data from a streaming media server through a real-time message transfer protocol (RTMP), wherein the streaming media server is not provided with a data cache region, and transmitting the data to a client in time by adopting the RTMP after the game data is encoded, so that the transmission efficiency of the data is improved; the client is provided with a data receiving buffer area for receiving game data from the streaming media server; the client reads game data from the data buffer and performs decoding processing, and performs image rendering processing on the decoded game data by using an image rendering technology of QT after audio and video synchronization processing, wherein the QT represents a cross-platform C + + graphical user interface application development framework;

the database server is used for storing the login information of the user;

the login server is connected with the database server and used for performing login verification on the user on the accessed client and distributing a matched game hall server for the client passing the login verification according to the login address of the client; wherein, the matching mode is as follows: the load of the game hall is minimum and the connection time delay with the client is shortest;

the game hall server is used for distributing the game server for the accessed client under the condition of meeting the preset load balance condition and returning the distribution information to the client;

the client initiates a game request to the allocated game server, and the allocated game server allocates a virtual machine for the requested client to run the networking game software selected by the client; the game server sends the corresponding relation between the client and the virtual machine to the streaming media server in real time; meanwhile, the virtual machines on the distributed game servers run the online game software in real time based on the operation control data from the client;

the streaming media server is used for collecting game data of the online game running on the game server in real time, and the collected game data comprises audio and video data; respectively encoding collected audio and video data in different threads by adopting a multithreading mode, packaging the encoded video and audio data and transmitting the packaged data to a corresponding client through a real-time message transfer protocol (RTMP); the client side decodes the received audio and video data in different threads in a multi-thread mode;

when a streaming media server carries out video data coding, the streaming media server uses a GPU to accelerate the video data coding, and the scheme of I frame and P frame coding is directly adopted for the video data; the I frame is an intra-frame coding frame and is an independent frame with all information, and the I frame can be independently decoded without referring to other images; the P frame is an interframe predictive coding frame and represents the difference between the current frame picture and the previous frame; when the streaming media server transmits game data, the congestion state of a network is detected in real time, if the congestion exists, the transmission rate of a code stream is reduced, and the compression ratio of an image is increased along with the congestion state by setting parameters during video data encoding so as to reduce the resolution of the image; if the congestion disappears, the transmission rate of the code stream is increased by an addition factor, and the compression ratio of the image is reduced to increase the resolution of the image when the video data is encoded.