WO2023125875A1

WO2023125875A1 - Correlation-based streaming method for game data

Info

Publication number: WO2023125875A1
Application number: PCT/CN2022/143652
Authority: WO
Inventors: Lei Xu; Wenbo Jin; Qingpeng ZHAO
Original assignee: Lei Xu
Priority date: 2021-12-30
Filing date: 2022-12-29
Publication date: 2023-07-06
Also published as: CN114356566B; CN114356566A

Abstract

A correlation-based streaming method for game data is provided. A sub-system for scanning associations between game logic and game files, that is, a plug-in of a Unity game engine, is established, to scan and record associations of multiple levels in a game, and establish an association model required for streaming. The method does not influence a game development process, and modeling for game streaming can be completed by only scanning correlations of game data before a game is released, to provide game services in the form of streaming media. Moreover, the method can be universally used in various game engines and various game platforms. The method overcomes the shortcomings of the prior art, greatly reduces capacity limits in game development, and greatly reduce the difficulty of game development.

Description

CORRELATION-BASED STREAMING METHOD FOR GAME DATA

TECHNICAL FIELD

The present invention relates to the field of game data technologies, and specifically, to a correlation-based streaming method for game data.

BACKGROUND

With the rapid development of computer technologies and the development of image processing technologies, the capacities of games explosively increase. The capacities of some large-scale games often exceed 200 GB, and the capacities of some mobile games also exceed 10 GB. More importantly, with the advent of open world games and the metaverse, the growth rate of the game capacity become higher. Therefore, on the conditions of existing distribution technologies (offline storage media such as compact discs, hard disks, and flash memories, and online download and installation) , distribution, installation, and storage of the games with large capacities become a problem. Hence, how to reduce acquisition costs and acquisition times of users becomes a problem to be resolved urgently.

Therefore, if access (mostly in a random access mode, but also in a sequential access mode) to game data can be serialized, data access can be served in a manner similar to the streaming media, thereby perfectly resolving the problems in game distribution and access.

SUMMARY OF THE INVENTION

The technical problem to be resolved by the present invention is to overcome the shortcomings of the prior art and provide a correlation-based streaming method for game data, to greatly reduce capacity limits in game development and greatly reduce the difficulty of game development.

To resolve the foregoing technical problems, the present invention provides the following technical solutions: A correlation-based streaming method for game data includes: establishing a sub-system for scanning associations between game logic and game files, that is, a plug-in of a Unity game engine, to scan and record associations of multiple levels in the game, and establish an association model required for streaming, where an association scanning process is as follows:

1) scanning dependencies between various scenes of the game and recording the dependencies, where the dependencies are manually modifiable;

2) scanning associations between a scene and resource files in the scene, stopping scanning if less than 10 resource files are referenced in the scene or a total length of resource files is less than 1 MB, recording all resources in the scene as a cluster, and at the same time, recording associations between the resources and the scene; and

3) scanning dependencies between various parts in the scene, and dividing a map in the scene into blocks according to a quadtree method: first dividing the scene into 5*5 sub-map blocks, recording coordinate ranges of the sub-map blocks in a world coordinate system, and then, scanning the sub-map blocks in sequence.

Further, to support streaming, a smallest resource file cluster is limited to 1 MB.

Further, in step 3) , scanning is stopped if less than 10 resource files are referenced in a sub-map block or a total length of resource files is less than 1 MB, a coordinate range of the sub-map block is recorded, and all resources in the sub-map block are recoded as a cluster, and at the same time, associations between the resources and the sub-map block are recorded; otherwise, the sub-map block is divided into 2*2 sub-map blocks, and the sub-map blocks are scanned and split in sequence until less than 10 resources files are referenced in all the sub-map blocks or a total length of resource files is less than 1 MB.

Further, a game data stream server sub-system is responsible for processing client resource prediction and a download request: mapping the resource files into a final packaging or file system structure, and recording correspondences; processing a resource prediction request from a client, where the client transmits a screen resolution, a list of recently accessed resource files, and user input sequence information, and for a game that has accessed a runtime SDK, the client further transmits information about a current scene, world coordinates, and a camera viewing angle; and processing a download request from a user: returning, in a compressed format, a file or a part of a file requested by the user.

Further, to calculate the current scene, the resource prediction process performed by the server includes:

1) calculating, if there is no scene information in the request from the client, a current possible scene according to a list of recently used resource files in the request with reference to correspondences between the scene and the resource files;

2) directly starting to calculate usage probability scores of the resource files if less than 10 resource files are referenced in the scene or a total length of resource files is less than 1 MB; otherwise, calculating a current sub-map block;

3) calculating the current sub-map block;

4) calculating resource files possibly needed in a next stage;

5) performing scoring according to probabilities that resources are used next: calculating scores of the resource files based on numbers of times that the resource files are referenced, distances from current coordinates, an angle with a camera viewing angle, and scoring information of other files in the cluster; and

6) returning predicted resource files and scores to the client.

Further, when the current sub-map block is calculated, if the request sent by the client does not include current world coordinates of the game, a current possible sub-map block is calculated according to the list of recently used resource files in the request with reference to associations between files and sub-map blocks output by the association scanning sub-system and a current scene. If the request sent by the client includes the current world coordinates, the current sub-map block is calculated with reference to the associations between the files and the sub-map blocs output by the association scanning sub-system.

Further, when resource files possibly needed in a next stage are calculated, a resource file candidate set is predicted: calculating a field of view of a user according to a screen resolution of the client and a unit pixel setting in the game, and then using the current sub-map block as a center to find a list of sub-map blocks within a range of 4 times a size of the field of view; finding, if there is information about a camera viewing angle, all sub-map blocs distant from the current coordinates by 4 times a height of the field of view along the camera viewing angle, and adding the sub-map blocks into the list of sub-map blocks; and then, searching for resource files corresponding to the sub-map blocks.

Further, the association model generated by the association scanning sub-system is packaged into a game program or the game runtime and is distributed to the client, and the client SDK predicts, based on a current context of the game, including information about a scene, world coordinates, and the camera viewing angle, resources to be used next, and then requests the corresponding resources from the server.

The present invention has the following advantages: With the support of the runtime, in the present invention, a sub-system for scanning associations between game logic and game files (specifically, a plug-in of a Unity game engine) is designed and implemented, to scan and record associations of multiple levels in a game, and establish an association model required for streaming.

The present invention does not influence a game development process, and modeling for game streaming can be completed by only scanning game data before a game is released, to provide game services in the form of streaming media. Moreover, the present invention can be universally used in various game engines and various game platforms.

The modeling process is based on static scanning, and methods, such as machine learning, are not used. Therefore, there is no need to collect user data, there is no model training process, and direct cold-starting is acceptable.

The distribution problems of games, particularly, large-capacity games, is resolved. With the support of the runtime, users can click and play games without downloading or installing the games.

Compared with the existing cloud game solutions, the present invention does not require server rendering, and greatly lowers traffic costs.

Requirements of games (programs) for storage capacities of clients (a mobile phone or a computer) are greatly reduced. In the past, a 128 GB mobile phone can store only 10 or more large-scale games (programs) . With the support of the present invention, a user can theoretically own an unlimited quantity of games.

The present invention greatly reduces capacity limits in game development, and is transparent to users and developers. A game developer can access a game resource of any capacity in a manner of accessing a local file. The present invention naturally supports an open-ended game, such as a metaverse game, and also greatly reduces the difficulty of game development (in the past, for dynamic resource loading, a game developer has to add resource downloading and loading logic to the game logic, which is no longer required because all resources are "local files" ) .

The present invention also supports streaming of an interactive video (AR/VR or the like) .

The present invention integrates access interfaces of cloud data and local data, to greatly lower the technical threshold of the cloud storage + local computation mode that only professional programmers can implement in the past, which really revitalizes cloud storage and makes the cloud storage more convenient.

In the present invention, the association model may also be stored in the client, and the client is responsible for predicting a next needed resource according to the context of the game during running, which can further reduce the network latency and the computing costs of the server.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic flowchart of association scanning according to the present invention;

FIG. 2 is a schematic flowchart of resource prediction performed on a server according to the present invention; and

FIG. 3 is a schematic flowchart of data access of a client runtime according to the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

The present invention is further described below in detail with reference to embodiments.

Concepts:

Game data refers to data, such as program code, textures, models, audio, and videos, used during running of a game distributed by a game maker, is usually released in the form of an installation package or an ISO, and is a directory and a file or in the form of an ISO package during running of the game.

Game file: A game file is a file form of game data in a non-packaged state, for example, a single texture file or an audio file.

Scene: A scene is a unit of a game plot. For example, a level of a level game may be a scene. For an open world game, there is no obvious scene division, but the open world game can be divided into several logical scenes according to locations in a map.

Resource: A resource refers to data, such as textures, models, audio, videos, and code, used in a game.

During development of a game, the company finds that although the game has a large capacity and most of access of the game to data is random access, the data used during running of the game has locality and associativity.

Locality is that at any moment, the game uses only a small part of data, and the data usually shows a specific degree of aggregation.

Associativity is that due to the certainty of game logic, data and a game scene, as well as data and data, are associated with each other.

During specific implementation, with the support of the runtime, in the present invention, a sub-system for scanning associations between game logic and game files (specifically, a plug-in of a Unity game engine) is designed and implemented, to scan and record associations of multiple levels in a game, and establish an association model required for streaming. To support streaming, a smallest resource file cluster is limited to 1 MB. The process is as follows:

1) Scan dependencies between various scenes of the game and record the dependencies, where the dependencies are manually modifiable.

2) Scan associations between a scene and resource files in the scene, stop scanning if less than 10 resource files are referenced in the scene or a total length of resource files is less than 1 MB, record all resources in the scene as a cluster, and at the same time, record associations between the resources and the scene.

3) Scan dependencies between various parts in the scene, and divide a map in the scene into blocks according to a quadtree method. Specifically, the scene is first divided into 5*5 sub-map blocks, coordinate ranges of the sub-map blocks in a world coordinate system are recorded, and then, the sub-map blocks are scanned in sequence.

Scanning is stopped if less than 10 resource files are referenced in a sub-map block or a total length of resource files is less than 1 MB, a coordinate range of the sub-map block is recorded, and all resources in the sub-map block are recoded as a cluster, and at the same time, associations between the resources and the sub-map block are recorded.

Otherwise, the sub-map block is divided into 2*2 sub-map blocks, and the sub-map blocks are scanned and split in sequence until less than 10 resources files are referenced in all the sub-map blocks or a total length of resource files is less than 1 MB.

A game data stream server sub-system is responsible for processing client resource prediction and a download request, and its functions are as follows:

mapping the resource files into a final packaging (or file system) structure, and recording correspondences.

A resource prediction request from a client is processed. The client transmits information such as a screen resolution, a list of recently accessed resource files, and user input sequence. For a game that has accessed a runtime SDK of the present invention, the client further transmits information such as a current scene, world coordinates, and a camera viewing angle.

1) The current scene is calculated: calculating, if there is no scene information in the request from the client, a current possible scene according to a list of recently used resource files in the request with reference to correspondences between the scene and the resource files;

2) Calculation of usage probability scores of the resource files is directly started if less than 10 resource files are referenced in the scene or a total length of resource files is less than 1 MB; otherwise, a current sub-map block is calculated.

3) The current sub-map block is calculated:

calculating, if the request sent by the client does not include the current world coordinates of the game, a possible sub-map block according to the list of recently used resource files in the request with reference to the associations between the files and the sub-map blocks output by the association scanning sub-system and the current scene; and

calculating, if the request sent by the client includes the current world coordinates, a current sub-map block with reference to the associations between the files and the sub-map blocks output by the association scanning sub-system.

(4) Resource files possibly needed in a next stage are calculated:

predicting a resource file candidate set: calculating a field of view of a user according to a screen resolution of the user and a unit pixel setting in the game, and then using the current sub-map block as a center to find a list of sub-map blocks within a range of 4 times a size of the field of view; finding, if there is information about a camera viewing angle, all sub-map blocs distant from the current coordinates by 4 times a height of the field of view along the camera viewing angle, and adding the sub-map blocks into the list of sub-map blocks; and then, searching for resource files corresponding to the sub-map blocks

5) Scoring is performed according to probabilities that resources are used next: calculating scores of the resource files based on numbers of times that the resource files are referenced, distances from current coordinates, the camera viewing angle, and scoring information of other files in the cluster.

6) Predicted resource files and scores are returned to the client.

A download request from a user is processed: returning, in a compressed format, a file or a part of a file requested by the user.

In the present invention, a runtime sub-system is established on the client, to take over a file access request during running of the game and map the file access request to access to a local file system or server, to satisfy data access requirements, and is transparent to users and game developers. Therefore, the game can be clicked and played without being downloaded or installed. Its main functions are as follows:

directly returning data for each data request if a local file system hits; otherwise, communicating with the server to request for currently needed data;

A prediction thread in the background communicates with the server periodically, uploads local device information, a list of recently used files, and information such as a user input sequence, and requests the server to predict a resource file list and perform scoring. Information, such as a current scene, world coordinates, and a camera viewing angle, is obtained if the game SDK of the present invention is used in a game, and added to the prediction request.

A download queue is maintained locally, and the download queue is updated when the server returns the predicted resource file list and score information, a weight of an original file is lowered, and a new file list and scores are added to the queue.

A download thread in the background obtains a to-be-downloaded file from the download queue, requests the server for downloading the file if the file does not exist in the local file system, decompresses the file, and writes the file in to the local file system.

To implement more accurate resource prediction, in the present invention, a game SDK is designed, which is specifically, a Unity game plug-in. If the SDK is introduced by the game developer during development, the client runtime sub-system can obtain information, such as a current scene, world coordinates, and a camera viewing angle, through the SDK during running of the game.

The association model generated by the association scanning sub-system in the present invention may also be packaged into a game program or the game runtime and distributed to the client. The client SDK predicts, based on the current context of the game (for example, information such as a scene, world coordinates, and the camera viewing angle) , a resource to be used next, and then requests the corresponding resource from the server.

Starting from game logic, in the present invention, associations between scenes of a game, between scenes and resources, between resources, between executable code and scenes and resources, and between code and code are analyzed, to construct a cluster of multiple levels and dependencies, so as to convert access of the game to data into a streaming manner. Based on this, an association scanning plug-in is implemented in the Unity game engine to scan dependencies between scenes and resources, as well as a clustering relationship between resources, and construct a streaming system for game data, to resolve the distribution problem of large-capacity games. Therefore, games can be clicked and played, and there is no need to wait for downloading.

Although the present invention has been described above in detail by using general descriptions and specific implementations, some modifications or improvements may be made to the present invention based on the present invention, which is obvious to a person skilled in the art. Therefore, the modifications or improvements made without departing from the spirit of the present invention shall all fall within the protection scope of the present invention.

Claims

A correlation-based streaming method for game data, comprising the following steps:

(1) constructing an association scanning sub-system, to scan associations of game data before a game is released, establish an association model required for streaming, and store the association model on a server, wherein the game data refers to program code, texture, model, audio, and video data used in running of a game released by a game maker, is usually released in the form of an installation package or an ISO, and is a directory and a file or in the form of an ISO package during running of the game, wherein

the association scanning sub-system is configured to scan and record associations of multiple levels in the game, and establish the association model required for streaming, wherein an association scanning process is as follows:

(1.1) scanning dependencies between various scenes of the game and recording the dependencies, wherein the dependencies are manually modifiable;

(1.2) scanning associations between a scene and resource files in the scene, stopping scanning if less than 10 resource files are referenced in the scene or a total length of resource files is less than 1 MB, recording all resources in the scene as a cluster, and at the same time, recording associations between the scene and the resources; and

(1.3) scanning dependencies between various parts in the scene, and dividing a map in the scene into blocks according to a quadtree method: first dividing the scene into 2*2 sub-map blocks, recording coordinate ranges of the sub-map blocks in a world coordinate system, and then, scanning the sub-map blocks in sequence; and stopping scanning if less than 10 resource files are referenced in a sub-map block or a total length of resource files is less than 1 MB, recording a coordinate range of the sub-map block, recording all resources in the sub-map block as a cluster, and at the same time, recording associations between the sub-map block and the resources; otherwise, further dividing the sub-map block into 2*2 sub-map blocks, scanning and splitting the sub-map blocks in sequence until less than 10 resources files are referenced in all the sub-map blocks or a total length of resource files is less than 1 MB;

(2) receiving, by a runtime sub-system arranged on a client, a data request from a user during running of the game, and directly returning data for each data request if a local file system on the client hits; otherwise, communicating with the server, to request for currently needed data;

(3) communicating, by the runtime sub-system in the background, with the server periodically, uploading local device information, a list of recently used files, and user input sequence information, and requesting the server to predict a resource file list and perform scoring;

(4) receiving, by the server, the local device information, the list of recently used files, and the user input sequence information transmitted from the client, performing resource file prediction and scoring based on the association model, and returning a predicted resource file list and scores to the client;

(5) updating, by the client, a download queue when the server returns the predicted resource file list and score information, adding a new resource file list and new scores to the download queue, and requesting the server for downloading if the new resource file list and new scores do not exist in a local system; and

(6) returning, by the server, a file requested by the user in a compressed format, and decompressing, by the client, the file and writing the file to the local file system.
The correlation-based streaming method for game data according to claim 1, wherein

in step (4) , the resource prediction and scoring process performed by the server comprises:

(4.1) calculating a current scene according to the list of recently used resource files in the request from the client with reference to corresponding associations between the scene and resource files;

(4.2) directly starting, if less than 10 resource files are referenced in the scene or a total length of resource files is less than 1 MB, to calculate usage probability scores of the resource files; otherwise, calculating a current sub-map block;

(4.3) calculating the current sub-map block; and calculating a current possible sub-map block according to the list of recently used resource files in the request from the client with reference to the associations between the resource files and the sub-map blocks in the association model and the current scene;

(4.4) calculating resource files possibly needed in a next stage;

(4.5) scoring probabilities that the resource files are used next; and

(4.6) returning predicted resource files and scores to the client.
The correlation-based streaming method for game data according to claim 2, wherein

in step (4.4) , the process of calculating resource files possibly needed in a next stage is: calculating a field of view of a user according to a screen resolution of the client and a unit pixel setting in the game, and then using the current sub-map block as a center to find a list of sub-map blocks within a range of 4 times a size of the field of view; adding the list of sub-map blocks into a list of sub-map blocks; and then, searching for resource files corresponding to the sub-map blocks.
The correlation-based streaming method for game data according to claim 2, wherein

in step (4.5) , the usage probability scores of the resource files are calculated based on numbers of times that the resource files are referenced, distances from current coordinates, and scoring information of other files in the cluster;
The correlation-based streaming method for game data according to claim 1, wherein

the association model is packaged into a game program at the same time, and distributed to the client during running of the game; a game SDK is designed on the client; and during running of the game, the runtime sub-system on the client obtains information about a current scene, world coordinates, and a camera viewing angle through the game SDK, and add the foregoing information to a prediction request.
The correlation-based streaming method for game data according to claim 5, wherein

in step (4) , the resource prediction and scoring process performed by the server comprises:

(4.1) receiving a current scene transmitted from the client;

(4.2) directly starting to calculate usage probability scores of the resource files if less than 10 resource files are referenced in the scene or a total length of resource files is less than 1 MB; otherwise, calculating a current sub-map block;

(4.3) calculating the current sub-map block; and calculating the current sub-map block with reference to the associations between the resource files and the sub-map blocks in the association model if a request sent by the client comprises current world coordinates;

(4.4) calculating resource files possibly needed in a next stage;

(4.5) scoring probabilities that the resource files are used next; and

(4.6) returning the predicted resource files and scores to the client.
The correlation-based streaming method for game data according to claim 6, wherein

in step (4.4) , the process of calculating resource files possibly needed in a next stage is: finding, according to information about a camera viewing angle, all sub-map blocs distant from current coordinates by 4 times a height of a field of view along the camera viewing angle, and adding the sub-map blocks into a list of sub-map blocks; and then, searching for resource files corresponding to the sub-map blocks.
The correlation-based streaming method for game data according to claim 6, wherein

in step (4.5) , usage probability scores of the resource files are calculated based on numbers of times that the resource files are referenced, distances from the current coordinates, an angle with the camera viewing angle, and scoring information of other files in the cluster.
A correlation-based streaming system for game data, comprising a client and a server, for implementing the correlation-based streaming method for game data according to any one of claims 1 to 8.