CN109840058A

CN109840058A - Cloud game collecting method and system

Info

Publication number: CN109840058A
Application number: CN201910011128.0A
Authority: CN
Inventors: 左成; 沈琼霞; 姜超; 刘强
Original assignee: Fiberhome Telecommunication Technologies Co Ltd
Current assignee: Fiberhome Telecommunication Technologies Co Ltd
Priority date: 2019-01-07
Filing date: 2019-01-07
Publication date: 2019-06-04

Abstract

The invention discloses a kind of cloud game collecting method and systems, it is related to cloud game field, method includes the following steps: customized DLL is injected into target game process, the customized MyEndScence function for being used to acquire data is inserted into the rendering pipeline of Direct3D；It jumps to target game process in MyEndScence function, the rear end caching BackBuffer of target game is copied in memory from the video memory of video card, to obtain original RGB pixel data, and original RGB pixel data is encoded.Independent of SDK provided by video card manufacturer when cloud game collecting method in the present invention acquires data, and time delay can be reduced, be beneficial to promote user experience.

Description

Cloud game collecting method and system

Technical field

The present invention relates to cloud game fields, and in particular to a kind of cloud game collecting method and system.

Background technique

With the continuous development of internet, economic benefits brought by cloud computing are inestimable.It is worth the phase as cloud computing To one of application, cloud game brings subversiveness to field of play and changes.

Cloud game operational mode refers to that all game are all run in server end, and the game picture compression after having rendered Afterwards by transmission of network to user.In client, the game station of user does not need any high-end processors and video card, is more not required to Download any game content of installation.Client only needs basic video compression ability and good network environment, so that it may Receive good game picture.

For the acquisition of game picture, existing technology relies on the SDK that video card manufacturer provides, and obtains from the video memory of video card Game data, or be output to desktop after game has rendered and then captured based on desktop.The former is overly dependent upon video card Manufacturer, and the latter's capture desktop will increase the time delay of game data transmission.And the experience effect of cloud game to data transmit when Prolong it is more demanding, based on desktop capture game picture mode can seriously affect user experience.

Summary of the invention

In view of the deficiencies in the prior art, the purpose of the present invention is to provide one kind is mentioned independent of video card manufacturer The SDK of confession, and time delay can be reduced, it is beneficial to promote the cloud game collecting method of user experience.

To achieve the above objectives, the technical solution adopted by the present invention is that:

A kind of cloud game collecting method, method includes the following steps:

Customized DLL is injected into target game process, by the customized MyEndScence for being used to acquire data Function is inserted into the rendering pipeline of Direct3D；

It jumps to target game process in MyEndScence function, the rear end of target game is cached into BackBuffer It copies in memory from the video memory of video card, to obtain original RGB pixel data, and original RGB pixel data is carried out Coding.

Based on the above technical solution, customized DLL is injected into target game process, specific steps packet It includes:

The game of process performance objective is created in suspended state；

Apply for memory headroom in target game process, stores the path of customized DLL；

Remote thread is created in target game process, loads customized DLL using remote thread；

Recovery target game process is operating status.

Based on the above technical solution, jump to target game process in MyEndScence function, specific steps Include:

IDirect3DDevice9 interface pointer object is created in customized DLL；

Using the base address of IDirect3DDevice9 interface pointer object, the address of Virtual Function EndScence is found, it will The address of Virtual Function EndScence replaces with a skip command, and specified jump address is customized MyEndScence letter Number address；

Hook Virtual Function EndScence executes Virtual Function EndScence.

Based on the above technical solution, the rear end caching BackBuffer of target game is multiple from the video memory of video card It makes in memory, to obtain original RGB pixel data, specific steps include:

The Surface object of off screen rendering is created in memory；

DirectX interface is called to copy BackBuffer in Surface object to；

Copy RGB pixel data original in Surface object in memory customized cyclic buffer In RINGBUFFER.

Based on the above technical solution, this method further includes judging to obtain whether original RGB pixel data succeeds The step of, it specifically includes:

Call GetRenderTargetData method, judge to copy to BackBuffer in Surface object whether at Function obtains original RGB pixel data failure if it is not, then returning to FAIL；If so, returning to OK, and call Surface object LockRect method, further judge whether the data area of Surface object locks success；If it is not, then returning to FAIL, obtain Original RGB pixel data is taken to fail；If so, returning to OK, original RGB pixel data success is obtained.

At the same time, it is another object of the present invention to provide one kind independent of SDK provided by video card manufacturer, and Time delay can be reduced, is beneficial to promote the cloud game data collection system of user experience.

A kind of cloud game data collection system, comprising:

Injection module is used to for customized DLL being injected into target game process；

It is inserted into module, is used to the customized MyEndScence function for being used to acquire data being inserted into Direct3D Rendering pipeline in；

Data capture module is used to that target game process to be made to jump in MyEndScence function, by target game Rear end caching BackBuffer is copied in memory from the video memory of video card, to obtain original RGB pixel data；And

Coding module is used to encode original RGB pixel data, and is sent in network.

Based on the above technical solution, customized DLL is injected into target game process by the injection module, tool Body process are as follows:

The game of process performance objective is created in suspended state；

Remote thread is created in target game process, loads customized DLL using remote thread.

Based on the above technical solution, the data capture module jumps to target game process In MyEndScence function, detailed process are as follows:

IDirect3DDevice9 interface pointer object is created in customized DLL；

Hook Virtual Function EndScence executes Virtual Function EndScence.

Based on the above technical solution, the rear end of target game is cached BackBuffer by the data capture module It is copied in memory from the video memory of video card, to obtain original RGB pixel data, detailed process are as follows:

The Surface object of off screen rendering is created in memory；

DirectX interface is called to copy BackBuffer in Surface object to；

Based on the above technical solution, the cloud game data collection system further includes judgment module, the judgement Module is used to judge to obtain whether original RGB pixel data succeeds, and judgment module judgement obtains original rgb pixel Whether data succeed, detailed process are as follows:

Compared with the prior art, the advantages of the present invention are as follows:

Cloud game collecting method of the invention, compared to traditional scheme using SDK provided by video card hardware manufacturer come Game data is acquired, is replicated from the video memory of video card in the present invention using by the rear end caching BackBuffer of target game Into memory, to obtain original RGB pixel data.So as to independent of SDK provided by video card manufacturer, as long as The capture of game data can be achieved in the game of DirectX.In addition, game picture is captured based on rear end caching, it can be defeated in picture It is just captured before arriving desktop out, is substantially reduced time delay, be beneficial to promote user experience.

Detailed description of the invention

Fig. 1 is the flow chart of medium cloud of embodiment of the present invention game data acquisition method；

Fig. 2 is the flow chart of the step S1 of medium cloud of embodiment of the present invention game data acquisition method；

Fig. 3 is the flow chart of the step S3 of medium cloud of embodiment of the present invention game data acquisition method；

Fig. 4 is the flow chart of the step S5 of medium cloud of embodiment of the present invention game data acquisition method；

Fig. 5 is the structural block diagram of medium cloud of embodiment of the present invention game data acquisition system.

Specific embodiment

Invention is further described in detail with reference to the accompanying drawings and embodiments.

Embodiment 1:

The present embodiment provides a kind of cloud game collecting methods, method includes the following steps:

Specifically, shown in Figure 1, the above method the specific implementation process is as follows:

S1. customized DLL is injected into target game process, and executed in target game process customized DLL；

S2. the customized MyEndScence function for being used to acquire data is inserted into the rendering pipeline of Direct3D；

Jump to target game process in MyEndScence function, by the interface of DirectX, by target game Rear end caching BackBuffer copied in memory from the video memory of video card, to obtain original RGB pixel data；

S4. judge to obtain whether original RGB pixel data succeeds, if so, executing step S5；If it is not, return step S3, Reacquire original RGB pixel data；

S5. original RGB pixel data is encoded, and be sent in network.

Game data is acquired using SDK provided by video card hardware manufacturer compared to traditional scheme, is used in the present embodiment Be to copy to the rear end of target game caching BackBuffer in memory from the video memory of video card, to obtain original RGB Pixel data.So as to independent of SDK provided by video card manufacturer, as long as the game of DirectX, can be achieved game The capture of data.

In addition, capturing game picture based on rear end caching, it can just capture, be substantially reduced before picture is output to desktop Time delay is beneficial to promote user experience.

Embodiment 2:

It is shown in Figure 2, as a preferable optional way, on the basis of embodiment 1, customized DLL is infused Enter into target game process, and execute customized DLL in target game process, specific steps include:

S11. the game of process performance objective is created in suspended state；

S12. apply for memory headroom in target game process, store the path of customized DLL；

S13. remote thread is created in target game process, loads customized DLL using remote thread；

In the present embodiment, which calls LoadLibrary function to be loaded into the DLL injected, LoadLibrary letter Several effects is module loading that will specify into the address space of calling process.The title of module can be library module (.dll File).

S14. restoring target game process is operating status.

After completing above-mentioned steps, it can realize and execute customized DLL in target game, and then can be in target Customized program in machine code is executed in game process, is inserted into customized code in the rendering process of DirectX, is based on these Code realizes the acquisition of game initial data, coding.

Embodiment 3:

As a preferable optional way, on the basis of embodiment 1, jump to target game process In MyEndScence function, specific steps include:

IDirect3DDevice9 interface pointer object is created in customized DLL；

IDirect3DDevice9 is device pointer interface in Direct3D, is represented hard for the physics that shows 3D rendering Part equipment.The resource of physical hardware devices (GPU) is called to complete the rendering pipeline of image by the pointer interface.

Hook Virtual Function EndScence executes Virtual Function EndScence.

Hook refers to by way of through hook, the message of intercepting system, message of function or some application, letter Number, modifies after interception, or jumps to and execute oneself desired operation in oneself code.It is to use in the present embodiment The mode of API HOOK intercepts the Virtual Function EndScence of IDirect3DDevice9.EndScence function is Direct3D A link in rendering pipeline indicates to be over when the rendering of previous frame image.And the render process of Direct3D for Applications be it is sightless, it is closed, can not intervene.So needing to intercept EndScence, customized generation is then branched to The collecting work to the image rendered is completed in code.After these work completion, then jump back to original EndScence letter Number, executes its own job function and (inserts a customized link, but not shadow in the rendering pipeline process of Direct3D Ring the rendering pipeline process).

It is by searching for EndScence function in IDirect3DDevice9 when executing EndScence function in program Then address in virtual table executes Jump instruction, jumps to the address.After Hook to EndScence function, mean that The address of EndScence has been found, the address of Virtual Function EndScence has then been replaced with into a skip command, and refer to Determine the address that address is customized MyEndScence function.It will be jumped to when program executes EndScence function in this way MyEndScence function.

Embodiment 4:

It is shown in Figure 3, as a preferable optional way, on the basis of embodiment 1, pass through connecing for DirectX Mouthful, the rear end caching BackBuffer of target game is copied in memory from the video memory of video card, to obtain original RGB picture Prime number evidence, specific steps include:

S31. the Surface object of off screen rendering is created in memory；

S32. DirectX interface is called to copy BackBuffer in Surface object to；

S33. RGB pixel data original in Surface object is copied in memory customized cyclic buffer In RINGBUFFER.

Further, shown in Figure 4, original RGB pixel data is encoded with FFMPEG, specific steps packet It includes:

S51. original RGB pixel data is read from RINGBUFFER, and reads the AVFrame structural body of FFMPEG In；

S52. original RGB pixel data is converted into yuv data, at the same carry out picture size scaling (original image with Size that user is locally displayed may be inconsistent, it is therefore desirable to which original image zooms to that device is locally displayed with user is consistent Size)；

S53. the yuv data after conversion is encoded.

After being encoded, it can be sent in network from the data after taking out coding in AVPacket structural body.This reality The yuv data after conversion can be encoded using coding modes such as H264, H265 by applying in example.

Surface represents one piece of linear display memory, and generally resides in the display memory of video card, Surface Also it can have in Installed System Memory.Surface in the present embodiment is present in Installed System Memory.

RINGBUFFER is a cyclic buffer of customized realization, is exactly one piece of region of memory.Effect be storage from The image data copied out in video card video memory.RINGBUFFER is designed to an annular, avoids write-in data and reads number When carrying out simultaneously according to two operations, there is a situation where data covers (to refer to data that the data to be read are currently written into covering Lid, leads to present frame image content mistake).

FFMPEG be it is a set of can be used to record, converted digital audio, video, and can be translated into stream open source calculate Machine program.AVFrame structural body is generally used for storage initial data, i.e. unpacked data, such as is YUV for video, RGB.YUV, is a kind of colour coding method, and rgb color mode is a kind of color standard of industry.AVPacket structural body is used In the data of storage compression, audio compression data, video compression data and subtitle compressed data have been respectively included.It is usually being solved Compressed data is stored after multiplexing operation, is then transmitted to decoder as input.Or multiplexing is then passed to by encoder output Device.Other encoding tools can also be used.

Embodiment 5:

As a preferable optional way, on the basis of embodiment 1, judge that obtaining original RGB pixel data is No success, specific steps include:

Call GetRenderTargetData method, judge to copy to BackBuffer in Surface object whether at Function obtains original RGB pixel data failure, if so, returning to OK, and calls Surface object if it is not, then returning to FAIL LockRect method, further judge whether the data area of Surface object locks success, if it is not, then returning to FAIL, obtain Original RGB pixel data is taken to fail；If so, returning to OK, original RGB pixel data success is obtained.

Specifically, the data copy of BackBuffer to Surface object (being located at Installed System Memory) is led to first The realization of calling GetRenderTargetData method is crossed, the effect of GetRenderTargetData method is by rendering objects Data copy in Installed System Memory from video memory.This method has return value, if achievement returns to OK, returns to FAIL if failure.

Then by the data copy in Surface object into RINGBUFFER, the two is respectively positioned in Installed System Memory.Pass through Call Surface object LockRect method lock it data area (region save be exactly from BackBuffer The image data that copy comes).After locking successfully, which can return to OK, and fail then FAIL.Then again from its data Region copies data to RINGBUFFER, uses conventional memory copying function memcpy etc..

Embodiment 6:

Shown in Figure 5, the present embodiment provides a kind of cloud game data collection systems, comprising:

Embodiment 7:

As a preferable optional way, on the basis of embodiment 6, the injection module injects customized DLL Into target game process, detailed process are as follows:

The game of process performance objective is created in suspended state；

Further, the data capture module jumps to target game process in MyEndScence function, specific mistake Journey are as follows:

IDirect3DDevice9 interface pointer object is created in customized DLL；

Hook Virtual Function EndScence executes Virtual Function EndScence.

Embodiment 8:

As a preferable optional way, on the basis of embodiment 6, the data capture module passes through DirectX Interface, the rear end of target game caching BackBuffer is copied in memory from the video memory of video card, it is original to obtain RGB pixel data, detailed process are as follows:

The Surface object of off screen rendering is created in memory；

DirectX interface is called to copy BackBuffer in Surface object to；

Further, the coding module encodes original RGB pixel data, detailed process are as follows:

Original RGB pixel data is read from RINGBUFFER, and is read in the AVFrame structural body of FFMPEG；

Original RGB pixel data is converted into yuv data, while carrying out picture size scaling；

Yuv data after conversion is encoded.

Embodiment 9:

As a preferable optional way, on the basis of embodiment 8, the cloud game data collection system further includes Judgment module, the judgment module is used to judge to obtain whether original RGB pixel data succeeds, if so, driving the volume Code module encodes original RGB pixel data, and is sent in network；If it is not, then driving the data capture module Reacquire original RGB pixel data.

Further, the judgment module judges to obtain whether original RGB pixel data succeeds, detailed process are as follows:

The present invention is not limited to the above-described embodiments, for those skilled in the art, is not departing from Under the premise of the principle of the invention, several improvements and modifications can also be made, these improvements and modifications are also considered as protection of the invention Within the scope of.The content being not described in detail in this specification belongs to the prior art well known to professional and technical personnel in the field.

Claims

1. a kind of cloud game collecting method, it is characterised in that: method includes the following steps:

Customized DLL is injected into target game process, by the customized MyEndScence function for being used to acquire data It is inserted into the rendering pipeline of Direct3D；

It jumps to target game process in MyEndScence function, the rear end of target game is cached into BackBuffer from aobvious It copies in memory in the video memory of card, to obtain original RGB pixel data, and original RGB pixel data is encoded.

2. cloud game collecting method as described in claim 1, which is characterized in that customized DLL is injected into target In game process, specific steps include:

The game of process performance objective is created in suspended state；

Recovery target game process is operating status.

3. cloud game collecting method as described in claim 1, which is characterized in that jump to target game process In MyEndScence function, specific steps include:

IDirect3DDevice9 interface pointer object is created in customized DLL；

Using the base address of IDirect3DDevice9 interface pointer object, the address of Virtual Function EndScence is found, by empty letter The address of number EndScence replaces with a skip command, and specified jump address is for customized MyEndScence function Location；

Hook Virtual Function EndScence executes Virtual Function EndScence.

4. cloud game collecting method as described in claim 1, which is characterized in that cache the rear end of target game BackBuffer is copied in memory from the video memory of video card, and to obtain original RGB pixel data, specific steps include:

The Surface object of off screen rendering is created in memory；

DirectX interface is called to copy BackBuffer in Surface object to；

5. cloud game collecting method as claimed in claim 4, which is characterized in that this method further includes that judgement acquisition is original The whether successful step of RGB pixel data, specifically include:

GetRenderTargetData method is called, judges to copy BackBuffer in Surface object and whether succeeds, If it is not, then returning to FAIL, original RGB pixel data failure is obtained；If so, returning to OK, and call Surface object LockRect method further judges whether the data area of Surface object locks success；If it is not, then returning to FAIL, obtain Original RGB pixel data failure；If so, returning to OK, original RGB pixel data success is obtained.

6. a kind of cloud game data collection system characterized by comprising

It is inserted into module, is used to for the customized MyEndScence function for being used to acquire data being inserted into the wash with watercolours of Direct3D It contaminates in pipeline；

Data capture module is used to that target game process to be made to jump in MyEndScence function, by the rear end of target game Caching BackBuffer is copied in memory from the video memory of video card, to obtain original RGB pixel data；And

7. cloud game data collection system as claimed in claim 6, which is characterized in that the injection module is by customized DLL It is injected into target game process, detailed process are as follows:

The game of process performance objective is created in suspended state；

8. cloud game data collection system as claimed in claim 6, which is characterized in that the data capture module swims target Play process jumps in MyEndScence function, detailed process are as follows:

IDirect3DDevice9 interface pointer object is created in customized DLL；

Hook Virtual Function EndScence executes Virtual Function EndScence.

9. cloud game data collection system as claimed in claim 6, which is characterized in that the data capture module swims target The rear end caching BackBuffer of play is copied in memory from the video memory of video card, to obtain original RGB pixel data, specifically Process are as follows:

The Surface object of off screen rendering is created in memory；

DirectX interface is called to copy BackBuffer in Surface object to；

10. cloud game data collection system as claimed in claim 9, which is characterized in that the cloud game data collection system It further include judgment module, the judgment module is used to judge to obtain whether original RGB pixel data succeeds, and the judgement mould Block judges whether the original RGB pixel data of acquisition succeeds, detailed process are as follows: