WO2019079940A1 - 图形处理方法及相关装置和设备 - Google Patents

图形处理方法及相关装置和设备

Info

Publication number
WO2019079940A1
WO2019079940A1 PCT/CN2017/107341 CN2017107341W WO2019079940A1 WO 2019079940 A1 WO2019079940 A1 WO 2019079940A1 CN 2017107341 W CN2017107341 W CN 2017107341W WO 2019079940 A1 WO2019079940 A1 WO 2019079940A1
Authority
WO
WIPO (PCT)
Prior art keywords
graphics
api
graphics api
library
client
Prior art date
Application number
PCT/CN2017/107341
Other languages
English (en)
French (fr)
Inventor
万双
黄巍
张福文
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to PCT/CN2017/107341 priority Critical patent/WO2019079940A1/zh
Priority to CN202010738340.XA priority patent/CN112068908B/zh
Priority to CN201780065686.4A priority patent/CN109983435B/zh
Priority to EP17929704.9A priority patent/EP3693850A4/en
Publication of WO2019079940A1 publication Critical patent/WO2019079940A1/zh
Priority to US16/854,929 priority patent/US11189003B2/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/451Execution arrangements for user interfaces
    • G06F9/452Remote windowing, e.g. X-Window System, desktop virtualisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/54Interprogram communication
    • G06F9/547Remote procedure calls [RPC]; Web services
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00General purpose image data processing
    • G06T1/20Processor architectures; Processor configuration, e.g. pipelining
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/363Graphics controllers
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00Indexing scheme for image data processing or generation, in general
    • G06T2200/16Indexing scheme for image data processing or generation, in general involving adaptation to the client's capabilities

Definitions

  • the present application relates to the field of information technology, and realizes remote interface rendering by a one-way data transmission from a server to a client.
  • an open graphics library (OpenGL) based on a client-server (CS) mode can implement remote display interface rendering of an application.
  • client settings There is a graphics processing unit (GPU), and an Opengl graphics library is installed, and a graphic application programming interface (API) is set in the Opengl graphics library.
  • the server does not have a GPU set up, and the server does not have an Opengl graphics library installed, but the application runs on the server, and the application needs to call the graphics API in the client's Opengl graphics library.
  • the server sends the graphics API that needs to be called to the client.
  • the client calls the graphics API in the local Opengl graphics library.
  • the client runs the graphics API drawing code.
  • the drawing code controls the client's local GPU for graphics rendering.
  • each frame of image requires multiple calls of the graphics API, and the call of the graphics API is mostly based on serial calls.
  • the application of the server needs to sequentially call two graphics APIs.
  • the application does not make two graphics API calls locally on the server, but sends two graphics APIs to the client respectively, and the client makes a call to the graphics API.
  • the application of the server sends the entry parameters of the first graphics API and the first graphics API to the client, and waits after the above data is sent to the client.
  • the client calls the first graphics API in the client-side Opengl graphics library according to the entry parameter of the first graphics API. After the call is completed, the client will notify the application that the first graphics API is called.
  • the processing notification is sent back to the server application, and the server application can end the waiting after receiving the processing notification sent by the client, and send the entry parameter of the second graphics API and the second graphics API to the client.
  • the above-mentioned remote graphics API calls have a network delay of less than 1 millisecond, and 1 second can complete more than 1000 calls.
  • the graphical interface to be drawn is not complicated, it can provide Good experience.
  • the network delay of the above remote graphics API call is between 100 and 300 milliseconds in a wide area network, especially in a wireless wide area network environment, such as a 3G or 4G network, then up to 10 calls are made within 1 second, only the local area network. 1% of the number of graphics APIs can be executed in 1 second. Therefore, the prior art has high requirements for network latency, and cannot be used in high-latency networks. Provide a good user experience.
  • the present application discloses a graphics processing method and related device and device, and the server transmits the calling information of the graphic API to the client in one-way data, which can reduce the requirement for network delay, and guarantee the application of the server in the state of network disconnection. Runs normally to improve the user experience.
  • the present application provides a graphics processing method, where the method is applied to a server.
  • the server is configured with a first graphics library and an application
  • the server has a network connection with the client
  • the client is configured with a second graphics library.
  • the first graphics library includes a plurality of graphics application programming interface APIs
  • the second graphics library includes a plurality of graphics APIs, wherein the first graphics library has at least M graphics APIs and M graphics APIs in the second graphics library.
  • the graphics processing method includes the following steps: acquiring an application-initiated first drawing instruction corresponding to the first graphics API.
  • the first graphics API of the first graphics library is invoked according to the first drawing instruction to execute the first drawing instruction and the first processing notification is sent to the application.
  • the first processing notification is used to notify the application that the first graphics API is called, and the calling information of the first graphics API is generated.
  • the calling information of the first graphics API is used to instruct the client to invoke the first graphic in the second graphics library.
  • the API, and the call information of the first graphics API includes parameters related to the first graphics API, and an identifier of the first graphics API. Finally, the call information of the first graphics API is sent to the client.
  • the application can immediately obtain the first processing notification for notifying the application that the first graphics API is called, so the application of the server
  • the program can run independently without relying on the client. Even if the client crashes or the connection between the client and the server is interrupted, the application of the server will not be affected, and the user experience can be improved.
  • the server generates call information of the first graphics API that carries the parameters involved in the first graphics API, and sends the call information of the first graphics API to the client, where the client may invoke the second according to the parameters involved in the first graphics API.
  • the first graphics API in the graphics library since the application of the server has received the first processing notification, after the client calls the first graphics API in the second graphics library, there is no need to send an application that processes the notification to the server, so One-way transmission between the server and the client enables the network latency to be reduced.
  • the graphics processing method further includes the steps of: determining whether the first drawing instruction needs to return an exit parameter, where the first drawing instruction needs to return an exit parameter, in the first
  • the processing notification further sets the exit parameter generated by the server to invoke the first graphics API, and the calling information of the first graphics API specifically includes an entry parameter of the first graphics API, an exit parameter generated by the server calling the first graphics API, and the first graphic. API identification code.
  • the first processing notification further sets the exit parameter generated by the server to invoke the first graphics API, so that the application generates, by the first processing, the acquireable server to invoke the first graphics API.
  • the exit parameter can be used as an entry parameter of other graphics APIs in subsequent calls to other graphics APIs whose input parameters are the exit parameters of the first graphics API, thereby ensuring the normal operation of the application.
  • the calling information of the first graphics API includes the first graphics API.
  • the entry parameter of the first graphics API and the identifier of the first graphics API are set in the call information of the first graphics API sent to the client, so that the client can
  • the entry parameter of the first graphics API and the identification code of the first graphics API locally call the first graphics API in the second graphics library.
  • the graphics processing method further includes the step of: receiving, by the application, a second drawing instruction initiated according to the first processing notification, specifically, The second drawing instruction corresponds to the second graphics API, and the entry parameter of the second graphics API is an exit parameter generated by the server to invoke the first graphics API.
  • the second graphics API of the first graphics library is invoked according to the second drawing instruction, and the second processing notification is sent to the application. Specifically, the second processing notification is used to notify the application that the second graphics API is called.
  • Generating call information of the second graphics API specifically, the calling information of the second graphics API is used to instruct the client to invoke the second graphics API in the second graphics library, and the calling information of the second graphics API includes the second graphics API The parameters involved, as well as the identification code of the second graphics API. After that, the call information of the second graphics API is sent to the client.
  • the first processing notification further sets an exit parameter generated by the server to invoke the first graphics API, so the application may invoke the first graphics API from the first processing notification acquisition server. Generating the exit parameter, and using the exit parameter generated by the first graphics API as an entry parameter of the second graphics API to invoke the second graphics API of the first graphics library, and sending a second processing notification to the application, thereby enabling the application
  • the program continues to run, so even if the second graphics API has a dependency on the first graphics API, the application does not have to wait for the client to return data, saving time, reducing network latency requirements, and improving the user experience.
  • the graphics processing method further includes: saving the calling information of the first graphics API and the calling information of the second graphics API in the advanced After the queue is fulfilled, the call information of the first graphics API in the FIFO queue and the call information of the second graphics API are simultaneously sent to the client.
  • the application runs faster.
  • multiple sets of call information are generated per unit time.
  • the package sends multiple sets of call information to the client, which is sent relative to each set of call information.
  • the implementation only needs to send data to the client once after the server makes multiple calls of the graphics API, thereby effectively saving processing overhead.
  • the predetermined condition is that the predetermined time is exceeded.
  • the predetermined condition is that the data in the FIFO queue exceeds a predetermined data size.
  • the graphics processing method further includes the steps of: writing the call information of the first graphics API and the call information of the second graphics API into the file, and storing the file.
  • the client can download the file from the server after a certain period of time (such as a few days later), and sequentially read the multiple sets of call information recorded in the file, thereby realizing playback of the drawing interface of the application.
  • the present application provides a graphics processing method, where the method is applied to a client, specifically, the client has a network connection with the server, the server is provided with a first graphics library and an application, and the client is configured with a second graphics library.
  • the first graphics library includes a plurality of graphics application programming interface APIs
  • the second graphics library includes a plurality of graphics APIs
  • At least M graphics APIs in a graphics library are in one-to-one correspondence with M graphics APIs in the second graphics library.
  • the graphics processing method specifically includes the following steps: receiving, by the server, the call information of the first graphics API in the first graphics library, where the call information of the first graphics API includes parameters involved in the first graphics API, And an identification code of the first graphics API.
  • the first graphics API in the second graphics library is invoked according to the identifier of the first graphics API in the call information of the first graphics API and the parameter involved in the first graphics API in the call information of the first graphics API.
  • the server After the first graphics API call of the first graphics library is completed, the server generates call information of the first graphics API that carries the parameters involved in the first graphics API, and sends the call information of the first graphics API to the client, the client. Receiving the call information of the first graphics API from the server, and calling the first graphics API in the second graphics library according to the parameter involved in the first graphics API, because the first graphics API call of the first graphics library of the server has been completed, therefore, After the client calls the first graphics API in the second graphics library, there is no need to send an application that processes the notification to the server, so a one-way transmission is implemented between the server and the client, thereby reducing network latency requirements.
  • the graphics processing method further includes: the parameter involved in the first graphics API in the call information of the first graphics API includes an exit parameter generated by the server calling the first graphics API In the case of obtaining the exit parameter generated by the client calling the first graphics API, and establishing a mapping relationship between the exit parameter generated by the server calling the first graphics API and the exit parameter generated by the client calling the first graphics API.
  • the exit parameter generated by the server calling the first graphics API is different from the exit parameter generated by the client calling the first graphics API, and the exit generated by the first graphics API is generated.
  • the client cannot directly use the exit parameter generated by the server of the first graphics API to invoke the first graphics API as an entry parameter of other graphics API, so the client establishes the server.
  • the client may obtain the client by calling the export parameter query mapping relationship generated by the first graphics API by the server.
  • the exit parameter generated by the first graphics API is called, and the exit parameter generated by the client calling the first graphics API is used as an entry parameter of other graphics API.
  • the method further includes: receiving, by the server, call information of the second graphics API, specifically, calling information of the second graphics API
  • the parameter involved in the second graphics API and the identifier of the second graphics API are included, and the entry parameter of the second graphics API is an exit parameter generated by the server to invoke the first graphics API.
  • Determining, according to the identifier of the second graphics API, the second graphics API, according to the entry parameter query mapping relationship of the second graphics API acquiring a parameter corresponding to the entry parameter of the second graphics API in the mapping relationship, specifically, the second graphics API
  • the corresponding parameter of the entry parameter in the mapping relationship is an exit parameter generated by the client calling the first graphics API.
  • the client cannot directly use the egress parameter generated by the server that is carried by the calling information of the first graphics API to invoke the first graphics API.
  • the entry parameter of the second graphics API so the client can obtain the exit parameter generated by the client calling the first graphics API by calling the export parameter query mapping relationship generated by the first graphics API, and the client invokes the first
  • the exit parameter generated by the graphics API serves as an entry parameter to the second graphics API.
  • the method further includes The receiving server simultaneously sends the calling information of the first graphics API and the calling information of the second graphics API.
  • the client receives the data sent by the server at the same time, and sequentially reads the call information of the first graphics API and the call information of the second graphics API from the data, without separately receiving the call information of the first graphics API and the second graphics API. Calling information can effectively save processing overhead.
  • the second parameter of the first graphics API in the call information of the first graphics API and the parameter of the first graphics API in the call information of the first graphics API are used to invoke the second
  • the first graphics API in the graphics library specifically includes: determining, according to the identifier of the first graphics API in the call information of the first graphics API, the first graphics API, and according to the first graphics API in the call information of the first graphics API The parameters call the first graphics API in the second graphics library.
  • the present application provides a graphics processing method, where the method is applied to a server.
  • the server is configured with a first graphics library and an application
  • the client is configured with a second graphics library, where the first graphics library includes multiple
  • the graphic application programming interface API includes a plurality of graphics APIs in the second graphics library, wherein at least the M graphics APIs in the first graphics library are in one-to-one correspondence with the M graphics APIs in the second graphics library.
  • the method includes the following steps: acquiring a first drawing instruction initiated by the application, specifically, the first drawing instruction corresponds to the first graphics API.
  • the first graphics API When determining that the first graphics API is a predetermined graphics API of the first graphics library, rejecting the first graphics API of the first graphics library, sending a first processing notification to the application, specifically, the first processing notification is used to notify the application A graphics API call is completed.
  • Generating call information of the first graphics API specifically, the calling information of the first graphics API is used to instruct the client to invoke the first graphics API in the second graphics library, and the calling information of the first graphics API includes the first graphics API The parameters involved, as well as the identification code of the first graphics API. After that, the call information of the first graphics API is sent to the client.
  • the server application can run independently without relying on the client. Even if the client crashes or the connection between the client and the server is interrupted, the application of the server will not be affected, and the user experience can be improved.
  • the server does not need to display the drawn graphics to the user, even if the first graphics API corresponding to the first drawing instruction initiated by the application of the server is rejected, so that the GPU of the server does not work, the user experience is not affected.
  • the server generates call information of the first graphics API that carries the parameters involved in the first graphics API, and sends the call information of the first graphics API to the client, where the client may invoke the second according to the parameters involved in the first graphics API.
  • the first graphics API in the graphics library since the application of the server has received the first processing notification, after the client calls the first graphics API in the second graphics library, there is no need to send an application that processes the notification to the server. Therefore, a one-way transmission between the server and the client is implemented, thereby reducing the network latency requirement.
  • the calling information of the first graphics API includes an entry parameter of the first graphics API and an identifier of the first graphics API.
  • the predetermined graphics API includes a shader startup graphics API, a rasterization startup graphics API, a clear buffer graphics API, or an exchange Any of the buffer graphics APIs.
  • the shader startup graphics API, the rasterization startup graphics API, the clear buffer graphics API, and the swap buffer graphics API corresponding to the drawing code are executed, the server GPU is required to perform a large number of operations, and the current tone is determined.
  • the first graphics API used is a predetermined graphics API, refusing to invoke the first graphics API can cause the GPU to avoid performing a large number of operations, thereby saving valuable computing resources on the server and enabling the server to run a larger number of applications.
  • the predetermined graphics API is used to control the GPU of the server for colorator startup, rasterization startup, clear buffer or swap buffer.
  • the predetermined graphics API is used to control the GPU of the server to perform a large number of graphics operations.
  • the application provides a graphics processing device, which is disposed on a server, where the server is further configured with a first graphics library and an application, the server has a network connection with the client, and the client is configured with a second graphics library, and the first graphic
  • the library includes a plurality of graphics application programming interface APIs, and the second graphics library includes a plurality of graphics APIs, wherein at least the M graphics APIs in the first graphics library are in one-to-one correspondence with the M graphics APIs in the second graphics library.
  • the graphics processing device includes: a drawing instruction acquiring module, configured to acquire a first drawing instruction initiated by the application, specifically, the first drawing instruction corresponds to the first graphics API, and the graphic API calling module is configured to invoke the first according to the first drawing instruction a first graphics API of the graphics library to execute the first drawing instruction, and send a first processing notification to the application, specifically, the first processing notification is used to notify the application that the first graphics API is called, and the information generating module is used to Generating call information of the first graphics API, specifically, the call information of the first graphics API is used to instruct the client to call the first in the second graphics library API-shaped, and, the API call information of the first pattern comprises a first parameter related to the graphics API, the graphics API and the first identification code, call information sending module, configured to send the information of the first graphics API calls to the client.
  • a drawing instruction acquiring module configured to acquire a first drawing instruction initiated by the application, specifically, the first drawing instruction corresponds to the first graphics API
  • the graphic API calling module is configured to invoke the first according
  • any one implementation manner is the device implementation corresponding to the first aspect or the implementation manner of any one of the first aspect, and the description in any one of the first aspect or the first aspect is applicable to the fourth aspect. Or any implementation manner of the fourth aspect, and details are not described herein again.
  • the application provides a graphics processing device, which is disposed on a client, the client has a network connection with the server, the server is configured with a first graphics library, and the client is configured with a second graphics library, where the first graphics library includes multiple a graphics application programming interface API, the second graphics library includes a plurality of graphics APIs, and at least the M graphics APIs in the first graphics library are in one-to-one correspondence with the M graphics APIs in the second graphics library, and the graphics processing device includes:
  • the call information receiving module is configured to receive call information of the first graphics API sent by the server, where the call information of the first graphics API includes parameters related to the first graphics API, and an identifier of the first graphics API, and a graphics API calling module, Determining the first graphics API according to the identifier of the first graphics API in the call information of the first graphics API, and calling the second graphics library according to parameters involved in the first graphics API in the call information of the first graphics API A graphics API.
  • the fifth aspect or the fifth aspect, any implementation manner is the apparatus implementation corresponding to the second aspect or the second aspect, and the description in any one of the second aspect or the second aspect is applicable to the fifth aspect. Or any implementation manner of the fifth aspect, and details are not described herein again.
  • the application provides a graphics processing device, which is disposed on a server, where the server is further configured with a first graphics library and an application, the server has a network connection with the client, and the client is configured with a second graphics library, and the first graphics library A plurality of graphics application programming interface APIs are included, and the second graphics library includes a plurality of graphics APIs, and at least the M graphics APIs in the first graphics library are in one-to-one correspondence with the M graphics APIs in the second graphics library, and the graphics processing is performed.
  • the device includes: a drawing instruction obtaining module, configured to acquire a first drawing instruction initiated by the application, where the first drawing instruction corresponds to the first graphic API, and the graphic API calling module is configured to determine that the first graphic API is a predetermined one of the first graphic library In the graphics API, the first graphics API of the first graphics library is rejected, and the graphics API calling module is further configured to send a first processing notification to the application, where the first processing notification is used to notify the application that the first graphics API is called, and the calling Information generation module for students Calling information of the first graphics API, specifically, the calling information of the first graphics API is used to instruct the client to call the first graphics API in the second graphics library, and the calling information of the first graphics API includes the first graphics API
  • the calling information sending module is configured to send the calling information of the first graphics API to the client.
  • any one of the sixth aspect or the sixth aspect is the device implementation corresponding to the third aspect or the third aspect, and the description in any one of the third aspect or the third aspect is applicable to the sixth aspect. Or any implementation manner of the sixth aspect, and details are not described herein again.
  • the present application provides a server, including a processor and a memory, the memory storing program instructions, and the processor running the program instructions to execute the graphics processing method provided by the first aspect or any one of the implementation manners of the first aspect.
  • the application provides a client, including a processor and a memory, wherein the memory stores program instructions, and the processor runs the program instructions to execute the graphics processing method provided by any one of the second aspect or the second aspect.
  • the present application provides a server, including a processor and a memory, the memory storing program instructions, and the processor running the program instructions to perform the graphics processing method provided by any one of the third aspect or the third aspect.
  • the application provides a storage medium storing program code, when the program code is executed by a storage controller, the storage controller performs the foregoing first aspect or any one of the first aspects.
  • the storage medium includes, but is not limited to, a read only memory, a random access memory, a flash memory, an HDD, or an SSD.
  • the application provides a storage medium, where the program code is stored, and when the program code is run by the storage controller, the storage controller performs any of the foregoing second aspect or the second aspect.
  • the graphical processing method provided by the method.
  • the storage medium includes, but is not limited to, a read only memory, a random access memory, a flash memory, an HDD, or an SSD.
  • the present application provides a storage medium storing program code, when the program code is executed by a storage controller, the storage controller performs any of the foregoing third aspect or the third aspect.
  • the storage medium includes, but is not limited to, a read only memory, a random access memory, a flash memory, an HDD, or an SSD.
  • the present application provides a computer program product, comprising: program code, when the computer program product is executed by a storage controller, the memory controller performs any of the foregoing first aspect or the first aspect A method of graphics processing provided by an implementation.
  • the computer program product may be a software installation package, and in the case of using the graphics processing method provided in any one of the foregoing first aspect or the first aspect, the computer program product may be downloaded to the storage controller and The computer program product runs on the storage controller.
  • the present application provides a computer program product, comprising: program code, when the computer program product is executed by a storage controller, the memory controller performs any of the foregoing second aspect or the second aspect A method of graphics processing provided by an implementation.
  • the computer program product may be a software installation package, and in the case of using the graphics processing method provided by any of the foregoing second aspect or the second aspect, the computer program product may be downloaded to the storage controller and stored in the storage The computer program product runs on the controller.
  • the present application provides a computer program product, comprising: program code, when the computer program product is executed by a storage controller, the storage controller performs any of the foregoing third aspect or the third aspect A method of graphics processing provided by an implementation.
  • the computer program product can be a software installation package, in need
  • the computer program product can be downloaded to a memory controller and run on the memory controller.
  • FIG. 1 is a schematic structural diagram of a remote interface display system according to an embodiment of the present invention.
  • FIG. 2 is a schematic diagram of an application calling graphics API according to an embodiment of the present invention.
  • FIG. 3 is a data interaction diagram of a graphics processing method according to an embodiment of the present invention.
  • FIG. 4 is a flow chart of a graphics processing method in accordance with an embodiment of the present invention.
  • FIG. 5 is another schematic diagram of an application calling graphics API according to an embodiment of the present invention.
  • FIG. 6 is a schematic diagram of a second graphics processing device calling a graphics API according to an embodiment of the present invention.
  • FIG. 7 is a data flow diagram of a server according to an embodiment of the present invention.
  • FIG. 8 is another flowchart of a graphics processing method according to an embodiment of the present invention.
  • FIG. 9 is a data flow diagram of a client according to an embodiment of the present invention.
  • FIG. 10 is a schematic structural diagram of an apparatus of a graphics processing apparatus according to an embodiment of the present invention.
  • FIG. 11 is a schematic structural diagram of another device of a graphics processing apparatus according to an embodiment of the present invention.
  • FIG. 12 is a schematic structural diagram of another device of a graphics processing apparatus according to an embodiment of the present invention.
  • FIG. 13 is a schematic structural diagram of an apparatus of a server according to an embodiment of the present invention.
  • FIG. 14 is a schematic structural diagram of a device of a client according to an embodiment of the present invention.
  • FIG. 1 is a schematic structural diagram of a remote interface display system according to an embodiment of the present invention.
  • the remote interface display system includes a server 10 and a client 20, and the client 20 is connected to the server 10 via the network 30.
  • the server 10 includes hardware 100, an operating system 102, a first graphics library 103, an application 104, and a first graphics processing device 105.
  • the hardware 10 is provided with a GPU 111, a network card 112, a CPU 113, and a memory 114.
  • the server 10 may not set the GPU 111, the CPU 113 simulates the GPU 111, and the CPU 113 implements the drawing function of the GPU 111.
  • the first graphics library 103 is provided with a plurality of graphics APIs, each graphics API corresponding to a specific drawing code, wherein the drawing code is used to control the GPU 111 to perform drawing, such as handle application, graphics rendering, coloring, update buffer, and display image.
  • the application 104 loads the first graphics library 103, and the application 104 can directly call the graphics API in the loaded first graphics library 103 during the running process.
  • the operating system 102 is provided with a network card driver of the network card 112.
  • the first graphics processing device 105 can control the network card 112 to receive or transmit data through the network 30 according to the network card driver of the network card 112.
  • the operating system 102 is further provided with a graphics card driver of the GPU 111.
  • the application 104 and the first graphics processing device 105 control the GPU 111 to perform drawing according to the graphics card driver of the GPU 111.
  • the first graphics library 103, the application 104, and the first graphics processing device 105 are respectively carried on the operating system 102.
  • the first graphics library 103 can be, for example, an Opengl graphics library, a Vulkan graphics library, or other graphics library.
  • the first graphics library 103 is an Opengl graphics library
  • the graphics API in the first graphics library 103 may be eglCreateWindowSurface, eglCreateContext, glCreateShader, glCompileShader, glGenBuffers, glBufferData, glDrawArrays, or eglSwapBuffers.
  • eglCreateWindowSurface is used to apply graphics buffer
  • eglCreateContext is used to apply context handle
  • glCreateShader is used to apply shader handle
  • glCompileShader is used to compile shader
  • glGenBuffers to apply vertex and texture buffer handle
  • glBufferData is used to send vertex and texture data to client
  • glDrawArrays is used to start drawing pixels to the back buffer
  • eglSwapBuffers is used to swap before and after buffers.
  • the application 104 is an application deployed by the client 20 on the server 10, such as a 2D game, a 3D game, a mailbox, and social software.
  • the N graphics APIs of the first graphics library 103 may be sequentially invoked according to the program code of the application 104, wherein the calling sequence of the N graphics APIs is pre-recorded in the application 104. In the program code.
  • program instructions may also be arranged to cause the application 104 to cyclically invoke N graphics APIs.
  • FIG. 2 is a schematic diagram of an application calling graphics API in accordance with an embodiment of the present invention.
  • the application 104 sequentially calls the graphics APIs in the first graphics library 103 according to its own program code: eglCreateWindowSurface, eglCreateContext, glCreateShader, glCompileShader, glGenBuffers, glBufferData, glDrawArrays, and eglSwapBuffers.
  • the exit parameters are generated for eglCreateWindowSurface, eglCreateContext, glCreateShader, and glGenBuffers, and the exit parameters are not generated for glCompileShader, glBufferData, glDrawArrays, and eglSwapBuffers, and are represented by void.
  • some graphics APIs have dependencies, that is, the entry parameters of some graphics APIs are export parameters of other graphics APIs.
  • the entry parameters of some graphics APIs are export parameters of other graphics APIs.
  • one of the entry parameters, surfaceA is an exit parameter of eglCreateWindowSurface.
  • eglCompileShader its entry parameter shader_handle is the exit parameter of glCreateshader.
  • eglSwapBuffers one of its entry parameters, surfaceA, is the exit parameter of eglCreateWindowSurface.
  • each graphics API shown in FIG. 2 egldisplay, eglconfig, eglwindow, attrib_list, eglcontexts, GL_FRAGMENT_SHADER, count, GL_ARRAY_BUFFER, size, Vertices, GL_STATIC_DRAW, GL_TRIANGLES, first are constants, and the specific values are pre-recorded in the program. In the code.
  • the entry parameter as a constant is pre-recorded in the program code, and the application 104 can directly acquire the entry parameter from the program code when calling the graphics API.
  • the entry parameter of the currently invoked graphics API is an exit parameter of another graphics API, that is, when there is a dependency between the currently invoked graphics API and other graphics APIs, the application 104 needs to be called after other graphics APIs are completed.
  • the exit parameter returned by other graphics APIs is used as the entry parameter of the currently invoked graphics API to call the graphics API currently called.
  • glCreateShader its entry parameter GL_FRAGMENT_SHADER can be obtained directly from the program code.
  • entry parameter shader_handle is the exit parameter of glCreateShader, so the application 104 needs to be called after glCreateShader
  • the export parameter shader_handle generated by glCreateShader can be used as the entry parameter of glCompileShader to call glCompileShader.
  • the first graphics library 103 may be a graphics library provided by a third-party vendor. In other examples, the first graphics 103 may be a customized graphics library. In this case, the first graphics processing The device 105 can be integrated with the first graphics library 103, indicated by the dashed box in FIG.
  • the client 20 is described below. Please continue to refer to FIG. 1.
  • the client 20 includes a hardware 200, an operating system 202, a second graphics library 203, and a second graphics processing device 204.
  • the client 20 may not set the GPU 211, the CPU 214 simulates the GPU 211, and the CPU 214 implements the drawing function of the GPU 211.
  • the second graphics library 203 is also provided with a plurality of graphics APIs.
  • Each graphics API of the second graphics library 203 corresponds to a specific graphics code, and the second graphics processing device 204 loads the second graphics library 203.
  • the second graphics processing device 204 can directly call the graphics API in the loaded second graphics library 203 to control the GPU 211 to draw.
  • the operating system 202 is provided with a network card driver of the network card 212.
  • the second graphics processing device 204 can control the network card 212 to receive or transmit data through the network 30 according to the network card driver of the network card 212.
  • the operating system 202 is also provided with a graphics card driver of the GPU 211, and the second graphics processing device 204 controls the GPU 211 for drawing by the graphics card driver of the GPU 211.
  • the operating system 202 is also provided with a display screen driver for the display screen 213. When the GPU 211 needs to display the drawn interface, the display driver control display screen 213 displays the drawn interface.
  • the second graphics library 203 and the second graphics processing device 204 are respectively run on the operating system 202.
  • the second graphics library 203 can be, for example, an Opengl graphics library, a Vulkan graphics library, or other commonly used graphics libraries.
  • the graphics API in the second graphics library 203 may be eglCreateWindowSurface, eglCreateContext, glCreateShader, glCompileShader, glGenBuffers, glBufferData, glDrawArrays, or eglSwapBuffers.
  • the first graphics library 103 and the second graphics library 203 are the same type of graphics library, and the first graphics library 103 has at least M graphics APIs and M graphics in the second graphics library 203. API one-to-one correspondence.
  • the first graphics library 103 and the second graphics library 203 are both Opengl graphics libraries, and are provided with the same graphics API: eglCreateWindowSurface, and the first graphics library 103 and the second graphics library 203 have the same function of eglCreateWindowSurface. It is used to apply for a graphics buffer, but the implementation details of the eglCreateWindowSurface of the first graphics library 103 and the eglCreateWindowSurface of the second graphics library 203 may be different.
  • the eglCreateWindowSurface of the first graphics library 103 can be implemented by the C++ language
  • the eglCreateWindowSurface of the second graphics library 203 can be implemented by the Java language.
  • the implementation details of the eglCreateWindowSurface of the first graphics library 103 and the eglCreateWindowSurface of the second graphics library 203 may be different, since the two functions are the same and are set in the same type of graphics library, the first graphics library
  • the eglCreateWindowSurface of 103 and the eglCreateWindowSurface of the second graphics library 203 have a one-to-one correspondence, which may be referred to as the same graphics API.
  • eglCreateContext glCreateShader in the first graphics library 103 and the second graphics library 203
  • glCompileShader glGenBuffers
  • glBufferData glDrawArrays
  • eglSwapBuffers all have a one-to-one correspondence.
  • graphics API introduced above is only a part of the OpenGL graphics library, and the embodiment of the present invention can also be applied to other graphics APIs in the Opengl graphics library, and can be applied to a Vulkan graphics library or other commonly used graphics libraries.
  • the graphics API is as long as the server 10 and the client 20 have the same type of graphics library.
  • the application 104 preloads the first graphics library 103. After loading the first graphics library 103, the application 104 learns the function address of the graphics API of the first graphics library 103, when the application 104 invokes the first When the graphics API of the graphics library 103, the drawing instruction carrying the entry parameter and the jump command of the graphics API is initiated, the jump command is used to jump to the function address of the graphics API to be called, and the application 104 is configured according to the jump command. Jump to the function address and execute the drawing code on the function address based on the entry parameters.
  • the second graphics processing device 204 preloads the second graphics library 203, after loading the second graphics library 203, the second graphics processing device 204 learns the function address of the graphics API of the second graphics library 203, when the second graphics processing device When the graphics API of the second graphics library 203 is invoked, a drawing instruction carrying an entry parameter and a jump command of the graphics API is used, the jump command is used to jump to a function address of the graphics API, and the second graphics processing device 204 Jump to the function address according to the jump command and execute the drawing code on the function address according to the entry parameters.
  • the graphics API is written in the C++ language, the graphics API is embodied as a function address of the graphics API that needs to be called at the server 10, where the function address is specifically a memory address.
  • the graphics API is written in the Java language
  • the graphics API is embodied as a function, method, or object of the graphics API that needs to be invoked in the server 10
  • the java virtual machine converts the function, method, or object of the graphics API to The function address of the server 10.
  • the graphic API is exemplified by using a function address as an example.
  • FIG. 3 is a data interaction diagram of a graphics processing method according to an embodiment of the present invention. As shown in FIG. 3, the graphics processing method of the embodiment of the present invention includes the following steps:
  • Step S1 The application 104 initiates a first drawing instruction, and the first graphics processing device 105 acquires a first drawing instruction, where the first drawing instruction corresponds to the first graphics API, and the first drawing instruction carries the entry parameter of the first graphics API. And a jump command for jumping to the function address of the first graphics API.
  • the first drawing instruction corresponds to the first graphics API of the first graphics library 103 and/or the first graphics API of the second graphics library 203.
  • the application 104 After the application 104 initiates the first drawing instruction, it waits until the execution result of the first drawing instruction is obtained to continue the operation.
  • the application 104 initiates a first drawing instruction and executes a first drawing instruction to jump to the function address to which the jump command is to jump, the first graphics processing device 105 intercepts the first drawing by a hook (HOOK) instruction.
  • HOOK hook
  • Step S2 The first graphics processing device 105 calls the first graphics API of the first graphics library according to the first drawing instruction to execute the first drawing instruction.
  • the first graphics processing device 105 jumps to the function address of the first graphics API according to the jump instruction carried by the first drawing instruction, and executes the function of the first graphics API according to the entry parameter of the first graphics API carried by the first drawing instruction.
  • the drawing code on the address when the drawing code is executed, can control the GPU 111 to perform image rendering, thereby completing the first The invocation of the first graphics API of the graphics library and the execution of the first drawing instruction.
  • the first graphics processing device 105 may further confirm whether the first graphics API is a predetermined graphics API of the first graphics library, where the predetermined graphics When the API is called, the GPU 111 needs to perform a large number of operations.
  • the first graphics processing device 105 determines that the first graphics API is not the predetermined graphics API of the first graphics library, the first graphics API of the first graphics library is invoked according to the first graphics instruction to execute the first graphics instruction. Moreover, when the first graphics processing device 105 determines that the first graphics API is a predetermined graphics API of the first graphics library, the first graphics API of the first graphics library is rejected.
  • Step S3 The first graphics processing device 105 generates a first processing notification, and sends a first processing notification to the application 104, wherein the first processing notification is used as an execution result of the first drawing instruction to notify the application 104 of the first graphics API. The call is completed.
  • the first graphics processing device 105 may further determine whether the first drawing instruction needs to return an exit parameter.
  • the first processing notification further includes the server calling the first graphics API. The resulting export parameters.
  • the first processing notification further carries the exit parameter generated by the server 10 calling the first graphics API, so that the application 104 notifies the acquireable server 10 to call the first through the first processing.
  • the exit parameter generated by the graphics API can be used as an entry parameter of other graphics APIs when the application 104 subsequently calls other graphics APIs whose input parameters are the exit parameters of the first graphics API, thereby ensuring the application. 104 is operating normally.
  • step S3 when the first graphics processing device 105 determines that the first graphics API is the predetermined graphics API of the first graphics library 103 and rejects the first graphics API of the first graphics library 103, in this step, The first graphics processing device 105 still generates a first processing notification for notifying the application 104 that the first graphics API is called, and sends a first processing notification to the application 104.
  • the predetermined graphics API is a graphics API that requires the GPU 111 to perform a large number of operations.
  • the graphics API includes any of a shader startup graphics API, a rasterization startup graphics API, a clear buffer graphics API, or an exchange buffer graphics API.
  • the graphics code corresponding to the shader startup graphics API, the rasterization startup graphics API, the clear buffer graphics API, and the swap buffer graphics API require the GPU 111 to start the shader, start the rasterization, and clear the buffer.
  • the area and the swap buffer, these operations will cause the GPU 111 to perform a large number of operations.
  • the refusal to invoke the first graphics API may cause the GPU 111 to avoid performing a large number of operations, thereby saving valuable operations on the server 10. Resources.
  • glDrawArrays is used to start drawing pixels to the back buffer.
  • eglSwapBuffers is used to exchange before and after buffers, so eglSwapBuffers is the swap buffer graphics API;
  • glDrawElements is used to start drawing, and the shader is started when drawing, so glDrawElements is also the shader startup graphics API;
  • glClearDepthf and glClearColor clears the buffer, so glClearDepthf and glClearColor are the clear buffer graphics API.
  • the first graphics processing device 105 determines that the first graphics API is a predetermined graphics API of the first graphics library 103, The first graphics API of the first graphics library 103 is rejected, so the GPU 111 does not need to perform a startup shader, initiate rasterization, clear buffers, or swap buffers to achieve null rendering. However, in the embodiment of the present invention, the first graphics processing device 105 still generates a first processing notification for notifying the application 104 that the first graphics API is called, and sends the first processing notification to the application 104, so that the application 104 The first processing notification can still be obtained to end the wait and continue to run.
  • the server 10 does not need to display the drawn graphics to the user, even if the GPU 111 of the server 10 does not work, the user experience is not affected.
  • null rendering disclosed above may allow the GPU 111 to avoid performing a large number of operations, thereby saving valuable computing resources on the server 10, allowing the server 10 to run a larger number of applications.
  • Step S4 The first graphics processing device 105 generates call information of the first graphics API, and transmits the call information of the first graphics API to the second graphics processing device 204 of the client 20 through the network 30.
  • the calling information of the first graphics API is used to instruct the client to invoke the first graphics API in the second graphics library, and the calling information of the first graphics API includes the parameters involved in calling the first graphics API by the server 10 in the first graphics library 103. And the identification code of the first graphics API.
  • the parameters involved in the first graphics API may include an entry parameter of the first graphics API, or the parameters involved in the graphics API may include an entry parameter and an exit parameter of the first graphics API.
  • a part of the graphics API has only an entry parameter and no exit parameter.
  • a graphics API for starting to draw a pixel to a background buffer glDrawArrays
  • its entry parameters Including GL_TRIANGLES, first and count, and no exit parameters and another part of the graphics API has both entry parameters and exit parameters, such as the graphics API for applying the graphics buffer: eglCreateWindowSurface
  • the entry parameters include egldisplay, eglconfig, eglwindow and attrib_list
  • Its exit parameter is surfaceA.
  • the above-mentioned entry parameter and exit parameter are specifically a value, a register address or a memory address.
  • surfaceA is a handle of a graphics buffer defined by the Opengl graphics library, that is, a memory address of the applied graphics buffer, surfaceA
  • the specific value depends on the memory 114 usage of the server 10. For example, if the memory 114 of the server 10 has a free memory space, the entry address of the free memory space can be assigned to the surfaceA.
  • Table 1 can be maintained by the first graphics processing device 105 of the server 10.
  • the function address A of the eglCreateWindowSurface at the server 10 is specifically used by the application 104 to implement the graphics after loading the first graphics library 103.
  • API The entry code to which the drawing code of eglCreateWindowSurface is assigned on the memory 114 of the server 10.
  • the entry address assigned by eglCreateWindowSurface in the memory 114 of the server 10 may be 0x00000001.
  • the application 104 calls the eglCreateWindowSurface of the first graphics library 103, it directly jumps to the memory address 0x00000001 of the server 10, and executes the drawing corresponding to the eglCreateWindowSurface from 0x00000001. Code.
  • the parameters involved in the first graphics API include an entry parameter of the first graphics API and an exit parameter generated by the server calling the first graphics API, first
  • the call information of the graphics API includes an entry parameter of the first graphics API, an exit parameter generated by the server calling the first graphics API, and an identifier of the first graphics API.
  • the parameter involved in the first graphics API includes an entry parameter of the first graphics API
  • the call information of the first graphics API includes an entry parameter of the first graphics API and the first The identifier of the graphics API.
  • the parameters involved in the first graphics API include the first The entry parameter of the graphics API, the call information of the first graphics API includes an entry parameter of the first graphics API and an identifier of the first graphics API.
  • the identification code shown in Table 1 is only an example, as long as the uniqueness of the identification code in the graphics library is guaranteed.
  • the name of the graphic API can also be used as the identification code, such as eglCreateWindowSurface. .
  • Step S5 The second graphics processing device 204 receives the call information of the first graphics API, and determines the first graphics API according to the identifier of the first graphics API in the call information of the first graphics API, according to the call information of the first graphics API.
  • the parameters involved in the first graphics API call the first graphics API in the second graphics library 203.
  • the second graphics processing device 204 pre-records the correspondence between the graphics API and the identifier of the graphics API.
  • Table 2 shows the second column shown in Table 1 above.
  • the table 2 can be maintained by the second graphics processing device 204 of the client 20.
  • the function address A' of the eglCreateWindowSurface at the client 20 is specifically the second graphics processing device 204 loading the second graphics library.
  • the graphics API is implemented: the drawing code of eglCreateWindowSurface is assigned to the entry address on the memory 215 of the client 20.
  • the memory entry address allocated by the eglCreateWindowSurface in the memory 215 of the client 20 may be 0xA000100F.
  • the second graphics processing device 204 calls the eglCreateWindowSurface of the second graphics library 203, it directly jumps to the memory address of the client 20. 0xA000100F, starting from 0xA000100F to execute the drawing code corresponding to eglCreateWindowSurface.
  • the second graphics processing device 204 determines the function address I' of the first graphics API at the client 20 according to the identification code 0009 of the first graphics API in the call information of the first graphics API, and executes the function address I'.
  • the drawing code controls the GPU 211 to perform graphics drawing, thereby completing the calling of the first graphics API in the second graphics library 203.
  • the second graphics processing device 204 obtains The client 20 invokes the exit parameter generated by the first graphics API, and establishes a mapping relationship between the exit parameter generated by the server calling the first graphics API and the exit parameter generated by the client calling the first graphics API.
  • the exit parameters generated by the server 10 invoking the first graphics API are not the same as the exit parameters generated by the client 20 calling the first graphics API. For example, if the function of the first graphics API is to apply for a buffer space, the exit parameter returned is the memory address of the cache space. Since the operating environment of the server 10 and the client 20 are completely different, the graphics API is on the server 10 and The exit parameters (ie, memory addresses) generated by the client 20 are also different.
  • the client 20 cannot directly use the exit parameter generated by the server 10 carried by the first graphics API to invoke the first graphics API as the exit parameter generated by the first graphics API. Entry parameters for other graphics APIs.
  • the client 20 establishes a mapping relationship between the egress parameter generated by the server 10 to invoke the first graphics API and the egress parameter generated by the client 20 to invoke the first graphics API, and the client 20 may be according to the server 10 received from the server 10. Calling the export parameter generated by the first graphics API to query the mapping relationship, to obtain the exit parameter generated by the client 20 calling the first graphics API, and using the obtained export parameter of the first graphics API by the client 20 as Entry parameters for other graphics APIs.
  • the technical problem that the exit parameters are not synchronized due to the difference in hardware configuration conditions between the client 20 and the server 10 is overcome.
  • the first graphics processing device 105 actually copies the parameters involved in the first graphics API, specifically, when the first graphics API has an exit parameter, the first graphics API.
  • the entry parameters and exit parameters are copied.
  • the first graphics API does not have an exit parameter, only the entry parameters of the first graphics API are copied.
  • the first graphics processing device 105 forms call information of the first graphics API based on the copied parameters and the identification code of the first graphics API.
  • Step S6 The application 104 initiates a second drawing instruction according to the first processing notification, and the first graphics processing apparatus 105 acquires a second drawing instruction, where the second drawing instruction corresponds to the second graphics API, and the second drawing instruction carries the second drawing instruction.
  • the second drawing instruction corresponds to the second graphics API of the first graphics library 103 and/or the second graphics API of the second graphics library 203.
  • the first graphics processing apparatus 105 may pass the hook. (HOOK) intercepts the second drawing instruction.
  • step S6 only needs to be guaranteed to be performed after step S3, so step S6 may also be directly after step S3 or step S4.
  • the application 104 stops waiting and continues to run, and runs to the program code of the second graphics API that needs to invoke the first graphics library 103.
  • the application 104 is completely independent of the client 20 and can be run independently at the server 10.
  • Step S7 The first graphics processing device 105 calls the second graphics API of the first graphics library 103 according to the second drawing instruction to execute the second drawing instruction.
  • the first graphics processing device 105 jumps to the function address of the second graphics API according to the jump instruction carried by the second drawing instruction, and executes the function of the first graphics API according to the entry parameter of the first graphics API carried by the second drawing instruction.
  • the drawing code on the address the drawing code controls the GPU 111 to perform a drawing operation, thereby completing the calling of the first graphics API of the first graphics library and the execution of the first drawing instruction.
  • Step S8 After the second drawing instruction is executed, the first graphics processing device 105 generates a second processing notification, and sends a second processing notification to the application 104, wherein the second processing notification is used as the execution result of the second drawing instruction.
  • the second graphics API call is completed in the notification application 104.
  • Step S9 The first graphics processing device 105 generates call information of the second graphics API, and transmits the call information of the second graphics API to the second graphics processing device 204 of the client 20 through the network 30.
  • the calling information of the second graphics API is used to instruct the client to invoke the second graphics API in the second graphics library, and the calling information of the second graphics API includes parameters involved in the second graphics API, and an identifier of the second graphics API. .
  • Step S10 The second graphics processing device 204 receives the call information of the second graphics API, and determines the second graphics API according to the identifier of the second graphics API in the call information of the second graphics API, according to the call information of the second graphics API.
  • the second graphics API involves parameters that call the first graphics API in the second graphics library.
  • the second graphics processing device 204 determines the function address J' of the first graphics API at the client 20 according to the identification code 000A of the second graphics API in the call information of the second graphics API, and executes the function address J'. Drawing code on the top, thereby completing the call of the second graphics API in the second graphics library 203.
  • the second graphics processing device 204 queries the mapping recorded in step S5 according to the entry parameter of the second graphics API. And a parameter corresponding to the entry parameter of the second graphics API in the mapping relationship, wherein the parameter corresponding to the entry parameter of the second graphics API in the mapping relationship is an exit parameter generated by the client 20 by calling the first graphics API.
  • the egress parameter generated by the first graphics API is the ingress parameter of the second graphics API
  • the egress parameter caused by the hardware configuration condition between the client 20 and the server 10 may be overcome. Synchronized technical issues.
  • the first graphics processing device 105 may save the call information of the first graphics API and the call information of the second graphics API in a first in first out queue, and after the predetermined condition is met, the first in first out The data in the queue is simultaneously sent to the second graphics processing device 204 of the client 20.
  • the predetermined condition may be, for example, that the timing time exceeds a preset time period, or the data in the first in first out queue exceeds a predetermined data size.
  • the application 104 runs faster, when the application 104 runs, multiple sets of call information are generated per unit time, and multiple sets of call information are simultaneously sent to the client 20 after the predetermined condition is satisfied by the package mode, with respect to each group generated.
  • the call information is sent to the client 20 once, and the processing overhead can be effectively saved.
  • the application 104 can locally obtain the processing notification for notifying the application 104 that the graphic API is completed, so the application of the server 10 is used.
  • the program 104 can operate independently without relying on the client 20, and even if the client 20 crashes, or the connection between the client 20 and the server 10 is interrupted, the operation of the application 104 of the server 10 is not affected, and the user experience can be improved.
  • the first graphics processing device 105 of the server 10 generates call information carrying parameters related to the graphics API, and sends the call information to the client 20, and the client 20 can invoke the second graphics according to parameters involved in the graphics API.
  • the application 104 of the server 10 has received the processing notification sent by the first graphics processing device 105 before the first graphics processing device 105 sends the call information. Therefore, after the client 20 calls the graphics API in the second graphics library 203, There is no need to send an application 104 to process the notification to the server 10, so that one-way transmission is implemented between the server 10 and the client 20, thereby reducing the requirement for network delay.
  • FIG. 4 is a flowchart of a graphics processing method according to an embodiment of the present invention, and FIG. 4 specifically describes a workflow of the server. As shown in FIG. 4, the graphics processing method specifically includes The following steps:
  • Step S101 The application 104 is started.
  • Step S102 The application 104 initiates a first drawing instruction. Specifically, when the application 104 runs to the program code that needs to invoke the first graphics API, the first drawing instruction is initiated.
  • the first drawing instruction carries an entry parameter of the first graphics API and a jump command for jumping to a function address of the first graphics API, where the function address is a function address I of the first graphics API at the server 10 and The entry parameter of the first graphics API.
  • Step S103 The first graphics processing device 105 acquires the first drawing instruction.
  • the first graphics processing device 105 intercepts the first drawing instruction with a hook (HOOK).
  • Step S104 The first graphics processing device 105 determines whether the graphics API corresponding to the graphics processing request needs to return an exit parameter. If yes, step S105 is performed, and if no, step S107 is performed.
  • the first graphics processing device 105 can pre-record which of all the graphics APIs in the first graphics library 103 needs to return the exit parameters, and which does not need to return the exit parameters.
  • the first graphics processing device 105 may pre-record whether each graphics API in the Opengl graphics library needs to return an exit parameter, and in the technical document disclosed by the Opengl graphics library, Specifically defines the type of the input parameters of the graphics API (such as integers, pointers, memory addresses, and register addresses), the number, and the sort order, and defines whether the graphics API needs to return the exit parameters.
  • the programmer can write the corresponding information by reading the technical documentation.
  • the program code calls the graphics API.
  • Step S105 The first graphics processing device 105 invokes the graphics API of the first graphics library 103, acquires an exit parameter, and sends a processing notification carrying the exit parameter to the application 104, wherein the processing notification is used to notify the application 104 of the current graphics.
  • the API call is complete.
  • Step S106 The first graphics processing device 105 sets an identification code for the graphics API, and sets the identification code, the entry parameter, and the exit parameter as a set of call information in the first in first out queue.
  • the first graphics processing device 105 can set an identifier for the graphics API according to Table 1 described above. Specifically, the first graphics processing device 105 acquires a jump command carried by the current drawing instruction, and searches in Table 1. The identification code corresponding to the function address to be jumped by the jump command uses the searched identifier as the identification code corresponding to the current graphics API.
  • Step S107 The first graphics processing device 105 determines whether the currently invoked graphics API is a predetermined graphics API. If yes, step S109 is executed to reject the graphics API of the first graphics library 103. If not, step S108 is executed to invoke the first The graphics API of the graphics library 103.
  • the predetermined graphics API includes a shader startup graphics API, a rasterization startup graphics API, a clear exchange buffer graphics API, and an exchange buffer graphics API.
  • the shader starts the graphics API, the rasterization startup graphics API, and the clear buffer graphics API.
  • the GPU 111 is required to perform a large number of operations.
  • Directly skipping step S108, refusing to invoke the predetermined graphics API may cause the GPU 111 to avoid performing a large number of graphics operations, thereby implementing null rendering, which may save valuable computing resources on the server 10.
  • the server 10 does not need to display the drawn graphics to the user, even if the GPU 111 does not work, it does not affect the user experience.
  • Step S108 The first graphics processing device 105 invokes the graphics API of the first graphics library 103.
  • the first graphics processing device 105 jumps to the function address of the jump command according to the jump command carried by the currently acquired drawing instruction, and uses the entry parameter carried by the current graphics processing request as the input parameter to execute the function address.
  • Drawing code
  • Step S109 After the first graphics processing device 105 invokes the graphics API of the first graphics library 103, the first graphics processing device 105 sends a processing notification to the application 104, wherein the processing notification is used to notify the application 104 that the graphics API call is completed. After the execution of this step is completed, the flow jumps to step S111 and step S110, respectively.
  • step S105 since the graphics API does not need to return the exit parameter, the sent processing notification of this step does not carry the exit parameter of the graphics API.
  • Step S110 The first graphics processing device 105 sets an identification code for the graphics API, and sets the identification code and the entry parameter as a group of call information in the first in first out queue.
  • the first graphics processing device 105 can set an identification code for the graphics API according to Table 1 described above. Specifically, the first graphics processing device 105 can search and jump in Table 1. The identification code corresponding to the function address to be jumped by the command, and the searched identification code is used as the identification code corresponding to the current graphics API.
  • Step S111 The application 104 continues to run the program instruction according to the received processing notification, and when running to the program code that needs to call another graphics API, initiates another drawing instruction, and then the flow jumps back to step S103.
  • the first graphics processing device 105 can sequentially call a plurality of graphics APIs locally for different drawing instructions, and call information of the called graphics API.
  • the FIFO queue is stored, wherein the call information of the invoked graphics API describes in detail the parameters involved in the call of the server 10 by the plurality of graphics APIs and the type of the graphics API.
  • Step S112 The first graphics processing device 105 sends the group call information in the FIFO queue to the client 20 every predetermined time after the first group of call information is stored in the first in first out queue.
  • the predetermined time may be, for example, 3 ms, and by setting the predetermined time, the recorded plurality of sets of call information may be automatically transmitted to the client 20 every predetermined time.
  • the application 104 runs faster, when the application 104 runs, multiple sets of call information are generated per unit time, and if a set of call information is generated and sent to the client 20, multiple calls need to be sent in a unit time.
  • the information while sending a plurality of sets of call information to the client 20 at the same time after the predetermined time has elapsed through the packaging mode, can effectively save the processing overhead by sending the call information to the client 20 every time a set of call information is generated.
  • the first graphics processing device 105 sends the N sets of call information to the client 20, if the application 104 continues to call the graphics API, the first graphics processing device 105 also sends the L group call information to the client after a predetermined time.
  • the terminal 20, wherein the values of L and N may be the same or different, and the number of call information depends on the number of graphics APIs that the first graphics processing device 105 calls within a predetermined time.
  • the flow shown in Figure 4 can process 50 graphics APIs, that is, the first 3ms corresponds to 50 sets of call information.
  • the flow shown in Figure 4 can process 70 graphics APIs, that is, the second 3ms correspondence. 70 groups of call information.
  • the first graphics processing device 105 can continuously record the identification code of the graphics API and the parameters involved in the application 104 during the running process with the operation of the application 104. Outgoing the queue, and periodically sending the plurality of sets of call information recorded by the FIFO queue to the client 20, since the first graphics processing device 105 sends a processing notification to the application 104 in time before each time the call information is recorded, the application 104 is in the process of obtaining After the notification, the operation can continue, so in the embodiment of the present invention, the client 20 does not have any influence on the operation of the application 104 at all.
  • the first graphics API and the second graphics API shown in Table 1 can be used as an example to illustrate the loop process shown in FIG. 4, assuming the entry parameter x of the first graphics API, y, z, there is a dependency relationship between the first graphics API and the second graphics API, the first graphics API is based on the entry parameter x, y, z, and the exit parameter generated by the server 10 is retA, and the second graphics API is based on the entry parameter.
  • the exit parameter generated by retA in the server 10 call is retB
  • the first graphics API generates an exit parameter retA' according to the entry parameter x, y, z at the client 20
  • the second graphics API is based on the entry parameter retA' at the client 20
  • the exit parameter generated by the call is retB', and it is assumed that the program code of the application 104 is set to only call the first graphics API and the second graphics API in a single serial.
  • FIG. 5 is another schematic diagram of an application calling graphics API according to an embodiment of the present invention.
  • the program code of the application 104 is set to respectively invoke the first graphics API.
  • a second graphics API wherein the entry parameter x, y, z is a constant set in the program code, and the entry parameter of the second graphics API is an exit parameter retA of the first graphics API.
  • FIG. 6 is a schematic diagram of a second graphics processing device calling a graphics API according to an embodiment of the present invention.
  • the second graphics processing device 204 respectively invokes a first graphics API and a second graphics API, where the first graphics
  • the entry parameters x, y, z of the API are obtained from the first set of call information, and the entry parameter retA' of the second graphics API is obtained from the mapping table, as will be described in more detail below.
  • the same graphics API generates different exit parameters after the server 10 and the client 20 are invoked.
  • the first graphics API is based on the entry parameters.
  • the exit parameter retA generated by x, y, z in the server 10 call and the first graphics API are different according to the entry parameter x, y, z, which is generated by the client 20 in response to the exit parameter retA'.
  • the reason is that the running environment of the server 10 and the client 20 are inconsistent.
  • the function of the graphics API is to apply for a buffer space, and the exit parameter returned is the memory address of the cache space. Since the operating environments of the server 10 and the client 20 are completely different, Therefore, the graphics API (and the memory address) generated by the graphics API and the client 20 are also different.
  • FIG. 3 is a data flow diagram of a server in accordance with an embodiment of the present invention.
  • Step S101 The application 104 is started.
  • Step S102 When the application 104 runs to the program code that needs to invoke the first graphics API, initiate the first Drawing instruction 1011 (shown in Figure 7), the first drawing instruction 1011 includes an entry parameter x, y, z of the first graphics API and a jump command for jumping to the function address I of the first graphics API.
  • Step S103 The first graphics processing device 105 acquires the first graphics processing request 1011, thereby acquiring a jump command and an entry parameter x, y, z that jump to the function address I of the first graphics API at the server 10.
  • Step S104 The first graphics processing device 105 determines whether the first graphics API needs to return an exit parameter.
  • the first graphics processing apparatus 105 can determine that the first graphics API needs to return an exit parameter, and therefore the flow jumps to step S105.
  • Step S105 The first graphics processing device 105 invokes the first graphics API of the first graphics library 103. Specifically, the first graphics processing device 105 jumps to the function address I of the first graphics API at the server 10, and sets the entry parameter x, y, z is used as an input parameter to run the drawing code on the function address I. After the drawing code is finished running, the exit parameter retA is obtained as the running result, and the first processing notification 1012 including the exit parameter retA is sent (as shown in FIG. 7). To the application 104, the flow then jumps to step S111 and step S106, respectively.
  • the first processing notification 1012 is used to notify the application 104 that the current first graphics API call is completed.
  • Step S106 The first graphics processing device 105 sets a first identification code 0009 for the first graphics API according to Table 1, and uses the first identification code 0009, the entry parameter x, y, z and the exit parameter retA as the calling information of the first graphics API. 1001 is set in the FIFO queue 1000.
  • Step S112 The first graphics processing device 105 starts timing after the first in first out queue 1000 is stored in the call information 1001 of the first graphics API.
  • the first in first out queue 1000 is based on 2 bytes, and the first graphics processing device 105 respectively stores the identification code 0009, the entry parameter x, the entry parameter y, the entry parameter z, and the exit parameter retA into advanced
  • the first byte of the queue 1000 is out of the eight bytes. Therefore, the call information 1001 of the first graphics API includes 8 bytes.
  • Step S111 The application 104 continues to run the program code according to the first processing notification 1012, and initiates the second graphics instruction 1013 when running to the program code that needs to invoke the second graphics API.
  • the application 104 after receiving the first processing notification 1012, the application 104 continues to run, and when running to the program code that needs to invoke the second graphics API, initiates a second drawing instruction 1013, where the second drawing instruction A jump command to jump to the function address J of the second graphics API at the server 10 and an entry parameter retA carried by the first processing notification 1012 are included.
  • Step S103 The first graphics processing device 105 acquires the second drawing instruction 1013, thereby acquiring a jump command and an entry parameter retA that jump to the function address J of the second graphics API at the server 10.
  • Step S104 The first graphics processing device 105 determines whether the second graphics API needs to return an exit parameter.
  • the second graphics API needs to return the exit parameter. Therefore, in this step, the first graphics processing apparatus 105 determines that the second graphics API needs to return the exit parameter, and the flow jumps to step S105.
  • Step S105 The first graphics processing device 105 invokes the second graphics API in the first graphics library 103. Specifically, the first graphics processing device 105 jumps to the function address J of the server 10 at the second graphics API according to the jump command.
  • the entry parameter retA is used as the input parameter to run the drawing code on the function address J, and after the drawing code is run, the exit parameter retB is obtained as the running result, the retB is returned to the application 104, and the exit parameter retB including the second graphics API is sent.
  • the second process notification 1014 to the application 104 and then jumps to step S111 and step S106, respectively.
  • the second processing notification 1014 is used to notify the application 104 that the current second graphics API call is completed.
  • Step S111 As shown in FIG. 5, since in the embodiment, the program instruction only calls the first graphics API and the second graphics API in a single time, the application 104 does not exist after the above two graphics APIs are called. Another graphics API, there is no need to continue to call the new graphics API, so another drawing instruction will not be initiated, and the application 104 exits the operation according to the second processing notification 1012, looping to this abort.
  • Step S106 The first graphics processing device 105 sets the identification code 000A for the second graphics API according to Table 1, and sets the identification code 000A, the entry parameter retA and the exit parameter retB as the call information 1002 of the second graphics API in the first in first out queue 1000. .
  • Step S112 The first graphics processing device 105 sends the two sets of call information in the FIFO queue 1000 to the client 20 through the network card 112 after the predetermined time arrives.
  • the predetermined time may be set to be long enough for the application 104 to complete the invocation of the first graphics API and the second graphics API, respectively.
  • the predetermined time may be, for example, 3 ms.
  • the first in first out queue 1000 includes call information 1001 and call information 1001.
  • the call information 1001 includes a first identification code 0009, an entry parameter x, y, z and an exit parameter retA
  • the call information 1002 includes a second identification code 000A, an entry. Parameter retA and exit parameter retB.
  • the first in first out queue 1000 is based on 2 bytes
  • the first graphics processing device 105 stores the first identification code 0009, the entry parameter x, the entry parameter y, the entry parameter z, and the exit parameter retA, respectively.
  • the call information 1001 includes 8 bytes
  • the second identification code 000A, the entry parameter retA and the exit parameter retB are respectively stored in the last 6 of the FIFO queue 1000.
  • the call information 1002 includes 6 bytes. Therefore, the total data length of the two sets of call information is 14 bytes.
  • the application 104 of the server 10 runs locally on the server 10, the server 10 sends N sets of data to the client 20 in one direction, and the server 10 does not need to receive data from the client 20, so the slave client can be omitted. 20
  • the waiting time for receiving data, the running speed of the application 104 is greatly improved.
  • FIG. 8 is another flowchart of a graphics processing method according to an embodiment of the present invention.
  • FIG. 8 illustrates a workflow of the client 20.
  • the graphics processing method specifically includes the following steps:
  • Step S301 The second graphics processing device 204 receives a plurality of sets of call information transmitted by the server 10.
  • the second graphics processing device 204 receives the plurality of sets of call information transmitted by the first graphics processing device 105 of the server 10 via the network card 212.
  • Step S302 The second graphics processing device 204 reads the identification code of the first group of call information.
  • the first set of call information is located at a starting position of the plurality of sets of call information, and the identification code is located at a start position of the first set of call information.
  • Step S303 The second graphics processing device 204 confirms the graphics API to be called according to the read identification code.
  • the second graphics processing device 204 looks up the table 2 based on the read identification code, thereby confirming the function address of the graphics API to be called at the client 20, thereby confirming the graphics API to be called.
  • Step S304 The graphics processing apparatus determines whether the entry parameter of the graphics API is an exit parameter of another graphics API. If yes, step S305 is performed, and if no, step S306 is performed.
  • the graphics API is divided into two types, one type is a type that can return an exit parameter, and the other type is a type that does not return an exit parameter.
  • the embodiment of the present invention is based on the type and relationship of the determined graphics API. Identification code is located The entry parameters of the same set of call information confirm the entry parameters of the graphics API to be called.
  • steps S305 and S306 respectively perform corresponding processing for the two types of graphics APIs.
  • Step S305 The second graphics processing device 204 reads the entry parameter of the same group of call information as the current identifier, searches for the corresponding exit parameter in the mapping table according to the entry parameter, and uses the searched exit parameter as the currently invoked graphics API. Entry parameters.
  • the mapping table records a mapping relationship between an exit parameter generated by the server calling a specific graphics API and an exit parameter generated by the client calling the specific graphics API, and the mapping table is generated according to step S309 described below.
  • Step S306 The second graphics processing device 204 reads the entry parameter of the same set of call information as the current identifier, and uses the read entry parameter as the entry parameter of the currently invoked graphics API.
  • Step S307 The second graphics processing device 204 invokes the graphics API of the second graphics library 203 according to the confirmed entry parameters.
  • Step S308 The second graphics processing device 204 determines whether the graphics API returns an exit parameter. If yes, step S309 is performed, and if no, step S310 is performed.
  • Step S309 The second graphics processing device 204 reads the exit parameter of the same group of call information as the current identifier, and records the read exit parameter and the returned exit parameter in the mapping table.
  • Step S310 The second graphics processing device 204 determines whether there is a next set of call information. If yes, step S312 is performed. If no, step S301 is performed. In step 301, the second graphics processing device 204 continues to receive another message sent by the server 10. A number of sets of call information are passed, and steps 301 through 312 are continued for another set of call information.
  • Step S312 The second graphics processing device 204 reads the identification code recorded in the next set of call information, and jumps to step S303, so as to continue to perform steps 303 to 312 for the next set of call information.
  • the client 20 can locally call a plurality of graphics APIs that are called by the server 10 according to the received group call information, so that the graphics are locally drawn, so that the user of the client 20 can be locally at the client 20.
  • the interface drawn when the application 104 of the server 10 is running is seen. Also, the client 20 does not need to transmit data to the server 10, thus implementing one-way transmission.
  • the first graphics API and the second graphics API shown in Table 1 and Table 2 may be used as an example, and the group calling information is specifically in FIG. 7 and its corresponding description.
  • the two sets of call information include the first set of call information 1001 and the second set of call information 1002.
  • FIG. 9 is a data flow diagram of the client according to an embodiment of the present invention.
  • Step S301 The second graphics processing device 204 receives two sets of call information sent by the server 10.
  • the two sets of call information includes a first set of call information 1001 and a second set of call information 1002, wherein the first set of call information 1001 is located in the two sets of call information.
  • the header, the identification code 0009 is located at the head of the first set of call information 1001.
  • Step S302 The second graphics processing device 204 reads the identification code 0009 of the first group of call information 1001 of the two sets of call information.
  • Step S303 The second graphics processing device 204 searches the table 2 according to the read identification code 0009, and obtains the function address I' of the first graphics API to be called at the client 20, thereby confirming the first graphics API to be called.
  • Step S304 The graphics processing device 204 determines whether the entry parameter of the first graphics API is an exit parameter of another graphics API, and if the determination result is no, step S306 is performed.
  • Step S306 The second graphics processing device 204 reads the entry parameters x, y, z of the same set of call information 1001 with the current identification code 0009, and uses the read entry parameters x, y, z as the first graphic currently called. API entry parameters.
  • Step S307 The second graphics processing device 204 calls the first graphics API of the second graphics library 203 according to the confirmed entry parameters x, y, z.
  • the second graphics processing device 204 jumps to the function address I' of the first graphics API at the client 20, and uses the entry parameter x, y, z as the input parameter to run the drawing code on the function address I', and the drawing code is used.
  • the control GPU 211 performs graphic drawing, and obtains an exit parameter retA' as a running result after the drawing code is finished running.
  • this step corresponds to step 1 of FIG.
  • Step S308 The second graphics processing device 204 determines whether the first graphics API returns an exit parameter, and if the result of the determination is yes, step S309 is performed.
  • Step S309 The second graphics processing device 204 reads the exit parameter retA of the same set of call information 1001 with the current identification code 0009, and records the mapping relationship between the read exit parameter retA and the returned exit parameter retA' in the mapping table 2041. (See Figure 9).
  • this step corresponds to step 2 of FIG.
  • Step S310 The second graphics processing device 204 determines whether there is a next set of call information. Since the second set of call information 1002 still exists, the determination result is yes, and step S312 is performed.
  • Step S312 The second graphics processing device 204 reads the identification code 000A recorded in the next set of call information 1002, and jumps to step S303 to cause the flow to be executed cyclically.
  • Step S303 The second graphics processing device 204 searches the table 2 based on the read identification code 000A, and obtains the function address J' of the second graphics API to be called at the client 20, thereby confirming the second graphics API to be called.
  • This step corresponds to the step of acquiring the API 2 based on the identification code in the step 3 of FIG.
  • Step S304 The second graphics processing device 204 determines whether the entry parameter of the second graphics API is an exit parameter of another graphics API, and if the determination result is yes, step S305 is performed.
  • the entry parameter retA' of the second graphics API when the client 20 is called is the exit parameter retA' when the first graphics API is called by the client 20, so the second graphics processing device 204 performs step 305.
  • Step 305 The second graphics processing device 204 reads the entry parameter retA of the same set of call information 1002 with the identification code 000A, searches for the corresponding exit parameter retA' in the mapping table 2041 according to the entry parameter retA, and searches for the exported parameter retA. 'As the entry parameter retA' of the second graphics API currently called.
  • this step corresponds to the step of reading the entry parameter retA and the step 4 in step 3 of FIG.
  • Step S307 The second graphics processing device 204 calls the second graphics API of the second graphics library 203 based on the confirmed entry parameter retA'.
  • This step corresponds to step 5 of FIG.
  • the second graphics processing device 204 can jump to the second graphics API at the client 20
  • the function address J', and the entry parameter retA' is used as an input parameter to execute the drawing code on the function address J'.
  • the drawing code is used to control the GPU 211 to perform graphic drawing, and obtain the exit parameter as the running result after the drawing code is finished running. retB'.
  • Step S308 The second graphics processing device 204 determines whether the second graphics API returns an exit parameter, and if the determination result is yes, step S309 is performed.
  • Step S309 The second graphics processing device 204 reads the exit parameter retB of the same group call information 1002 as the current identification code 000A, and records the read exit parameter retB and the returned exit parameter retB' in the mapping table 2041.
  • Step S310 The second graphics processing device 204 determines whether there is a next set of call information. Since it is assumed that the group call information is specifically two sets of call information, the next set of call information does not exist, so the determination result is no, and step S301 is performed. Continue to receive another set of call information for similar processing.
  • the second graphics processing device 204 can locally call the two graphics APIs according to the received two sets of call information as shown in FIG. 6, thereby completing the graphics processing.
  • the first graphics processing device 105 may separately generate the generated sets of call information into a file and store the file.
  • the client 20 can download the file from the server 10 after a certain period of time (for example, a few days later), and the second graphics processing device 204 sequentially reads the plurality of sets of call information recorded in the file according to the flow shown in FIG. Thereby playback of the drawing interface of the application 104 is achieved.
  • FIG. 10 is a schematic structural diagram of a device of a graphics processing apparatus according to an embodiment of the present invention.
  • the graphics processing apparatus is disposed on a server 10, as shown in FIG.
  • the graphics processing device 105 includes:
  • the drawing instruction obtaining module 1051 is configured to acquire a first drawing instruction initiated by the application 104, where the first drawing instruction corresponds to the first graphics API;
  • the graphic API calling module 1052 is configured to invoke the first graphics API of the first graphics library 103 according to the first drawing instruction to execute the first drawing instruction, and send a first processing notification to the application 104, where the first processing notification is used to notify the application
  • the first graphics API of the program 104 is called;
  • the call information generating module 1053 is configured to generate call information of the first graphics API, where the calling information of the first graphics API is used to instruct the client 20 to invoke the first graphics API in the second graphics library 203, and the calling information of the first graphics API Include parameters involved in the first graphics API, and an identification code of the first graphics API;
  • the information sending module 1054 is configured to send the calling information of the first graphics API to the client 20.
  • the call information generating module 1053 is further configured to: determine whether the first drawing instruction needs to return an exit parameter; in a case that the first drawing instruction needs to return an exit parameter, the first processing notification further includes the server 10 calling the first graphics API.
  • the generated exit parameter, the call information of the first graphics API includes an entry parameter of the first graphics API, an exit parameter generated by the server 10 to invoke the first graphics API, and an identifier of the first graphics API.
  • the call information of the first graphics API includes an entry parameter of the first graphics API and an identifier of the first graphics API.
  • the drawing instruction obtaining module 1051 is configured to acquire a second drawing instruction initiated by the application 104 according to the first processing notification, where the second drawing instruction corresponds to the second graphic API, and the entry parameter of the second graphic API is called by the server 10
  • An export parameter generated by a graphics API a graphics API calling module 1052, configured to invoke a second graphics API of the first graphics library 103 according to the second drawing instruction, and send a second processing notification to the application 104, where the second processing notification is Yu
  • the notification application 104 is configured to call the second graphics API.
  • the call information generating module 1053 is configured to generate call information of the second graphics API, where the call information of the second graphics API is used to instruct the client 20 to call the second graphics library 203.
  • the second graphics API, the call information of the second graphics API includes the parameters involved in the second graphics API, and the identifier of the second graphics API, and the call information sending module 1054 is configured to send the call information of the second graphics API to the client 20 .
  • the information sending module 1054 is configured to: save the calling information of the first graphics API and the calling information of the second graphics API in a first in first out queue; and after the predetermined condition is met, the first in first out queue The data is simultaneously sent to the client 20.
  • the application can immediately obtain the first processing notification for notifying the application that the first graphics API is called, so the application of the server
  • the program can run independently without relying on the client. Even if the client crashes or the connection between the client and the server is interrupted, the application of the server will not be affected, and the user experience can be improved.
  • the server generates call information of the first graphics API that carries the parameters involved in the first graphics API, and sends the call information of the first graphics API to the client, where the client may invoke the second according to the parameters involved in the first graphics API.
  • the first graphics API in the graphics library since the application of the server has received the first processing notification, after the client calls the first graphics API in the second graphics library, there is no need to send an application that processes the notification to the server. Therefore, one-way transmission between the server and the client is implemented, thereby reducing the requirement for network delay.
  • FIG. 11 is a schematic structural diagram of another apparatus of a graphics processing apparatus according to an embodiment of the present invention.
  • the graphics processing apparatus is disposed on the client 20, as shown in FIG.
  • the graphics processing device 204 includes:
  • the call information receiving module 2041 is configured to receive call information of the first graphics API sent by the server 10, where the call information of the first graphics API includes parameters related to the first graphics API, and an identifier of the first graphics API;
  • the graphic API calling module 2042 is configured to determine the first graphics API according to the identifier of the first graphics API in the call information of the first graphics API, and invoke the parameter according to the first graphics API in the call information of the first graphics API.
  • the first graphics API in the second graphics library 203 is configured to determine the first graphics API according to the identifier of the first graphics API in the call information of the first graphics API, and invoke the parameter according to the first graphics API in the call information of the first graphics API.
  • the first graphics API in the second graphics library 203 is configured to determine the first graphics API according to the identifier of the first graphics API in the call information of the first graphics API, and invoke the parameter according to the first graphics API in the call information of the first graphics API.
  • the first graphics API in the second graphics library 203 is configured to determine the first graphics API according to the identifier of the first graphics API in the call information of the first graphics API, and invoke the parameter according to the first graphics API in the call information of the first graphics API.
  • the graphics processing device 204 further includes a mapping relationship establishing module 2043, where the parameter involved in the first graphics API in the calling information of the first graphics API includes a case where the server 10 invokes an exit parameter generated by the first graphics API.
  • the exit parameter generated by the client 20 to invoke the first graphics API is obtained, and a mapping relationship between the exit parameter generated by the server 10 to invoke the first graphics API and the exit parameter generated by the client 20 to invoke the first graphics API is established.
  • the call information receiving module 2041 is further configured to receive call information of the second graphics API sent by the server 10, where the call information of the second graphics API includes a parameter related to the second graphics API and an identifier of the second graphics API,
  • the entry parameter of the second graphics API is the export parameter generated by the server 10 to invoke the first graphics API;
  • the graphics API calling module 2042 is further configured to determine the second graphics API according to the identifier of the second graphics API;
  • the parameter is also used to query the mapping relationship according to the entry parameter of the second graphics API, and obtain the parameter corresponding to the entry parameter of the second graphics API in the mapping relationship, and the parameter corresponding to the entry parameter of the second graphics API is called by the client 20 in the mapping relationship.
  • the information receiving module 2041 is configured to: receive data sent by the server 10, where the data includes call information of the first graphics API and call information of the second graphics API.
  • the client receives the call information of the first graphics API from the server, and may invoke the first graphics API in the second graphics library according to the parameters involved in the first graphics API, because the first graphics API call of the first graphics library of the server has been completed, Therefore, after the client calls the first graphics API in the second graphics library, there is no need to send an application that processes the notification to the server, so a one-way transmission is implemented between the server and the client, thereby reducing network delay. Claim.
  • FIG. 12 is a schematic structural diagram of another apparatus of a graphics processing apparatus according to an embodiment of the present invention.
  • the graphics processing apparatus is disposed on the server 10, as shown in FIG.
  • the graphics processing device 105 includes:
  • a drawing instruction obtaining module 1061 configured to acquire a first drawing instruction initiated by the application 104, where the first drawing instruction corresponds to the first graphics API;
  • the graphics API calling module 1062 is configured to: when determining that the first graphics API is a predetermined graphics API of the first graphics library, rejecting the first graphics API of the first graphics library;
  • the graphics API invoking module 1062 is further configured to send a first processing notification to the application 104, where the first processing notification is used to notify the application 104 that the first graphics API call is completed;
  • the call information generating module 1063 is configured to generate call information of the first graphics API, where the calling information of the first graphics API is used to instruct the client to invoke the first graphics API in the second graphics library, where the calling information of the first graphics API includes a parameter involved in a graphics API, and an identification code of the first graphics API;
  • the information sending module 1064 is configured to send the calling information of the first graphics API to the client.
  • the calling information of the first graphics API includes an entry parameter of the first graphics API and an identifier of the first graphics API.
  • the predetermined graphics API includes any one of a shader startup graphics API, a rasterization startup graphics API, or an exchange buffer graphics API.
  • the server application can run independently without relying on the client. Even if the client crashes or the connection between the client and the server is interrupted, the application of the server will not be affected, and the user experience can be improved.
  • the server does not need to display the drawn graphics to the user, even if the first graphics API corresponding to the first drawing instruction initiated by the application of the server is rejected, so that the GPU of the server does not work, the user experience is not affected.
  • FIG. 13 is a schematic structural diagram of a device according to an embodiment of the present invention.
  • the server 10 includes a processor 121, a bus 122, and a memory 123.
  • the processor 121 and the memory 123 are connected to the bus 122.
  • the memory 123 stores program instructions, and the processor 121 executes program instructions to perform the functions of the first graphics processing device 105 described above.
  • FIG. 14 is a schematic structural diagram of a device of a client according to an embodiment of the present invention.
  • the client 20 includes a processor 221, a bus 222, and a memory 223, and a processor 221 and a memory 223 and a bus 222.
  • memory 223 stores program instructions
  • processor 221 executes program instructions to perform the functions of second graphics processing device 204 described above.
  • any of the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as the unit may be or may be It is not a physical unit, it can be located in one place, or it can be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • the connection relationship between the modules indicates that there is a communication connection between them, and specifically, one or more communication buses or signal lines can be realized.
  • the present invention can be implemented by means of software plus necessary general hardware, and of course, dedicated hardware, dedicated CPU, dedicated memory, dedicated memory, Special components and so on.
  • functions performed by computer programs can be easily implemented with the corresponding hardware, and the specific hardware structure used to implement the same function can be various, such as analog circuits, digital circuits, or dedicated circuits. Circuits, etc.
  • software program implementation is a better implementation in more cases.
  • the technical solution of the present invention which is essential or contributes to the prior art, can be embodied in the form of a software product stored in a readable storage medium, such as a floppy disk of a computer.
  • U disk mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., including a number of commands to make a computer device (may be A personal computer, server, or network device, etc.) performs the methods described in various embodiments of the present invention.
  • a computer device may be A personal computer, server, or network device, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Information Transfer Between Computers (AREA)
  • Image Generation (AREA)
  • Stored Programmes (AREA)

Abstract

本申请公开了一种图形处理方法及相关装置和设备,该方法包括:获取应用程序发起的对应第一图形API的第一绘图指令;根据第一绘图指令调用第一图形库的第一图形API以执行第一绘图指令,并向应用程序发送第一处理通知;生成第一图形API的调用信息,第一图形API的调用信息用于指示客户端调用第二图形库中的第一图形API,第一图形API的调用信息包括第一图形API涉及的参数,以及第一图形API的识别码;将第一图形API的调用信息发送至客户端。通过以上方案,服务器向客户端单向数据传输图形API的调用信息,可降低对网络延时的要求,且在网络断线状态下保证服务器的应用程序正常运行,从而提高用户体验。

Description

图形处理方法及相关装置和设备 技术领域
本申请涉及信息技术领域,通过服务器对客户端的单向数据传输来实现远程界面绘制。
背景技术
随着移动互联网和云计算的发展,由于云计算便于管理、存储容量大、高稳定性等特点,将应用程序从客户端迁移到服务器成为未来发展的趋势,在将客户端的应用程序迁移到服务器后,在服务器运行的应用程序的图形界面(User Interface,UI)需通过广域网显示到终端,使得客户端和服务器的数据交互越来越频繁。
现有技术中,基于客户端-服务器(client-server,C-S)模式的开放图形库(Open Graphics Library,Opengl)可实现应用程序的远程显示界面绘制,在客户端-服务器架构中,客户端设置有图形处理器(Graphics Processing Unit,GPU),且安装有Opengl图形库,在Opengl图形库中设置有图形应用程序编程接口(Application Programming Interface,API)。而服务器没有设置GPU,服务器也没有安装Opengl图形库,但服务器上运行有应用程序,该应用程序需要在客户端的Opengl图形库调用图形API。当应用程序需要调用图形API时,服务器将需要调用的图形API发送至客户端,客户端在本地的Opengl图形库调用图形API,在图形API调用过程中,客户端运行图形API的绘图代码,该绘图代码控制客户端本地的GPU进行图形绘制。
在现有技术中,每帧图像的生成,需要多次图形API的调用,且图形API的调用大多是以串行调用为主,举例而言,服务器的应用程序需要依序调用2个图形API,应用程序在服务器本地不进行2个图形API的调用,而是分别将2个图形API发送至客户端,由客户端进行图形API的调用。
在现有技术中,服务器的应用程序将第一个图形API及第一图形API的入口参数发送至客户端,并在上述数据发送至客户端之后进行等待。在应用程序等待期间,客户端根据第一图形API的入口参数在客户端本地的Opengl图形库调用第一个图形API,在调用完毕之后,客户端将用于通知应用程序第一图形API调用完毕的处理通知发回服务器的应用程序,服务器的应用程序在接收到客户端发送的处理通知后,才能结束等待,将第二图形API的入口参数和第二图形API发送至客户端。
因此,现有技术中,服务器与客户端之间每进行一次图形API调用时均需要进行一次双向的数据传输,服务器需花费时间来等待客户端返回的处理通知。
在低延时网络,如局域网,上述的远程图形API调用的网络延时在1毫秒以内,而1秒可以完成大于1000次调用,在所需绘制的图形界面不复杂的情况下,可以提供较好的体验。
但是,如果在广域网,尤其是在无线广域网的环境下,如3G或4G网络,上述远程图形API调用的网络延时在100~300毫秒之间,则1秒内最多完成10次调用,只有局域网1秒内可执行图形API数量的1%,因此,现有技术对网络延时要求较高,在高延时网络无法 提供良好的用户体验。
发明内容
本申请公开了一种图形处理方法及相关装置和设备,服务器向客户端单向数据传输图形API的调用信息,可降低对网络延时的要求,且在网络断线状态下保证服务器的应用程序正常运行,从而提高用户体验。
第一方面,本申请提供一种图形处理方法,该方法应用于服务器,具体的,服务器设置有第一图形库和应用程序,服务器与客户端具有网络连接,客户端设置有第二图形库,第一图形库中包括多个图形应用程序编程接口API,第二图形库中包括多个图形API,第一图形库中至少有M个图形API与第二图形库中的M个图形API一一对应。基于以上配置环境,该图形处理方法包括以下步骤:获取应用程序发起的对应第一图形API的第一绘图指令。根据第一绘图指令调用第一图形库的第一图形API以执行第一绘图指令,并向应用程序发送第一处理通知。第一处理通知用于通知应用程序第一图形API调用完毕,生成第一图形API的调用信息,具体地,第一图形API的调用信息用于指示客户端调用第二图形库中的第一图形API,并且,第一图形API的调用信息包括第一图形API涉及的参数,以及第一图形API的识别码。最后,将第一图形API的调用信息发送至客户端。
由于服务器的应用程序发起的第一绘图指令对应的第一图形API在调用完毕之后,应用程序可马上在本地获取用于通知应用程序第一图形API调用完毕的第一处理通知,因此服务器的应用程序可不依赖于客户端而独立运行,即便客户端死机,或客户端与服务器之间的连接中断,也不会影响服务器的应用程序的运行,可提高用户体验。
进一步,服务器生成携带有第一图形API涉及的参数的第一图形API的调用信息,并将第一图形API的调用信息发送至客户端,客户端可根据第一图形API涉及的参数调用第二图形库中的第一图形API,由于服务器的应用程序已经收到第一处理通知,客户端调用完第二图形库中的第一图形API之后,无需再发送处理通知给服务器的应用程序,因此服务器与客户端之间实现了单向传输,从而可降低对网络延时的要求。
在第一方面的第一种可能的实现方式中,该图形处理方法还包括以下步骤:确定第一绘图指令是否需要返回出口参数,在第一绘图指令需要返回出口参数的情况下,在第一处理通知进一步设置服务器调用第一图形API所产生的出口参数,并且,第一图形API的调用信息具体包括第一图形API的入口参数、服务器调用第一图形API所产生的出口参数和第一图形API的识别码。
在第一绘图指令需要返回出口参数的情况下,在第一处理通知进一步设置服务器调用第一图形API所产生的出口参数,使得应用程序通过第一处理通知可获取服务器调用第一图形API所产生的出口参数,在后续调用到输入参数为第一图形API的出口参数的其他图形API时,可将获取到的出口参数作为其他图形API的入口参数,从而保证应用程序正常运行。
并且,将第一图形API的入口参数、服务器调用第一图形API所产生的出口参数和第一图形API的识别码设置在发送至客户端的第一图形API的调用信息中,使得客户端可根据第一图形API的入口参数、服务器调用第一图形API所产生的出口参数和第一图形API 的识别码在本地调用第二图形库中的第一图形API。
根据第一方面的第一种可能的实现方式,在第二种可能的实现方式中,在第一图形API不需要返回出口参数的情况下,第一图形API的调用信息包括第一图形API的入口参数和第一图形API的识别码。
在第一绘图指令不需要返回出口参数的情况下,将第一图形API的入口参数和第一图形API的识别码设置在发送至客户端的第一图形API的调用信息中,使得客户端可根据第一图形API的入口参数和第一图形API的识别码本地调用第二图形库中的第一图形API。
根据第一方面的第一种可能的实现方式,在第三种可能的实现方式中,该图形处理方法还包括以下步骤:接收应用程序根据第一处理通知发起的第二绘图指令,具体地,第二绘图指令对应第二图形API,第二图形API的入口参数为服务器调用第一图形API所产生的出口参数。根据第二绘图指令调用第一图形库的第二图形API,并向应用程序发送第二处理通知,具体地,第二处理通知用于通知应用程序第二图形API调用完毕。生成第二图形API的调用信息,具体地,第二图形API的调用信息用于指示客户端调用第二图形库中的第二图形API,并且,第二图形API的调用信息包括第二图形API涉及的参数,以及第二图形API的识别码。之后,将第二图形API的调用信息发送至客户端。
由于在第一方面的第一种可能的实现方式中,第一处理通知进一步设置服务器调用第一图形API所产生的出口参数,因此,应用程序可从第一处理通知获取服务器调用第一图形API所产生的出口参数,并将第一图形API所产生的出口参数作为第二图形API的入口参数来调用第一图形库的第二图形API,并向应用程序发送第二处理通知,从而使得应用程序继续运行,因此,即便第二图形API与第一图形API存在依赖关系,应用程序也无需等待客户端返回数据,从而节约时间,降低对网络延时的要求,提高用户体验。
根据第一方面的第三种可能的实现方式,在第四种可能的实现方式中,该图形处理方法还包括:将第一图形API的调用信息和第二图形API的调用信息保存在先进先出队列,在满足预定条件后,将先进先出队列中的第一图形API的调用信息和第二图形API的调用信息同时发送给客户端。
应用程序的运行速度较快,应用程序运行时,单位时间内产生多组调用信息,通过打包方式在满足预定条件后同时发送多组调用信息至客户端,相对于每产生一组调用信息就发送一次调用信息给客户端而言,本实现方式在服务器进行多次图形API的调用之后,仅需向客户端发送一次数据,因此可有效节约处理开销。
在一种可能的实现方式中,预定条件为超出预定时间。
在一种可能的实现方式中,预定条件为先进先出队列中的数据超出预定数据大小。
在一种可能的实现方式中,该图形处理方法还包括以下步骤:将第一图形API的调用信息和第二图形API的调用信息写入文件中,并存储该文件。
客户端可隔一定时间后(如几天后)再从服务器下载该文件,依次读取该文件中记录的多组调用信息,从而实现对应用程序的绘图界面的回放。
第二方面,本申请提供一种图形处理方法,该方法应用于客户端,具体地,客户端与服务器具有网络连接,服务器设置有第一图形库和应用程序,客户端设置有第二图形库,第一图形库中包括多个图形应用程序编程接口API,第二图形库中包括多个图形API,第 一图形库中至少有M个图形API与第二图形库中的M个图形API一一对应。基于以上配置环境,图形处理方法具体包括以下步骤:接收服务器发送的第一图形库中的第一图形API的调用信息,具体地,第一图形API的调用信息包括第一图形API涉及的参数,以及第一图形API的识别码。根据第一图形API的调用信息中的第一图形API的识别码和第一图形API的调用信息中的第一图形API涉及的参数调用第二图形库中的第一图形API。
服务器在第一图形库的第一图形API调用完成之后,生成携带有第一图形API涉及的参数的第一图形API的调用信息,并将第一图形API的调用信息发送至客户端,客户端从服务器接收第一图形API的调用信息,可根据第一图形API涉及的参数调用第二图形库中的第一图形API,由于服务器的第一图形库的第一图形API调用已经完成,因此,客户端调用完第二图形库中的第一图形API之后,无需再发送处理通知给服务器的应用程序,因此服务器与客户端之间实现了单向传输,因此可降低对网络延时的要求。
在第二方面的第一种可能的实现方式中,该图形处理方法还包括:在第一图形API的调用信息中的第一图形API涉及的参数包括服务器调用第一图形API所产生的出口参数的情况下,获取客户端调用第一图形API所产生的出口参数,并建立服务器调用第一图形API所产生的出口参数与客户端调用第一图形API所产生的出口参数的映射关系。
由于客户端与服务器之间的硬件配置条件不同,因此,服务器调用第一图形API所产生的出口参数与客户端调用第一图形API产生的出口参数也不相同,在第一图形API产生的出口参数为其他图形API的入口参数的情况下,客户端不能将第一图形API的调用信息携带的服务器调用第一图形API所产生的出口参数直接作为其他图形API的入口参数,因此客户端建立服务器调用第一图形API所产生的出口参数与客户端调用第一图形API所产生的出口参数的映射关系,客户端可通过服务器调用第一图形API所产生的出口参数查询映射关系,来获取客户端调用第一图形API所产生的出口参数,并将客户端调用第一图形API所产生的出口参数作为其他图形API的入口参数。从而克服由于客户端与服务器之间的硬件配置条件不同而造成的出口参数不同步的技术问题。
根据第二方面的第一种可能的实现方式,在第二种可能的实现方式中,该方法还包括:接收服务器发送的第二图形API的调用信息,具体地,第二图形API的调用信息包括第二图形API涉及的参数以及第二图形API的识别码,并且,第二图形API的入口参数为服务器调用第一图形API所产生的出口参数。根据第二图形API的识别码确定第二图形API,根据第二图形API的入口参数查询映射关系,获取第二图形API的入口参数在映射关系中对应的参数,具体的,第二图形API的入口参数在映射关系中对应的参数为客户端调用第一图形API所产生的出口参数。
具体地,在第一图形API产生的出口参数为第二图形API的入口参数的情况下,客户端不能将第一图形API的调用信息携带的服务器调用第一图形API所产生的出口参数直接作为第二图形API的入口参数,因此客户端可通过服务器调用第一图形API所产生的出口参数查询映射关系,来获取客户端调用第一图形API所产生的出口参数,并将客户端调用第一图形API所产生的出口参数作为第二图形API的入口参数。从而克服由于客户端与服务器之间的硬件配置条件不同而造成的出口参数不同步的技术问题。
根据第二方面的第二种可能的实现方式中,在第三种可能的实现方式中,该方法还包 括:接收服务器同时发送第一图形API的调用信息和第二图形API的调用信息。
客户端接收服务器同时发送的数据,并从该数据中依次读取到第一图形API的调用信息和第二图形API的调用信息,无需分别接收第一图形API的调用信息和第二图形API的调用信息,可有效节约处理开销。
在第二方面的一种可能的实现方式中,根据第一图形API的调用信息中的第一图形API的识别码和第一图形API的调用信息中的第一图形API涉及的参数调用第二图形库中的第一图形API具体包括:根据第一图形API的调用信息中的第一图形API的识别码确定第一图形API,并根据第一图形API的调用信息中的第一图形API涉及的参数调用第二图形库中的第一图形API。
第三方面,本申请提供一种图形处理方法,该方法应用于服务器,具体地,服务器设置有第一图形库和应用程序,客户端设置有第二图形库,第一图形库中包括多个图形应用程序编程接口API,第二图形库中包括多个图形API,第一图形库中至少有M个图形API与第二图形库中的M个图形API一一对应。基于以上配置环境,该方法包括以下步骤:获取应用程序发起的第一绘图指令,具体地,第一绘图指令对应第一图形API。确定第一图形API为第一图形库的预定图形API时,拒绝调用第一图形库的第一图形API,向应用程序发送第一处理通知,具体地,第一处理通知用于通知应用程序第一图形API调用完毕。生成第一图形API的调用信息,具体地,第一图形API的调用信息用于指示客户端调用第二图形库中的第一图形API,并且,第一图形API的调用信息包括第一图形API涉及的参数,以及第一图形API的识别码。之后,将第一图形API的调用信息发送至客户端。
当第一图形API为预定图形API时,拒绝调用服务器的应用程序发起的第一绘图指令对应的第一图形API,并发送第一处理通知给应用程序,以通知应用程序第一图形API调用完毕,因此服务器的应用程序可不依赖于客户端而独立运行,即便客户端死机,或客户端与服务器之间的连接中断,也不会影响服务器的应用程序的运行,可提高用户体验。
并且,由于服务器无需将绘制好的图形显示给用户,因此即使拒绝调用服务器的应用程序发起的第一绘图指令对应的第一图形API,从而使得服务器的GPU不工作,也不会影响用户体验。
进一步,服务器生成携带有第一图形API涉及的参数的第一图形API的调用信息,并将第一图形API的调用信息发送至客户端,客户端可根据第一图形API涉及的参数调用第二图形库中的第一图形API,由于服务器的应用程序已经收到第一处理通知,因此,客户端调用完第二图形库中的第一图形API之后,无需再发送处理通知给服务器的应用程序,因此服务器与客户端之间实现了单向传输,因此可降低对网络延时的要求。
在第三方面的第一种可能的实现方式中,第一图形API的调用信息包括第一图形API的入口参数和第一图形API的识别码。
根据第三方面或第三方面的第一种可能的实现方式,在第二种可能的实现方式中,预定图形API包括着色器启动图形API、光栅化启动图形API、清除缓冲区图形API或交换缓冲区图形API中的任意一种。
着色器启动图形API、光栅化启动图形API、清除缓冲区图形API以及交换缓冲区图形API所对应的绘图代码在执行时,需要服务器的GPU进行大量的运算,在判断到当前调 用的第一图形API为预定图形API时,拒绝调用第一图形API,可使得GPU避免进行大量运算,从而可节约服务器上宝贵的运算资源,使得服务器可以运行数量更多的应用程序。
在一种可能的实现方式中,预定图形API用于控制服务器的GPU进行色器启动、光栅化启动、清除缓冲区或交换缓冲区。
在一种可能的实现方式中,预定图形API用于控制服务器的GPU进行大量的图形运算。第四方面,本申请提供一种图形处理装置,设置于服务器,该服务器还设置有第一图形库和应用程序,服务器与客户端具有网络连接,客户端设置有第二图形库,第一图形库中包括多个图形应用程序编程接口API,第二图形库中包括多个图形API,第一图形库中至少有M个图形API与第二图形库中的M个图形API一一对应,该图形处理装置包括:绘图指令获取模块,用于获取应用程序发起的第一绘图指令,具体地,第一绘图指令对应第一图形API,图形API调用模块,用于根据第一绘图指令调用第一图形库的第一图形API以执行第一绘图指令,并向应用程序发送第一处理通知,具体地,第一处理通知用于通知应用程序第一图形API调用完毕,调用信息生成模块,用于生成第一图形API的调用信息,具体地,第一图形API的调用信息用于指示客户端调用第二图形库中的第一图形API,并且,第一图形API的调用信息包括第一图形API涉及的参数,以及第一图形API的识别码,调用信息发送模块,用于将第一图形API的调用信息发送至客户端。
第四方面或第四方面任意一种实现方式是第一方面或第一方面任意一种实现方式对应的装置实现,第一方面或第一方面任意一种实现方式中的描述适用于第四方面或第四方面任意一种实现方式,在此不再赘述。
第五方面,本申请提供一种图形处理装置,设置于客户端,客户端与服务器具有网络连接,服务器设置有第一图形库,客户端设置有第二图形库,第一图形库中包括多个图形应用程序编程接口API,第二图形库中包括多个图形API,第一图形库中至少有M个图形API与第二图形库中的M个图形API一一对应,图形处理装置包括:调用信息接收模块,用于接收服务器发送的第一图形API的调用信息,第一图形API的调用信息包括第一图形API涉及的参数,以及第一图形API的识别码,图形API调用模块,用于根据第一图形API的调用信息中的第一图形API的识别码确定第一图形API,并根据第一图形API的调用信息中的第一图形API涉及的参数调用第二图形库中的第一图形API。
第五方面或第五方面任意一种实现方式是第二方面或第二方面任意一种实现方式对应的装置实现,第二方面或第二方面任意一种实现方式中的描述适用于第五方面或第五方面任意一种实现方式,在此不再赘述。
第六方面,本申请提供一种图形处理装置,设置于服务器,服务器还设置有第一图形库和应用程序,服务器与客户端具有网络连接,客户端设置有第二图形库,第一图形库中包括多个图形应用程序编程接口API,第二图形库中包括多个图形API,第一图形库中至少有M个图形API与第二图形库中的M个图形API一一对应,图形处理装置包括:绘图指令获取模块,用于获取应用程序发起的第一绘图指令,其中第一绘图指令对应第一图形API,图形API调用模块,用于确定第一图形API为第一图形库的预定图形API时,拒绝调用第一图形库的第一图形API,图形API调用模块,还用于向应用程序发送第一处理通知,第一处理通知用于通知应用程序第一图形API调用完毕,调用信息生成模块,用于生 成第一图形API的调用信息,具体地,第一图形API的调用信息用于指示客户端调用第二图形库中的第一图形API,并且,第一图形API的调用信息包括第一图形API涉及的参数,以及第一图形API的识别码,调用信息发送模块,用于将第一图形API的调用信息发送至客户端。
第六方面或第六方面任意一种实现方式是第三方面或第三方面任意一种实现方式对应的装置实现,第三方面或第三方面任意一种实现方式中的描述适用于第六方面或第六方面任意一种实现方式,在此不再赘述。
第七方面,本申请提供一种服务器,包括处理器和存储器,存储器存储有程序指令,处理器运行程序指令以执行第一方面或第一方面任意一种实现方式提供的图形处理方法。
第八方面,本申请提供一种客户端,包括处理器和存储器,存储器存储有程序指令,处理器运行程序指令以执行第二方面或第二方面任意一种实现方式提供的图形处理方法。
第九方面,本申请提供一种服务器,包括处理器和存储器,存储器存储有程序指令,处理器运行程序指令以执行第三方面或第三方面任一一种实现方式提供的图形处理方法。
第十方面,本申请提供了一种存储介质,该存储介质中存储了程序代码,该程序代码被存储控制器运行时,该存储控制器执行前述第一方面或第一方面的任意一种实现方式提供的图形处理方法。该存储介质包括但不限于只读存储器,随机访问存储器,快闪存储器、HDD或SSD。
第十一方面,本申请提供了一种存储介质,该存储介质中存储了程序代码,该程序代码被存储控制器运行时,该存储控制器执行前述第二方面或第二方面任意一种实现方式提供的图形处理方法。该存储介质包括但不限于只读存储器,随机访问存储器,快闪存储器、HDD或SSD。
第十二方面,本申请提供了一种存储介质,该存储介质中存储了程序代码,该程序代码被存储控制器运行时,该存储控制器执行前述第三方面或第三方面任意一种实现方式中提供的图形处理方法。该存储介质包括但不限于只读存储器,随机访问存储器,快闪存储器、HDD或SSD。
第十三方面,本申请提供了一种计算机程序产品,该计算机程序产品包括程序代码,当该计算机程序产品被存储控制器执行时,该存储控制器执行前述第一方面或第一方面的任意一种实现方式提供的图形处理方法。该计算机程序产品可以为一个软件安装包,在需要使用前述第一方面或第一方面的任意一种实现方式提供的图形处理方法的情况下,可以下载该计算机程序产品至存储控制器并在该存储控制器上运行该计算机程序产品。
第十四方面,本申请提供了一种计算机程序产品,该计算机程序产品包括程序代码,当该计算机程序产品被存储控制器执行时,该存储控制器执行前述第二方面或第二方面任意一种实现方式提供的图形处理方法。该计算机程序产品可以为一个软件安装包,在需要使用前述第二方面或第二方面任意一种实现方式提供的图形处理方法的情况下,可以下载该计算机程序产品至存储控制器并在该存储控制器上运行该计算机程序产品。
第十五方面,本申请提供了一种计算机程序产品,该计算机程序产品包括程序代码,当该计算机程序产品被存储控制器执行时,该存储控制器执行前述第三方面或第三方面的任意一种实现方式提供的图形处理方法。该计算机程序产品可以为一个软件安装包,在需 要使用前述第三方面或第三方面任意一种实现方式提供的图形处理方法的情况下,可以下载该计算机程序产品至存储控制器并在该存储控制器上运行该计算机程序产品。
附图说明
图1是根据本发明实施例的远程界面显示系统的结构示意图;
图2是根据本发明实施例的应用程序调用图形API的示意图;
图3是根据本发明实施例的图形处理方法的数据交互图;
图4是根据本发明实施例的图形处理方法的流程图;
图5是根据本发明实施例的应用程序调用图形API的另一示意图;
图6是根据本发明实施例的第二图形处理装置调用图形API的示意图;
图7是根据本发明实施例的服务器的数据流向图;
图8是根据本发明实施例的图形处理方法的另一流程图;
图9是根据本发明实施例的客户端的数据流向图;
图10是根据本发明实施例的图形处理装置的装置结构示意图;
图11是根据本发明实施例的图形处理装置的另一装置结构示意图;
图12是根据本发明实施例的图形处理装置的另一装置结构示意图;
图13是根据本发明实施例的服务器的装置结构示意图;
图14是根据本发明实施例的客户端的装置结构示意图。
具体实施方式
下面将结合附图,对本发明实施例中的技术方案进行清楚、完整地描述。显然,所描述的实施例仅仅是本发明一部分实施例,而不是全部的实施例。
首先请参见图1,图1是根据本发明实施例的远程界面显示系统的结构示意图。如图1所示,远程界面显示系统包括服务器10和客户端20,客户端20通过网络30与服务器10连接。
服务器10包括硬件100、操作系统102、第一图形库103、应用程序104以及第一图形处理装置105,硬件10设置有GPU 111、网卡112、CPU 113以及内存114。
值得注意的是,在另一些示例中,服务器10可不设置GPU 111,由CPU 113对GPU 111进行模拟,通过CPU 113来实现GPU 111的绘图功能。
第一图形库103设置有多个图形API,每一图形API对应特定的绘图代码,其中,绘图代码用于控制GPU 111进行绘图,如句柄申请、图形渲染、着色、更新缓冲区、以及显示图像等,应用程序104加载第一图形库103,应用程序104在运行过程中,可直接调用已加载的第一图形库103中的图形API。
操作系统102设置有网卡112的网卡驱动程序,第一图形处理装置105可根据网卡112的网卡驱动程序控制网卡112通过网络30接收或发送数据,操作系统102还设置有GPU 111的显卡驱动程序,应用程序104以及第一图形处理装置105根据GPU 111的显卡驱动程序控制GPU 111进行绘图。
第一图形库103、应用程序104以及第一图形处理装置105分别在操作系统102上运 行,第一图形库103可例如为Opengl图形库、Vulkan图形库或其他图形库。
举例而言,第一图形库103为Opengl图形库,第一图形库103中的图形API可为eglCreateWindowSurface、eglCreateContext、glCreateShader、glCompileShader、glGenBuffers、glBufferData、glDrawArrays、或eglSwapBuffers。
其中,eglCreateWindowSurface用于申请图形缓冲区,eglCreateContext用于申请上下文句柄,glCreateShader用于申请着色器句柄,glCompileShader用于编译着色器,glGenBuffers申请顶点和纹理缓冲句柄,glBufferData用于发送顶点和纹理数据到客户端,glDrawArrays用于开始绘制像素到后台缓冲区,eglSwapBuffers用于交换前后缓冲区。
在本发明实施例中,应用程序104为客户端20部署在服务器10的应用程序,例如为2D游戏、3D游戏、邮箱以及社交软件等。
在本发明实施例中,应用程序104启动后,可根据应用程序104的程序代码依次调用第一图形库103的N个图形API,其中,N个图形API的调用顺序预先记录在应用程序104的程序代码中。
在另一些示例中,程序指令也可设置为使得应用程序104循环调用N个图形API。
举例而言,可进一步参见图2,图2是根据本发明实施例的应用程序调用图形API的示意图。如图2所示,应用程序104根据自身的程序代码依次调用第一图形库103中的图形API:eglCreateWindowSurface、eglCreateContext、glCreateShader、glCompileShader、glGenBuffers、glBufferData、glDrawArrays、以及eglSwapBuffers。
并且,在调用图形API的过程中,针对eglCreateWindowSurface、eglCreateContext、glCreateShader、以及glGenBuffers产生出口参数,针对glCompileShader、glBufferData、glDrawArrays、以及eglSwapBuffers不产生出口参数,以空(void)表示。
并且,有些图形API之间存在依赖关系,即部分图形API的入口参数为其他图形API的出口参数,例如针对eglCreateContext而言,其入口参数之一surfaceA为eglCreateWindowSurface的出口参数。针对glCompileShader而言,其入口参数shader_handle为glCreateshader的出口参数。针对eglSwapBuffers而言,其入口参数之一surfaceA为eglCreateWindowSurface的出口参数。
进一步,图2所示的各图形API的入参:egldisplay,eglconfig,eglwindow,attrib_list,eglcontexts,GL_FRAGMENT_SHADER,count,GL_ARRAY_BUFFER,size,Vertices,GL_STATIC_DRAW,GL_TRIANGLES,first均为常数,其具体数值预先记录在程序代码中。
在程序员进行编程过程中,作为常数的入口参数预先记录在程序代码中,应用程序104在调用图形API时可直接从程序代码获取该入口参数。而在当前调用的图形API的入口参数为其他图形API的出口参数时,即当前调用的图形API与其他图形API之间存在依赖关系时,应用程序104需在其他图形API调用完毕后之后,才能将其他图形API返回的出口参数作为当前调用的图形API的入口参数,来调用当前需调用的图形API。
举例而言,请继续参见图2,针对glCreateShader,其入口参数GL_FRAGMENT_SHADER可直接从程序代码获得,针对glCompileShader,其入口参数shader_handle为glCreateShader的出口参数,因此应用程序104需在glCreateShader调用完 毕并产生出口参数shader_handle之后,才能将glCreateShader产生的出口参数shader_handle作为glCompileShader的入口参数,来调用glCompileShader。
值得注意的是,在一些示例中,第一图形库103可为第三方厂商提供的图形库,在另一些示例中,第一图形103可为自定义的图形库,此时,第一图形处理装置105可与第一图形库103整合在一起,在图1中以虚线框表示。
以下对客户端20进行介绍,请继续参见图1,如图1所示,客户端20包括硬件200、操作系统202、第二图形库203以及第二图形处理装置204,其中,硬件200设置有GPU 211、网卡212、显示屏213、CPU 214以及内存215。
值得注意的是,在另一些示例中,客户端20可不设置GPU 211,由CPU 214对GPU211进行模拟,通过CPU 214实现GPU 211的绘图功能。
与第一图形库103类似,第二图形库203同样设置有多个图形API,第二图形库203的每一图形API对应特定的绘图代码,第二图形处理装置204加载第二图形库203,第二图形处理装置204可直接调用已加载的第二图形库203中的图形API,从而控制GPU 211进行绘图。
操作系统202设置有网卡212的网卡驱动程序,第二图形处理装置204可根据网卡212的网卡驱动程序控制网卡212通过网络30接收或发送数据。操作系统202还设置有GPU 211的显卡驱动程序,第二图形处理装置204通过GPU 211的显卡驱动程序控制GPU 211进行绘图。操作系统202还设置有显示屏213的显示屏驱动程序,当GPU 211需要显示绘制好的界面时,显示屏驱动程序控制显示屏213显示绘制好的界面。
第二图形库203以及第二图形处理装置204分别在操作系统202上运行,第二图形库203可例如为Opengl图形库、Vulkan图形库或其他常用的图形库。
举例而言,与第一图形库103类似,在第二图形库203为Opengl图形库时,第二图形库203中的图形API可为eglCreateWindowSurface、eglCreateContext、glCreateShader、glCompileShader、glGenBuffers、glBufferData、glDrawArrays、或eglSwapBuffers。
其中,上述图形API的功能于上文已经得到描述,于此不作赘述。
在本发明实施例中,第一图形库103和第二图形库203为相同类型的图形库,第一图形库103中至少有M个图形API与所述第二图形库203中的M个图形API一一对应。
举例而言,第一图形库103和第二图形库203均为Opengl图形库,且设置有相同的图形API:eglCreateWindowSurface,第一图形库103和第二图形库203中的eglCreateWindowSurface的作用相同,均用于申请图形缓冲区,但第一图形库103的eglCreateWindowSurface和第二图形库203的eglCreateWindowSurface的实现细节有可能不一样。具体而言,第一图形库103的eglCreateWindowSurface可由C++语言实现,而第二图形库203中的eglCreateWindowSurface可由java语言实现。在本发明实施例中,虽然第一图形库103的eglCreateWindowSurface和第二图形库203的eglCreateWindowSurface实现细节可能不一样,但由于二者作用相同,且设置在相同类型的图形库,因此第一图形库103的eglCreateWindowSurface和第二图形库203的eglCreateWindowSurface具有一一对应关系,可称为相同的图形API。
同理,第一图形库103和第二图形库203中的eglCreateContext、glCreateShader、 glCompileShader、glGenBuffers、glBufferData、glDrawArrays、或eglSwapBuffers均具有一一对应关系。
值得注意的是,上述介绍的图形API仅为Opengl图形库中的一部分,本发明实施例亦可适用于Opengl图形库中的其他图形API,并可以适用于Vulkan图形库或其他常用的图形库中的图形API,只要保证服务器10和客户端20具有同一类型的图形库即可。
在本发明实施例中,应用程序104预先加载第一图形库103,在加载第一图形库103之后,应用程序104获知第一图形库103的图形API的函数地址,当应用程序104调用第一图形库103的图形API时,发起携带有该图形API的入口参数和跳转命令的绘图指令,该跳转命令用于跳转至所要调用的图形API的函数地址,应用程序104根据跳转命令跳转至该函数地址并根据入口参数执行函数地址上的绘图代码。
进一步,第二图形处理装置204预先加载第二图形库203,在加载第二图形库203之后,第二图形处理装置204获知第二图形库203的图形API的函数地址,当第二图形处理装置204调用第二图形库203的图形API时,发起携带有该图形API的入口参数和跳转命令的绘图指令,该跳转命令用于跳转至图形API的函数地址,第二图形处理装置204根据跳转命令跳转至该函数地址并根据入口参数执行函数地址上的绘图代码。
在一些示例中,若图形API以C++语言编写,图形API具体表现为需要调用的图形API在服务器10的函数地址,其中,该函数地址具体为内存地址。
在另一些示例中,若图形API以java语言编写,则图形API具体表现为需要调用的图形API在服务器10的函数、方法或对象,java虚拟机将图形API的函数、方法或对象转换为在服务器10的函数地址。
值得注意的是,在本发明实施例中,以函数地址为例对图形API进行举例说明。
为了进一步清楚说明,以下请参见图3,图3是根据本发明实施例的图形处理方法的数据交互图。如图3所示,本发明实施例的图形处理方法包括以下步骤:
步骤S1:应用程序104发起第一绘图指令,第一图形处理装置105获取第一绘图指令,其中,第一绘图指令对应于第一图形API,第一绘图指令携带有第一图形API的入口参数和用于跳转至第一图形API的函数地址的跳转命令。
第一绘图指令对应于第一图形库103的第一图形API和/或第二图形库203中的第一图形API。
其中,应用程序104发起第一绘图指令后,进行等待,直到获取第一绘图指令的执行结果才能继续运行。
在一些示例中,应用程序104发起第一绘图指令,并执行第一绘图指令,跳转到跳转命令所要跳转的函数地址时,第一图形处理装置105通过钩子(HOOK)截获第一绘图指令。
步骤S2:第一图形处理装置105根据第一绘图指令调用第一图形库的第一图形API以执行第一绘图指令。
其中,第一图形处理装置105根据第一绘图指令携带的跳转指令跳转至第一图形API的函数地址,根据第一绘图指令携带的第一图形API的入口参数执行第一图形API的函数地址上的绘图代码,在绘图代码执行时,可控制GPU 111进行图像渲染,从而完成对第一 图形库的第一图形API的调用以及第一绘图指令的执行。
在本步骤中,第一图形处理装置105根据第一绘图指令调用第一图形库的第一图形API之前,可进一步确认第一图形API是否为第一图形库的预定图形API,其中,预定图形API调用时,GPU 111需进行大量的运算。
在本步骤中,而第一图形处理装置105确定第一图形API非第一图形库的预定图形API时,根据第一绘图指令调用第一图形库的第一图形API以执行第一绘图指令。并且,第一图形处理装置105确定第一图形API为第一图形库的预定图形API时,拒绝调用第一图形库的第一图形API。
步骤S3:第一图形处理装置105产生第一处理通知,并发送第一处理通知至应用程序104,其中,第一处理通知作为第一绘图指令的执行结果用于通知应用程序104第一图形API调用完毕。
在本步骤中,第一图形处理装置105可进一步确定第一绘图指令是否需要返回出口参数,在第一绘图指令需要返回出口参数的情况下,第一处理通知还包括服务器调用第一图形API所产生的出口参数。
在第一绘图指令需要返回出口参数的情况下,在第一处理通知进一步携带有服务器10调用第一图形API所产生的出口参数,使得应用程序104通过第一处理通知可获取服务器10调用第一图形API所产生的出口参数,在应用程序104后续调用到输入参数为第一图形API的出口参数的其他图形API时,可将获取到的出口参数作为其他图形API的入口参数,从而保证应用程序104正常运行。
并且,当在步骤S3中,第一图形处理装置105确定第一图形API为第一图形库103的预定图形API,并拒绝调用第一图形库103的第一图形API时,在本步骤中,第一图形处理装置105仍产生用于通知应用程序104第一图形API调用完毕的第一处理通知,并发送第一处理通知至应用程序104。
其中,预定图形API为需要GPU 111进行大量运算的图形API。
图形API包括着色器启动图形API、光栅化启动图形API、清除缓冲区图形API或交换缓冲区图形API中的任意一种。
值得注意的是,着色器启动图形API、光栅化启动图形API、清除缓冲区图形API以及交换缓冲区图形API所对应的绘图代码在执行时,需要GPU 111启动着色器、启动光栅化、清除缓冲区以及交换缓冲区,这些操作会使得GPU 111进行大量的运算。第一图形处理装置105在判断到当前调用的第一图形API为上述的预定图形API时,拒绝调用该第一图形API,可使得GPU 111避免进行大量运算,从而可节约服务器10上宝贵的运算资源。
举例而言,针对Opengl图形库,glDrawArrays用于开始绘制像素到后台缓冲区,在绘制像素到后台缓冲区时,需要启动着色器,并由着色器启动光栅化,因此glDrawArrays同时为着色器启动图形API和光栅化启动图形API;eglSwapBuffers用于交换前后缓冲区,因此eglSwapBuffers为交换缓冲区图形API;glDrawElements用于开始绘制,而绘制时会启动着色器,因此glDrawElements也是着色器启动图形API;glClearDepthf和glClearColor会清除缓冲区,因此glClearDepthf和glClearColor为清除缓冲区图形API。
由于第一图形处理装置105确定第一图形API为第一图形库103的预定图形API时, 拒绝调用第一图形库103的第一图形API,因此GPU 111无需执行启动着色器、启动光栅化、清除缓冲区或交换缓冲区,从而实现空渲染。但是,在本发明实施例中,第一图形处理装置105仍产生用于通知应用程序104第一图形API调用完毕的第一处理通知,并发送第一处理通知至应用程序104,使得应用程序104仍可获取第一处理通知得以结束等待,继续运行。
由于服务器10无需将绘制好的图形显示给用户,因此即使服务器10的GPU 111不工作,也不会影响用户体验。
因此,上述揭示的空渲染可使得GPU 111避免进行大量运算,从而可节约服务器10上宝贵的运算资源,使得服务器10可以运行数量更多的应用程序。
步骤S4:第一图形处理装置105生成第一图形API的调用信息,并通过网络30将第一图形API的调用信息发送至客户端20的第二图形处理装置204。
其中,第一图形API的调用信息用于指示客户端调用第二图形库中的第一图形API,第一图形API的调用信息包括服务器10在第一图形库103调用第一图形API涉及的参数,以及第一图形API的识别码。
可选地,第一图形API的涉及的参数可包括第一图形API的入口参数,或者,图形API的涉及的参数可包括第一图形API的入口参数和出口参数。
在第一图形库103和第二图形库203中,一部分图形API只有入口参数而没有出口参数,以Opengl图形库为例,用于开始绘制像素到后台缓冲区的图形API:glDrawArrays,其入口参数包括GL_TRIANGLES、first以及count,且没有出口参数,而另一部分图形API同时具有入口参数和出口参数,例如用于申请图形缓冲区的图形API:eglCreateWindowSurface,其入口参数包括egldisplay、eglconfig、eglwindow以及attrib_list,其出口参数是surfaceA。
其中,上述的入口参数和出口参数具体为数值、寄存器地址或内存地址,举例而言,surfaceA为Opengl图形库定义的图形缓存区的句柄,即为申请到的图形缓存区的内存地址,surfaceA的具体数值取决于服务器10的内存114使用情况,例如服务器10的内存114存在空闲的内存空间,则可将该空闲内存空间的入口地址分配给surfaceA。
进一步,在第一图形库103和第二图形库203均为Opengl图形库时,以下的表1举例示出第一图形库103的图形API在服务器10的函数地址与识别码的对应关系:
Figure PCTCN2017107341-appb-000001
Figure PCTCN2017107341-appb-000002
表1
其中,表1可由服务器10的第一图形处理装置105维护,举例而言,在表1中,eglCreateWindowSurface在服务器10的函数地址A具体为应用程序104在加载第一图形库103之后用于实现图形API:eglCreateWindowSurface的绘图代码在服务器10的内存114上分配到的入口地址。
eglCreateWindowSurface在服务器10的内存114所分配到的入口地址可以是0x00000001,当应用程序104调用第一图形库103的eglCreateWindowSurface时,直接跳转至服务器10的内存地址0x00000001,从0x00000001开始执行eglCreateWindowSurface对应的绘图代码。
因此,在本步骤中,在第一绘图指令需要返回出口参数的情况下,第一图形API涉及的参数包括第一图形API的入口参数和服务器调用第一图形API所产生的出口参数,第一图形API的调用信息包括第一图形API的入口参数、服务器调用第一图形API所产生的出口参数和第一图形API的识别码。
并且,在第一绘图指令不需要返回出口参数的情况下,第一图形API涉及的参数包括第一图形API的入口参数,第一图形API的调用信息包括第一图形API的入口参数和第一图形API的识别码。
进一步,当第一图形处理装置105确定第一图形API为第一图形库103的预定图形API,并拒绝调用第一图形库103的第一图形API时,第一图形API涉及的参数包括第一图形API的入口参数,第一图形API的调用信息包括第一图形API的入口参数和第一图形API的识别码。
值得注意的是,表1中所示的识别码仅为示例说明,只要保证识别码在图形库中的唯一性即可,在另一些示例中,也可以图形API的名称作为识别码,如eglCreateWindowSurface。
步骤S5:第二图形处理装置204接收第一图形API的调用信息,根据第一图形API的调用信息中的第一图形API的识别码确定第一图形API,根据第一图形API的调用信息中 的第一图形API涉及的参数调用第二图形库203中的第一图形API。
在本发明实施例中,第二图形处理装置204预先记录图形API与图形API的识别码之间的对应关系,示例性地,以下的表2示出上文表1第二列中所示的识别码与第二图形库203的图形API在客户端20的函数地址的对应关系:
Figure PCTCN2017107341-appb-000003
表2
其中,表2可由客户端20的第二图形处理装置204维护,举例而言,在表2中,eglCreateWindowSurface在客户端20的函数地址A’具体为第二图形处理装置204在加载第二图形库203之后实现图形API:eglCreateWindowSurface的绘图代码在客户端20的内存215上分配到的入口地址。
举例而言,eglCreateWindowSurface在客户端20的内存215所分配到的内存入口地址可以是0xA000100F,当第二图形处理装置204调用第二图形库203的eglCreateWindowSurface时,直接跳转至客户端20的内存地址0xA000100F,从0xA000100F开始执行eglCreateWindowSurface对应的绘图代码。
在本步骤中,第二图形处理装置204根据第一图形API的调用信息中的第一图形API的识别码0009确定第一图形API在客户端20的函数地址I’,并执行函数地址I’上的绘图代码,绘图代码控制GPU 211进行图形绘制,从而完成第二图形库203中的第一图形API的调用。
可选地,在本步骤中,在第一图形API的调用信息中的第一图形API涉及的参数包括服务器10调用第一图形API所产生的出口参数的情况下,第二图形处理装置204获取客户端20调用第一图形API所产生的出口参数,并建立服务器调用第一图形API所产生的出口参数与客户端调用第一图形API所产生的出口参数的映射关系。
由于客户端20与服务器10之间的硬件配置条件不同,因此,服务器10调用第一图形API所产生的出口参数与客户端20调用第一图形API产生的出口参数也不相同。举例而言,若第一图形API的作用为申请一段缓存空间,其返回的出口参数为缓存空间的内存地址,由于服务器10和客户端20的运行环境完全不同,因此该图形API在服务器10和客户端20产生的出口参数(即内存地址)也不相同。
因此,在第一图形API产生的出口参数为其他图形API的入口参数的情况下,客户端20不能将第一图形API的调用信息携带的服务器10调用第一图形API所产生的出口参数直接作为其他图形API的入口参数。对应地,客户端20建立服务器10调用第一图形API所产生的出口参数与客户端20调用第一图形API所产生的出口参数的映射关系,客户端20可根据从服务器10接收到的服务器10调用第一图形API所产生的出口参数查询该映射关系,来获取客户端20调用第一图形API所产生的出口参数,并将获取到的客户端20调用第一图形API所产生的出口参数作为其他图形API的入口参数。从而克服由于客户端20与服务器10之间的硬件配置条件不同而造成的出口参数不同步的技术问题。
值得注意的是,本发明实施例中,第一图形处理装置105实际上是对第一图形API涉及的参数进行复制,具体而言,当第一图形API具有出口参数时,对第一图形API的入口参数和出口参数进行复制,当第一图形API不具有出口参数时,仅对第一图形API的入口参数进行复制。第一图形处理装置105根据复制所得的参数和第一图形API的识别码,形成第一图形API的调用信息。
步骤S6:应用程序104根据第一处理通知发起第二绘图指令,第一图形处理装置105获取第二绘图指令,其中,第二绘图指令对应于第二图形API,第二绘图指令携带有第二图形API的入口参数和用于跳转至第二图形API的函数地址的跳转命令。
第二绘图指令对应于第一图形库103的第二图形API和/或第二图形库203中的第二图形API。
可选地,应用程序104发起第二绘图指令,并执行第二绘图指令,跳转至第二绘图指令中的跳转命令所要跳转到的函数地址时,第一图形处理装置105可通过钩子(HOOK)截获第二绘图指令。
应该理解的是,步骤S6只需保证在步骤S3之后执行,因此步骤S6也可直接位于步骤S3或步骤S4之后。
在本步骤中,应用程序104在接收到第一图形处理装置105发送的第一处理通知之后,停止等待,并继续运行,在运行至需要调用第一图形库103的第二图形API的程序代码时,发起第二绘图指令。因此,应用程序104完全不依赖于客户端20,可在服务器10独立运行。
步骤S7:第一图形处理装置105根据第二绘图指令调用第一图形库103的第二图形API以执行第二绘图指令。
其中,第一图形处理装置105根据第二绘图指令携带的跳转指令跳转至第二图形API的函数地址,根据第二绘图指令携带的第一图形API的入口参数执行第一图形API的函数地址上的绘图代码,绘图代码控制GPU 111进行绘图运算,从而完成对第一图形库的第一图形API的调用以及第一绘图指令的执行。
步骤S8:第一图形处理装置105在第二绘图指令执行完毕之后,产生第二处理通知,并发送第二处理通知至应用程序104,其中,第二处理通知作为第二绘图指令的执行结果用于通知应用程序104第二图形API调用完毕。
步骤S9:第一图形处理装置105生成第二图形API的调用信息,并通过网络30将第二图形API的调用信息发送至客户端20的第二图形处理装置204。
其中,第二图形API的调用信息用于指示客户端调用第二图形库中的第二图形API,第二图形API的调用信息包括第二图形API涉及的参数,以及第二图形API的识别码。
步骤S10:第二图形处理装置204接收第二图形API的调用信息,根据第二图形API的调用信息中的第二图形API的识别码确定第二图形API,根据第二图形API的调用信息中的第二图形API涉及的参数调用第二图形库中的第一图形API。
在本步骤中,第二图形处理装置204根据第二图形API的调用信息中的第二图形API的识别码000A确定第一图形API在客户端20的函数地址J’,并执行函数地址J’上的绘图代码,从而完成第二图形库203中的第二图形API的调用。
可选地,在本步骤中,在第二图形API的入口参数为第一图形API的出口参数的情况下,第二图形处理装置204根据第二图形API的入口参数查询步骤S5中记录的映射关系,并获取第二图形API的入口参数在映射关系中对应的参数,其中,第二图形API的入口参数在映射关系中对应的参数为客户端20调用第一图形API所产生的出口参数。
因此,在第一图形API产生的出口参数为第二图形API的入口参数的情况下,通过以上处理方式,可克服由于客户端20与服务器10之间的硬件配置条件不同而造成的出口参数不同步的技术问题。
值得注意的是,在一些示例中,第一图形处理装置105可将第一图形API的调用信息和第二图形API的调用信息保存在先进先出队列,在满足预定条件后,将先进先出队列中的数据同时发送给客户端20的第二图形处理装置204。其中,预定条件可例如为计时时间超过预设时间段,或先进先出队列中的数据超出预定数据大小。
由于应用程序104的运行速度较快,应用程序104运行时,单位时间内产生多组调用信息,通过打包方式在满足预定条件后同时发送多组调用信息至客户端20,相对于每产生一组调用信息就发送一次调用信息给客户端20而言,可有效节约处理开销。
综上,由于服务器10的应用程序104发起的绘图指令对应的图形API在调用完毕之后,应用程序104可马上在本地获取用于通知应用程序104图形API调用完毕的处理通知,因此服务器10的应用程序104可不依赖于客户端20而独立运行,即便客户端20死机,或客户端20与服务器10之间的连接中断,也不会影响服务器10的应用程序104的运行,可提高用户体验。
进一步,服务器10的第一图形处理装置105生成携带有图形API涉及的参数的调用信息,并将调用信息发送至客户端20,客户端20可根据图形API涉及的参数调用第二图形 库中的第一图形API。而服务器10的应用程序104在第一图形处理装置105发送调用信息之前,已经接收到第一图形处理装置105发送的处理通知,因此,客户端20调用完第二图形库203中的图形API之后,无需再发送处理通知给服务器10的应用程序104,因此服务器10与客户端20之间实现了单向传输,从而可降低对网络延时的要求。
为进一步对上述步骤进行清楚说明,请参见图4,图4是根据本发明实施例的图形处理方法的流程图,图4具体描述服务器的工作流程,如图4所示,图形处理方法具体包括以下步骤:
步骤S101:应用程序104启动。
步骤S102:应用程序104发起第一绘图指令。具体而言,应用程序104运行至需要调用第一图形API的程序代码时,发起第一绘图指令。
其中,第一绘图指令携带有第一图形API的入口参数和用于跳转至第一图形API的函数地址的跳转命令,其中,函数地址为第一图形API在服务器10的函数地址I以及第一图形API的入口参数。
步骤S103:第一图形处理装置105获取第一绘图指令。
在一些示例中,第一图形处理装置105利用钩子(HOOK)截获第一绘图指令。
步骤S104:第一图形处理装置105判断图形处理请求对应的图形API是否需要返回出口参数,如果是,执行步骤S105,如果否,执行步骤S107。
在本步骤中,第一图形处理装置105可预先记录第一图形库103中的所有图形API中何者需要返回出口参数,何者不需要返回出口参数。
举例而言,第一图形库103为Opengl图形库时,第一图形处理装置105可预先记录Opengl图形库中的每一图形API是否需要返回出口参数,而在Opengl图形库公开的技术文档中,具体定义了图形API的输入参数的类型(如整数、指针、内存地址以及寄存器地址等)、数量、以及排列顺序,并定义了图形API是否需要返回出口参数,程序员通过阅读技术文档可以编写对应的程序代码调用图形API。
步骤S105:第一图形处理装置105调用第一图形库103的图形API,获取出口参数,发送携带有出口参数的处理通知至应用程序104,其中,该处理通知用于通知应用程序104当前的图形API调用完毕。
步骤S106:第一图形处理装置105为图形API设置识别码,将识别码、入口参数及出口参数作为一组调用信息设置在先进先出队列。
举例而言,第一图形处理装置105可根据上文所述的表1为图形API设置识别码,具体地,第一图形处理装置105获取当前的绘图指令携带的跳转命令,在表1搜索与跳转命令所要跳转的函数地址对应的识别码,将搜索到的识别码作为当前的图形API对应的识别码。
步骤S107:第一图形处理装置105判断当前调用的图形API是否是预定图形API,如果是,执行步骤S109,拒绝调用第一图形库103的该图形API,如果否,执行步骤S108,调用第一图形库103的该图形API。其中,预定图形API包括着色器启动图形API、光栅化启动图形API、清除交换缓冲区图形API以及交换缓冲区图形API。
值得注意的是,着色器启动图形API、光栅化启动图形API、清除缓冲区图形API以 及交换缓冲区图形API所对应的绘图代码在执行时,需要GPU 111进行大量的运算,在本步骤中,第一图形处理装置105在判断到当前调用的图形API为上述的预定图形API时,直接跳过步骤S108,拒绝调用预定图形API,可使得GPU 111避免进行大量图形运算,从而实现空渲染,可节约服务器10上宝贵的运算资源。
由于服务器10无需将绘制好的图形显示给用户,因此即使GPU 111不工作,也不会影响用户体验。
步骤S108:第一图形处理装置105调用第一图形库103的图形API。
在本步骤中,第一图形处理装置105根据当前获取的绘图指令携带的跳转命令跳转至跳转命令的函数地址,并将当前的图形处理请求携带的入口参数作为输入参数执行函数地址上的绘图代码。
步骤S109:第一图形处理装置105调用第一图形库103的图形API之后,第一图形处理装置105发送处理通知至应用程序104,其中,该处理通知用于通知应用程序104图形API调用完毕,本步骤执行完毕后,流程分别跳转至步骤S111和步骤S110。
值得注意的是,与步骤S105相比,在本步骤中,由于图形API不需要返回出口参数,因此,本步骤的所发送的处理通知没有携带有图形API的出口参数。
步骤S110:第一图形处理装置105为图形API设置识别码,将识别码、入口参数作为一组调用信息设置在先进先出队列。
与步骤S105类似,在本步骤中,第一图形处理装置105可根据上文所述的表1为图形API设置识别码,具体而言,第一图形处理装置105可在表1搜索与跳转命令所要跳转的函数地址对应的识别码,将搜索到的识别码作为当前的图形API对应的识别码。
步骤S111:应用程序104根据接收到的处理通知继续运行程序指令,且在运行至需要调用另一图形API的程序代码时,发起另一绘图指令,然后流程跳转回步骤S103。
因此,随着应用程序104根据程序代码的运行依次发起多个绘图指令,第一图形处理装置105可针对不同的绘图指令在本地依次调用多个图形API,并将调用过的图形API的调用信息存入先进先出队列,其中,调用过的图形API的调用信息详细描述了多个图形API在服务器10的调用时涉及的参数以及图形API的类型。
步骤S112:第一图形处理装置105在先进先出队列存入第一组调用信息后每隔预定时间将先进先出队列中的若干组调用信息发送至客户端20。
在本步骤中,预定时间可例如为3ms,通过设置预定时间,使得记录到的若干组调用信息可每隔预定时间自动发送至客户端20。
由于应用程序104的运行速度较快,应用程序104运行时,单位时间内产生多组调用信息,若每产生一组调用信息就发送给客户端20,则在单位时间内,需要发送多次调用信息,而通过打包方式在经过预定时间后同时发送多组调用信息至客户端20,相对于每产生一组调用信息就发送一次调用信息给客户端20而言,可有效节约处理开销。
进一步,第一图形处理装置105在发送N组调用信息至客户端20之后,若应用程序104继续调用图形API,则在预定时间后,第一图形处理装置105还会发送L组调用信息至客户端20,其中,L与N的数值可以相同,也可以不相同,而调用信息的数量取决于第一图形处理装置105在预定时间内调用的图形API的数量。举例而言,在第一个3ms内, 图4所示的流程可处理50个图形API,即第一个3ms对应50组调用信息,在第二个3ms内,图4所示的流程可处理70个图形API,即第二个3ms对应70组调用信息,。
因此,在图4所示的流程中,第一图形处理装置105可随着应用程序104的运行不断将应用程序104在运行过程中调用过的图形API的识别码和涉及的参数记录于先进先出队列,并定时发送先进先出队列记录的多组调用信息至客户端20,由于第一图形处理装置105在每次记录调用信息之前及时发送处理通知至应用程序104,应用程序104在获取处理通知后可继续运行,因此在本发明实施例中,客户端20完全不会对应用程序104的运行造成任何影响。
即便客户端20停机,所造成的后果只是服务器10发送至客户端20的调用信息不能被客户端20接收,而服务器10中的应用程序104不会受到任何影响,应用程序104仍可正常运行,因此,可极大地提高用户体验。
为进一步解释图4所示流程,于此可以表1中所示的第一图形API和第二图形API为例对图4所示的循环过程进行说明,假设第一图形API的入口参数x,y,z,第一图形API与第二图形API之间存在依赖关系,第一图形API根据入口参数x,y,z在服务器10调用产生的出口参数为retA,第二图形API在根据入口参数retA在服务器10调用产生的出口参数为retB,第一图形API根据入口参数x,y,z在客户端20调用产生的出口参数为retA’,第二图形API根据入口参数retA’在客户端20调用产生的出口参数为retB’,并且,假设应用程序104的程序代码设定为只会单次串行地调用第一图形API和第二图形API。
可结合图5和图6进行理解,图5是根据本发明实施例的应用程序调用图形API的另一示意图,如图5所示,应用程序104的程序代码设定为分别调用第一图形API和第二图形API,其中,入口参数x,y,z为设置在程序代码中的常数,第二图形API的入口参数为第一图形API的出口参数retA。
而图6是根据本发明实施例的第二图形处理装置调用图形API的示意图,如图6所示,第二图形处理装置204分别调用第一图形API和第二图形API,其中,第一图形API的入口参数x,y,z从第一组调用信息获得,第二图形API的入口参数retA’从映射表中获得,于下文将会具体介绍。
值得注意的是,在本发明实施例中,即便输入的入口参数相同,相同的图形API在服务器10和客户端20调用后产生的出口参数不相同,举例而言,第一图形API根据入口参数x,y,z在服务器10调用产生的出口参数retA和第一图形API根据入口参数x,y,z在客户端20调用产生的出口参数retA’不相同。
原因在于,服务器10和客户端20的运行环境不一致,例如图形API的作用为申请一段缓存空间,其返回的出口参数为缓存空间的内存地址,由于服务器10和客户端20的运行环境完全不同,因此该图形API在服务器10和客户端20产生的出口参数(即内存地址)也不相同。
以下请结合图5至图7提供的实例继续参考图3,其中图7是根据本发明实施例的服务器的数据流向图。
步骤S101:应用程序104启动。
步骤S102:应用程序104运行至需要调用第一图形API的程序程序代码时,发起第一 绘图指令1011(如图7所示),该第一绘图指令1011包括第一图形API的入口参数x,y,z和用于跳转至第一图形API的函数地址I的跳转命令。
步骤S103:第一图形处理装置105获取该第一图形处理请求1011,从而获取跳转至第一图形API在服务器10的函数地址I的跳转命令和入口参数x,y,z。
步骤S104:第一图形处理装置105判断该第一图形API是否需要返回出口参数。
根据图5可知,第一图形处理装置105可判断到第一图形API需要返回出口参数,因此流程跳转至步骤S105。
步骤S105:第一图形处理装置105调用第一图形库103的第一图形API,具体地,第一图形处理装置105跳转至第一图形API在服务器10的函数地址I,将入口参数x,y,z作为输入参数运行函数地址I上的绘图代码,在绘图代码运行完毕后获取作为运行结果的出口参数retA,并将包括出口参数retA的第一处理通知1012(如图7所示)发送给应用程序104,然后流程分别跳转至步骤S111和步骤S106。
其中,该第一处理通知1012用于通知应用程序104当前的第一图形API调用完毕。
步骤S106:第一图形处理装置105根据表1为第一图形API设置第一识别码0009,将第一识别码0009、入口参数x,y,z和出口参数retA作为第一图形API的调用信息1001设置在先进先出队列1000。
步骤S112:第一图形处理装置105在先进先出队列1000存入第一图形API的调用信息1001后开始计时。
在一些示例中,先进先出队列1000以2个字节为基本单位,第一图形处理装置105分别将识别码0009、入口参数x、入口参数y、入口参数z和出口参数retA分别存入先进先出队列1000的8个字节中,因此,第一图形API的调用信息1001包括8个字节。
步骤S111:应用程序104根据第一处理通知1012继续运行程序代码,且在运行至需要调用第二图形API的程序代码时,发起第二图形指令1013。
具体而言,在本步骤中,应用程序104接收到第一处理通知1012后,继续运行,在运行至需要调用第二图形API的程序代码时,发起第二绘图指令1013,其中第二绘图指令包括跳转至第二图形API在服务器10的函数地址J的跳转命令和第一处理通知1012携带的入口参数retA。
此时,流程跳转回步骤S103。
步骤S103:第一图形处理装置105获取第二绘图指令1013,从而获取跳转至第二图形API在服务器10的函数地址J的跳转命令和入口参数retA。
步骤S104:第一图形处理装置105判断该第二图形API是否需要返回出口参数。
根据图5可知,第二图形API需要返回出口参数,因此在本步骤中,第一图形处理装置105判断到第二图形API需要返回出口参数,流程跳转至步骤S105。
步骤S105:第一图形处理装置105调用第一图形库103中的第二图形API,具体地,第一图形处理装置105根据跳转命令跳转至第二图形API在服务器10的函数地址J,将入口参数retA作为输入参数运行函数地址J上的绘图代码,并在绘图代码运行完毕后获取作为运行结果的出口参数retB,将retB返回应用程序104,并发送包括第二图形API的出口参数retB的第二处理通知1014至应用程序104,然后分别跳转至步骤S111和步骤S106。
其中,该第二处理通知1014用于通知应用程序104当前的第二图形API调用完毕。
步骤S111:如图5所示,由于在本实施例中,程序指令只是分别单次调用第一图形API和第二图形API,因此,应用程序104在调用完以上两个图形API之后,不存在另一图形API,无需继续调用新的图形API,因此不会发起另一绘图指令,应用程序104根据第二处理通知1012退出运行,循环到此中止。
步骤S106:第一图形处理装置105根据表1为第二图形API设置识别码000A,将识别码000A、入口参数retA和出口参数retB作为第二图形API的调用信息1002设置在先进先出队列1000。
步骤S112:第一图形处理装置105在预定时间到达后将先进先出队列1000中的两组调用信息通过网卡112发送至客户端20。
值得注意的是,在本实施例中,预定时间可设置为足够长以使得应用程序104可分别完成第一图形API和第二图形API的调用。其中,预定时间可例如为3ms。
综上,先进先出队列1000包括调用信息1001和调用信息1002,调用信息1001包括第一识别码0009、入口参数x,y,z和出口参数retA,调用信息1002包括第二识别码000A、入口参数retA和出口参数retB。
在一些示例中,先进先出队列1000以2个字节为基本单位,第一图形处理装置105分别将第一识别码0009、入口参数x、入口参数y、入口参数z和出口参数retA分别存入先进先出队列1000的前8个字节中,即调用信息1001包括8个字节,并将第二识别码000A、入口参数retA和出口参数retB分别存入先进先出队列1000的后6个字节中,即调用信息1002包括6个字节。因此,2组调用信息的总数据长度为14个字节。
根据上述公开内容可知,服务器10的应用程序104在服务器10本地运行,服务器10单向发送N组数据至客户端20,而且服务器10无需从客户端20接收数据,因此,可省去从客户端20接收数据的等待时间,应用程序104的运行速度得到极大提升。
以下请参见图8,图8是根据本发明实施例的图形处理方法的另一流程图,图8描述了客户端20的工作流程,如图8所示,图形处理方法具体包括以下步骤:
步骤S301:第二图形处理装置204接收服务器10发送的若干组调用信息。
具体而言,第二图形处理装置204通过网卡212接收到服务器10的第一图形处理装置105发送的若干组调用信息。
步骤S302:第二图形处理装置204读取若第一组调用信息的识别码。
可选地,第一组调用信息位于若干组调用信息的起始位置,并且识别码位于第一组调用信息的起始位置。
步骤S303:第二图形处理装置204根据读取到的识别码确认要调用的图形API。
具体地,第二图形处理装置204根据读取到的识别码查找表2,从而确认要调用的图形API在客户端20的函数地址,从而确认要调用的图形API。
步骤S304:图形处理装置判断图形API的入口参数是否是其他图形API的出口参数,如果是,执行步骤S305,如果否,执行步骤S306。
在本步骤中,图形API分为两种类型,一种类型是可返回出口参数的类型,另一种类型是不返回出口参数的类型,本发明实施例根据判断到的图形API的类型和与识别码位于 同一组调用信息的入口参数确认要调用的图形API的入口参数。
以下步骤S305和步骤S306分别针对两种类型的图形API进行相应处理。
步骤S305:第二图形处理装置204读取与当前识别码位于同一组调用信息的入口参数,根据该入口参数在映射表搜索对应的出口参数,将搜索到的出口参数作为当前调用的图形API的入口参数。
其中,映射表记录有服务器调用特定图形API所产生的出口参数与客户端调用该特定图形API所产生的出口参数的映射关系,并且映射表根据下文所述的步骤S309生成。
步骤S306:第二图形处理装置204读取与当前识别码位于同一组调用信息的入口参数,将读取到的入口参数作为当前调用的图形API的入口参数。
值得注意的是,针对具有多个入口参数的图形API而言,若多个入口参数中的一部分入口参数不是其他图形API的出口参数,且多个入口参数中的另一部分入口参数是其他图形API的出口参数,则针对是其他图形API的出口参数的入口参数可通过执行步骤S305获得,针对不是其他图形API的入口参数可通过执行步骤S306获得。
步骤S307:第二图形处理装置204根据已确认的入口参数调用第二图形库203的图形API。
步骤S308:第二图形处理装置204判断图形API是否返回出口参数,如果是,执行步骤S309,如果否,执行步骤S310。
步骤S309:第二图形处理装置204读取与当前识别码位于同一组调用信息的出口参数,将读取到的出口参数与返回的出口参数记录在映射表。
步骤S310:第二图形处理装置204判断是否存在下一组调用信息,如果是,执行步骤S312,如果否,执行步骤S301,在步骤301中,第二图形处理装置204继续接收服务器10发送的另一若干组调用信息,并针对另一若干组调用信息继续执行步骤301至步骤312。
步骤S312:第二图形处理装置204读取下一组调用信息中记录的识别码,并跳转至步骤S303,从而针对下一组调用信息继续执行步骤303至步骤312。
综上,客户端20可根据接收到的若干组调用信息,依序在本地调用若干个服务器10调用过的图形API,从而在本地进行图形绘制,使得客户端20的用户可在客户端20本地看到服务器10的应用程序104运行时所绘制的界面。并且,客户端20无需向服务器10发送数据,因此实现了单向传输。
为进一步解释图8所示流程,于此可以表1和表2中所示的第一图形API和第二图形API为例进行说明,并且,若干组调用信息具体为图7及其对应描述中的2组调用信息,该2组调用信息包括第一组调用信息1001和第二组调用信息1002。
并请结合图9继续参考图8所示的流程,其中,图9是根据本发明实施例的客户端的数据流向图。
步骤S301:第二图形处理装置204接收服务器10发送的2组调用信息,2组调用信息包括第一组调用信息1001和第二组调用信息1002,其中第一组调用信息1001位于2组调用信息的头部,识别码0009位于第一组调用信息1001的头部。
步骤S302:第二图形处理装置204读取2组调用信息的第一组调用信息1001的识别码0009。
步骤S303:第二图形处理装置204根据读取到的识别码0009查找表2,获得要调用的第一图形API在客户端20的函数地址I’,从而确认要调用的第一图形API。
步骤S304:图形处理装置204判断第一图形API的入口参数是否是其他图形API的出口参数,判断结果为否,执行步骤S306。
步骤S306:第二图形处理装置204读取与当前识别码0009位于同一组调用信息1001的入口参数x,y,z,将读取到的入口参数x,y,z作为当前调用的第一图形API的入口参数。
步骤S307:第二图形处理装置204根据已确认的入口参数x,y,z调用第二图形库203的第一图形API。
具体地,第二图形处理装置204跳转至第一图形API在客户端20的函数地址I’,将入口参数x,y,z作为输入参数运行函数地址I’上的绘图代码,绘图代码用于控制GPU 211进行图形绘制,并在绘图代码运行完毕后获取作为运行结果的出口参数retA’。
可结合图9进行理解,具体而言,该步骤对应于图9的步骤1。
步骤S308:第二图形处理装置204判断第一图形API是否返回出口参数,并且判断结果为是,执行步骤S309。
步骤S309:第二图形处理装置204读取与当前识别码0009位于同一组调用信息1001的出口参数retA,将读取到的出口参数retA与返回的出口参数retA’的映射关系记录在映射表2041(参见图9)。
具体而言,该步骤对应于图9的步骤2。
步骤S310:第二图形处理装置204判断是否存在下一组调用信息,由于还存在第二组调用信息1002,因此判断结果为是,执行步骤S312。
步骤S312:第二图形处理装置204读取下一组调用信息1002中记录的识别码000A,并跳转至步骤S303,使得本流程循环执行。
步骤S303:第二图形处理装置204根据读取到的识别码000A查找表2,获得要调用的第二图形API在客户端20的函数地址J’,从而确认要调用的第二图形API。
该步骤对应于图9的步骤3中根据识别码获取API2的步骤。
步骤S304:第二图形处理装置204判断第二图形API的入口参数是否是其他图形API的出口参数,并且判断结果为是,执行步骤S305。
在本步骤中,第二图形API在客户端20调用时的入口参数retA’为第一图形API在客户端20调用时的出口参数retA’,因此第二图形处理装置204执行步骤305。
步骤305:第二图形处理装置204读取与识别码000A位于同一组调用信息1002的入口参数retA,根据该入口参数retA在映射表2041搜索对应的出口参数retA’,将搜索到的出口参数retA’作为当前调用的第二图形API的入口参数retA’。
可结合图9进行理解,具体而言,该步骤对应于图9的步骤3中读取入口参数retA的步骤以及步骤4。
步骤S307:第二图形处理装置204根据已确认的入口参数retA’调用第二图形库203的第二图形API。
该步骤对应于图9的步骤5。
具体而言,在调用过程中,第二图形处理装置204可跳转至第二图形API在客户端20 的函数地址J’,并将入口参数retA’作为输入参数执行函数地址J’上的绘图代码,绘图代码用于控制GPU 211进行图形绘制,并在绘图代码运行完毕后获取作为运行结果的出口参数retB’。
步骤S308:第二图形处理装置204判断第二图形API是否返回出口参数,判断结果为是,执行步骤S309。
步骤S309:第二图形处理装置204读取与当前识别码000A位于同一组调用信息1002的出口参数retB,将读取到的出口参数retB与返回的出口参数retB’记录在映射表2041。
步骤S310:第二图形处理装置204判断是否存在下一组调用信息,由于前面假设若干组调用信息具体为2组调用信息,不存在下一组调用信息,因此判断结果为否,执行步骤S301,继续接收另一若干组调用信息进行类似处理。
因此,第二图形处理装置204可如图6所示根据接收到的2组调用信息在本地调用2个图形API,从而完成图形处理。
可选地,一些示例中,第一图形处理装置105可将生成的多组调用信息分别写入文件中,并存储该文件。客户端20可隔一定时间后(如几天后)再从服务器10下载该文件,并由第二图形处理装置204根据图8所示的流程依次读取该文件中记录的多组调用信息,从而实现对应用程序104的绘图界面的回放。
请参见图10,本发明实施例进一步提供一种图形处理装置,图10是根据本发明实施例的图形处理装置的装置结构示意图,该图形处理装置设置在服务器10,如图10所示,该图形处理装置105包括:
绘图指令获取模块1051,用于获取应用程序104发起的第一绘图指令,第一绘图指令对应第一图形API;
图形API调用模块1052,用于根据第一绘图指令调用第一图形库103的第一图形API以执行第一绘图指令,并向应用程序104发送第一处理通知,第一处理通知用于通知应用程序104第一图形API调用完毕;
调用信息生成模块1053,用于生成第一图形API的调用信息,第一图形API的调用信息用于指示客户端20调用第二图形库203中的第一图形API,第一图形API的调用信息包括第一图形API涉及的参数,以及第一图形API的识别码;
调用信息发送模块1054,用于将第一图形API的调用信息发送至客户端20。
可选地,调用信息生成模块1053还用于:确定第一绘图指令是否需要返回出口参数;在第一绘图指令需要返回出口参数的情况下,第一处理通知还包括服务器10调用第一图形API所产生的出口参数,第一图形API的调用信息包括第一图形API的入口参数、服务器10调用第一图形API所产生的出口参数和第一图形API的识别码。
可选地,在第一图形API不需要返回出口参数的情况下,第一图形API的调用信息包括第一图形API的入口参数和第一图形API的识别码。
可选地,绘图指令获取模块1051,用于获取应用程序104根据第一处理通知发起的第二绘图指令,第二绘图指令对应第二图形API,第二图形API的入口参数为服务器10调用第一图形API所产生的出口参数;图形API调用模块1052,用于根据第二绘图指令调用第一图形库103的第二图形API,并向应用程序104发送第二处理通知,第二处理通知用于 通知应用程序104第二图形API调用完毕;调用信息生成模块1053,用于生成第二图形API的调用信息,第二图形API的调用信息用于指示客户端20调用第二图形库203中的第二图形API,第二图形API的调用信息包括第二图形API涉及的参数,以及第二图形API的识别码;调用信息发送模块1054,用于将第二图形API的调用信息发送至客户端20。
可选地,调用信息发送模块1054,具体用于:将第一图形API的调用信息和第二图形API的调用信息保存在先进先出队列;在满足预定条件后,将先进先出队列中的数据同时发送给客户端20。
由于服务器的应用程序发起的第一绘图指令对应的第一图形API在调用完毕之后,应用程序可马上在本地获取用于通知应用程序第一图形API调用完毕的第一处理通知,因此服务器的应用程序可不依赖于客户端而独立运行,即便客户端死机,或客户端与服务器之间的连接中断,也不会影响服务器的应用程序的运行,可提高用户体验。
进一步,服务器生成携带有第一图形API涉及的参数的第一图形API的调用信息,并将第一图形API的调用信息发送至客户端,客户端可根据第一图形API涉及的参数调用第二图形库中的第一图形API,由于服务器的应用程序已经收到第一处理通知,因此,客户端调用完第二图形库中的第一图形API之后,无需再发送处理通知给服务器的应用程序,因此服务器与客户端之间实现了单向传输,从而可降低对网络延时的要求。
以下请参见图11,本发明实施例进一步提供一种图形处理装置,图11是根据本发明实施例的图形处理装置的另一装置结构示意图,该图形处理装置设置在客户端20,如图11所示,图形处理装置204,包括:
调用信息接收模块2041,用于接收服务器10发送的第一图形API的调用信息,第一图形API的调用信息包括第一图形API涉及的参数,以及第一图形API的识别码;
图形API调用模块2042,用于根据第一图形API的调用信息中的第一图形API的识别码确定第一图形API,并根据第一图形API的调用信息中的第一图形API涉及的参数调用第二图形库203中的第一图形API。
可选地,图形处理装置204还包括映射关系建立模块2043,用于在第一图形API的调用信息中的第一图形API涉及的参数包括服务器10调用第一图形API所产生的出口参数的情况下,获取客户端20调用第一图形API所产生的出口参数,并建立服务器10调用第一图形API所产生的出口参数与客户端20调用第一图形API所产生的出口参数的映射关系。
可选地,调用信息接收模块2041,还用于接收服务器10发送的第二图形API的调用信息,第二图形API的调用信息包括第二图形API涉及的参数以及第二图形API的识别码,第二图形API的入口参数为服务器10调用第一图形API所产生的出口参数;图形API调用模块2042,还用于根据第二图形API的识别码确定第二图形API;图形API调用模块2042,还用于根据第二图形API的入口参数查询映射关系,获取第二图形API的入口参数在映射关系中对应的参数,第二图形API的入口参数在映射关系中对应的参数为客户端20调用第一图形API所产生的出口参数。
可选地,调用信息接收模块2041,具体用于:接收服务器10发送的数据,该数据包括第一图形API的调用信息和第二图形API的调用信息。
客户端从服务器接收第一图形API的调用信息,可根据第一图形API涉及的参数调用第二图形库中的第一图形API,由于服务器的第一图形库的第一图形API调用已经完成,因此,客户端调用完第二图形库中的第一图形API之后,无需再发送处理通知给服务器的应用程序,因此服务器与客户端之间实现了单向传输,因此可降低对网络延时的要求。
请参见图12,本发明实施例进一步提供一种图形处理装置,图12是根据本发明实施例的图形处理装置的另一装置结构示意图,该图形处理装置设置在服务器10,如图12所示,图形处理装置105包括:
绘图指令获取模块1061,用于获取应用程序104发起的第一绘图指令,第一绘图指令对应第一图形API;
图形API调用模块1062,用于确定第一图形API为第一图形库的预定图形API时,拒绝调用第一图形库的第一图形API;
图形API调用模块1062,还用于向应用程序104发送第一处理通知,第一处理通知用于通知应用程序104第一图形API调用完毕;
调用信息生成模块1063,用于生成第一图形API的调用信息,第一图形API的调用信息用于指示客户端调用第二图形库中的第一图形API,第一图形API的调用信息包括第一图形API涉及的参数,以及第一图形API的识别码;
调用信息发送模块1064,用于将第一图形API的调用信息发送至客户端。
可选地,第一图形API的调用信息包括第一图形API的入口参数和第一图形API的识别码。
可选地,预定图形API包括着色器启动图形API、光栅化启动图形API或交换缓冲区图形API中的任意一种。
当第一图形API为预定图形API时,拒绝调用服务器的应用程序发起的第一绘图指令对应的第一图形API,并发送第一处理通知给应用程序,以通知应用程序第一图形API调用完毕,因此服务器的应用程序可不依赖于客户端而独立运行,即便客户端死机,或客户端与服务器之间的连接中断,也不会影响服务器的应用程序的运行,可提高用户体验。
并且,由于服务器无需将绘制好的图形显示给用户,因此即使拒绝调用服务器的应用程序发起的第一绘图指令对应的第一图形API,从而使得服务器的GPU不工作,也不会影响用户体验。
以下请参见图13,图13是根据本发明实施例的服务器的装置结构示意图,如图13所示,服务器10包括处理器121、总线122以及存储器123,处理器121和存储器123与总线122连接,存储器123存储有程序指令,处理器121执行程序指令以完成上文中介绍的第一图形处理装置105的功能。
以下请参见图14,图14是根据本发明实施例的客户端的装置结构示意图,如图14所示,客户端20包括处理器221、总线222以及存储器223,处理器221和存储器223与总线222连接,存储器223存储有程序指令,处理器221执行程序指令以完成上文中介绍的第二图形处理装置204的功能。
需说明的是,以上描述的任意装置实施例都仅仅是示意性的,其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以 不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。另外,本发明提供的装置实施例附图中,模块之间的连接关系表示它们之间具有通信连接,具体可以实现为一条或多条通信总线或信号线。本领域普通技术人员在不付出创造性劳动的情况下,即可以理解并实施。
通过以上的实施方式的描述,所属领域的技术人员可以清楚地了解到本发明可借助软件加必需的通用硬件的方式来实现,当然也可以通过专用硬件包括专用集成电路、专用CPU、专用存储器、专用元器件等来实现。一般情况下,凡由计算机程序完成的功能都可以很容易地用相应的硬件来实现,而且,用来实现同一功能的具体硬件结构也可以是多种多样的,例如模拟电路、数字电路或专用电路等。但是,对本发明而言更多情况下软件程序实现是更佳的实施方式。基于这样的理解,本发明的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品存储在可读取的存储介质中,如计算机的软盘,U盘、移动硬盘、只读存储器(ROM,Read-Only Memory)、随机存取存储器(RAM,Random Access Memory)、磁碟或者光盘等,包括若干命令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行本发明各个实施例所述的方法。
所属领域的技术人员可以清楚地了解到,上述描述的系统、装置或单元的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。
以上所述,仅为本发明的具体实施方式,但本发明的保护范围并不局限于此,任何熟悉本技术领域的技术人员在本发明揭露的技术范围内,可轻易想到变化或替换,都应涵盖在本发明的保护范围之内。因此,本发明的保护范围应以所述权利要求的保护范围为准。

Claims (27)

  1. 一种图形处理方法,其特征在于,所述方法应用于服务器,所述服务器设置有第一图形库和应用程序,客户端设置有第二图形库,所述第一图形库中包括多个图形应用程序编程接口API,所述第二图形库中包括多个图形API,所述第一图形库中至少有M个图形API与所述第二图形库中的M个图形API一一对应,所述方法包括:
    获取所述应用程序发起的第一绘图指令,所述第一绘图指令对应第一图形API;
    根据所述第一绘图指令调用所述第一图形库的所述第一图形API以执行所述第一绘图指令,并向所述应用程序发送第一处理通知,所述第一处理通知用于通知所述应用程序所述第一图形API调用完毕;
    生成第一图形API的调用信息,所述第一图形API的调用信息用于指示所述客户端调用所述第二图形库中的所述第一图形API,所述第一图形API的调用信息包括所述第一图形API涉及的参数,以及所述第一图形API的识别码;
    将所述第一图形API的调用信息发送至所述客户端。
  2. 根据权利要求1所述的方法,其特征在于,所述方法还包括:
    确定所述第一绘图指令是否需要返回出口参数;
    在所述第一绘图指令需要返回出口参数的情况下,所述第一处理通知还包括所述服务器调用所述第一图形API所产生的出口参数,所述第一图形API的调用信息包括所述第一图形API的入口参数、所述服务器调用所述第一图形API所产生的出口参数和所述第一图形API的识别码。
  3. 根据权利要求2所述的方法,其特征在于,在所述第一图形API不需要返回出口参数的情况下,所述第一图形API的调用信息包括所述第一图形API的入口参数和所述第一图形API的识别码。
  4. 根据权利要求2所述的方法,其特征在于,所述方法还包括:
    接收所述应用程序根据所述第一处理通知发起的第二绘图指令,所述第二绘图指令对应第二图形API,所述第二图形API的入口参数为所述服务器调用所述第一图形API所产生的出口参数;
    根据所述第二绘图指令调用所述第一图形库的所述第二图形API,并向所述应用程序发送第二处理通知,所述第二处理通知用于通知所述应用程序所述第二图形API调用完毕;
    生成第二图形API的调用信息,所述第二图形API的调用信息用于指示所述客户端调用所述第二图形库中的所述第二图形API,所述第二图形API的调用信息包括所述第二图形API涉及的参数,以及所述第二图形API的识别码;
    将所述第二图形API的调用信息发送至所述客户端。
  5. 根据权利要求4所述的方法,其特征在于,所述方法还包括:
    将所述第一图形API的调用信息和所述第二图形API的调用信息保存在先进先出队列;
    在满足预定条件后,将所述先进先出队列中的所述第一图形API的调用信息和所述第二图形API的调用信息同时发送给所述客户端。
  6. 一种图形处理方法,其特征在于,所述方法应用于所述客户端,服务器设置有第一图形库,所述客户端设置有第二图形库,所述第一图形库中包括多个图形应用程序编程接口API,所述第二图形库中包括多个图形API,所述第一图形库中至少有M个图形API与所述第二图形库中的M个图形API一一对应,所述方法包括:
    接收所述服务器发送的第一图形API的调用信息,所述第一图形API的调用信息包括所述第一图形API涉及的参数,以及所述第一图形API的识别码;
    根据所述第一图形API的调用信息中的所述第一图形API的识别码和所述第一图形API的调用信息中的所述第一图形API涉及的参数调用所述第二图形库中的所述第一图形API。
  7. 根据权利要求6所述的方法,其特征在于,所述方法还包括:
    在所述第一图形API的调用信息中的所述第一图形API涉及的参数包括所述服务器调用所述第一图形API所产生的出口参数的情况下,获取所述客户端调用所述第一图形API所产生的出口参数,并建立所述服务器调用所述第一图形API所产生的出口参数与所述客户端调用所述第一图形API所产生的出口参数的映射关系。
  8. 根据权利要求7所述的方法,其特征在于,所述方法还包括:
    接收所述服务器发送的第二图形API的调用信息,所述第二图形API的调用信息包括所述第二图形API涉及的参数以及所述第二图形API的识别码,所述第二图形API的入口参数为所述服务器调用所述第一图形API所产生的出口参数;
    根据所述第二图形API的识别码确定所述第二图形API;
    根据所述第二图形API的入口参数查询所述映射关系,获取所述第二图形API的入口参数在所述映射关系中对应的参数,所述第二图形API的入口参数在所述映射关系中对应的参数为所述客户端调用所述第一图形API所产生的出口参数。
  9. 根据权利要求8所述的方法,其特征在于,所述方法还包括:
    接收所述服务器同时发送的所述第一图形API的调用信息和所述第二图形API的调用信息。
  10. 一种图形处理方法,其特征在于,所述方法应用于服务器,所述服务器设置有第一图形库和应用程序,客户端设置有第二图形库,所述第一图形库中包括多个图形应用程序编程接口API,所述第二图形库中包括多个图形API,所述第一图形库中至少有M个图形API与所述第二图形库中的M个图形API一一对应,所述方法包括:
    获取所述应用程序发起的第一绘图指令,所述第一绘图指令对应第一图形API;
    确定所述第一图形API为所述第一图形库的预定图形API时,拒绝调用所述第一图形库的所述第一图形API;
    向所述应用程序发送第一处理通知,所述第一处理通知用于通知所述应用程序所述第一图形API调用完毕;
    生成第一图形API的调用信息,所述第一图形API的调用信息用于指示所述客户端调用所述第二图形库中的所述第一图形API,所述第一图形API的调用信息包括所述第一图形API涉及的参数,以及所述第一图形API的识别码;
    将所述第一图形API的调用信息发送至所述客户端。
  11. 根据权利要求10所述的方法,其特征在于,所述第一图形API的调用信息包括所述第一图形API的入口参数和所述第一图形API的识别码。
  12. 根据权利要求10或11所述的方法,其特征在于,所述预定图形API包括着色器启动图形API、光栅化启动图形API、清除缓冲区图形API或交换缓冲区图形API中的任意一种。
  13. 一种图形处理装置,其特征在于,设置于服务器,所述服务器还设置有第一图形库和应用程序,客户端设置有第二图形库,所述第一图形库中包括多个图形应用程序编程接口API,所述第二图形库中包括多个图形API,所述第一图形库中至少有M个图形API与所述第二图形库中的M个图形API一一对应,所述图形处理装置包括:
    绘图指令获取模块,用于获取所述应用程序发起的第一绘图指令,所述第一绘图指令对应第一图形API;
    图形API调用模块,用于根据所述第一绘图指令调用所述第一图形库的所述第一图形API以执行所述第一绘图指令,并向所述应用程序发送第一处理通知,所述第一处理通知用于通知所述应用程序所述第一图形API调用完毕;
    调用信息生成模块,用于生成第一图形API的调用信息,所述第一图形API的调用信息用于指示所述客户端调用所述第二图形库中的所述第一图形API,所述第一图形API的调用信息包括所述第一图形API涉及的参数,以及所述第一图形API的识别码;
    调用信息发送模块,用于将所述第一图形API的调用信息发送至所述客户端。
  14. 根据权利要求13所述的装置,其特征在于,所述调用信息生成模块还用于:
    确定所述第一绘图指令是否需要返回出口参数;
    在所述第一绘图指令需要返回出口参数的情况下,所述第一处理通知还包括所述服务器调用所述第一图形API所产生的出口参数,所述第一图形API的调用信息包括所述第一图形API的入口参数、所述服务器调用所述第一图形API所产生的出口参数和所述第一图形API的识别码。
  15. 根据权利要求14所述的装置,其特征在于,在所述第一图形API不需要返回出口参数的情况下,所述第一图形API的调用信息包括所述第一图形API的入口参数和所述第一图形API的识别码。
  16. 根据权利要求14所述的装置,其特征在于,
    所述绘图指令获取模块,用于获取所述应用程序根据所述第一处理通知发起的第二绘图指令,所述第二绘图指令对应第二图形API,所述第二图形API的入口参数为所述服务器调用所述第一图形API所产生的出口参数;
    所述图形API调用模块,用于根据所述第二绘图指令调用所述第一图形库的所述第二图形API,并向所述应用程序发送第二处理通知,所述第二处理通知用于通知所述应用程序所述第二图形API调用完毕;
    所述调用信息生成模块,用于生成第二图形API的调用信息,所述第二图形API的调用信息用于指示所述客户端调用所述第二图形库中的所述第二图形API,所述第二图形API的调用信息包括所述第二图形API涉及的参数,以及所述第二图形API的识别码;
    所述调用信息发送模块,用于将所述第二图形API的调用信息发送至所述客户端。
  17. 根据权利要求16所述的装置,其特征在于,所述调用信息发送模块,具体用于:
    将所述第一图形API的调用信息和所述第二图形API的调用信息保存在先进先出队列;
    在满足预定条件后,将所述先进先出队列中的所述第一图形API的调用信息和所述第二图形API的调用信息同时发送给所述客户端。
  18. 一种图形处理装置,其特征在于,设置于客户端,服务器设置有第一图形库,所述客户端设置有第二图形库,所述第一图形库中包括多个图形应用程序编程接口API,所述第二图形库中包括多个图形API,所述第一图形库中至少有M个图形API与所述第二图形库中的M个图形API一一对应,所述图形处理装置包括:
    调用信息接收模块,用于接收所述服务器发送的第一图形API的调用信息,所述第一图形API的调用信息包括所述第一图形API涉及的参数,以及所述第一图形API的识别码;
    图形API调用模块,用于根据所述第一图形API的调用信息中的所述第一图形API的识别码确和所述第一图形API的调用信息中的所述第一图形API涉及的参数调用所述第二图形库中的所述第一图形API。
  19. 根据权利要求18所述的装置,其特征在于,所述装置还包括映射关系建立模块,用于在所述第一图形API的调用信息中的所述第一图形API涉及的参数包括所述服务器调用所述第一图形API所产生的出口参数的情况下,获取所述客户端调用所述第一图形API所产生的出口参数,并建立所述服务器调用所述第一图形API所产生的出口参数与所述客户端调用所述第一图形API所产生的出口参数的映射关系。
  20. 根据权利要求19所述的装置,其特征在于,
    所述调用信息接收模块,还用于接收所述服务器发送的第二图形API的调用信息,所述第二图形API的调用信息包括所述第二图形API涉及的参数以及所述第二图形API的识别码,所述第二图形API的入口参数为所述服务器调用所述第一图形API所产生的出口参数;
    所述图形API调用模块,还用于根据所述第二图形API的识别码确定所述第二图形API;
    所述图形API调用模块,还用于根据所述第二图形API的入口参数查询所述映射关系,获取所述第二图形API的入口参数在所述映射关系中对应的参数,所述第二图形API的入口参数在所述映射关系中对应的参数为所述客户端调用所述第一图形API所产生的出口参数。
  21. 根据权利要求20所述的装置,其特征在于,所述调用信息接收模块,具体用于:
    接收所述服务器同时发送所述第一图形API的调用信息和所述第二图形API的调用信息。
  22. 一种图形处理装置,其特征在于,设置于服务器,所述服务器还设置有第一图形库和应用程序,客户端设置有第二图形库,所述第一图形库中包括多个图形应用程序编程接口API,所述第二图形库中包括多个图形API,所述第一图形库中至少有M个图形API与所述第二图形库中的M个图形API一一对应,所述图形处理装置包括:
    绘图指令获取模块,用于获取所述应用程序发起的第一绘图指令,所述第一绘图指令 对应第一图形API;
    图形API调用模块,用于确定所述第一图形API为所述第一图形库的预定图形API时,拒绝调用所述第一图形库的所述第一图形API;
    所述图形API调用模块,还用于向所述应用程序发送第一处理通知,所述第一处理通知用于通知所述应用程序所述第一图形API调用完毕;
    调用信息生成模块,用于生成第一图形API的调用信息,所述第一图形API的调用信息用于指示所述客户端调用所述第二图形库中的所述第一图形API,所述第一图形API的调用信息包括所述第一图形API涉及的参数,以及所述第一图形API的识别码;
    调用信息发送模块,用于将所述第一图形API的调用信息发送至所述客户端。
  23. 根据权利要求22所述的装置,其特征在于,所述第一图形API的调用信息包括所述第一图形API的入口参数和所述第一图形API的识别码。
  24. 根据权利要求22或23所述的装置,其特征在于,所述预定图形API包括着色器启动图形API、光栅化启动图形API、清除缓冲区图形API或交换缓冲区图形API中的任意一种。
  25. 一种服务器,其特征在于,包括处理器和存储器,所述存储器存储有程序指令,所述处理器运行所述程序指令以执行权利要求1至5任一项所述的步骤。
  26. 一种客户端,其特征在于,包括处理器和存储器,所述存储器存储有程序指令,所述处理器运行所述程序指令以执行权利要求6至9任一项所述的步骤。
  27. 一种服务器,其特征在于,包括处理器和存储器,所述存储器存储有程序指令,所述处理器运行所述程序指令以执行权利要求10至12任一项所述的步骤。
PCT/CN2017/107341 2017-10-23 2017-10-23 图形处理方法及相关装置和设备 WO2019079940A1 (zh)

Priority Applications (5)

Application Number Priority Date Filing Date Title
PCT/CN2017/107341 WO2019079940A1 (zh) 2017-10-23 2017-10-23 图形处理方法及相关装置和设备
CN202010738340.XA CN112068908B (zh) 2017-10-23 2017-10-23 图形处理方法及相关装置和设备
CN201780065686.4A CN109983435B (zh) 2017-10-23 2017-10-23 图形处理方法及相关装置和设备
EP17929704.9A EP3693850A4 (en) 2017-10-23 2017-10-23 GRAPHIC PROCESSING PROCESS AND ASSOCIATED DEVICE AND DEVICE
US16/854,929 US11189003B2 (en) 2017-10-23 2020-04-22 Graphics processing method and related apparatus, and device for unidirectionally transmitting calling information of a graphics API to a client

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/107341 WO2019079940A1 (zh) 2017-10-23 2017-10-23 图形处理方法及相关装置和设备

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/854,929 Continuation US11189003B2 (en) 2017-10-23 2020-04-22 Graphics processing method and related apparatus, and device for unidirectionally transmitting calling information of a graphics API to a client

Publications (1)

Publication Number Publication Date
WO2019079940A1 true WO2019079940A1 (zh) 2019-05-02

Family

ID=66247125

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/107341 WO2019079940A1 (zh) 2017-10-23 2017-10-23 图形处理方法及相关装置和设备

Country Status (4)

Country Link
US (1) US11189003B2 (zh)
EP (1) EP3693850A4 (zh)
CN (2) CN112068908B (zh)
WO (1) WO2019079940A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11947993B2 (en) * 2021-06-22 2024-04-02 International Business Machines Corporation Cooperative input/output of address modes for interoperating programs

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103164838A (zh) * 2011-12-12 2013-06-19 扬智科技股份有限公司 图形数据处理方法
CN103593184A (zh) * 2013-10-31 2014-02-19 福州瑞芯微电子有限公司 图像显示系统及图像显示方法
CN104536758A (zh) * 2014-12-29 2015-04-22 北京奇艺世纪科技有限公司 一种图形生成方法及装置

Family Cites Families (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06325182A (ja) * 1993-05-10 1994-11-25 Hitachi Ltd グラフィックス描画方法及びその装置と計算機システム
US5903278A (en) * 1996-08-02 1999-05-11 Hewlett-Packard Company Primitive colorization of three dimensional graphics to provide feedback to a developer
EP1051694A1 (en) * 1998-01-26 2000-11-15 UNIF/X Inc. A transaction execution system interface and enterprise system architecture thereof
US7274368B1 (en) 2000-07-31 2007-09-25 Silicon Graphics, Inc. System method and computer program product for remote graphics processing
US20060080406A1 (en) 2004-08-20 2006-04-13 Boris Sacks XWindows++
US8035636B1 (en) * 2005-09-08 2011-10-11 Oracle America, Inc. Software system for efficient data transport across a distributed system for interactive viewing
EP2018027A1 (en) * 2005-11-03 2009-01-21 KTFreetel Co., Ltd. Business logic device and processing method
US7898554B2 (en) * 2007-06-07 2011-03-01 Apple Inc. Asymmetric two-pass graphics scaling
US8180905B2 (en) 2008-12-09 2012-05-15 Microsoft Corporation User-mode based remote desktop protocol (RDP) encoding architecture
KR101520445B1 (ko) * 2008-12-16 2015-05-14 삼성전자주식회사 그래픽 데이터 처리 방법
US8307103B2 (en) 2009-03-09 2012-11-06 Microsoft Corporation Tear-free remote desktop protocol (RDP) display
US8171154B2 (en) 2009-09-29 2012-05-01 Net Power And Light, Inc. Method and system for low-latency transfer protocol
US8799357B2 (en) * 2010-11-08 2014-08-05 Sony Corporation Methods and systems for use in providing a remote user interface
US8941673B2 (en) * 2011-11-08 2015-01-27 Red Hat, Inc. Rendering images in a remote web browser
US8922569B1 (en) * 2011-12-30 2014-12-30 hopTo Inc. Cloud based system for and method of translating between disparate 3D graphics languages in client-server computing environments
US9064292B1 (en) * 2011-12-30 2015-06-23 hopTo, Inc. System for and method of classifying and translating graphics commands in client-server computing systems
CN103457965B (zh) 2012-05-29 2018-11-06 索尼公司 虚拟机系统和远程显示方法
WO2013180729A1 (en) * 2012-05-31 2013-12-05 Intel Corporation Rendering multiple remote graphics applications
KR20140027741A (ko) * 2012-08-27 2014-03-07 한국전자통신연구원 응용 서비스 제공 시스템 및 방법, 응용 서비스를 위한 서버 장치 및 클라이언트 장치
CN104904230B (zh) * 2012-10-18 2018-10-09 Lg电子株式会社 处理交互服务的设备和方法
CN104065679B (zh) 2013-03-21 2018-04-20 华为技术有限公司 一种远程桌面操作的方法及客户端
WO2016045729A1 (en) * 2014-09-25 2016-03-31 Huawei Technologies Co.,Ltd. A server for providing a graphical user interface to a client and a client
US9876880B2 (en) * 2014-12-05 2018-01-23 Red Hat, Inc. Creation of a binding based on a description associated with a server
US10225570B2 (en) 2015-11-12 2019-03-05 Vmware, Inc. Split framebuffer encoding
CN106161630A (zh) 2016-07-20 2016-11-23 中霆云计算科技(上海)有限公司 一种基于远程桌面协议的客户端屏幕更新显示方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103164838A (zh) * 2011-12-12 2013-06-19 扬智科技股份有限公司 图形数据处理方法
CN103593184A (zh) * 2013-10-31 2014-02-19 福州瑞芯微电子有限公司 图像显示系统及图像显示方法
CN104536758A (zh) * 2014-12-29 2015-04-22 北京奇艺世纪科技有限公司 一种图形生成方法及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3693850A4 *

Also Published As

Publication number Publication date
US20200250790A1 (en) 2020-08-06
CN109983435B (zh) 2020-08-07
US11189003B2 (en) 2021-11-30
CN112068908B (zh) 2024-01-30
EP3693850A4 (en) 2020-10-21
CN109983435A (zh) 2019-07-05
EP3693850A1 (en) 2020-08-12
CN112068908A (zh) 2020-12-11

Similar Documents

Publication Publication Date Title
JP4489399B2 (ja) プロセッサでのデータ処理方法及びデータ処理システム
JP4597553B2 (ja) コンピュータ・プロセッサ及び処理装置
US9734546B2 (en) Split driver to control multiple graphics processors in a computer system
EP3951595A1 (en) Method and apparatus for graphics rendering
US7523157B2 (en) Managing a plurality of processors as devices
US7549145B2 (en) Processor dedicated code handling in a multi-processor environment
KR101394094B1 (ko) 응용 프로그램 화상의 표시 방법 및 장치
CN111737019B (zh) 一种显存资源的调度方法、装置及计算机存储介质
CN114020470B (zh) 资源分配方法、装置、可读介质及电子设备
KR20010014944A (ko) 계산기 시스템
WO2021013019A1 (zh) 一种图片处理方法及装置
CN114972607B (zh) 加速图像显示的数据传输方法、装置及介质
US20060061578A1 (en) Information processing apparatus for efficient image processing
US10949226B2 (en) Display method of multi-application based on Android system, and terminal device
US20240143395A1 (en) Method and device for identifying android system drawing thread, and mobile terminal and storage medium
US8368704B2 (en) Graphic processor and information processing device
CN107077376B (zh) 帧缓存实现方法、装置、电子设备和计算机程序产品
EP1768024A1 (en) Processing management device, computer system, distributed processing method, and computer program
CN115629884A (zh) 一种线程调度方法、电子设备及存储介质
CN113886019B (zh) 虚拟机创建方法、装置、系统、介质和设备
CN114168301A (zh) 线程调度方法、处理器以及电子装置
EP1647887A2 (en) Apparatus for efficient image processing
WO2023221822A1 (zh) 数据处理方法、电子设备和可读存储介质
WO2019079940A1 (zh) 图形处理方法及相关装置和设备
CN117130571A (zh) 基于多核异构系统的显示方法、设备、芯片、存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17929704

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2017929704

Country of ref document: EP

Effective date: 20200505