WO2017129105A1 - 一种图形界面更新方法和装置 - Google Patents

一种图形界面更新方法和装置 Download PDF

Info

Publication number
WO2017129105A1
WO2017129105A1 PCT/CN2017/072408 CN2017072408W WO2017129105A1 WO 2017129105 A1 WO2017129105 A1 WO 2017129105A1 CN 2017072408 W CN2017072408 W CN 2017072408W WO 2017129105 A1 WO2017129105 A1 WO 2017129105A1
Authority
WO
WIPO (PCT)
Prior art keywords
instruction
dirty
graphical interface
intersection
data
Prior art date
Application number
PCT/CN2017/072408
Other languages
English (en)
French (fr)
Inventor
王伟光
曹文升
韩祥
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to US15/762,266 priority Critical patent/US10645005B2/en
Publication of WO2017129105A1 publication Critical patent/WO2017129105A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • G06F3/1462—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay with means for detecting differences between the image stored in the host and the images displayed on the remote displays
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/445—Program loading or initiating
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/12—Avoiding congestion; Recovering from congestion
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/30—Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
    • A63F13/35—Details of game servers
    • A63F13/358—Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/50—Controlling the output signals based on the game progress
    • A63F13/53—Controlling the output signals based on the game progress involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
    • A63F13/533—Controlling the output signals based on the game progress involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game for prompting the player, e.g. by displaying a game menu
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/50—Controlling the output signals based on the game progress
    • A63F13/53—Controlling the output signals based on the game progress involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
    • A63F13/537—Controlling the output signals based on the game progress involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game using indicators, e.g. showing the condition of a game character on screen
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/60—Software deployment
    • G06F8/65—Updates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/131—Protocols for games, networked simulations or virtual reality
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/20—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterised by details of the game platform
    • A63F2300/209—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterised by details of the game platform characterized by low level software layer, relating to hardware management, e.g. Operating System, Application Programming Interface
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/40—Filling a planar surface by adding surface attributes, e.g. colour or texture
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2350/00—Solving problems of bandwidth in display systems

Definitions

  • the present application relates to the field of cloud technologies, and in particular, to a method and device for updating a graphical interface.
  • the cloud interaction system usually includes a server and multiple user devices.
  • the server can be a cluster server, and various interactive applications can be used. Run in a cloud interactive system.
  • Servers and user devices may require an interactive graphical interface when running an interactive application.
  • the server sends the updated cache data to the user equipment end each time the cached data of the graphical interface is updated.
  • the embodiment of the invention provides a method and device for updating a graphical interface, so as to help reduce network traffic between the server and the user equipment, and improve the operating efficiency of the server.
  • a first aspect of the present application provides a method for updating a graphical interface, the method comprising: determining a dirty region, where the graphical region needs to be changed; acquiring a drawing instruction, the drawing instruction indicating a drawing area; calculating a drawing area and An intersection of the dirty regions; an instruction data stream indicating the intersection is sent to the user equipment, wherein the user equipment updates the graphical interface in accordance with the instruction data stream.
  • the second aspect of the present application provides a graphical interface updating apparatus, where the apparatus includes: a first determining module, configured to determine a dirty area, where the dirty area is an area that the graphic interface needs to be changed; and a first acquiring module, configured to acquire a drawing An instruction, the drawing instruction instructs a drawing area, and a calculation module for calculating a drawing area An intersection of the domain and the dirty region; a sending module, configured to send, to the user equipment, an instruction data stream indicating the intersection, wherein the user equipment updates the graphical interface according to the instruction data stream.
  • the server is used to record the area where the graphic interface needs to be changed as a dirty area, and when the graphic interface needs to be updated, the intersection of the drawing area and the dirty area is calculated.
  • the technical solution that the user equipment sends the intersection of the instruction data stream achieves the following effects:
  • the intersection of the instruction data stream is sent, that is, the intersection stream is reduced, and the number of streams is reduced; and the stream is only the intersection, not the entire dirty area, and the stream data is reduced; Moreover, some useless data streams do not intersect with the data interval indicated by the drawing instruction, and the stream is not streamed, thereby further reducing the streamed data; thus, network traffic is optimized, and the network traffic of the stream can be effectively reduced;
  • 1 is a schematic flow chart of synchronous streaming
  • FIG. 2 is a schematic flow chart of asynchronous streaming
  • 3 is a schematic diagram of pseudo code updated for a one-dimensional vertex buffer
  • FIG. 4 is a schematic diagram of data usage when the graphical interface is updated
  • FIG. 5 is a schematic structural diagram of an interactive application system according to an embodiment of the present invention.
  • FIG. 6 is a schematic flowchart of a method for updating a graphical interface according to an embodiment of the present invention
  • FIG. 7 is a schematic flowchart of a method for updating a graphical interface in a specific embodiment of the present invention.
  • FIG. 8 is a specific flowchart when a drawing function is invoked according to an embodiment of the present invention.
  • FIG. 9 is a schematic diagram of a process of a dirty region algorithm according to an embodiment of the present invention.
  • FIG. 10 is a schematic structural diagram of a graphic interface updating apparatus according to an embodiment of the present invention.
  • FIG. 11 is a schematic structural diagram of another graphic interface updating apparatus according to an embodiment of the present invention.
  • FIG. 12 is a schematic structural diagram of a server according to an embodiment of the present invention.
  • Interactive applications include servers and user devices, and when running interactive applications such as cloud games, servers and user devices may require an interactive graphical interface.
  • Clients running interactive applications on servers and user devices are often referred to as Trace and Retrace.
  • the Trace end refers to the game client running on the cloud game server, that is, the cloud game cloud, executing the main logical operation of the game, and transmitting the graphical interface related to the user interaction to the Retrace terminal via the network.
  • the Retrace side refers to the client running on the user device, that is, the client of the cloud game, which is used to express the game graphical interface of the Trace side, input the user's mouse, keyboard and other devices, and transmit it to the Trace end via the network.
  • the Trace interface and the Retrace side interact with the graphical interface
  • the data of the Application Programming Interface (Api) are encoded by the network and sent to the peer end, which may be referred to as streaming.
  • streaming There are currently two streaming technologies, synchronous and asynchronous.
  • FIG. 1 is a schematic flow chart of synchronous streaming.
  • the Trace side Api stream is generally generated when the Api is called.
  • the function call generally enters the Hook (hook injection) Api, then calls the real Direct3D Api, and finally returns the Api function, parameters, and return. Values are encoded into data and streamed to the Retrace side.
  • Direct3D is a three-dimensional (3D) graphics API based on Microsoft's Common Object Mode (Common Object Mode), which is part of DirectX (Direct eXtension, DX).
  • DirectX is a multimedia programming interface created by Microsoft Corporation. It consists of many APIs. According to its nature, it can be divided into four parts: display part, sound part, input part and network part. Display part For DirectDraw and Direct3D, the former is responsible for two-dimensional (2D) image acceleration, and the latter is responsible for the display of 3D effects.
  • Hook technology replaces the original function execution process with a custom function. Hook can touch the Api function entry point that needs to be modified, and change its address to point to the new custom function.
  • FIG. 2 is a schematic flow chart of asynchronous streaming.
  • the asynchronous streaming technology is for updating cached data, for example, for a one-dimensional vertex buffer (VertexBuffe). The process is as shown in FIG. 2, including:
  • the call to Lock usually first enters Hook Api, then calls the real Direct3D Api; then, the Lock Api data to User equipment.
  • the cache data update of VertexBuffe is performed in the locked data interval, that is, Update (ppbDate), and the update method may use a memory copy (Memcpy) function, and other methods may be used.
  • Figure 3 shows the pseudo code of the asynchronous stream process described above.
  • IDirect3DVertexBuffer9::Lock() includes four parameters, namely: OffsetToLock (locked memory start address), SizeToLock (locked memory size), ppbData (return memory pointer address), flags (locked attribute).
  • Update(ppbDate) is not the Api of DirectX3D. Therefore, after locking a data interval, the Trace side does not know how to update the data of this interval. Therefore, as shown in Figure 2, when IDirect3DVertexBuffer9::UnLock() is called, it is necessary to simulate generating a Memcpy function to stream the cached data of the game update. Specifically, find the locked data range of VertexBuffe from the record. And copy the data in this interval from Direct3D to slow In Temp, Hook Api calls the simulated Memcpy and streams the data in Temp as source data to the user device. Finally, the Api data of UnLock() is also streamed.
  • UnLock() function call will stream two function data. Therefore, each time you update the cached data, you have to stream three functions: Lock(), Memcpy(), UnLock(). In addition, when UnLock() streams data, it is concatenated by the size of the data range locked by Lock().
  • the cached data of the graphical interface generally includes: a one-dimensional vertex buffer, an index cache, a two-dimensional map, and a stereoscopic map for 3D games.
  • the game logic layer in Lock may indicate to lock a large data interval, but in fact may only update a small piece of data, the updated data is smaller than the locked data interval, but in UnLock () streaming data At the same time, it is required to stream according to the data interval indicated by Lock(), so that some data that does not need to be streamed is also streamed, which leads to waste of network traffic.
  • the reason for the analysis is that the 3D game engine provides functions such as Lock(), UnLock(), LockRect(), UnLockRect() for one-dimensional vertex buffer, index cache, two-dimensional texture, and stereo map, but does not provide direct update cache.
  • Data Api in addition to the data pointer that can be obtained by Lock(), the game can update any size data in any way, and the amount of updated data can be much smaller than the interval specified by Lock(); Hook Api is not known when UnLock() How much data is updated in the game, and how much data is used when it is to be drawn, so data can only be streamed according to the interval of Lock().
  • the 3D game engine provides lock/unlock functions for one-dimensional vertex buffers, index caches, two-dimensional textures, and three-dimensional maps, such as one-dimensional Lock() and UnLock() for two-dimensional LockRect().
  • Functions such as UnLockRect() when updating data, consider copying data from the game logic layer to system memory or graphics card memory, and the 3D game engine and graphics card are optimized for such a usage model, but in the instruction string.
  • the process used in the streaming technology scheme to access data from the graphics card to the memory is slower and affects the computing speed of the Graphics Processing Unit (GPU); therefore, when a large number of UnLock() functions are called, a large number of The graphics card data into the memory will cause the Trace side to run inefficiently.
  • GPU Graphics Processing Unit
  • the updated cached data stream is streamed to the Retrace side.
  • the lock function is first called to lock a data interval to obtain access rights, and then the update operation can be performed, and then the unlock function is called to release the locked data interval.
  • the updated cache data needs to be streamed by using the simulated memory copy (Memcpy) function, that is, each time the cache data is updated, the following three functions must be streamed: Lock(), Memcpy(), UnLock ().
  • the embodiment of the present invention provides a method and device for updating the graphical interface. The detailed description will be respectively made below through specific embodiments.
  • Embodiments of the present invention provide a method for updating a graphical interface.
  • the method can be applied to an interactive application system as shown in FIG. 5, the interactive application system including a server and a user equipment.
  • the Trace terminal is running on the server, and the Retrace terminal is running on the user equipment.
  • the Trace end is the cloud of an interactive application such as a cloud game, performs the main logical operation of the game, and transmits a graphical interface related to the user interaction to the Retrace end via the network.
  • the transmission process includes:
  • the game logic calls Direct3D Api to generate a graphical interface.
  • the function call generally enters Hook Api and then calls the real Direct3D Api; the server extracts the API data and streams it to the user device.
  • the Retrace side is a client running on the user device. It can receive API data, play back Direct3D Api, and generate and display a graphical interface.
  • the method may include:
  • the server records the area of the graphical interface that needs to be changed or updated as a dirty area.
  • the dirty area is a graphical interface area that needs to be updated but is not necessarily updated in the end.
  • the server determines the area that the graphic interface needs to be updated, it is not necessary to immediately stream the data of the area to the user equipment, but only records the area as a dirty area.
  • a lock command such as a Lock() function may be invoked to lock a data interval to obtain access rights of the data interval, and then in the data interval in which the access right is obtained. Update the cached data, and finally call the unlock command, such as the UnLock() function, to release the locked data interval.
  • the server in order to reduce the number of streams, records the data interval indicated by the locking instruction as a dirty area when acquiring the locking instruction, because the area that the graphic interface needs to be updated is included in the adding The data interval indicated by the lock command. After executing the locking instruction, the server does not need to perform streaming, that is, determines that the instruction data stream corresponding to the locking instruction is not sent to the user equipment.
  • the server records the data interval indicated by the locking instruction as a dirty area, and after executing the locking instruction, acquires cache data required for the graphic interface update to update the cached data, and obtains the updated data after the update. And execute the unlock command to release the data interval locked by the lock command.
  • the streaming since the dirty area has been recorded, after the unlocking instruction is executed, the streaming does not need to be performed, that is, it is determined that the cache data and the instruction data stream corresponding to the unlocking instruction are not sent to the user equipment.
  • the "dirty area” in the embodiment of the present invention is derived from the “dirty rectangle” algorithm and is an extension of the "dirty rectangle".
  • “Dirty rectangle” means: when the 2D graphical interface of the screen is drawn, when only the image in a small rectangle needs to be updated, if the entire screen is redrawn, it will inevitably waste resources; therefore, it is only marked first. Its rectangular position, not immediately updated, is marked as “dirty”, the rectangle marked as “dirty” is called “dirty rectangle”; when updating the interface, all the rectangular areas of the record are updated once, by "dirty” Change To be “clean”, this method is called the “dirty rectangle” algorithm.
  • the "dirty rectangle” is for a 2D graphical interface.
  • the "dirty area” in the embodiment of the present invention is no longer a 2D graphical interface, and includes: a one-dimensional array interval, a two-dimensional plane rectangle, a three-dimensional space region, and also when there is data to be updated, The "dirty area” is recorded first, and when it is updated, the “dirty area” is made clean.
  • the server does not stream lock instructions such as the Lock() function, and no longer simulates the generation of the Memcpy function, nor does it stream the UnLock() function.
  • the drawing instruction may be first acquired, then the intersection of the drawing area indicated by the drawing instruction and the dirty area may be calculated, and finally the instruction data stream indicating the intersection is sent to the user equipment, and the user equipment updates the graphic according to the instruction data stream. interface.
  • the stream occurs when the graphical interface is updated.
  • the interactive application for example, the cloud game
  • the server may issue a drawing instruction.
  • the data interval indicated by the drawing instruction may be calculated according to the parameter of the drawing instruction, and the data interval is a drawing area.
  • the server can calculate the intersection of the drawing area and the dirty area, or calculate the intersection of the data section indicated by the drawing instruction and the dirty area, and only stream the intersection, only to the user equipment. Transmitting the instruction data stream of the intersection, and the server further needs to stream the drawing instruction, and send the instruction data stream of the drawing instruction to the user equipment.
  • the intersecting instruction data stream and the instruction data stream of the drawing instruction are used by the user equipment to update a picture in a dirty area on the user equipment side.
  • the intersection may be first determined to be empty. If the intersection is empty, no instruction data stream is sent. If the intersection is not empty, an analog locking instruction such as Lock is generated for the intersection. a () function, a memory copy instruction such as a Memcpy function and a simulated unlock instruction such as an UnLock() function, the analog lock instruction instructing to lock the intersection, the memory copy instruction being generated based on cached data that needs to be updated in the dirty region; The server performs streaming for the intersection, that is, sends the analog locking instruction, the memory copy instruction, and the instruction data stream corresponding to the simulated unlocking instruction to the user equipment.
  • an analog locking instruction such as Lock is generated for the intersection.
  • a () function a memory copy instruction such as a Memcpy function and a simulated unlock instruction such as an UnLock() function
  • the server performs streaming for the intersection, that is, sends the analog locking instruction, the memory copy instruction, and the instruction data stream corresponding to the simulated unlocking instruction to the user equipment.
  • the cached data Before updating the graphical interface, the cached data may have been updated multiple times. Therefore, there may be multiple dirty regions, and there may be multiple intersections obtained in this step.
  • a Memcpy function Before sending the instruction data stream corresponding to the intersection, A Memcpy function can be generated separately for multiple intersections. A string is sent with a Lock() function, multiple Memcpy functions, and an instruction data stream corresponding to an UnLock() function.
  • the dirty area needs to be updated, including: removing the intersection of the already streamed, removing the dirty area of the record, and leaving the remaining dirty area as the current dirty area, thereby completing the dirty area. Update.
  • the command data stream corresponding to the intersection is sent to the user equipment, and is used by the user equipment to update the screen in the dirty area of the graphical interface of the user equipment side.
  • the server On the server side, the server also needs to execute the drawing instruction to update the screen in the dirty area of the server side graphical interface using the intersection data. At this point, the update of the graphical interface is completed on both sides of the server and the user device.
  • the intersection of the instruction data stream is sent, that is, the intersection stream is not necessarily streamed when the cache data is updated, reducing the number of streams; and the stream is only an intersection, not an The entire dirty area reduces the streamed data; and, some useless data streams do not intersect with the data interval indicated by the drawing instruction, and the stream is not streamed, thereby further reducing the streamed data; thus, the network traffic is optimized. , can effectively reduce the network traffic of the stream;
  • a method for updating a one-dimensional vertex buffer (VertexBuffer) running in an interactive application system may include:
  • the game calls the Lock() function, ie, IDirect3DVertexBuffer9::Lock; due to the Hook technique, the Lock() function call first enters the Hook Api; in the Hook Api, the real first is called. Lock() function; then the data indicated by the Lock() function The interval is associated with the VertexBuffer object, saved and recorded as a dirty area.
  • Lock() function ie, IDirect3DVertexBuffer9::Lock
  • the game updates the VertexBuffer data.
  • the server has no operation.
  • the update method may be to use the Memcpy function, or other methods may be used.
  • UnLock () function that is, IDirect3DVertexBuffer9::UnLock; due to the use of Hook technology, the Lock() function call first enters Hook Api; in Hook Api, the real Unlock() function is directly transferred.
  • the game calls the DrawPrimitive or DrawIndexedPrimitive drawing function.
  • the drawing function When calling the drawing function, first enter the Hook Api, analyze the data interval of the VertexBuffer that the current Api needs to use; if the data interval is dirty in the record, simulate a Lock(), several times of Memcpy function, one UnLock() function, and Stream them to the user device. Finally, in Hook Api, call the real DrawPrimitive or DrawIndexedPrimitive drawing function. For further explanation of this step, please refer to Figure 8.
  • Figure 8 is a specific flow chart when calling DrawPrimitive or DrawIndexedPrimitive drawing function, including:
  • A1 first calculate the data interval of the VertexBuffer that needs to be used by the drawing function by drawing the parameters of the function;
  • A4 call the real DrawPrimitive or DrawIndexedPrimitive.
  • the current Api is the data interval of the VertexBuffer that needs to be used for the drawing function.
  • the simulation generates three kinds of Api calls, namely, Lock. (), Memcpy, UnLock () three functions call, where Lock (), and UnLock () can be: IDirect3DVertexBuffer9::Lock, IDirect3DVertexBuffer9::UnLock; Finally call the real DrawPrimitive or DrawIndexedPrimitive.
  • the DrawPrimitive or DrawIndexedPrimitive function is called multiple times by calling the Lock() and UnLock() functions for one VertexBuffer as an example to describe the dirty region algorithm used in the embodiment of the present invention:
  • the game calls Lock(), UnLock() on the same VertexBuffer for the second time, and updates the "dirty area" of the VertexBuffer ⁇ [a,b],[c,d] ⁇ after updating the interval [c,d];
  • the data of the stream is not immediately applied by the Retrace side, such as graphics being drawn, music being played, etc.; when another API is called, Only the Api data that has been streamed before it is used; the technical solution of the present application can be applied.
  • an embodiment of the present invention provides a graphical interface updating apparatus 200, which is applied to a server of an interactive application system, where the interactive application system includes a user equipment and the server, and the apparatus 200 includes :
  • the first determining module 210 is configured to determine a dirty area, where the dirty area is an area that the graphic interface needs to be changed, and a calculating module 220, configured to calculate an intersection of the drawing area and the dirty area;
  • the sending module 230 is configured to send, to the user equipment, an instruction data stream that indicates the intersection, where the user equipment updates the graphical interface according to the instruction data stream.
  • the device 200 further includes: a second obtaining module 240; configured to acquire a locking instruction;
  • the first determining module 210 determines the dirty area according to the data interval indicated by the locking instruction.
  • the second obtaining module 240 is further configured to acquire cache data required for the graphical interface update, and obtain an unlocking instruction
  • the device 200 further includes: a second determining module 250, configured to determine not to send the instruction data stream corresponding to the locking instruction to the user equipment; determining not to send the cache data to the user equipment The instruction data stream corresponding to the unlocking instruction.
  • a second determining module 250 configured to determine not to send the instruction data stream corresponding to the locking instruction to the user equipment; determining not to send the cache data to the user equipment The instruction data stream corresponding to the unlocking instruction.
  • the apparatus 200 further includes:
  • a first obtaining module 260 configured to acquire a drawing instruction, where the drawing area is a data interval indicated by the drawing instruction
  • the sending module 230 is further configured to send the instruction data stream of the drawing instruction to the user equipment.
  • the apparatus 200 further includes:
  • the determining module 270 is configured to determine whether the intersection is empty
  • a generating module 280 configured to generate an analog locking instruction, a memory copy instruction, and a simulated unlocking instruction for the intersection if the intersection is not empty;
  • the sending module 230 is further configured to send, to the user equipment, the analog locking instruction, the memory copy instruction, and the instruction data stream corresponding to the simulated unlocking instruction.
  • the first determining module is further configured to remove the intersection from the dirty area to complete the update of the dirty area.
  • the apparatus 200 further includes:
  • the drawing module 290 is configured to execute the drawing instruction, and update the picture in the dirty area by using the intersection data.
  • the intersection of the instruction data stream is sent, that is, the intersection stream is not necessarily streamed when the cache data is updated, reducing the number of streams; and the stream is only an intersection, not an The entire dirty area reduces the streamed data; and, some useless data streams do not intersect with the data interval indicated by the drawing instruction, and the stream is not streamed, thereby further reducing the streamed data; thus, the network traffic is optimized. , can effectively reduce the network traffic of the stream;
  • Embodiments of the present invention also provide a computer readable storage medium storing one or more programs, the one or more programs, when executed by a server including one or more processors, causing the server to perform the method implementation as described above
  • the graphical interface update method described in the example is also provided.
  • an embodiment of the present invention further provides a computing device 500.
  • a server which may include:
  • the memory 520 is used to store a program 550; the processor 510 is configured to execute the program in the memory.
  • the processor 510 executes the program 550 stored by the memory 520 to cause the server 500 to perform the graphical interface update method as described in the method embodiment above.
  • the processor 510 is configured to determine an area that the graphic interface needs to be updated as a dirty area; and when the graphic interface needs to be updated, calculate an intersection of the drawing area and the dirty area;
  • the communication interface 530 can be configured to send the intersection of the instruction data stream to the user equipment, the intersection instruction data stream being used by the user equipment to update a picture in a dirty area on the user equipment side.
  • the bus may be an Industry Standard Architecture (ISA) bus or a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA). ) Bus, etc.
  • ISA Industry Standard Architecture
  • PCI Peripheral Component
  • EISA Extended Industry Standard Architecture
  • the bus can be divided into one or more of an address bus, a data bus, and a control bus. For ease of representation, only one thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.
  • the memory may include a high speed RAM (Ramdom Access Memory) memory.
  • the memory may further include a non-volatile memory.
  • the memory can include a disk storage.
  • the processor may be a central processing unit (CPU), or the processor may be an application specific integrated circuit (Application Specific Integrated Circuit, Referring to ASIC for short, or the processor may be one or more integrated circuits configured to implement embodiments of the present invention.
  • CPU central processing unit
  • ASIC Application Specific Integrated Circuit
  • This graphical interface update method can be used in a server-client architecture.
  • the server in the server-client architecture is a cloud server
  • the server is also called the Trace side, which performs the main logical operations of the application
  • the client also called the Retrace side
  • Receive user input is also called the Trace side, which performs the main logical operations of the application.
  • the graphical interface can be broken down into a number of graphics, and the server stores the vertex data for those graphics.
  • a graphical interface can be drawn based on these vertex data.
  • the server can store these vertex data in a buffer (the buffer that stores these vertex data is simply referred to as a vertex buffer) to improve access speed.
  • vertex data is only one example of data that can be used to draw a graphical interface, and the present invention is not limited thereto.
  • the server will find that one or more intervals of the vertex buffer (or vertex data) need to be updated.
  • the server can first mark these intervals as dirty intervals (the graphical interface area corresponding to the dirty interval is dirty). Area), then update the dirty interval. After updating the dirty interval, if the instruction to draw the graphical interface is not received (that is, the graphical interface that is displayed is not yet required to be redrawn or updated), the server does not send the updated data interval to the user device (ie, the client). ).
  • one or more regions of the graphical interface may need to be redrawn (or updated), also referred to as drawing regions.
  • the section of the vertex buffer (or vertex data) corresponding to the drawing area is called a drawing section. Only the data in the vertex data that is in the drawing interval and the dirty interval (that is, the intersection of the dirty interval and the drawing interval) is required for re-rendering the graphical interface.
  • the server only sends the intersection data to the user equipment to redraw the graphical interface (ie, update the graphical interface).
  • the graphical interface can be redrawn on the server, and then the server sends the drawn graphical interface to the user device for display.
  • the program may be stored in a computer readable storage medium, and the storage medium may include: ROM, RAM, disk or CD.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Optics & Photonics (AREA)
  • Computer Security & Cryptography (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • User Interface Of Digital Computer (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

一种图形界面更新方法和装置，有助于减少服务器和用户设备交互的网络流量，并提高服务器的运行效率。所述方法包括：确定脏区域，脏区域为图形界面需要改变的区域；获取绘图指令，绘图指令指示绘图区域；计算绘图区域与所述脏区域的交集，向所述用户设备发送指示所述交集的指令数据流，用户设备根据所述指令数据流更新所述图形界面。

Description

一种图形界面更新方法和装置
本申请要求于2016年1月29日提交中国专利局，申请号为201610064040.1，发明名称为“一种图形界面更新方法和装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及云技术领域，具体涉及一种图形界面更新方法和装置。
背景技术
随着近年来终端技术和网络带宽的高速发展，云计算技术的日趋成熟催生了云交互系统，云交互系统通常包括服务器和多个用户设备，服务器可以是集群服务器，多种交互式应用都可以运行在云交互系统中。
在运行交互式应用时，服务器和用户设备可能需要交互图形界面。其中，服务器端每次更新图形界面的缓存数据时，都将更新的缓存数据发送到用户设备端。
实践发现，上述图形界面交互技术需要消耗的网络流量比较大，并且频繁的图形界面交互容易影响服务器端的运行效率。
发明内容
本发明实施例提供一种图形界面更新方法和装置，以有助于减少服务器和用户设备交互的网络流量，并提高服务器的运行效率。
本申请第一方面提供一种图形界面更新方法，所述方法包括：确定脏区域，所述脏区域为图形界面需要改变的区域；获取绘图指令，所述绘图指令指示绘图区域；计算绘图区域与所述脏区域的交集；向所述用户设备发送指示所述交集的指令数据流，其中所述用户设备根据所述指令数据流更新所述图形界面。
本申请第二方面提供一种图形界面更新装置，所述装置包括：第一确定模块，用于确定脏区域，所述脏区域为图形界面需要改变的区域；第一获取模块，用于获取绘图指令，所述绘图指令指示绘图区域；计算模块，用于计算绘图区 域与所述脏区域的交集；发送模块，用于向用户设备发送指示所述交集的指令数据流，其中所述用户设备根据所述指令数据流更新所述图形界面。
由上可见，在本申请的一些可行的实施方式中，采用服务器将图形界面需要改变的区域记录为脏区域，在需要更新所述图形界面时，计算绘图区域与所述脏区域的交集，向所述用户设备发送所述交集的指令数据流的技术方案，取得了以下效果：
在实际更新图形界面时，才发送交集的指令数据流，也就是对交集串流，减少了串流的次数；而且串流的仅仅是交集，而不是整个脏区域，减少了串流的数据；并且，有些无用的数据流与绘图指令指示的数据区间没有交集，就不进行串流，进一步减少了串流的数据；从而，实现了网络流量的优化，可以有效减少串流的网络流量；
另外，由于不必在更新缓存数据时进行串流，服务器上可以减少大量的内存拷贝操作，从而可以有效提高服务器的运行效率。
附图说明
为了更清楚地说明本发明实施例技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其它的附图。
图1是同步串流的流程示意图；
图2是异步串流的流程示意图；
图3是对一维顶点缓存更新的伪代码的示意图；
图4是图形界面更新时的数据使用示意图；
图5是本发明实施例提供的一种交互式应用系统的架构示意图；
图6是本发明实施例提供的一种图形界面更新方法的流程示意图；
图7是本发明一具体场景实施例中的图形界面更新方法的流程示意图；
图8是根据本发明实施例调用绘制函数时的具体流程图；
图9是根据本发明实施例的脏区域算法的过程示意图；
图10是本发明实施例提供的一种图形界面更新装置的结构示意图；
图11是本发明实施例提供的另一种图形界面更新装置的结构示意图；
图12是本发明实施例提供的一种服务器的结构示意图。
具体实施方式
为了使本技术领域的人员更好地理解本申请方案，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分的实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本发明保护的范围。
交互式应用系统包括服务器和用户设备，在运行云游戏等交互式应用时，服务器和用户设备可能需要交互图形界面。交互式应用分别运行在服务器和用户设备的客户端通常被称为Trace端和Retrace端。以云游戏为例，Trace端是指：运行在云游戏服务器上的游戏客户端，即云游戏的云端，执行游戏的主要逻辑运行，把与用户交互有关的图形界面，经网络传送给Retrace端。Retrace端是指：运行在用户设备上的客户端，即云游戏的用户端，用于表现Trace端的游戏图形界面，把用户的鼠标、键盘等设备输入，经网络传送给Trace端。
Trace端和Retrace端交互图形界面时，把应用程序编程接口(Application Programming Interface，Api)、输入设备、输出设备的数据，经网络编码后，发送给对端，可称为串流。目前有两种串流技术，即同步串流和异步串流。
图1是同步串流的流程示意图。同步串流技术中，Trace端对Api的串流，一般发生在调用Api的时候，函数调用一般先进入Hook(钩子注入)Api，接着调用真实的Direct3D Api，最后把Api的函数、参数、返回值，都编码成数据，串流到Retrace端。
其中，Direct3D是基于微软的通用对象模式COM(Common Object Mode)的三维(3D)图形API，是DirectX(Direct eXtension，DX)的一部分。DirectX是微软公司创建的一种多媒体编程接口，由很多API组成的，按照性质分类，可以分为四大部分，显示部分、声音部分、输入部分和网络部分。显示部分分 为DirectDraw和Direct3D，前者负责二维(2D)图像加速，后者负责3D效果的显示。
Hook技术是把原本的函数执行过程替换成自定义的函数，Hook可以接触到需要修改的Api函数入口点，改变它的地址指向新的自定义的函数。
图2是异步串流的流程示意图。以Direct3D SDK(Software Development Kit，软件开发工具包)为例，异步串流技术针对缓存数据例如针对一维顶点缓存(VertexBuffe)的更新，其过程如图2所示，包括：
首先调用加锁(Lock)函数例如IDirect3DVertexBuffer9::Lock()锁定一个数据区间以获得访问权限，对Lock的调用通常先进入Hook Api，接着调用真实的Direct3D Api；然后，串流Lock的Api数据到用户设备。
然后在锁定的数据区间对VertexBuffe进行缓存数据更新，即Update(ppbDate)，更新方法可采用内存拷贝(Memcpy)函数，也可以采用其它方法。
图3示出了上述异步串流过程的伪代码。
需要说明的是，调用Lock之后，调用解锁(UnLock)之前，游戏还有可能调用其它Api，值得指出的是，每次调用Api通常都是先进入Hook Api，接着调用真实的Api，且每次调用API之后，就立即将该Api的数据串流到用户设备。
再调用解锁(UnLock)函数例如IDirect3DVertexBuffer9::UnLock()释放锁定的数据区间。对UnLock的调用也是先进入Hook Api，接着调用真实的Direct3D Api。其中，IDirect3DVertexBuffer9::Lock()包括四个参数，分别是：OffsetToLock(加锁内存起始地址),SizeToLock(加锁内存大小),ppbData(返回内存指针地址),flags(加锁属性)。
需要说明的是，Update(ppbDate)并不是DirectX3D的Api，所以在锁定了一个数据区间后，在Trace端并不知道如何更新这个区间的数据。因此如图2所示，在IDirect3DVertexBuffer9::UnLock()调用时，就要模拟生成一个Memcpy函数，用来串流游戏更新的缓存数据，具体的，从记录中找出VertexBuffe的锁定的数据区间，并把该区间内的数据从Direct3D中拷贝到缓 存(Temp)中，Hook Api调用模拟生成的Memcpy，把Temp中的数据作为源数据，串流到用户设备。最后还要串流UnLock()的Api数据。
于是，一个UnLock()函数调用，会串流两个函数数据。因此，每次更新缓存数据时，都要串流三个函数：Lock()、Memcpy()、UnLock()。另外，在UnLock()串流数据时要按Lock()锁定的数据区间的大小来串流。
交互式应用例如云游戏，其图形界面的缓存数据一般包括：一维顶点缓存、索引缓存，二维贴图，针对3D游戏，还包括立体贴图。
实践发现，上述指令串流技术，在串流一维顶点缓存(VertexBuffer)、索引缓存(IndexxBuffer)，二维贴图，立体贴图等数据时，有如下缺陷：
一、浪费流量：
以VertexBuffer为例，从上述异步串流技术可知：
1、每一次调用UnLock()时，都会异步串流一个Memcpy函数，其数据大小为Lock()时指定的SizeToLock，那么当游戏每次对一个VertexBuffer的数据进行更新的时候，都要串流三个函数：Lock()、Memcpy()、UnLock()；
2、请参考如图4所示的图形界面更新时的数据使用示意图。假设游戏对一个VertexBuffer多次更新，例如两次，这两次更新锁定或者说使用的数据区间分别是[a，b]，[c,d]，但在对图形界面进行绘制更新时使用的VertexBuffer区间是[e，f]，如果[e，f]与[a，b]，[c,d]并没有交集，那么，串流的数据区间[a,b]、[c,d]实际并不会被使用，则在这次绘制中就浪费了网络流量。
3、游戏逻辑层在Lock()时，可能会指示锁定一个较大的数据区间，而实际上可能只更新了一小段数据，更新的数据小于锁定的数据区间，但在UnLock()串流数据时，却要按Lock()指示的数据区间大小来串流，使得一些无需串流的数据也被串流了，这就会导致网络流量的浪费。
分析原因在于：3D游戏引擎为一维顶点缓存、索引缓存，二维贴图，立体贴图等，提供了Lock()、UnLock()、LockRect()、UnLockRect()等函数，但是没有提供直接更新缓存数据的Api，游戏除了可以通过Lock()时，得到的数据指针，可以用任意的方式，更新任意大小的数据，而且更新的数据量，可以比Lock()时指定的区间小很多；这样导致在UnLock()时，Hook Api无法得知 游戏到底更新了多少数据，以及有多少数据在要绘制时使用，所以只能按Lock()的区间大小，进行数据串流。
二、效率低下：
3D游戏引擎为一维顶点缓存、索引缓存，二维贴图，立体贴图等，提供的加锁/解锁函数，例如用于一维的Lock()、UnLock()，用于二维的LockRect()、UnLockRect()等函数，在更新数据时，考虑的都是把数据从游戏逻辑层向系统内存或显卡内存拷贝，且3D游戏引擎及显卡都为这样的使用模型进行过优化，但在指令串流技术方案中使用的，从显卡内存取数据到内存的过程，速度比较慢，且影响图形处理器(Graphics Processing Unit，GPU)的运算速度；所以当大量UnLock()函数调用时，会导致大量的显卡数据到内存，就会导致Trace端运行效率低下。
由上可见，采用上述串流技术，Trace端每次更新图形界面的缓存数据时，都将更新的缓存数据串流到Retrace端。特别的，采用异步串流技术时：每次更新缓存数据，首先要调用加锁(Lock)函数锁定一个数据区间以获得访问权限，才能进行更新操作，然后还要调用解锁函数释放锁定的数据区间。其中，更新的缓存数据需要利用模拟生成的内存拷贝(Memcpy)函数来串流，也就是说，每次更新缓存数据时，都要串流以下三种函数：Lock()、Memcpy()、UnLock()。并且，串流时是将锁定的整个数据区间串流到Retrace端。可见，上述串流技术需要消耗的网络流量比较大，并且，大量的Memcpy函数导致大量的内存拷贝操作，会影响Trace端的运行效率。
为了解决上述串流技术导致的Trace端网络流量消耗大且运行效率不高的技术问题，本发明实施例提供一种图形界面更新方法和装置。下面通过具体实施例，分别进行详细的说明。
本发明实施例提供一种图形界面更新方法。
所述方法可以应用于如图5所示的交互式应用系统，所述交互式应用系统包括服务器和用户设备。所述服务器上运行有Trace端，所述用户设备上运行有Retrace端。Trace端是交互式应用例如云游戏的云端，执行游戏的主要逻辑运行，把与用户交互有关的图形界面，经网络传送给Retrace端，传送过程包括： 游戏逻辑调用Direct3D Api来生成图形界面，函数调用一般先进入Hook Api，接着调用真实的Direct3D Api；服务器提取API数据，串流到用户设备。Retrace端是运行在用户设备上的用户端，可以接收API数据，回放Direct3D Api，生成并显示图形界面。
请参考图6，所述方法可包括：
110、确定脏区域，所述脏区域为图形界面需要改变的区域。
在这个步骤，服务器将图形界面需要改变或更新的区域记录为脏区域。需要说明的是，脏区域是需要更新但是最终并不一定会被更新的图形界面区域。
本发明实施例中，当服务器确定图形界面需要更新的区域时，不需要立即将该区域的数据串流到用户设备，而是仅仅将该区域记录为脏区域。
本发明实施例中，，服务器更新图形界面的缓存数据时，首先可以调用加锁指令例如Lock()函数以锁定一个数据区间从而获得该数据区间的访问权限，然后在获得访问权限的数据区间中进行缓存数据的更新，最后再调用解锁指令例如UnLock()函数释放锁定的数据区间。
本发明实施例中，为了减少串流的次数，所述服务器在获取加锁指令时，将所述加锁指令指示的数据区间记录为脏区域，因为图形界面需要更新的区域包含于所述加锁指令指示的数据区间。服务器在执行该加锁指令后，不需要要进行串流，即，确定不向所述用户设备发送所述加锁指令对应的指令数据流。
进一步的，所述服务器将加锁指令指示的数据区间记录为脏区域，执行该加锁指令之后，还获取所述图形界面更新所需要的缓存数据以对缓存数据进行更新，并在更新后获取和执行解锁指令，释放加锁指令锁定的数据区间。本发明实施例中，由于已记录脏区域，在执行解锁指令后，不需要进行串流，即，确定不向所述用户设备发送所述缓存数据和所述解锁指令对应的指令数据流。
本发明实施例中的“脏区域”源自“脏矩形”算法，是对“脏矩形”的扩展。
“脏矩形”是指：屏幕的2D图形界面绘制的时候，当只有一小块矩形内的图象需要更新时，如果把整个屏幕都重新绘制的话，势必造成资源的浪费；于是，只是先标记它的矩形位置，且不立即更新，记为“脏”，该记为“脏”的矩形称为“脏矩形”；当更新界面的时候，把记录的所有矩形区域都更新一次，由“脏”变 成“干净”，这种方法叫做“脏矩形”算法。“脏矩形”针对的是2D图形界面。
本发明实施例中的“脏区域”，针对的不再是2D图形界面，它包括：一维数组的区间、2维平面的矩形、3维空间的区域，且同样在有数据需要更新时，先记录“脏区域”，当更新时，才把“脏区域”变成干净的。
由于采用脏区域技术，服务器不对加锁指令例如Lock()函数进行串流，不再模拟生成Memcpy函数，也不对UnLock()函数进行串流。
120、在需要更新所述图形界面时，计算绘图区域与所述脏区域的交集，向所述用户设备发送所述交集的指令数据流，所述交集的指令数据流用于所述用户设备更新所述用户设备侧的脏区域内的画面。
在该步骤中，可以首先获取绘图指令，然后计算绘图指令指示的绘图区域与所述脏区域的交集，最后向用户设备发送指示所述交集的指令数据流，用户设备则根据指令数据流更新图形界面。
本发明实施例中，串流发生在更新图形界面时。在需要更新所述图形界面时，交互式应用例如云游戏会发出绘图指令，服务器获取绘图指令后，可以根据绘图指令的参数计算出绘图指令指示的数据区间，该数据区间为绘图区域。本步骤中，服务器在可以计算绘图区域与所述脏区域的交集，或者说，计算绘图指令指示的数据区间与所述脏区域的交集，仅对该交集进行串流，仅向所述用户设备发送所述交集的指令数据流，并且，服务器还需要对该绘图指令进行串流，向所述用户设备发送所述绘图指令的指令数据流。所述交集的指令数据流和所述绘图指令的指令数据流，用于所述用户设备更新所述用户设备侧的脏区域内的画面。
本发明实施例中，可以在串流之前，首先判断交集是否为空，如果交集为空，则不发送任何指令数据流，如果交集不为空，则针对所述交集生成模拟加锁指令例如Lock()函数，内存拷贝指令例如Memcpy函数和模拟解锁指令例如UnLock()函数，所述模拟加锁指令指示锁定所述交集，所述内存拷贝指令根据脏区域内需要更新的缓存数据生成；然后，服务器针对交集进行串流，即，向所述用户设备发送所述模拟加锁指令，内存拷贝指令和模拟解锁指令对应的指令数据流。
在更新所述图形界面之前，可能已经更新过多次缓存数据，因此，脏区域可能会有多个，本步骤中得到的交集可能也会有多个，则发送交集对应的指令数据流之前，可以针对多个交集分别模拟生成一个Memcpy函数，串流时发送的是一个Lock()函数，和多个Memcpy函数，以及一个UnLock()函数对应的指令数据流。
进一步的，每次更新图形界面时，还需要更新脏区域，包括：将已经串流的交集，从记录的脏区域中去除，剩下的脏区域作为当前的脏区域，从而完成对脏区域的更新。
其中，交集对应的指令数据流发送到用户设备，被用户设备用来更新用户设备侧图形界面脏区域内的画面。在服务器侧，服务器也需要执行所述绘图指令，使用所述交集的数据更新服务器侧图形界面脏区域内的画面。至此，在服务器和用户设备两侧都完成对图形界面的更新。由上可见，在本申请的一些可行的实施方式中，取得了以下效果：
在实际更新图形界面时，才发送交集的指令数据流，也就是对交集串流，而不必在更新缓存数据时进行串流，减少了串流的次数；而且串流的仅仅是交集，而不是整个脏区域，减少了串流的数据；并且，有些无用的数据流与绘图指令指示的数据区间没有交集，就不进行串流，进一步减少了串流的数据；从而，实现了网络流量的优化，可以有效减少串流的网络流量；
另外，由于不必在更新缓存数据时进行串流，服务器上可以减少大量的内存拷贝操作，从而可以有效提高服务器的运行效率。
为便于更好的理解本发明实施例提供的技术方案，下面通过一个具体场景下的实施方式为例进行介绍。
请参考图7，以交互式应用系统中运行的游戏更新一维顶点缓存(VertexBuffer)为例，本发明实施例的另一种图形界面更新方法，可包括：
S1、在需要更新图形界面的缓存数据时，游戏调用Lock()函数，即IDirect3DVertexBuffer9::Lock；由于采用Hook技术，Lock()函数调用先进入Hook Api中；在Hook Api中，先调用真实的Lock()函数；再把Lock()函数指示的数据 区间与VertexBuffer对象关联起来，保存并记录为脏区域。
S2、游戏更新VertexBuffer数据，此时，服务器无操作。其中，更新方法可以是采用Memcpy函数，也可以是采用其它方法。
S3、游戏调用UnLock()函数，即IDirect3DVertexBuffer9::UnLock；由于采用Hook技术，Lock()函数调用先进入Hook Api中；在Hook Api中，直接转调真实的Unlock()函数。
S4、在需要更新图形界面时，游戏调用DrawPrimitive或者DrawIndexedPrimitive绘制函数。调用绘制函数时先进入Hook Api，分析当前Api需要使用的VertexBuffer的数据区间；如果该数据区间在记录中是脏的，则模拟一次Lock()、若干次Memcpy函数、一次UnLock()函数，并把它们串流到用户设备。最后，在Hook Api中，调用真实的DrawPrimitive或者DrawIndexedPrimitive绘制函数。关于本步骤的进一步的说明，请参考图8.
请参考图8，是调用DrawPrimitive或者DrawIndexedPrimitive绘制函数时的具体流程图，包括：
a1、先通过绘制函数的参数，计算绘制函数需要使用的VertexBuffer的数据区间；
a2、计算这个数据区间与之前记录的脏区域的交集；如果交集不为空，则模拟生成以下函数：一次Lock()、若干次Memcpy函数、一次UnLock()函数，并把它们串流，更新脏区域；如果为空，则不串流数据；
a3、串流DrawPrimitive或者DrawIndexedPrimitive；
a4、调用真实的DrawPrimitive或者DrawIndexedPrimitive。
可见，在绘制过程中，分析当前Api即绘制函数的需要使用的VertexBuffer的数据区间，在该数据区间为脏或者说该数据区间与脏区域有交集时，模拟生成三种Api的调用，即Lock()、Memcpy、UnLock()三种函数的调用，其中，Lock()、和UnLock()具体可以分别是：IDirect3DVertexBuffer9::Lock，IDirect3DVertexBuffer9::UnLock；最后再调用真实的DrawPrimitive或者DrawIndexedPrimitive。
下面，举例说明所述脏区域的算法：
如图9所示，一帧内，以对一个VertexBuffer多次调用Lock()、UnLock()函数，多次调用DrawPrimitive或者DrawIndexedPrimitive函数为例，说明本发明实施例所用的脏区域算法：
b1、游戏第一次调用Lock()、UnLock()，更新区间[a,b]后，更新VertexBuffer的“脏区域”{[a,b]}；
b2、游戏第二次对相同的VertexBuffer调用Lock()、UnLock()，更新区间[c,d]后，更新VertexBuffer的“脏区域”{[a,b],[c,d]}；
b3、游戏第一次调用DrawPrimitive或者DrawIndexedPrimitive绘制函数时：
假设绘制函数使用区间[e,f]，“脏区域”{[a,b],[c,d]}，且e<a,c<f<d，如图所示，有两个区间为“脏”的：[a,b]，[c,f]，于是执行以下步骤：
模拟一次Lock()函数、两次Memcpy函数，分别串流[a,b]，[c,f]区间的数据，一次UnLock()函数；
更新“脏区域”{[f,d]}；
串流DrawPrimitive或者DrawIndexedPrimitive；
调用真实的DrawPrimitive或者DrawIndexedPrimitive；
b4、游戏第二次调用DrawPrimitive或者DrawIndexedPrimitive绘制函数时：
假设绘制函数使用区间[e,f]，“脏区域”{[f,d]}，它们没有交集，即当前绘制函数使用的数据都是“干净”的，不需要模拟Lock()、Memcpy、UnLock()函数，于是：
串流DrawPrimitive或者DrawIndexedPrimitive；
调用真实的DrawPrimitive或者DrawIndexedPrimitive。
由上可见，本申请一些可行的实施方式中，通过采用脏区域算法，可以取得以下技术效果：
一、减少串流的网络流量：本发明实施例技术方案应用在一维顶点缓存、索引缓存，二维贴图，立体贴图等缓存数据的Lock()、UnLock()、LockRect()、UnLockRect()等函数时，可以减少串流的Api数量；并且，这些函数，只在DrawPrimitive或者DrawIndexedPrimitive绘制函数调用时，才会被串流，且有脏区域算法管理，不会多次串流干净的区域，减少了串流的网络流量；尤其对 Lock()区间大，更新缓存数据区间小的情况，可进一步减少串流的网络流量；从而，实现了对串流的网络流量优化。
二、提高服务器或者说Trace端的运行效率：由于减少了串流的Api数量，所以减少了从显卡到内存的内存拷贝操作，间接提高了Trace端的运行效率。
需要说明的，上文中时以一维顶点缓存(VertexBuffer)为例进行的说明，但在3D游戏中，还可以应用在索引缓存(IndexBuffer)、二维贴图、立体贴图的相应Lock、UnLock函数上，方法相同，不再赘述。
另外，在其它基于指令串流的远程方案中，只要Api满足以下条件：串流的数据，不会立即被Retrace端应用，如图形被绘制、音乐被播放等；当另一个Api被调用时，才使用它之前曾串流过的Api数据；就可以应用本申请技术方案。
为了更好的实施本发明实施例的上述方案，下面还提供用于配合实施上述方案的相关装置。
请参考图10，本发明实施例提供一种图形界面更新装置200，所述装置200应用于交互式应用系统的服务器，所述交互式应用系统包括用户设备和所述服务器，所述装置200包括：
第一确定模块210，用于确定脏区域，所述脏区域为图形界面需要改变的区域；计算模块220，用于计算绘图区域与所述脏区域的交集；
发送模块230，用于向所述用户设备发送指示所述交集的指令数据流，其中所述用户设备根据所述指令数据流更新所述图形界面。
请参考图11，在一些实施例中，
所述装置200还包括：第二获取模块240；用于获取加锁指令；
所述第一确定模块210，根据加锁指令指示的数据区间确定脏区域。
在一些实施例中，
所述第二获取模块240，还用于获取所述图形界面更新所需要的缓存数据，以及获取解锁指令；
所述装置200还包括：第二确定模块250，用于确定不向所述用户设备发送所述加锁指令对应的指令数据流；确定不向所述用户设备发送所述缓存数据和 所述解锁指令对应的指令数据流。
请参考图11，在一些实施例中，所述装置200还包括：
第一获取模块260，用于获取绘图指令，所述绘图区域是所述绘图指令指示的数据区间；
所述发送模块230，还用于向所述用户设备发送所述绘图指令的指令数据流。
请参考图11，在一些实施例中，所述装置200还包括：
判断模块270，用于判断所述交集是否为空；
生成模块280，用于若所述交集不为空，针对所述交集生成模拟加锁指令，内存拷贝指令和模拟解锁指令；
所述发送模块230，还可以用于向所述用户设备发送所述模拟加锁指令，内存拷贝指令和模拟解锁指令对应的指令数据流。
在一些实施例中，所述第一确定模块，还用于将所述交集从脏区域中去除，完成对脏区域的更新。
在一些实施例中，所述装置200还包括：
绘图模块290，用于执行所述绘图指令，使用所述交集的数据更新所述脏区域内的画面。
可以理解，本发明实施例装置的各个功能模块的功能可根据上述方法实施例中的方法具体实现，其具体实现过程可参照上述方法实施例中的相关描述，此处不再赘述。
由上可见，在本申请的一些可行的实施方式中，提供了一种图形界面更新装置，取得了以下效果：
在实际更新图形界面时，才发送交集的指令数据流，也就是对交集串流，而不必在更新缓存数据时进行串流，减少了串流的次数；而且串流的仅仅是交集，而不是整个脏区域，减少了串流的数据；并且，有些无用的数据流与绘图指令指示的数据区间没有交集，就不进行串流，进一步减少了串流的数据；从而，实现了网络流量的优化，可以有效减少串流的网络流量；
另外，由于不必在更新缓存数据时进行串流，服务器上可以减少大量的内 存拷贝操作，从而可以有效提高服务器的运行效率。
本发明实施例还提供一种存储一个或多个程序的计算机可读存储介质，所述一个或多个程序当被包括一个或多个处理器的服务器执行时使所述服务器执行如上文方法实施例所述的图形界面更新方法。
请参考图12，本发明实施例还提供一种计算装置500，所述装置的一个例子是服务器，可包括：
处理器510，存储器520，通信接口530，总线540；所述处理器510，存储器520，通信接口530通过所述总线540相互的通信；所述通信接口530，用于接收和发送数据；所述存储器520用于存储程序550；所述处理器510用于执行所述存储器中的所述程序。当所述服务器500运行时，所述处理器510执行所述存储器520存储的所述程序550，以使所述服务器500执行如上文方法实施例所述的图形界面更新方法。
其中，处理器510可用于将图形界面需要更新的区域确定为脏区域；在需要更新所述图形界面时，计算绘图区域与所述脏区域的交集；
通信接口530可用于向所述用户设备发送所述交集的指令数据流，所述交集的指令数据流用于所述用户设备更新所述用户设备侧的脏区域内的画面。
其中，所述总线可以是工业标准体系结构(Industry Standard Architecture，简称为ISA)总线或外部设备互连(Peripheral Component，简称为PCI)总线或扩展工业标准体系结构(Extended Industry Standard Architecture，简称为EISA)总线等。所述总线可以分为地址总线、数据总线、控制总线中的一种或多种。为便于表示，图中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
所述存储器可以包含高速RAM(Ramdom Access Memory)存储器。可选地，所述存储器还可以还包括非易失性存储器(non-volatile memory)。例如所述存储器可以包括磁盘存储器。
所述处理器可以是一个中央处理器(Central Processing Unit，简称为CPU)，或者所述处理器可以是特定集成电路(Application Specific Integrated Circuit， 简称为ASIC)，或者所述处理器可以是被配置成实施本发明实施例的一个或多个集成电路。
上文描述了本申请的一个或多个实施例的图形界面更新方法和装置。该图形界面更新方法可用于服务器-客户端架构中。在服务器-客户端架构中的服务器是云服务器的情况下，该服务器也称作Trace端，执行应用程序的主要逻辑运算；客户端，也称作Retrace端，用于呈现应用程序的图形界面，接收用户输入。
本领域的技术人员可以理解，图形界面能够分解为很多个图形，服务器会存储这些图形的顶点数据。根据这些顶点数据可以绘制图形界面。服务器可以在缓冲区中存储这些顶点数据(存储这些顶点数据的缓冲区简称为顶点缓冲区)，以提高访问速度。当然顶点数据只是可以用于绘制图形界面的数据的一个例子，本发明不限于此。
随着应用程序的进行，服务器会发现需要对顶点缓冲区(或顶点数据)的一个或多个区间进行更新，服务器可以先标记这些区间为脏区间(与脏区间对应的图形界面区域即为脏区域)，然后对脏区间进行更新。对脏区间进行更新后，如果没有收到绘制图形界面的指令(即暂时还不需要对显示的图形界面重新进行绘制或更新)，服务器不发送这些更新过的数据区间给用户设备(即客户端)。
而随着应用程序的继续进行，可能需要对图形界面的一个或多个区域重新进行绘制(或更新)，这些区域也称作绘图区域。与绘图区域对应的顶点缓冲区(或顶点数据)的区间称作绘图区间。顶点数据中只有那些既在绘图区间又在脏区间的数据(即脏区间与绘图区间的交集)才是重新绘制图形界面所需要的。为了节省网络流量，服务器仅将交集数据发送给用户设备，以重新绘制图形界面(即更新图形界面)。当然，也可以在服务器重新绘制图形界面，然后服务器将绘制好的图形界面发送给用户设备进行显示。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详 细描述的部分，可以参见其它实施例的相关描述。
需要说明的是，对于前述的各方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本发明并不受所描述动作顺序的限制，因为依据本发明，某些步骤可以采用其它顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作和模块并不一定是本发明所必须的。
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，该程序可以存储于一计算机可读存储介质中，存储介质可以包括：ROM、RAM、磁盘或光盘等。
以上对本发明实施例所提供的图形界面更新方法和装置进行了详细介绍，本文中应用了具体个例对本发明的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本发明的方法及其核心思想；同时，对于本领域的一般技术人员，依据本发明的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本发明的限制。

Claims (19)

  1. 一种图形界面更新方法，包括：
    确定脏区域，所述脏区域为图形界面需要改变的区域；
    获取绘图指令，所述绘图指令指示绘图区域；
    计算所述绘图区域与所述脏区域的交集；
    向用户设备发送指示所述交集的指令数据流，其中所述用户设备根据所述指令数据流更新所述图形界面。
  2. 根据权利要求1所述的方法，其中，确定脏区域包括：
    获取加锁指令，根据所述加锁指令指示的数据区间确定脏区域。
  3. 根据权利要求2所述的方法，其中，
    根据所述加锁指令指示的数据区间确定脏区域之前，所述方法还包括：
    确定不向所述用户设备发送所述加锁指令对应的指令数据流；
    根据所述加锁指令指示的数据区间确定脏区域之后，所述方法还包括：
    获取所述图形界面更新所需要的缓存数据，以及获取解锁指令；
    确定不向所述用户设备发送所述缓存数据和所述解锁指令对应的指令数据流。
  4. 根据权利要求1所述的方法，还包括：向所述用户设备发送所述绘图指令的指令数据流。
  5. 根据权利要求1所述的方法，其中，向用户设备发送指示所述交集的指令数据流之前，所述方法还包括：
    判断所述交集是否为空，若所述交集不为空，针对所述交集生成模拟加锁指令，内存拷贝指令和模拟解锁指令。
  6. 根据权利要求1所述的方法，其中，所述方法还包括：
    将所述交集从所述脏区域中去除，以更新所述脏区域。
  7. 根据权利要求1所述的方法，其中，所述方法应用于服务器，所述服务器与所述用户设备连接，
    其中，所述用户设备根据所述指令数据流更新所述图形界面包括：
    执行所述绘图指令，使用所述交集更新所述脏区域。
  8. 根据权利要求1所述的方法，其中所述图形界面由多个图形组成，所述方法还包括：
    生成顶点缓冲区，所述顶点缓冲区包括所述图形界面的各图形的顶点数据。
  9. 根据权利要求8所述的方法，其中确定脏区域包括：
    确定所述顶点缓冲区中的一个或多个数据区间为脏区间，其中与脏区间对应的图形界面区域为脏区域；
    对所述脏区间进行更新。
  10. 根据权利要求9所述的方法，其中所述绘图指令指示顶点缓冲区的另外的一个或多个数据区间为绘图区间，其中与绘图区间对应的图形界面区域为绘图区域。
  11. 一种图形界面更新装置，包括：
    第一确定模块，用于确定脏区域，所述脏区域为图形界面需要改变的区域；
    第一获取模块，用于获取绘图指令，所述绘图指令指示绘图区域；
    计算模块，用于计算所述绘图区域与所述脏区域的交集；
    发送模块，用于向用户设备发送指示所述交集的指令数据流，其中所述用户设备根据所述指令数据流更新所述图形界面。
  12. 根据权利要求11所述的装置，还包括：第二获取模块，用于获取加锁指令；
    其中，所述第一确定模块，根据所述加锁指令指示的数据区间确定脏区域。
  13. 根据权利要求12所述的装置，其中，
    所述第二获取模块，还用于获取所述图形界面更新所需要的缓存数据，以及获取解锁指令；
    所述装置还包括：第二确定模块，用于确定不向所述用户设备发送所述加锁指令对应的指令数据流以及确定不向所述用户设备发送所述缓存数据和所述解锁指令对应的指令数据流。
  14. 根据权利要求11所述的装置，
    其中，所述发送模块，还用于向所述用户设备发送所述绘图指令的指令数据流。
  15. 根据权利要求11所述的装置，还包括：
    判断模块，用于判断所述交集是否为空；
    生成模块，用于若所述交集不为空，针对所述交集生成模拟加锁指令，内存拷贝指令和模拟解锁指令。
  16. 根据权利要求11至15中任一所述的装置，其中，
    所述第一确定模块，还用于将所述交集从所述脏区域中去除，以更新所述脏区域。
  17. 根据权利要求11至15中任一所述的装置，还包括：
    绘图模块，用于执行所述绘图指令，使用所述交集更新所述脏区域。
  18. 一种图形界面更新设备，包括：
    处理器；和
    存储器，其中所述存储器存储有程序指令，所述指令当由所述处理器执行时，配置所述装置执行根据权利要求1-10任一项所述的方法。
  19. 一种计算机可读存储介质，包括程序指令，所述指令当由处理器执行时执行根据权利要求1-10任一项所述的方法。
PCT/CN2017/072408 2016-01-29 2017-01-24 一种图形界面更新方法和装置 WO2017129105A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US15/762,266 US10645005B2 (en) 2016-01-29 2017-01-24 GUI updating method and device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610064040.1A CN105511890B (zh) 2016-01-29 2016-01-29 一种图形界面更新方法和装置
CN201610064040.1 2016-01-29

Publications (1)

Publication Number Publication Date
WO2017129105A1 true WO2017129105A1 (zh) 2017-08-03

Family

ID=55719900

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/072408 WO2017129105A1 (zh) 2016-01-29 2017-01-24 一种图形界面更新方法和装置

Country Status (3)

Country Link
US (1) US10645005B2 (zh)
CN (1) CN105511890B (zh)
WO (1) WO2017129105A1 (zh)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105511890B (zh) 2016-01-29 2018-02-23 腾讯科技（深圳）有限公司 一种图形界面更新方法和装置
CN110286992A (zh) * 2019-07-02 2019-09-27 中国工商银行股份有限公司 界面图形重绘的方法及装置
CN111309357B (zh) * 2020-02-24 2022-07-12 腾讯科技(成都)有限公司 云游戏软件的更新方法、装置、存储介质和云游戏系统
CN111714894B (zh) * 2020-06-08 2023-07-21 咪咕互动娱乐有限公司 游戏数据的获取方法、装置、电子设备及存储介质
CN114064017B (zh) * 2020-08-04 2024-08-30 腾讯科技（深圳）有限公司 一种绘制方法及相关设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009020723A (ja) * 2007-07-12 2009-01-29 Casio Comput Co Ltd サーバ装置およびプログラム
US20090100483A1 (en) * 2007-10-13 2009-04-16 Microsoft Corporation Common key frame caching for a remote user interface
CN101847269A (zh) * 2009-03-27 2010-09-29 上海科泰世纪科技有限公司 多图层动画渲染系统及方法
CN105511890A (zh) * 2016-01-29 2016-04-20 腾讯科技（深圳）有限公司 一种图形界面更新方法和装置

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5666551A (en) * 1994-06-30 1997-09-09 Digital Equipment Corporation Distributed data bus sequencing for a system bus with separate address and data bus protocols
US5758149A (en) * 1995-03-17 1998-05-26 Unisys Corporation System for optimally processing a transaction and a query to the same database concurrently
US7055151B1 (en) * 1998-04-03 2006-05-30 Applied Micro Circuits Corporation Systems and methods for multi-tasking, resource sharing and execution of computer instructions
WO2004002044A2 (en) * 2002-02-01 2003-12-31 John Fairweather A system for exchanging binary data
GB0502891D0 (en) * 2005-02-12 2005-03-16 Next Device Ltd User interfaces
US7535475B2 (en) * 2005-11-01 2009-05-19 Adobe Systems Incorporated Virtual view tree
US7847798B1 (en) * 2006-10-24 2010-12-07 Adobe Systems Incorporated Clipping and culling artwork using a graphics processing unit
US20090240880A1 (en) * 2008-03-21 2009-09-24 Hitachi, Ltd. High availability and low capacity thin provisioning
US7962458B2 (en) * 2008-06-12 2011-06-14 Gravic, Inc. Method for replicating explicit locks in a data replication engine
US8847970B2 (en) * 2012-04-18 2014-09-30 2236008 Ontario Inc. Updating graphical content based on dirty display buffers
US8994750B2 (en) * 2012-06-11 2015-03-31 2236008 Ontario Inc. Cell-based composited windowing system
US10410312B2 (en) * 2014-01-21 2019-09-10 Elektraglide Ltd Method and system for interactive graphics streaming

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009020723A (ja) * 2007-07-12 2009-01-29 Casio Comput Co Ltd サーバ装置およびプログラム
US20090100483A1 (en) * 2007-10-13 2009-04-16 Microsoft Corporation Common key frame caching for a remote user interface
CN101847269A (zh) * 2009-03-27 2010-09-29 上海科泰世纪科技有限公司 多图层动画渲染系统及方法
CN105511890A (zh) * 2016-01-29 2016-04-20 腾讯科技（深圳）有限公司 一种图形界面更新方法和装置

Also Published As

Publication number Publication date
US10645005B2 (en) 2020-05-05
CN105511890A (zh) 2016-04-20
CN105511890B (zh) 2018-02-23
US20180278529A1 (en) 2018-09-27

Similar Documents

Publication Publication Date Title
WO2017129105A1 (zh) 一种图形界面更新方法和装置
US11344806B2 (en) Method for rendering game, and method, apparatus and device for generating game resource file
CN112614202B (zh) Gui渲染显示的方法、终端、服务器、电子设备及存储介质
KR101692193B1 (ko) 크라우드소싱된 비디오 렌더링 시스템
US10535171B2 (en) Evaluation manager for 3D animation scenes
KR101267120B1 (ko) 성능 분석 동안 관련된 그래픽스 데이터에 대한 그래픽스 명령들의 매핑
CN103970518B (zh) 一种逻辑窗口的3d渲染方法和装置
WO2021146930A1 (zh) 显示处理方法、显示处理装置、电子设备及存储介质
WO2005078571A2 (en) A method and graphics subsystem for a computing device
US20100073379A1 (en) Method and system for rendering real-time sprites
CN103679813B (zh) 三维数字地球的构建方法以及三维数字地球系统
JP2013120438A (ja) グラフィックスコマンド生成装置およびグラフィックスコマンド生成方法
CN105144243A (zh) 数据可视化
CN112181633B (zh) 用于图形处理的资产感知计算体系结构
CN110930492B (zh) 模型渲染的方法、装置、计算机可读介质及电子设备
CN113495933A (zh) 一种矢量瓦片显示方法及系统
CN108345492A (zh) 一种虚拟化环境中的数据通信的方法、装置及处理器
CN113419806B (zh) 图像处理方法、装置、计算机设备和存储介质
CN114241094A (zh) 动画绘制方法、装置、存储介质及电子设备
CN111243069B (zh) 一种Unity3D引擎的场景切换方法和系统
CN116843802A (zh) 虚拟形象的处理方法及相关产品
CN115705668A (zh) 一种视图绘制的方法、装置及存储介质
CN117112950B (zh) 电子地图中对象的渲染方法、装置、终端及存储介质
CN112930556A (zh) 集成用于可视化大数据集的第三方几何结构的系统和方法
WO2015035129A2 (en) Rendering engine for multi-dimensional tracking, reconstruction and mapping

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17743720

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 15762266

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17743720

Country of ref document: EP

Kind code of ref document: A1