WO2017129118A1 - 图形指令数据的处理方法和装置及系统 - Google Patents

图形指令数据的处理方法和装置及系统 Download PDF

Info

Publication number
WO2017129118A1
WO2017129118A1 PCT/CN2017/072538 CN2017072538W WO2017129118A1 WO 2017129118 A1 WO2017129118 A1 WO 2017129118A1 CN 2017072538 W CN2017072538 W CN 2017072538W WO 2017129118 A1 WO2017129118 A1 WO 2017129118A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
data
key frame
instruction data
user equipment
Prior art date
Application number
PCT/CN2017/072538
Other languages
English (en)
French (fr)
Inventor
杜幸阁
聂志明
李从兵
艾风
顾煜
王杨军
曹文升
叶劲峰
王伟光
韩祥
程东哲
陈国乐
周贺楠
庄志伟
沈超
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to EP17743733.2A priority Critical patent/EP3410302B1/en
Publication of WO2017129118A1 publication Critical patent/WO2017129118A1/zh
Priority to US15/926,547 priority patent/US10645391B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication
    • G06F9/543—User-generated data transfer, e.g. clipboards, dynamic data exchange [DDE], object linking and embedding [OLE]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/30—Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
    • A63F13/35—Details of game servers
    • A63F13/355—Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H04N19/126—Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/48—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/54—Indexing scheme relating to G06F9/54
    • G06F2209/541—Client-server
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L5/00—Arrangements affording multiple use of the transmission path
    • H04L5/003—Arrangements for allocating sub-channels of the transmission path
    • H04L5/0058—Allocation criteria
    • H04L5/0064—Rate requirement of the data, e.g. scalable bandwidth, data priority

Definitions

  • the present invention relates to the field of cloud technologies, and in particular, to a method, device, and system for processing graphics command data.
  • cloud interaction systems typically include a server and multiple user devices.
  • the server can be a cluster server, and a variety of interactive applications can run in the cloud interaction system.
  • Servers and user devices may require an interactive graphical interface when running interactive applications such as cloud games.
  • the graphics command data of the graphical interface is transmitted in frames.
  • the amount of graphics command data transmission is relatively large, and the network transmission has high bandwidth requirements, which may easily lead to the phenomenon that the picture on the user equipment side is not smooth.
  • Embodiments of the present invention provide a method, an apparatus, and a system for processing graphics instruction data, to help reduce the amount of graphics instruction data transmission and reduce bandwidth requirements.
  • a first aspect of the present invention provides a method for processing graphics instruction data, where the method is applied to a cloud interaction system, where the cloud interaction system includes a server and a user equipment, and the method includes: The server acquires graphic instruction data of the frame; determines whether the frame meets a key frame setting condition; if not, compares the graphic instruction data of the frame with the graphic instruction data of the current key frame to obtain difference data; The difference data is sent to the user equipment, so that the user equipment acquires graphics instruction data of the frame according to the current key frame and the difference data.
  • a second aspect of the present invention provides a method for processing graphics instruction data, where the method is applied to a cloud interaction system, where the cloud interaction system includes a server and a user equipment, and the method includes: the user equipment receives the Data of the frame; identifying whether the frame is a key frame; if not, acquiring a current key frame, determining that the data of the frame is difference data with the current key frame; according to the data of the frame and the current key A frame acquires graphics instruction data of the frame.
  • a third aspect of the present invention provides a method for processing graphics instruction data, where the method is applied to a cloud interaction system, where the cloud interaction system includes a server and a user equipment, and the method includes: the server acquiring graphics instruction data of a frame, Determining whether the frame meets a key frame setting condition, and if not, comparing the graphic instruction data of the frame with the graphic instruction data of the current key frame to obtain difference data, and transmitting the difference data to the user equipment Receiving, by the user equipment, data of the frame sent by the server, identifying whether the frame is a key frame; if not, acquiring the current key frame, determining that the data of the frame is related to the current key frame The difference data, according to the data of the frame and the current key frame, acquire graphics instruction data of the frame, and generate a graphic interface of the frame according to the graphics instruction data of the frame.
  • a fourth aspect of the present invention provides a method for processing graphics instruction data, where the method is applied to a cloud interaction system, where the cloud interaction system includes a server and a user equipment, and the method includes: the server acquiring graphics instruction data of a frame, and determining the location Whether the frame meets the key frame setting condition, and if not, compares the graphic instruction data of the frame with the graphic instruction data of the current key frame to obtain difference data, and sends the difference data to the user equipment; Receiving, by the user equipment, data of the frame sent by the server, identifying whether the frame is a key frame; if not, acquiring a current key frame, determining that the data of the frame is difference data with the current key frame, According to The data of the frame and the current key frame acquire graphics command data of the frame, and generate a graphical interface of the frame according to the graphics instruction data of the frame.
  • a fifth aspect of the present invention provides a processing apparatus for graphics command data, where the apparatus is applied to a user equipment of a cloud interaction system, the cloud interaction system includes a server and the user equipment, and the apparatus includes: a receiving module, configured to: Receiving data of a frame sent by the server; an identifying module, configured to identify whether the frame is a key frame; and a determining module, configured to acquire a current key frame if the recognition result of the identifying module is negative, and determine the frame The data is difference data from the current key frame; a restoration module is configured to acquire graphics instruction data of the frame according to the data of the frame and the current key frame.
  • a sixth aspect of the present invention provides a cloud interaction system, where the cloud interaction system includes a server and a user equipment, where the server is configured to acquire graphics command data of a frame, and determine whether the frame meets a key frame setting condition. If not, the graphics command data of the frame is compared with the graphics command data of the current key frame to obtain the difference data, and the difference data is sent to the user equipment; the user equipment is configured to receive the server.
  • Transmitting the data of the frame identifying whether the frame is a key frame; if not, acquiring a current key frame, determining that the data of the frame is difference data with the current key frame, according to the data of the frame
  • the current key frame acquires graphics instruction data of the frame, and generates a graphical interface of the frame according to the graphics instruction data of the frame.
  • the difference data between the graphics instruction data of the frame and the graphics instruction data of the current key frame is obtained, and the difference data is used as the data to be transmitted of the frame. Since the size of the difference data is much smaller than the original graphics command data, the data transmission amount can be effectively reduced, the bandwidth requirement can be reduced, and the efficient and smooth communication can be realized, and the smoothness of the image surface can be ensured.
  • Figure 1 is a flow chart of interaction between the Trace side and the Retrace side
  • Figure 2 is a schematic diagram of the distribution of instruction data amount per frame of a game
  • FIG. 3 is a schematic structural diagram of a cloud interaction system
  • FIG. 4 is a schematic flowchart of a method for processing graphics command data according to an embodiment of the present invention.
  • Figure 5 is a schematic flowchart of establishing a key frame policy on the Trace side
  • FIG. 6 is a schematic diagram of a processing flow of graphics command data on the Trace end and the Retrace side;
  • FIG. 7 is a schematic diagram of processing timing of graphics instructions according to an embodiment of the present invention.
  • Figure 8 is a schematic diagram showing a comparison of the amount of instruction data per frame of a game
  • FIG. 9 is a schematic structural diagram of a device for processing graphics command data according to an embodiment of the present invention.
  • FIG. 10 is a schematic structural diagram of a device for processing graphics command data according to another embodiment of the present invention.
  • FIG. 11 is a schematic structural diagram of a server according to an embodiment of the present invention.
  • FIG. 12 is a schematic structural diagram of a user equipment according to an embodiment of the present invention.
  • FIG. 3 shows a schematic diagram of the architecture of a cloud interaction system.
  • Cloud interaction systems include servers and user devices for running interactive applications, such as cloud games.
  • the clients of the server and user equipment that the interactive application runs separately are often referred to as the Trace side and the Retrace side.
  • the cloud game means that the real client of the game runs on the cloud server, and the game screen is displayed on the player machine, and the game is realized by remotely redirecting the player's local keyboard and mouse control to the cloud server. control.
  • the Trace end refers to the game client running on the cloud game server, that is, the cloud game cloud, executing the main logic operation of the game, and transmitting the relevant graphical interface with the user to the Retrace via the network. end.
  • the Retrace side refers to the client running on the user device, that is, the client of the cloud game, which is used to present the game graphical interface of the Trace end, and input the user's mouse, keyboard and other devices, and transmit it to the Trace terminal via the network.
  • Servers and user devices may require an interactive graphical interface when running interactive applications such as cloud games.
  • FIG. 1 is a flow chart of interaction between the Trace end and the Retrace end.
  • the cloud game runs on the Trace side.
  • the Trace side can capture the operation of drawing the graphics picture each time, and record its rendering instructions, generate graphics instruction data, and send the graphics instruction data to the Retrace terminal through the network.
  • the Retrace side receives the instruction data and parses the playback rendering instruction, and calls the execution related graphics drawing API (Application Programming Interface) to draw the image and play the cloud game screen.
  • the graphics command data is transmitted in frames.
  • the frequency of calling the graphics rendering API per frame is extremely high.
  • the data volume distribution map of each frame of a game shows that the average data volume of one frame exceeds 500K Byte, and the peak data volume reaches 3400K Byte. Smooth game rendering speed of 60 frames per second, then its network transmission traffic reaches an average of 30MB / s, peak value of more than 200MB / s.
  • the Trace terminal on the server needs to send the graphics command data of each frame of the graphical interface to the Retrace terminal on the user equipment, so that the Retrace side can receive the graphics according to the received Command data generation and playback of graphical interfaces for cloud games.
  • the amount of graphics command data transmission is relatively large, and the network transmission has high bandwidth requirements, which may easily lead to the phenomenon that the picture is not smooth on the Retrace side.
  • the embodiment of the invention provides a method, a device and a system for processing graphics instruction data.
  • an embodiment of the present invention provides a method for processing graphics command data, which is applied to a cloud interaction system as shown in FIG. 3, where the cloud interaction system includes a server and a user equipment.
  • the cloud interaction system is used to run interactive applications such as cloud games.
  • This article uses cloud games as an example.
  • a cloud interactive system can be understood as a cloud gaming system.
  • the method can include the following steps.
  • the server acquires graphics instruction data of a frame.
  • the server When the cloud game is running, the server is usually responsible for image drawing. After the drawing is completed, the drawn image frame is sent to the user equipment. After receiving the drawn image frame, the user equipment decodes the image frame and then displays the image frame. The Trace end can capture the drawing operation of each frame of the graphic interface, and record the graphic drawing rendering instruction, generate the graphic instruction data, and put the graphic instruction data into the processing queue. When frame data needs to be transmitted, graphics instruction data of the frame can be acquired from the processing queue.
  • a key frame policy is pre-established, and according to the key frame policy, it is determined whether the frame meets the key frame setting condition to determine whether to update the current key frame.
  • the frame is the first frame, it can be directly recorded as the current key frame. If the frame is not the first frame, the degree of difference between the frame and the recorded current key frame may be calculated, and whether the frame meets the key frame setting condition is determined according to whether the degree of difference exceeds a threshold range.
  • the degree of difference may refer to a graphic of the frame and the recorded current key frame.
  • the ratio of the size of the instruction data Specifically, when the difference degree exceeds the threshold range, if the difference between the two frames is considered to be large, it is determined that the frame meets the key frame setting condition; when the difference degree falls within the threshold range, the difference between the two frames is considered to be small. It is determined that the frame does not meet the key frame setting condition.
  • the threshold range can be set to (0.3, 2.0).
  • the current key frame is not updated, and the difference data of the current key frame is obtained by using the recorded current key frame, and only the difference data is sent to the user equipment, without Send all data of the frame.
  • the difference data may also be compressed, and the compressed difference data is sent to the user equipment.
  • the method may further include:
  • the key frame update is performed, the frame is recorded as the current key frame, and the graphic instruction data of the frame is sent to the user equipment, so that the user equipment generates a graphical interface of the frame.
  • the result of S120 When the result of S120 is consistent, it indicates that the difference between the two frames is large. If the current key frame recorded in the original record is used, the difference data obtained is also large, and the effect of reducing the transmission traffic cannot be better. Therefore, the key is needed.
  • a frame update that replaces the recorded current key frame with the frame.
  • the difference data is not calculated, but the graphics command data of the frame is directly sent to the user equipment.
  • the graphics instruction data of the frame may be compressed and sent to the user equipment.
  • the server may insert a special identifier in the data of the frame to indicate that the frame is a key frame for the user equipment to identify.
  • zlib's DEFLATE compression algorithm can be used.
  • other compression algorithms may also be used, which is not limited herein.
  • the method can include the following steps.
  • the user equipment receives data of a frame sent by the server.
  • the server sends data to the user equipment in frames, and the user equipment receives data in frames. If the received data is compressed data, the user equipment decompresses the compressed data of the received frame.
  • the user equipment first needs to identify the data of the frame to determine whether the frame is a key frame.
  • whether the frame is a key frame can be determined by identifying a particular identifier in the data of the frame.
  • S230 If not, acquiring the recorded current key frame, and determining that the data of the frame is difference data with the current key frame.
  • the recognition result of S220 is YES, determining that the data of the frame is graphics instruction data of the frame; performing key frame update, and recording the frame as a current key frame.
  • the data of the frame received by the user equipment may be the original graphics instruction data, or may be the difference data between the original graphics instruction data and the recorded current key frame.
  • the data of the frame received by the user equipment is the original graphics instruction data, and the user equipment needs to perform key frame update to update the current key frame of the local record to the frame.
  • the difference data is received.
  • the user equipment can acquire the original graphics instruction data of the frame using the current key frame and the difference data.
  • the method may further include: playing back the graphics instruction data of the frame to generate a graphical interface of the frame.
  • difference data of the graphics instruction data of the frame and the current key frame may be acquired when the cloud game is running, and the difference number is obtained. According to the data to be transmitted as the frame. Since the size of the difference data is much smaller than the original graphics instruction data, the data transmission amount of the frame can be effectively reduced, the bandwidth requirement can be reduced, and the efficient and smooth communication between the Trace end and the Retrace end can be realized, and the smoothness of the image surface can be ensured. .
  • FIG. 5 and FIG. 6 illustrate a method for processing graphics command data according to an embodiment of the present invention, including a compression process and a decompression process, which are respectively applied to the Trace side of the server side and the Retrace end of the user equipment side.
  • the method may include:
  • a keyframe strategy is established to determine keyframes.
  • the server sends the image frame data of the cloud game to the user equipment, it is important to determine which frame data is set as the key frame. Since each subsequent frame data is based on the key frame data to obtain the difference data, the desired difference data is exhausted. It may be small, then the key frame must be better represented, and the key frame strategy is used to find the most representative one frame as the key frame.
  • the specific process of establishing a key frame policy on the Trace side may include:
  • S505 setting a difference interval (ScaleMin, ScaleMax), determining a difference ratio DiffScale Whether the value is in this interval; if the difference ratio DiffScale value is in this interval, it is considered that the key frame setting condition is not met, and the key frame does not need to be updated; if the difference ratio DiffScale value is not in the interval, the frame is considered to be in the key frame setting.
  • the condition is that the key frame needs to be updated, so the frame NewFrameData can be recorded as a new current key frame, replacing the original current key frame.
  • the difference interval (ScaleMin, ScaleMax) is set at (0.3, 2.0) to achieve better results.
  • the method for processing the graphics command data on the Trace side may include:
  • the Trace side captures the drawing operation of each frame of the graphic picture, and records the graphic drawing rendering instruction to generate the graphic instruction data of the frame.
  • step S2 determining whether the frame meets the key frame setting condition; if not, indicating that the key frame does not need to be updated, proceeding to step S3; if yes, indicating that the frame update key frame is required Then, the process proceeds to step S4.
  • the graphics instruction data of the frame is compared with the graphics instruction data of the current key frame, the difference data therein is found, and the difference data is recorded. Since the data difference of similar frames is generally small, the difference data will be much smaller than the original graphics command data, which can greatly reduce the amount of data transmitted by the network.
  • the frame data block difference value or the exclusive OR value can be simply obtained.
  • a The data difference comparison algorithm can find the instruction data moving forward or backward in the frame data.
  • the process of calculating the interframe difference data by using the data difference comparison algorithm may include:
  • the current key frame of the graphics command data (referred to as the current key frame data) as a a character array KeyFrameData[m] of length m
  • the graphics instruction data (referred to as frame data) of the frame is regarded as a string array NewFrameData[n] of length n;
  • NewFrameData[n] set the value of k from 0 to n-1. For each k value, take the data of NewFrameData[k to n-1] and traverse it to find this segment in KeyFrameData[m]. For the data, the first t characters must be identical, that is, the NewFrameData[k to k+t] segment data can be found in KeyFrameData[m]; find the largest t is the best match length, and record the best match length. The position in KeyFrameData;
  • the original key frame data NewFrameData can be restored according to the data of the best matching length in the difference data and the offset position in the KeyFrameData.
  • the frame is used in this step to update the current key frame, and the recorded current key frame is replaced with the frame, that is, the frame is recorded as a new current key. frame.
  • the Trace end may add a special signal (Flag) identifier to the key frame data to indicate that the frame is a key frame.
  • the calculated difference data is determined to be data to be compressed.
  • the frame is determined to be data to be compressed.
  • the difference data of the frame and the current key frame or the frame itself is compressed, and the obtained compressed data is used as the data to be sent of the frame, and is sent to the Retrace end.
  • Compress data may be compressed using zlib's DEFLATE compression algorithm or other algorithms to minimize the amount of data that needs to be transferred.
  • the processing method of the graphic instruction data of the Retrace side may include:
  • S8 Determine whether the decompressed data is key frame data, or is the difference data between the frame and the current key frame.
  • the key frame policy on the Trace side determines which frame is used as the key frame. We can find out whether the special flag is included by parsing the decompressed data, and determine whether the flag is a key frame by judging the flag.
  • Cloud games require high real-time performance and require this process to ensure sufficient fluency for a better cloud gaming experience.
  • Performing interframe difference data analysis and compression on graphics instruction data is a relatively time consuming process. Therefore, a processing queue of graphics instruction data can be established. After capturing the original graphics instruction data, it is placed in the processing queue, waiting for difference analysis and compression. Send to avoid problems such as the graphical interface, such as catastrophic effects.
  • the main thread of the Trace side used to capture the drawing operation of each frame of the graphic picture, and record the graphic
  • the rendering instruction is drawn, the graphics instruction data of the frame is generated, and the graphics instruction data is processed and queued.
  • differential data analysis and compression threads can be created, and data difference analysis calculations and compressions are put into this thread for separate processing. Handling differential data analysis and compression through independent threads will not affect the speed of the main thread. This thread can pass the results of the difference analysis and compression directly to the background network sending thread to send, or can be placed in the queue to be sent waiting for the background network sending thread to send for better fluency.
  • the original rendering instruction is recorded to obtain the graphics instruction data and placed in the processing queue of the graphics instruction data;
  • the difference analysis and compression thread extracts the graphics instruction data from the processing queue, performs differential data analysis and applies the compression algorithm.
  • the result is sent directly or placed in the queue to be sent waiting for the background network sending thread to send.
  • FIG. 7 is a timing sequence of graphics instruction processing according to an embodiment of the present invention.
  • the activity sequence of the cloud game system running cycle will be performed in order between the cloud game, the graphics instruction frame data, the interframe difference data, and the compressed/decompressed data.
  • the processing method of the graphics command data provided by the embodiments of the present invention can greatly reduce the data size by using a key frame strategy, and significantly reduce network transmission traffic, reduce bandwidth requirements, and can significantly reduce data traffic.
  • the peak value reduces the time for sending and receiving network data, making the game run more smoothly and fluently.
  • the original graphics command data, key frame and difference data, and compressed data size distribution are compared. It can be seen that the difference data has been greatly reduced compared to the original graphics command data, and the data is further reduced after the data is compressed.
  • the average data of one frame after compression is about 12kbyte, which is greatly reduced compared with the original data of 500kbyte or more.
  • embodiments of the present invention can also be used for other graphics rendering
  • the client receives applications in the form of rendering instructions, such as a scene roaming program that receives rendering data from the server side, and a virtual reality (VR) device that receives pictures from the cloud server.
  • rendering instructions such as a scene roaming program that receives rendering data from the server side
  • VR virtual reality
  • an embodiment of the present invention provides a device for processing graphics command data, which is applied to a server of a cloud interaction system, where the cloud interaction system includes the server and user equipment.
  • the device 300 may include:
  • An obtaining module 310 configured to acquire graphics instruction data of a frame
  • the determining module 320 is configured to determine whether the frame meets a key frame setting condition
  • the comparing module 330 is configured to compare the graphic instruction data of the frame with the graphic instruction data of the current key frame to obtain the difference data if the determining module determines that the content is not met;
  • the sending module 350 is configured to send the difference data to the user equipment, so that the user equipment acquires graphics instruction data of the frame according to the current key frame and the difference data.
  • the apparatus 300 further includes:
  • the updating module 360 is configured to record the frame as a current key frame if the determining module determines that the content is consistent;
  • the sending module 350 is further configured to send graphics instruction data of the frame to the user equipment.
  • the determining module 320 includes:
  • a calculating unit configured to calculate a difference between the frame and the current key frame
  • a determining unit configured to determine, according to whether the degree of difference exceeds a threshold range, whether the frame meets a key frame setting condition.
  • the apparatus 300 further includes:
  • a compression module 340 configured to compress the difference data
  • the sending module 350 is configured to send the compressed difference data to the user equipment.
  • the apparatus 300 further includes:
  • a data processing module 370 configured to capture a drawing operation of each frame of the graphical interface, generate graphics instruction data, and put the graphics instruction data into a processing queue;
  • the obtaining module 310 is specifically configured to acquire graphics instruction data of the frame from the processing queue.
  • the device of the embodiment of the present invention may be a server.
  • the difference data between the graphics command data of the frame and the graphics command data of the current key frame is acquired and compressed, and the compressed data is used as the data to be transmitted of the frame. Since the size of the difference data is much smaller than the original graphics command data, the amount of data transmission can be effectively reduced, the bandwidth requirement can be reduced, and the efficient and smooth communication between the server and the user equipment can be realized, and the smoothness of the image surface can be ensured.
  • an embodiment of the present invention further provides a processing device 400 for graphics command data, which is applied to a user equipment of a cloud interaction system, where the cloud interaction system includes a server and the user equipment; and the apparatus 400 may include:
  • the receiving module 410 is configured to receive data of a frame sent by the server;
  • An identification module 420 configured to identify whether the frame is a key frame
  • the determining module 430 is configured to: if the recognition result of the identification module is negative, acquire a current key frame, and determine that the data of the frame is difference data with the current key frame;
  • the restoration module 440 is configured to acquire graphics instruction data of the frame according to the data of the frame and the current key frame.
  • the apparatus 400 further includes:
  • the updating module 450 is configured to determine, if the identification module identifies that, the data of the frame is graphics instruction data of the frame; and record the frame as a current key frame.
  • the apparatus 400 further includes:
  • the playback module 460 is configured to play back the graphics instruction data of the frame to generate a graphical interface of the frame.
  • the data of the frame is compressed data
  • the receiving module is further configured to decompress the compressed data
  • the device in the embodiment of the present invention may be a user equipment.
  • difference data of graphics instruction data of a frame and graphics instruction data of a current key frame is acquired, and the difference data is used as data to be transmitted of the frame. Since the size of the difference data is much smaller than the original graphics command data, the amount of data transmission can be effectively reduced, the bandwidth requirement can be reduced, and the efficient and smooth communication between the server and the user equipment can be realized, and the smoothness of the image surface can be ensured.
  • the embodiment of the present invention further provides a cloud interaction system.
  • the cloud interaction system includes a server and a user equipment, where
  • the server is configured to acquire graphics command data of the frame, determine whether the frame meets a key frame setting condition, and if not, compare the graphic instruction data of the frame with the graphic instruction data of the current key frame to obtain a difference. Data, the difference data is sent to the user equipment;
  • the user equipment is configured to receive data of the frame sent by the server, to identify whether the frame is a key frame; if not, acquire a current key frame, and determine that the data of the frame is the current key frame And the difference data, according to the difference data and the key frame, acquiring graphics instruction data of the frame.
  • the server is further configured to: when the frame does not meet the key frame setting condition, record the frame as a current key frame, and send graphics instruction data of the frame to the user equipment;
  • the user equipment is further configured to: when the frame is a current key frame, record the frame as a current key frame.
  • the data transmitted between the server and the user equipment may not be all the original graphics instruction data, but may be replaced by the difference data with the key frame. Therefore, the amount of data transmission can be effectively reduced, the bandwidth requirement can be reduced, and the efficient and smooth communication between the server and the user equipment can be realized, and the smoothness of the image surface can be ensured.
  • Embodiments of the present invention also provide a computer readable storage medium storing one or more programs, the one or more programs, when executed by a server including one or more processors, causing the server to perform the method implementation as described above Steps 110-130 of the method for processing graphics command data.
  • Embodiments of the present invention also provide another computer readable storage medium storing one or more programs that, when executed by a user device including one or more processors, cause the server to perform as above
  • the method implements steps 210-240 of the method of processing graphics command data.
  • an embodiment of the present invention further provides a server 500, which may include:
  • the memory 520 is used to store a program 550; the processor 510 is configured to execute the program in the memory.
  • the processor 510 executes the program 550 stored by the memory 520 to cause the server 500 to perform step 110 of the processing method of the graphics instruction data as described in the above method embodiment. 130.
  • an embodiment of the present invention further provides a user equipment 600, which may include:
  • a processor 610 a memory 620, a communication interface 630, a bus 640; the processor 610, a memory 620, and a communication interface 630 communicating with each other through the bus 640; the communication interface 630, for receiving and transmitting data; the memory 620 is used to store a program 650; the processor 610 is configured to execute the program 650 in the memory.
  • the processor 610 executes the program 650 stored by the memory 620 to cause the user equipment 600 to perform the steps of the processing method of the graphics instruction data as described in the above method embodiment. 210-240.
  • the processor 610 is configured to obtain graphics instruction data of the frame, determine whether the frame meets a key frame setting condition, and if not, compare the graphic instruction data of the frame with the graphic instruction data of the current key frame to obtain Difference data;
  • the communication interface 630 can be configured to send the difference data to the user equipment, so that the user equipment acquires graphics instruction data of the frame according to the current key frame and the difference data.
  • the bus may be an Industry Standard Architecture (ISA) bus or a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA). ) Bus, etc.
  • ISA Industry Standard Architecture
  • PCI Peripheral Component
  • EISA Extended Industry Standard Architecture
  • the bus can be divided into one or more of an address bus, a data bus, and a control bus. For ease of representation, only one thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.
  • the memory may include a high speed RAM (Ramdom Access Memory) memory.
  • the memory may further include a non-volatile memory.
  • the memory can include a disk storage.
  • the processor may be a central processing unit (CPU), or the processor may be an Application Specific Integrated Circuit (ASIC), or the processor may be configured.
  • CPU central processing unit
  • ASIC Application Specific Integrated Circuit
  • the program may be stored in a computer readable storage medium, and the storage medium may include: ROM, RAM, disk or CD.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Information Transfer Between Computers (AREA)
  • User Interface Of Digital Computer (AREA)
  • Computer And Data Communications (AREA)
  • Processing Or Creating Images (AREA)

Abstract

一种图形指令数据的处理方法、装置及系统，有助于减少图形指令数据的传输量，降低对带宽的要求。在本发明一实施例中，所述方法应用于云交互系统，所述云交互系统包括服务器和用户设备，所述方法包括：所述服务器获取帧的图形指令数据；判断所述帧是否符合关键帧设定条件；若不符合，将所述帧的图形指令数据与当前关键帧的图形指令数据进行对比，获得差异数据；将所述差异数据发送给所述用户设备，以便所述用户设备根据所述当前关键帧和所述差异数据获取所述帧的图形指令数据。

Description

图形指令数据的处理方法和装置及系统
本申请要求于2016年01月29日提交中国专利局、申请号为201610067797.6、发明名称为“图形指令数据的处理方法和装置及系统”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及云技术领域，具体涉及一种图形指令数据的处理方法、装置及系统。
背景技术
随着近年来终端技术和网络带宽的高速发展，云计算技术的日趋成熟催生了云交互系统。现有的云交互系统通常包括服务器和多个用户设备。服务器可以是集群服务器，多种交互式应用都可以运行在云交互系统中。
在运行云游戏等交互式应用时，服务器和用户设备可能需要交互图形界面。其中，图形界面的图形指令数据是按帧传输的。
在实际交互中，图形指令数据传输量比较大，网络传送对带宽要求很高，容易导致用户设备端出现画面卡顿不流畅等现象。
发明内容
本发明实施例提供一种图形指令数据的处理方法、装置及系统，以有助于减少图形指令数据的传输量，降低对带宽的要求。
本发明第一方面提供一种图形指令数据的处理方法，所述方法应用于云交互系统，所述云交互系统包括服务器和用户设备，所述方法包括： 所述服务器获取帧的图形指令数据；判断所述帧是否符合关键帧设定条件；若不符合，将所述帧的图形指令数据与当前关键帧的图形指令数据进行对比，获得差异数据；将所述差异数据发送给所述用户设备，以便所述用户设备根据所述当前关键帧和所述差异数据获取所述帧的图形指令数据。
本发明第二方面提供一种图形指令数据的处理方法，所述方法应用于云交互系统，所述云交互系统包括服务器和用户设备，所述方法包括：所述用户设备接收所述服务器发送的帧的数据；识别所述帧是否为关键帧；若否，则获取当前关键帧，确定所述帧的数据是与所述当前关键帧的差异数据；根据所述帧的数据和所述当前关键帧，获取所述帧的图形指令数据。
本发明第三方面提供一种图形指令数据的处理方法，所述方法应用于云交互系统，所述云交互系统包括服务器和用户设备，所述方法包括：所述服务器获取帧的图形指令数据，判断所述帧是否符合关键帧设定条件，若不符合，将所述帧的图形指令数据与当前关键帧的图形指令数据进行对比，获得差异数据，将所述差异数据发送给所述用户设备；所述用户设备接收所述服务器发送的所述帧的数据，识别所述帧是否为关键帧；若否，则获取所述当前关键帧，确定所述帧的数据是与所述当前关键帧的差异数据，根据所述帧的数据和所述当前关键帧，获取所述帧的图形指令数据，根据所述帧的图形指令数据生成帧的图形界面。
本发明第四方面提供图形指令数据的处理方法，所述方法应用于云交互系统，所述云交互系统包括服务器和用户设备，所述方法包括：所述服务器获取帧的图形指令数据，判断所述帧是否符合关键帧设定条件，若不符合，将所述帧的图形指令数据与当前关键帧的图形指令数据进行对比，获得差异数据，将所述差异数据发送给所述用户设备；所述用户设备接收所述服务器发送的所述帧的数据，识别所述帧是否为关键帧；若否，则获取当前关键帧，确定所述帧的数据是与所述当前关键帧的差异数据，根据 所述帧的数据和所述当前关键帧，获取所述帧的图形指令数据，根据所述帧的图形指令数据生成帧的图形界面。
本发明第五方面提供一种图形指令数据的处理装置，所述装置应用于云交互系统的用户设备，所述云交互系统包括服务器和所述用户设备；所述装置包括：接收模块，用于接收所述服务器发送的帧的数据；识别模块，用于识别所述帧是否为关键帧；确定模块，用于若所述识别模块识别结果为否，则获取当前关键帧，确定所述帧的数据是与所述当前关键帧的差异数据；还原模块，用于根据所述帧的数据和所述当前关键帧，获取所述帧的图形指令数据。
本发明第六方面提供一种云交互系统，所述云交互系统包括服务器和用户设备，其中，所述服务器，用于获取帧的图形指令数据，判断所述帧是否符合关键帧设定条件，若不符合，将所述帧的图形指令数据与当前关键帧的图形指令数据进行对比，获得差异数据，将所述差异数据发送给所述用户设备；所述用户设备，用于接收所述服务器发送的所述帧的数据，识别所述帧是否为关键帧；若否，则获取当前关键帧，确定所述帧的数据是与所述当前关键帧的差异数据，根据所述帧的数据和所述当前关键帧，获取所述帧的图形指令数据，根据所述帧的图形指令数据生成帧的图形界面。
由上可见，在本发明的一些可行的实施方式中，获取帧的图形指令数据与当前关键帧的图形指令数据的差异数据，并将所述差异数据作为帧的待发送数据。由于差异数据的大小远小于原始的图形指令数据，因此，可以有效减少数据传输量，降低对带宽的要求，有助于实现高效流畅通信，保障图像图面的流畅度。
附图说明
为了更清楚地说明本发明实施例技术方案，下面将对实施例和现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其它的附图。
图1是一种Trace端和Retrace端交互的流程图；
图2是一款游戏的每帧指令数据量分布示意图；
图3是云交互系统的架构示意图；
图4是本发明实施例提供的图形指令数据的处理方法的流程示意图；
图5是Trace端建立关键帧策略的流程示意图；
图6是Trace端和Retrace端图形指令数据的处理流程示意图；
图7是本发明实施例的图形指令处理时序的示意图；
图8是一款游戏的每帧指令数据量分布对比示意图；
图9是本发明一个实施例提供的图形指令数据的处理装置的结构示意图；
图10是本发明另一实施例提供的图形指令数据的处理装置的结构示意图；
图11是本发明实施例提供的服务器的结构示意图；
图12是本发明实施例提供的用户设备的结构示意图。
具体实施方式
为了使本技术领域的人员更好地理解本发明方案，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分的实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本发明保护的范围。
图3示出了云交互系统的架构示意图。云交互系统包括服务器和用户设备，用于运行交互式应用，例如云游戏。交互式应用分别运行的服务器和用户设备的客户端通常被称为Trace端和Retrace端。其中，云游戏是指：游戏的真实客户端在云服务器上运行，同时将游戏的画面在玩家机器上进行显示，通过将玩家本地的键盘和鼠标控制远程重定向到云服务器来实现对游戏的控制。以云游戏为例，Trace端是指：运行在云游戏服务器上的游戏客户端，即云游戏的云端，执行游戏的主要逻辑运行，并把与用户交互的有关图形界面，经网络传送给Retrace端。Retrace端是指：运行在用户设备上的客户端，即云游戏的用户端，用于呈现Trace端的游戏图形界面，并把用户的鼠标、键盘等设备输入，经网络传送给Trace端。在运行云游戏等交互式应用时，服务器和用户设备可能需要交互图形界面。
请参考图1，是一种Trace端和Retrace端交互的流程图。如图1所示，云游戏运行在Trace端。Trace端可以捕获每次绘制图形画面的操作，并录制其渲染指令，产生图形指令数据，将图形指令数据通过网络发送到Retrace端。Retrace端接收到指令数据并解析回放渲染指令，调用执行相关的图形绘制API(Application Programming Interface，应用程序编程接口)绘制图像，播放云游戏的画面。其中，图形指令数据是按帧传输的。
在游戏运行过程中，每帧画面调用图形渲染API的频率是极高的，渲染指令的参数繁多，录制时除了需要记录这些指令和参数外，还需要记录所有的纹理数据、顶点数据、索引数据等。如图2所示的一款游戏的每帧指令数据量分布图，可以看出一帧的均值数据量超过500K Byte，峰值数据量达到3400K Byte。流畅游戏每秒60帧的渲染速度，那么其对网络传送流量达到均值30MB/s，峰值达到200MB/s以上。
当云游戏运行时，服务器上的Trace端需要把每一帧图形界面的图形指令数据发送给用户设备上的Retrace端，以便Retrace端根据收到的图形 指令数据生成和播放云游戏的图形界面。可见，图形指令数据传输量比较大，其网络传送对带宽要求很高，容易导致Retrace端出现画面卡顿不流畅等现象。为了降低图形指令数据的传输量，降低对带宽的要求，本发明实施例提供一种图形指令数据的处理方法、装置及系统。
下面通过具体实施例，分别进行详细的说明。
实施例一
请参考图4，本发明实施例提供一种图形指令数据的处理方法，应用于如图3所示的云交互系统，所述云交互系统包括服务器和用户设备。云交互系统用于运行云游戏等交互式应用，本文以云游戏为例进行说明。云交互式系统可以理解为是云游戏系统，
在服务器侧，所述方法可以包括以下步骤。
S110、所述服务器获取帧的图形指令数据。
云游戏运行时，通常由服务器负责图像绘制，绘制完成后，将绘制好的图像帧发送给用户设备。用户设备接收到绘制好的图像帧后，解码图像帧，然后显示图像画面。其中，Trace端可以捕获每一帧图形界面的绘制操作，并录制图形绘制渲染指令，产生图形指令数据，并将所述图形指令数据放入处理队列中。需要发送帧数据时，可以从所述处理队列中获取帧的图形指令数据。
S120、判断所述帧是否符合关键帧设定条件。
本发明实施例中，预先建立关键帧策略，根据该关键帧策略判断帧是否符合关键帧设定条件，以决定是否更新当前关键帧。可选的，如果帧是第一帧，则可以直接记录为当前关键帧。如果帧不是第一帧，则可以计算该帧与已记录的当前关键帧的差异度，根据所述差异度是否超出阈值范围判断所述帧是否符合关键帧设定条件。
其中，所说的差异度可以是指所述帧与已记录的当前关键帧的图形 指令数据的大小的比值。具体的，当差异度超出阈值范围时，则认为两帧差异较大，则判断为所述帧符合关键帧设定条件；当差异度落入阈值范围内时，则认为两帧差异较小，判断为所述帧不符合关键帧设定条件。举例来说，阈值范围可以设定为(0.3,2.0)。
S130、若所述帧不符合关键帧设定条件，将所述帧的图形指令数据与当前关键帧的图形指令数据进行对比，获得差异数据；并将所述差异数据发送给用户设备。
当S120判断结果为不符合时，不更新当前关键帧，使用已记录的当前关键帧，获取所述帧与已记录的当前关键帧的差异数据，仅将该差异数据发送给用户设备，而不必发送所述帧的全部数据。
在一种实现方式中，还可以对差异数据进行压缩，将压缩后的差异数据发送给用户设备。
进一步的，所述方法还可以包括：
若S120判断结果为符合，则进行关键帧更新，将所述帧记录为当前关键帧，并将所述帧的图形指令数据发送给所述用户设备，以便所述用户设备生成帧的图形界面。
当S120判断结果为符合时，说明两帧差异较大，如果继续使用原来记录的当前关键帧，得到的差异数据也较大，不能较好的起到减少传输流量的效果，于是，需要进行关键帧更新，使用所述帧替换已记录的当前关键帧。并且，对于该帧，不再计算差异数据，而是直接将该帧的图形指令数据发送给用户设备。可选地，可以将该帧的图形指令数据进行压缩后发送给用户设备。可选的，服务器可以在该帧的数据中插入特殊的标识，用来表示该帧为关键帧，以便用户设备识别。
进行数据压缩时，可以采用zlib的DEFLATE压缩算法。当然，也可以采用其它压缩算法，本文对此不予限定。
在用户设备侧，所述方法可以包括以下步骤。
S210、所述用户设备接收所述服务器发送的帧的数据。
服务器按帧发送数据给用户设备，用户设备按帧接收数据。如果所接收的数据是压缩数据，用户设备对接收到的帧的压缩数据进行解压缩。
S220、识别所述帧是否为关键帧。
用户设备首先需要对帧的数据进行识别，以判断该帧是否为关键帧。在一些实施例中，可以通过识别帧的数据中特殊的标识，来判断该帧是否为关键帧。
S230、若否，则获取已记录的当前关键帧，确定所述帧的数据是与所述当前关键帧的差异数据。可选的，若S220的识别结果为是，则确定所述帧的数据是所述帧的图形指令数据；进行关键帧更新，并将所述帧记录为当前关键帧。
用户设备接收到的帧的数据，可能是原始的图形指令数据，也可能是原始的图形指令数据与已记录的当前关键帧的差异数据。在所述帧为关键帧时，用户设备接收到的帧的数据是原始的图形指令数据，用户设备需要进行关键帧更新，将本地记录的当前关键帧更新为所述帧。在所述帧不是关键帧时，接收到的是差异数据。
S240、根据所述帧的数据和所述当前关键帧，获取所述帧的图形指令数据。
如果接收到的是差异数据，则用户设备可以利用当前关键帧和该差异数据获取所述帧的原始的图形指令数据。
得到所述帧的图形指令数据后，进一步的，所述方法还可以包括：回放所述帧的图形指令数据，生成所述帧的图形界面。
由上可见，在本发明的一些可行的实施方式中，可以在云游戏运行时，获取所述帧与当前关键帧的图形指令数据的差异数据，并将该差异数 据作为所述帧的待发送数据。由于差异数据的大小远小于原始的图形指令数据，因此，可以有效减少帧的数据传输量，降低对带宽的要求，有助于实现Trace端和Retrace端的高效流畅通信，保障图像图面的流畅度。
为便于更好的理解本发明实施例提供的技术方案，下面通过一个具体场景下的实施方式为例进行介绍。
请参考图5和图6，其示出了本发明实施例的一种图形指令数据的处理方法，包括压缩流程及解压缩流程，分别应用于服务器侧的Trace端，以及用户设备侧的Retrace端，具体可以包括：
一、预先建立关键帧策略。
本步骤中建立关键帧策略，以确定关键帧。服务器向用户设备发送云游戏的图像帧数据时，确定把哪一帧数据设定为关键帧是重要的，由于后面的每帧数据是根据关键帧数据来求差异数据，希望得到的差异数据尽可能小，那么关键帧就必须具有更好的代表性，关键帧策略用于找到最有代表性的一帧作为关键帧。
请参考图5，Trace端建立关键帧策略的具体过程可包括：
S501、判断帧NewFrameData是否为第一帧(或者说起始帧)；
S502、如果是第一帧，则认为符合关键帧设定条件，可直接设定为当前关键帧，记录为KeyFrameData，并且，可以在帧数据中添加特殊的标识进行标记，记录当前关键帧KeyFrameData的大小keyFrameSize；
S503、如果不是第一帧，计算所述帧原始的图形指令数据NewFrameData的大小newFrameSize；
S504、计算newFrameSize与当前关键帧的图形指令数据的大小keyFrameSize的比值，将该比值记为差异比率DiffScale，该比值表示两帧的差异度；
S505、设定差异度区间(ScaleMin,ScaleMax)，判断差异比率DiffScale 值是否在此区间；如果差异比率DiffScale值在此区间，则认为不符合关键帧设定条件，不需要更新关键帧；如果差异比率DiffScale值没在此区间，则认为所述帧符合关键帧设定条件，需要进行更新关键帧，于是可以将帧NewFrameData记录为新的当前关键帧，替换原来的当前关键帧。可选的，差异度区间(ScaleMin,ScaleMax)设定在(0.3,2.0)可取得较好的效果。
二、图形指令数据的压缩流程。
如图6所示，Trace端图形指令数据的处理方法，具体流程可以包括：
S1、获取帧的图形指令数据。
云游戏运行时，Trace端捕获每帧图形画面的绘制操作，并录制图形绘制渲染指令，产生该帧的图形指令数据。
S2、根据上文所述的关键帧策略，判断所述帧是否符合关键帧设定条件；若不符合，说明不需要更新关键帧，则进入步骤S3；若符合，说明需要使用帧更新关键帧，则进入步骤S4。
S3、计算所述帧与当前关键帧的差异数据。
将所述帧的图形指令数据与当前关键帧的图形指令数据进行对比，找出其中的差异数据，并记录差异数据。由于相近几帧的数据差异一般较小，所以差异数据比起原始的图形指令数据将会小很多，可以极大的降低网络传送的数据量。
计算数据帧与当前关键帧的差异数据，可以简单的获取帧数据块差值或者异或值。但是，因为图形指令数据的每帧都在变化，相同的一个图形指令数据也经常发生在帧数据块中的前移或后移，为了取得更好的效果，本发明一些实例中，提供一种数据差异对比算法，能够找出帧数据中前移或后移的指令数据。
采用该数据差异对比算法计算帧间差异数据的流程可以包括：
a、将当前关键帧的图形指令数据(简称当前关键帧数据)看作一个 长度m的字符数组KeyFrameData[m]，所述帧的图形指令数据(简称帧数据)看作一个长度n的字符串数组NewFrameData[n]；
b、对NewFrameData[n]，设定k取值从0到n-1，对每一个k值，取出NewFrameData[k到n-1]的这段数据，在KeyFrameData[m]中遍历查找此段数据，则必存在前t个字符完全相同，即NewFrameData[k到k+t]段数据能在KeyFrameData[m]中找到；找出最大的一个t为最佳匹配长度，记录最佳匹配长度在KeyFrameData中的位置；
c、设定一个相同长度的阀值T，只有当t大于该阀值T时，才寻找最佳匹配长度，并将所有最佳匹配长度的数据和偏移位置记录下来，作为与关键帧的差异数据；小于阀值t的数据可直接拷贝到差异数据尾部；
至此，计算得到所述帧与当前关键帧的差异数据。
需要说明的是，在接收端，可以根据差异数据中的最佳匹配长度的数据和其在KeyFrameData中的偏移位置，还原出原始的关键帧数据NewFrameData。
S4、更新当前关键帧。
在所述帧符合关键帧设定条件时，本步骤中使用所述帧来更新当前关键帧，将已记录的当前关键帧替换为所述帧，即，将所述帧记录为新的当前关键帧。进一步的，Trace端可以为关键帧数据加入特殊信号(Flag)标识，以表示该帧为关键帧。
S5、确定待压缩数据。
在所述帧不是关键帧时，确定计算得到的差异数据为待压缩数据。
在所述帧被记录为新的当前关键帧时，确定所述帧为待压缩数据。
S6、对待压缩数据进行压缩并发送。
本步骤中，对所述帧与当前关键帧的差异数据或所述帧本身，进行压缩，将得到的压缩数据作为所述帧的待发送数据，并向Retrace端发送该 压缩数据。一些实施例中，可以使用zlib的DEFLATE压缩算法或其它算法对待压缩数据进行压缩，尽可能减少需要传送的数据量。
三、图形指令数据的解压缩流程。
请继续参考图6，Retrace端图形指令数据的处理方法，具体过程可包括：
S7、接收Trace端发来的帧的压缩数据，并解压缩；
S8、判断解压缩得到的数据，是否是关键帧数据，抑或是帧和当前关键帧的差异数据。Trace端的关键帧策略决定哪一帧作为关键帧，我们可以通过解析解压后的数据寻找其中是否包括特殊Flag标识，通过判断该标识位可以识别是否为关键帧。
S9、如果确定所述帧是关键帧，则记录或更新关键帧，如果已记录有当前关键帧，则进行当前关键帧替换，将所述帧数据记录为新的当前关键帧。
S10、如果确定所述帧不是关键帧，解压缩得到的数据是与当前关键帧的差异数据，则获取已记录的当前关键帧。
S11、根据帧的差异数据和所述当前关键帧，获取所述帧的原始数据，原始数据是指原始的图形指令数据。
通过回放指令数据，得到绘制的云游戏的图形界面。
四、建立图形指令数据的处理队列。
云游戏对实时性要求较高，需要此过程保证足够的流畅度，才能有更好的云游戏体验。对图形指令数据进行帧间差异数据分析和压缩是相对耗时的过程，因此可以建立图形指令数据的处理队列，当捕获到原始的图形指令数据后先放入处理队列中，等待差异分析和压缩发送，以避免出现图形界面卡顿等影响游戏效果的问题。
Trace端的主线程，用于捕获每帧图形画面的绘制操作，并录制图形 绘制渲染指令，产生该帧的图形指令数据，并将图形指令数据处理队列。由于差异数据分析和压缩是耗时的一个过程，在主线程执行将影响录制图形指令时游戏的流畅度。优选的，可以创建差异数据分析和压缩线程，将数据差异分析计算和压缩放入此线程单独处理。通过独立线程处理差异数据分析和压缩，将不会影响主线程的运行速度。此线程可将差异分析和压缩的结果直接传递给后台网络发送线程来发送，也可以放入待发送队列中等待后台网络发送线程发送，以获得更好的流畅度。
由上可见，在主线程中，录制原始渲染指令得到图形指令数据并放入图形指令数据的处理队列中；差异分析和压缩线程从处理队列中取出图形指令数据，进行差异数据分析和应用压缩算法，结果直接发送或放入待发送队列等待后台网络发送线程发送。
请参考图7，是本发明实施例一实施例技术方案的图形指令处理时序。应用本发明实施例的技术方案后，云游戏系统运行周期的活动时序，将在云游戏、图形指令帧数据、帧间差异数据和压缩/解压数据之间按照次序进行。
由上可见，本发明各个实施例提供的图形指令数据的处理方法，通过采用关键帧策略，可以极大压缩数据大小，并显著减少网络传输流量，降低对带宽的要求；也可以显著降低数据流量峰值，减少网络数据收发时间，使得游戏运行更平稳，流畅度更佳。
如图8所示，以某款游戏为例，对比原始的图形指令数据、关键帧和差异数据、压缩后数据大小分布。可以看出，差异数据比起原始的图形指令数据已经极大的减少了，压缩数据后进一步减少。压缩后一帧的均值数据为12kbyte左右，比起原始数据的500kbyte以上已经有极大的减少效果了。
本发明实施例除了应用于云游戏外，还可用于其他将图形渲染放在 服务器端，客户端接收渲染指令形式的应用，如从服务器端接收渲染数据的场景漫游类程序、从云端服务器接收画面的虚拟现实(VR)设备等。
为了更好的实施本发明实施例的上述方案，下面还提供用于配合实施上述方案的相关装置。
请参考图9，本发明实施例提供一种图形指令数据的处理装置300，应用于云交互系统的服务器，所述云交互系统包括所述服务器和用户设备；所述装置300可包括：
获取模块310，用于获取帧的图形指令数据；
判断模块320，用于判断所述帧是否符合关键帧设定条件；
比较模块330，用于若所述判断模块判断不符合，将所述帧的图形指令数据与当前关键帧的图形指令数据进行对比，获得差异数据；
发送模块350，用于将所述差异数据发送给所述用户设备，以便所述用户设备根据当前关键帧和所述差异数据获取所述帧的图形指令数据。
在一些实施例中，所述装置300还包括：
更新模块360，用于若所述判断模块判断符合，则将所述帧记录为当前关键帧；
所述发送模块350，还用于将所述帧的图形指令数据发送给所述用户设备。
在一些实施例中，所述判断模块320包括：
计算单元，用于计算所述帧与所述当前关键帧的差异度；
判断单元，用于根据所述差异度是否超出阈值范围判断所述帧是否符合关键帧设定条件。
在一些实施例中，所述装置300还包括：
压缩模块340，用于对所述差异数据进行压缩；
所述发送模块350用于将压缩后的差异数据发送给所述用户设备。
在一些实施例中，所述装置300还包括：
数据处理模块370，用于捕获每一帧图形界面的绘制操作，产生图形指令数据，并将所述图形指令数据放入处理队列中；
所述获取模块310，具体用于从所述处理队列中获取所述帧的图形指令数据。
可以理解，本发明实施例装置的各个功能模块的功能可根据上述方法实施例中的方法具体实现，其具体实现过程可参照上述方法实施例中的相关描述，此处不再赘述。本发明实施例装置可以是服务器。
由上可见，在本发明的一些可行的实施方式中，获取帧的图形指令数据与当前关键帧的图形指令数据的差异数据，并进行压缩，将压缩数据作为所述帧的待发送数据。由于差异数据的大小远小于原始的图形指令数据，因此，可以有效减少数据传输量，降低对带宽的要求，有助于实现服务器和用户设备的高效流畅通信，保障图像图面的流畅度。
请参考图10，本发明实施例还提供一种图形指令数据的处理装置400，应用于云交互系统的用户设备，所述云交互系统包括服务器和所述用户设备；所述装置400可包括：
接收模块410，用于接收所述服务器发送的帧的数据；
识别模块420，用于识别所述帧是否为关键帧；
确定模块430，用于若所述识别模块识别结果为否，则获取当前关键帧，确定所述帧的数据是与所述当前关键帧的差异数据；
还原模块440，用于根据所述帧的数据和所述当前关键帧，获取所述帧的图形指令数据。
在一些实施例中，所述装置400还包括：
更新模块450，用于若所述识别模块识别为是，则确定所述帧的数据是所述帧的图形指令数据；将所述帧记录为当前关键帧。
在一些实施例中，所述装置400还包括：
回放模块460，用于回放所述帧的图形指令数据，生成所述帧的图形界面。
在一些实施例中，所述帧的数据为压缩数据，所述接收模块还用于对所述压缩数据进行解压缩。
可以理解，本发明实施例装置的各个功能模块的功能可根据上述方法实施例中的方法具体实现，其具体实现过程可参照上述方法实施例中的相关描述，此处不再赘述。本发明实施例装置可以是用户设备。
由上可见，在本发明的一些可行的实施方式中，获取帧的图形指令数据与当前关键帧的图形指令数据的差异数据，并将所述差异数据作为所述帧的待发送数据。由于差异数据的大小远小于原始的图形指令数据，因此，可以有效减少数据传输量，降低对带宽的要求，有助于实现服务器和用户设备的高效流畅通信，保障图像图面的流畅度。
本发明实施例还提供一种云交互系统，如图3所示，所述云交互系统包括服务器和用户设备，其中，
所述服务器，用于获取帧的图形指令数据，判断所述帧是否符合关键帧设定条件，若不符合，将所述帧的图形指令数据与当前关键帧的图形指令数据进行对比，获得差异数据，将所述差异数据发送给所述用户设备；
所述用户设备，用于接收所述服务器发送的所述帧的数据，识别所述帧是否为关键帧；若否，则获取当前关键帧，确定所述帧的数据是与所述当前关键帧的差异数据，根据所述差异数据和所述关键帧，获取所述帧的图形指令数据。
在一些实施例中，
所述服务器，还用于判断所述帧不符合关键帧设定条件时，将所述帧记录为当前关键帧，将所述帧的图形指令数据发送给所述用户设备；
所述用户设备，还用于识别所述帧为当前关键帧时，将所述帧记录为当前关键帧。
由上可见，在本发明的一些可行的实施方式中，服务器和用户设备之间传输的数据可以不再全部是原始的图形指令数据，而是利用与关键帧的差异数据代替。因此，可以有效减少数据传输量，降低对带宽的要求，有助于实现服务器和用户设备的高效流畅通信，保障图像图面的流畅度。
本发明实施例还提供一种存储一个或多个程序的计算机可读存储介质，所述一个或多个程序当被包括一个或多个处理器的服务器执行时使所述服务器执行如上文方法实施所述的图形指令数据的处理方法的步骤110-130。
本发明实施例还提供另一种存储一个或多个程序的计算机可读存储介质，所述一个或多个程序当被包括一个或多个处理器的用户设备执行时使所述服务器执行如上文方法实施所述的图形指令数据的处理方法的步骤210-240。
请参考图11，本发明实施例还提供一种服务器500，可包括：
处理器510，存储器520，通信接口530，总线540；所述处理器510，存储器520，通信接口530通过所述总线540相互的通信；所述通信接口530，用于接收和发送数据；所述存储器520用于存储程序550；所述处理器510用于执行所述存储器中的所述程序。当所述服务器500运行时，所述处理器510执行所述存储器520存储的所述程序550，以使所述服务器500执行如上文方法实施例所述的图形指令数据的处理方法的步骤110-130。
请参考图12，本发明实施例还提供一种用户设备600，可包括：
处理器610，存储器620，通信接口630，总线640；所述处理器610，存储器620，通信接口630通过所述总线640相互的通信；所述通信接口 630，用于接收和发送数据；所述存储器620用于存储程序650；所述处理器610用于执行所述存储器中的所述程序650。当所述用户设备600运行时，所述处理器610执行所述存储器620存储的所述程序650，以使所述用户设备600执行如上文方法实施例所述的图形指令数据的处理方法的步骤210-240。
其中，处理器610可用于获取帧的图形指令数据；判断所述帧是否符合关键帧设定条件；若不符合，将所述帧的图形指令数据与当前关键帧的图形指令数据进行对比，获得差异数据；
通信接口630可用于将差异数据发送给所述用户设备，以便所述用户设备根据当前关键帧和所述差异数据获取所述帧的图形指令数据。
其中，所述总线可以是工业标准体系结构(Industry Standard Architecture，简称为ISA)总线或外部设备互连(Peripheral Component，简称为PCI)总线或扩展工业标准体系结构(Extended Industry Standard Architecture，简称为EISA)总线等。所述总线可以分为地址总线、数据总线、控制总线中的一种或多种。为便于表示，图中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
所述存储器可以包含高速RAM(Ramdom Access Memory)存储器。可选地，所述存储器还可以还包括非易失性存储器(non-volatile memory)。例如所述存储器可以包括磁盘存储器。
所述处理器可以是一个中央处理器(Central Processing Unit，简称为CPU)，或者所述处理器可以是特定集成电路(Application Specific Integrated Circuit，简称为ASIC)，或者所述处理器可以是被配置成实施本发明实施例的一个或多个集成电路。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详细描述的部分，可以参见其它实施例的相关描述。
需要说明的是，对于前述的各方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本发明并不受所描述动作顺序的限制，因为依据本发明，某些步骤可以采用其它顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作和模块并不一定是本发明所必须的。
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，该程序可以存储于一计算机可读存储介质中，存储介质可以包括：ROM、RAM、磁盘或光盘等。
以上对本发明实施例所提供的图形指令数据的处理方法和装置及系统进行了详细介绍，本文中应用了具体个例对本发明的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本发明的方法及其核心思想；同时，对于本领域的一般技术人员，依据本发明的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本发明的限制。

Claims (22)

  1. 一种图形指令数据的处理方法，其特征在于，所述方法应用于云交互系统，所述云交互系统包括服务器和用户设备，所述方法包括：
    所述服务器获取帧的图形指令数据；
    判断所述帧是否符合关键帧设定条件；
    若不符合，将所述帧的图形指令数据与当前关键帧的图形指令数据进行对比，获得差异数据；
    将所述差异数据发送给所述用户设备，以便所述用户设备根据所述当前关键帧和所述差异数据获取所述帧的图形指令数据。
  2. 根据权利要求1所述的方法，其特征在于，所述判断所述帧是否符合关键帧设定条件之后，还包括：
    若符合，则将所述帧记录为当前关键帧；
    将所述帧的图形指令数据发送给所述用户设备。
  3. 根据权利要求1所述的方法，其特征在于，所述判断所述帧是否符合关键帧设定条件包括：
    计算所述帧与所述当前关键帧的差异度，根据所述差异度是否超出阈值范围判断所述帧是否符合关键帧设定条件。
  4. 根据权利要求1所述的方法，其特征在于，还包括：
    获得差异数据后，对所述差异数据进行压缩，将压缩后的差异数据发送给所述用户设备。
  5. 根据权利要求1-4任一项所述的方法，其特征在于，还包括：
    捕获每一帧图形界面的绘制操作，产生图形指令数据，并将所述图形指令数据放入处理队列中；
    所述获取帧的图形指令数据包括：
    从所述处理队列中获取所述帧的图形指令数据。
  6. 一种图形指令数据的处理方法，其特征在于，所述方法应用于云交互系统，所述云交互系统包括服务器和用户设备，所述方法包括：
    所述用户设备接收所述服务器发送的帧的数据；
    识别所述帧是否为关键帧；
    若不是关键帧，则获取当前关键帧，确定所述帧的数据是与所述当前关键帧的差异数据；
    根据所述帧的数据和所述当前关键帧，获取所述帧的图形指令数据。
  7. 根据权利要求6所述的方法，其特征在于，所述识别所述帧是否为关键帧之后，还包括：
    若是关键帧，则确定所述帧的数据是所述帧的图形指令数据；
    将所述帧记录为当前关键帧。
  8. 根据权利要求6所述的方法，其特征在于，还包括：
    回放所述帧的图形指令数据，生成所述帧的图形界面。
  9. 根据权利要求6所述的方法，其特征在于，所述帧的数据为压缩数据，所述方法还包括：
    在接收到所述服务器发送的帧的压缩数据之后，对所述压缩数据进行解压缩。
  10. 一种图形指令数据的处理方法，其特征在于，所述方法应用于云交互系统，所述云交互系统包括服务器和用户设备，所述方法包括：
    所述服务器获取帧的图形指令数据，判断所述帧是否符合关键帧设定条件，若不符合，将所述帧的图形指令数据与当前关键帧的图形指令数据进行对比，获得差异数据，将所述差异数据发送给所述用户设备；
    所述用户设备接收所述服务器发送的所述帧的数据，识别所述帧是否为关键帧；若否，则获取当前关键帧，确定所述帧的数据是与所述当前关键帧的差异数据，根据所述帧的数据和所述当前关键帧，获取所述帧的图形指令数据。
  11. 根据权利要求10所述的方法，其特征在于，还包括：
    所述服务器判断所述帧符合关键帧设定条件时，将所述帧记录为当前关键帧，并将所述帧的图形指令数据发送给所述用户设备；
    所述用户设备识别所述帧为关键帧时，将所述帧记录为当前关键帧。
  12. 一种图形指令数据的处理装置，其特征在于，所述装置应用于云交互系统的服务器，所述云交互系统包括所述服务器和用户设备；
    所述装置包括：
    获取模块，用于获取帧的图形指令数据；
    判断模块，用于判断所述帧是否符合关键帧设定条件；
    比较模块，用于若所述判断模块判断不符合，将所述帧的图形指令数据与当前关键帧的图形指令数据进行对比，获得差异数据；
    发送模块，用于将所述差异数据发送给所述用户设备，以便所述用户设备根据所述当前关键帧和所述差异数据获取所述帧的图形指令数据。
  13. 根据权利要求12所述的装置，其特征在于，还包括：
    更新模块，用于若所述判断模块判断符合，则将所述帧记录为当前关键帧；
    所述发送模块，还用于将所述帧的图形指令数据发送给所述用户设备。
  14. 根据权利要求12所述的装置，其特征在于，所述判断模块包括：
    计算单元，用于计算所述帧与所述当前关键帧的差异度；
    判断单元，用于根据所述差异度是否超出阈值范围判断所述帧是否符合关键帧设定条件。
  15. 根据权利要求12所述的装置，其特征在于，还包括：
    压缩模块，用于对所述差异数据进行压缩；
    所述发送模块用于将压缩后的差异数据发送给所述用户设备。
  16. 根据权利要求12-15任一项所述的装置，其特征在于，还包括：
    数据处理模块，用于捕获每一帧图形界面的绘制操作，产生图形指令数据，并将所述图形指令数据放入处理队列中；
    所述获取模块，用于从所述处理队列中获取所述帧的图形指令数据。
  17. 一种图形指令数据的处理装置，其特征在于，所述装置应用于云交互系统的用户设备，所述云交互系统包括服务器和所述用户设备；
    所述装置包括：
    接收模块，用于接收所述服务器发送的帧的数据；
    识别模块，用于识别所述帧是否为关键帧；
    确定模块，用于若所述识别模块识别结果为否，则获取当前关键帧，确定所述帧的数据是与所述当前关键帧的差异数据；
    还原模块，用于根据所述帧的数据和所述当前关键帧，获取所述帧的图形指令数据。
  18. 根据权利要求17所述的装置，其特征在于，还包括：
    更新模块，用于若所述识别模块识别为是，则确定所述帧的数据是所述帧的图形指令数据；将所述帧记录为当前关键帧。
  19. 根据权利要求17所述的装置，其特征在于，还包括：
    回放模块，用于回放所述帧的图形指令数据，生成所述帧的图形界面。
  20. 根据权利要求17所述的方法，其特征在于，所述帧的数据为压缩数据，所述接收模块还用于对所述压缩数据进行解压缩。
  21. 一种云交互系统，其特征在于，所述云交互系统包括服务器和用户设备，其中，
    所述服务器，用于获取帧的图形指令数据，判断所述帧是否符合关键帧设定条件，若不符合，将所述帧的图形指令数据与当前关键帧的图形指令数据进行对比，获得差异数据，将所述差异数据发送给所述用户设备；
    所述用户设备，用于接收所述服务器发送的所述帧的数据，识别所述帧是否为关键帧；若否，则获取当前关键帧，确定所述帧的数据是与所述当前关键帧的差异数据，根据所述帧的数据和所述当前关键帧，获取所述帧的图形指令数据。
  22. 根据权利要求21所述的系统，其特征在于，
    所述服务器，还用于判断所述帧符合关键帧设定条件时，将所述帧记录为当前关键帧，将所述帧的图形指令数据发送给所述用户设备；
    所述用户设备，还用于识别所述帧为关键帧时，将所述帧记录为当前关键帧。
PCT/CN2017/072538 2016-01-29 2017-01-25 图形指令数据的处理方法和装置及系统 WO2017129118A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP17743733.2A EP3410302B1 (en) 2016-01-29 2017-01-25 Graphic instruction data processing method, apparatus
US15/926,547 US10645391B2 (en) 2016-01-29 2018-03-20 Graphical instruction data processing method and apparatus, and system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610067797.6A CN105760238B (zh) 2016-01-29 2016-01-29 图形指令数据的处理方法和装置及系统
CN201610067797.6 2016-01-29

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/926,547 Continuation US10645391B2 (en) 2016-01-29 2018-03-20 Graphical instruction data processing method and apparatus, and system

Publications (1)

Publication Number Publication Date
WO2017129118A1 true WO2017129118A1 (zh) 2017-08-03

Family

ID=56342844

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/072538 WO2017129118A1 (zh) 2016-01-29 2017-01-25 图形指令数据的处理方法和装置及系统

Country Status (4)

Country Link
US (1) US10645391B2 (zh)
EP (1) EP3410302B1 (zh)
CN (1) CN105760238B (zh)
WO (1) WO2017129118A1 (zh)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105760238B (zh) 2016-01-29 2018-10-19 腾讯科技（深圳）有限公司 图形指令数据的处理方法和装置及系统
CN107977272A (zh) * 2016-10-25 2018-05-01 腾讯科技（深圳）有限公司 应用运行的方法及装置
US11100041B2 (en) * 2018-03-09 2021-08-24 Microsoft Technology Licensing, Llc Techniques for tracking independent hardware graphics processing unit (GPU) performance
CN108769715B (zh) * 2018-05-22 2022-05-13 深圳市腾讯网络信息技术有限公司 图形指令数据的处理方法及装置
CN110737669A (zh) * 2019-10-18 2020-01-31 北京百度网讯科技有限公司 数据存储方法、装置、设备及存储介质
CN113055729B (zh) * 2019-12-27 2024-05-28 中兴通讯股份有限公司 一种云化机顶盒图像差分处理方法及装置
CN113559506B (zh) * 2021-09-24 2022-01-25 深圳易帆互动科技有限公司 一种帧同步的自动化测试方法、装置及可读存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040151390A1 (en) * 2003-01-31 2004-08-05 Ryuichi Iwamura Graphic codec for network transmission
CN101031085A (zh) * 2007-03-30 2007-09-05 中国联合通信有限公司 一种移动终端帧动画的处理方法
CN102566910A (zh) * 2011-12-14 2012-07-11 广州杰赛科技股份有限公司 刷新虚拟桌面客户端屏显的方法及虚拟桌面系统
CN103440612A (zh) * 2013-08-27 2013-12-11 华为技术有限公司 一种gpu虚拟化中图像处理方法和装置
CN105760238A (zh) * 2016-01-29 2016-07-13 腾讯科技（深圳）有限公司 图形指令数据的处理方法和装置及系统

Family Cites Families (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5828370A (en) * 1996-07-01 1998-10-27 Thompson Consumer Electronics Inc. Video delivery system and method for displaying indexing slider bar on the subscriber video screen
JP3788677B2 (ja) * 1997-11-14 2006-06-21 富士通株式会社 連続録画システム及び方法並びに長時間録画プログラムを記録したコンピュータ読み取り可能な記録媒体
JPH11298890A (ja) * 1998-04-13 1999-10-29 Hitachi Ltd 画像データの圧縮又は伸張方法とその装置、並びに、それを用いた画像伝送システム及び監視システム
US7158681B2 (en) * 1998-10-01 2007-01-02 Cirrus Logic, Inc. Feedback scheme for video compression system
WO2002007440A2 (en) * 2000-07-15 2002-01-24 Filippo Costanzo Audio-video data switching and viewing system
US20020104094A1 (en) * 2000-12-01 2002-08-01 Bruce Alexander System and method for processing video data utilizing motion detection and subdivided video fields
US7916147B2 (en) * 2002-03-01 2011-03-29 T5 Labs Ltd. Centralised interactive graphical application server
US7539393B2 (en) * 2003-12-05 2009-05-26 Microsoft Corporation Method and system for reverse playback of compressed data
US20110157196A1 (en) * 2005-08-16 2011-06-30 Exent Technologies, Ltd. Remote gaming features
US8948260B2 (en) * 2005-10-17 2015-02-03 Qualcomm Incorporated Adaptive GOP structure in video streaming
US8692837B1 (en) * 2006-09-21 2014-04-08 Nvidia Corporation Screen compression for mobile applications
US8106909B2 (en) * 2007-10-13 2012-01-31 Microsoft Corporation Common key frame caching for a remote user interface
JP4978575B2 (ja) * 2008-06-25 2012-07-18 富士通株式会社 シンクライアントシステムにおける画像符号化方法及び画像符号化プログラム
US8473628B2 (en) * 2008-08-29 2013-06-25 Adobe Systems Incorporated Dynamically altering playlists
US8639086B2 (en) * 2009-01-06 2014-01-28 Adobe Systems Incorporated Rendering of video based on overlaying of bitmapped images
US8619198B1 (en) * 2009-04-28 2013-12-31 Lucasfilm Entertainment Company Ltd. Adjusting frame rates for video applications
US9049271B1 (en) * 2009-07-16 2015-06-02 Teradici Corporation Switch-initiated congestion management method
US8321905B1 (en) * 2009-10-02 2012-11-27 Adobe Systems Incorporated Fast switching of media streams
US8451906B1 (en) * 2011-02-25 2013-05-28 Adobe Systems Incorporated Reconstructing efficiently encoded video frames in a distributed video coding environment
US8908775B1 (en) * 2011-03-30 2014-12-09 Amazon Technologies, Inc. Techniques for video data encoding
WO2012159640A1 (en) * 2011-05-20 2012-11-29 Dream Chip Technologies Gmbh Method for transmitting digital scene description data and transmitter and receiver scene processing device
US8872895B2 (en) * 2011-09-30 2014-10-28 Deutsche Telekom Ag Real-time video coding using graphics rendering contexts
KR101335711B1 (ko) * 2012-01-20 2013-12-04 연세대학교 산학협력단 서버, 연산 처리 방법 및 연산 처리 시스템
KR20140013652A (ko) * 2012-07-26 2014-02-05 삼성전자주식회사 시스템 온 칩 및 이를 포함하는 전자 기기
WO2014065844A1 (en) * 2012-10-26 2014-05-01 Sheng Yang A cloud-based system for flash content streaming
JP5411385B1 (ja) * 2013-08-12 2014-02-12 株式会社 ディー・エヌ・エー ゲームを提供するサーバ及び方法
US10491916B2 (en) * 2013-10-01 2019-11-26 Advanced Micro Devices, Inc. Exploiting camera depth information for video encoding
US9271048B2 (en) * 2013-12-13 2016-02-23 The Directv Group, Inc. Systems and methods for immersive viewing experience
US9679348B2 (en) * 2014-05-15 2017-06-13 Disney Enterprises, Inc. Storage and compression methods for animated images
CN106717007B (zh) * 2014-07-30 2020-04-03 Sk 普兰尼特有限公司 云端流媒体服务器
US9436876B1 (en) * 2014-12-19 2016-09-06 Amazon Technologies, Inc. Video segmentation techniques
US9583144B2 (en) * 2015-02-24 2017-02-28 Plaay, Llc System and method for creating a sports video
US10244234B2 (en) * 2015-11-20 2019-03-26 Ellation, Inc. Adaptive compression rate control

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040151390A1 (en) * 2003-01-31 2004-08-05 Ryuichi Iwamura Graphic codec for network transmission
CN101031085A (zh) * 2007-03-30 2007-09-05 中国联合通信有限公司 一种移动终端帧动画的处理方法
CN102566910A (zh) * 2011-12-14 2012-07-11 广州杰赛科技股份有限公司 刷新虚拟桌面客户端屏显的方法及虚拟桌面系统
CN103440612A (zh) * 2013-08-27 2013-12-11 华为技术有限公司 一种gpu虚拟化中图像处理方法和装置
CN105760238A (zh) * 2016-01-29 2016-07-13 腾讯科技（深圳）有限公司 图形指令数据的处理方法和装置及系统

Also Published As

Publication number Publication date
EP3410302C0 (en) 2023-08-09
CN105760238A (zh) 2016-07-13
US10645391B2 (en) 2020-05-05
EP3410302A4 (en) 2020-03-25
EP3410302A1 (en) 2018-12-05
US20180213232A1 (en) 2018-07-26
CN105760238B (zh) 2018-10-19
EP3410302B1 (en) 2023-08-09

Similar Documents

Publication Publication Date Title
WO2017129118A1 (zh) 图形指令数据的处理方法和装置及系统
US11909984B2 (en) Video encoding and decoding for cloud gaming
CN111954053B (zh) 获取蒙版帧数据的方法、计算机设备及可读存储介质
US20160198208A1 (en) Desktop-cloud-based media control method and device
CN110971929A (zh) 云游戏视频处理方法、电子设备及存储介质
WO2022257699A1 (zh) 图像画面显示方法、装置、设备、存储介质及程序产品
US20140321553A1 (en) Systems and methods for using pre-calculated block hashes for image block matching
US9438917B2 (en) System and method for adaptive bandwidth management
CN112184739B (zh) 一种图像检测方法、装置及计算机可读存储介质
US9940732B2 (en) Implementing reduced video stream bandwidth requirements when remotely rendering complex computer graphics scene
CN111491208B (zh) 视频处理方法、装置、电子设备及计算机可读介质
CN110401866B (zh) 直播视频的显示方法、装置、终端和存储介质
CN112055254A (zh) 视频播放的方法、装置、终端及存储介质
CN112929728A (zh) 一种视频渲染方法、装置、系统、电子设备及存储介质
WO2023024563A1 (zh) 视频处理方法、服务器和计算机可读存储介质
CN111359220B (zh) 游戏广告生成方法、装置和计算机设备
JP2016524730A (ja) 情報処理装置並びにその制御方法、及びプログラム
CN113825020B (zh) 视频清晰度切换方法、装置、设备、存储介质及程序产品
US20240098316A1 (en) Video encoding method and apparatus, real-time communication method and apparatus, device, and storage medium
CN113082703A (zh) 一种光标显示方法、装置及介质
TW201842775A (zh) 用於在視訊編碼中的順延後處理的系統及方法
WO2024124670A1 (zh) 视频播放方法、装置、计算机设备及计算机可读存储介质
CN113810755B (zh) 全景视频预览的方法、装置、电子设备及存储介质
KR102231875B1 (ko) 응용 프로그램의 스트리밍 영상 제공 장치 및 방법
CN111954082B (zh) 蒙版文件结构、蒙版文件的读取方法、计算机设备及可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17743733

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE