WO2019015543A1 - 一种虚拟桌面数据显示的方法及系统 - Google Patents

一种虚拟桌面数据显示的方法及系统 Download PDF

Info

Publication number
WO2019015543A1
WO2019015543A1 PCT/CN2018/095764 CN2018095764W WO2019015543A1 WO 2019015543 A1 WO2019015543 A1 WO 2019015543A1 CN 2018095764 W CN2018095764 W CN 2018095764W WO 2019015543 A1 WO2019015543 A1 WO 2019015543A1
Authority
WO
WIPO (PCT)
Prior art keywords
desktop
frame
small
virtual desktop
server
Prior art date
Application number
PCT/CN2018/095764
Other languages
English (en)
French (fr)
Inventor
刘少华
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Priority to EP18835791.7A priority Critical patent/EP3657316A4/en
Priority to RU2019143752A priority patent/RU2742860C1/ru
Publication of WO2019015543A1 publication Critical patent/WO2019015543A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/003—Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto
    • G09G5/006—Details of the interface to the display terminal
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/08—Protocols specially adapted for terminal emulation, e.g. Telnet
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • G06F3/1462—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay with means for detecting differences between the image stored in the host and the images displayed on the remote displays
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • G06F9/452—Remote windowing, e.g. X-Window System, desktop virtualisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/167—Position within a video image, e.g. region of interest [ROI]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/40—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
    • H04N7/147—Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/15—Conference systems
    • H04N7/152—Multipoint control units therefor
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2350/00—Solving problems of bandwidth in display systems
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00—Aspects of data communication
    • G09G2370/02—Networking aspects
    • G09G2370/022—Centralised management of display operation, e.g. in a server instead of locally
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00—Aspects of data communication
    • G09G2370/10—Use of a protocol of communication by packets in interfaces along the display data pipeline
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00—Aspects of data communication
    • G09G2370/16—Use of wireless transmission of display information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/142—Detection of scene cut or scene change
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/423—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements

Definitions

  • the present disclosure relates to the field of wireless communication technologies, and in particular, to a method and system for displaying virtual desktop data.
  • Cloud desktop system is an important achievement of cloud computing technology in the fields of government, enterprise and education.
  • the smoothness of cloud desktop operations will directly affect the user experience, and it is also the most basic and important performance indicator of cloud desktop systems.
  • Each cloud client user logs in to his or her own cloud desktop to ensure the smoothness of desktop operations. Its fluency is limited by two basic conditions: network bandwidth and hardware performance of the server, especially the processing power of the CPU (Central Processing Unit).
  • CPU Central Processing Unit
  • the first is to send the desktop data to the cloud client without any processing or after a simple compression encoding process (such as Mjpeg). Because there is no compression or a low compression ratio, the CPU pressure on the server is relatively small. When the network bandwidth is sufficient, the desktop data can be quickly sent to the client for processing, and the client can also easily perform desktop data. Processing (decompression decoding is faster or not necessary at all).
  • the shortcomings are also obvious, that is, the network bandwidth requirements are too high, because the amount of desktop data transmitted is relatively large.
  • a server host will map several virtual cloud desktops at the same time. When multiple users log in to the cloud desktop at the same time, it will inevitably occupy the network bandwidth, causing the desktop data transmission rate to be restricted, and desktop operation delays and other problems. .
  • the second solution is to perform high compression ratio encoding (such as H264/H265) on the entire desktop data (desktop frame), and then transmit it to the client for processing. Because of the high compression, the amount of desktop data will be significantly reduced. It can significantly reduce the bandwidth usage.
  • high compression ratio coding is a great burden on the server CPU, and the time-consuming and efficient capability of decompressing and decoding the client is also a great challenge.
  • a method and system for displaying virtual desktop data is provided.
  • a method for displaying virtual desktop data including:
  • the server divides the virtual desktop into multiple virtual desktop blocks of equal size according to the virtual desktop resolution, so that each virtual desktop block corresponds to one small desktop stream and one small desktop frame;
  • the server compresses and encodes a small desktop frame corresponding to each virtual desktop block, and encapsulates into a small stream frame corresponding to the small desktop stream;
  • the server sends the encapsulated small stream frame to the client, so that the client reproduces the virtual desktop image of each frame of the server by decompressing, integrating and displaying the received small stream frame.
  • a system for displaying virtual desktop data including a server and a client, where the server includes:
  • the segmentation module is configured to divide the virtual desktop into a plurality of virtual desktop blocks of equal size according to the virtual desktop resolution, so that each virtual desktop block corresponds to one small desktop stream and one small desktop frame;
  • the compression coding module is configured to perform compression coding on a small desktop frame corresponding to each virtual desktop block, and encapsulate the small stream frame corresponding to the small desktop stream;
  • the sending module is configured to send the encapsulated small stream frame to the client, so that the client reproduces the virtual desktop image of each frame of the server by decompressing, integrating, and displaying the received small stream frame.
  • a computer readable storage medium storing computer executable instructions that, when executed by a processor, implement the method described above.
  • FIG. 1 is a flowchart of a method for displaying virtual desktop data according to an embodiment of the present disclosure
  • FIG. 2 is a schematic diagram of a system for displaying virtual desktop data according to an embodiment of the present disclosure
  • FIG. 3 is a structural diagram of a virtual desktop data display system according to an embodiment of the present disclosure.
  • FIG. 4 is a flowchart of generating server stream data according to an embodiment of the present disclosure
  • FIG. 5 is a flowchart of a client multi-stream process according to an embodiment of the present disclosure.
  • FIG. 1 is a flowchart of a method for displaying virtual desktop data according to an embodiment of the present disclosure. As shown in FIG. 1 , the method includes:
  • Step S101 The server divides the virtual desktop into multiple virtual desktop blocks of equal size according to the virtual desktop resolution, so that each virtual desktop block corresponds to one small desktop stream and one small desktop frame;
  • a desktop image in each desktop block area is a small desktop frame, and small desktop frames in different desktop block areas belong to different small desktop streams.
  • Step S102 The server compresses and encodes a small desktop frame corresponding to each virtual desktop block, and encapsulates into a small stream frame corresponding to the small desktop stream.
  • Each small stream frame contains information about the small desktop stream, such as the stream ID.
  • Step S103 The server sends the encapsulated small stream frame to the client, so that the client reproduces the virtual desktop screen of each frame of the server by decompressing, integrating, and displaying the received small stream frame.
  • the server further includes: the server detects the current frame desktop screen of each virtual desktop block, and determines the current frame desktop screen of the virtual desktop block. Whether there is a change compared with the desktop picture of the previous frame; if it is determined that the current frame desktop picture of the virtual desktop block changes compared with the desktop picture of the previous frame, the server corresponds to a frame corresponding to the virtual desktop block.
  • the small desktop frame is compression-encoded, encapsulated into a small stream frame corresponding to the small desktop stream, and the small stream frame is sent to the client.
  • the server performs compression coding on a small desktop frame corresponding to the virtual desktop block, including: the server creates a compression coding thread and an encoder for each virtual desktop block according to the segmented virtual desktop block; The end uses a compression coding thread and an encoder created for the virtual desktop block to compress and encode a small desktop frame corresponding to the virtual desktop block.
  • the client re-compresses, integrates, and displays the received small stream frame, and the virtual desktop image of each frame of the server is: the client receives the small stream frame sent by the server, and is The small stream frame creates a decoding display thread and a decoder, and sends the received small stream frame to the corresponding buffer queue; the client uses the decoding display thread and the decoder created for the small stream frame, and the corresponding buffer queue
  • the small stream frame in the process performs decompression, integration, and display processing, and reproduces each frame of the virtual desktop screen of the server.
  • the client uses the decoding display thread and the decoder created for the small stream frame to decompress, integrate, and display the small stream frame in the corresponding buffer queue, including: the client uses the small The decoding display thread created by the flow frame extracts the small stream frame from the corresponding buffer queue, and decompresses the small stream frame by using a decoder to obtain desktop transcoding data corresponding to the small stream frame; the client will All desktop transcoding data is integrated to obtain the entire desktop transcoding data, and the obtained desktop transcoding data is displayed and processed.
  • the server 201 includes a server 201 and a client 202.
  • the server 201 includes a segmentation module 2011 configured to be distinguished according to the virtual desktop.
  • the compression encoding module 2012 is set to pass each virtual desktop A small desktop frame corresponding to the block is compression-encoded and encapsulated into a small stream frame corresponding to the small desktop stream;
  • the sending module 2013 is configured to send the encapsulated small stream frame to the client, so that the client passes the pair
  • the received small stream frame is subjected to decompression, integration and display processing, and each virtual desktop picture of the server is reproduced.
  • the sending module 2013 includes: a detecting unit, configured to determine whether a desktop frame of the current frame of the virtual desktop block is compared with a desktop image of a previous frame by detecting a current frame desktop image of each virtual desktop block. a sending unit configured to: when determining that the current frame desktop picture of the virtual desktop block changes compared to the previous frame desktop picture, the server compresses a small desktop frame corresponding to the virtual desktop block The encoding is encapsulated into a small stream frame corresponding to the small desktop stream, and the small stream frame is sent to the client.
  • the compression encoding module 2012 includes: a creating unit, configured to create a compression encoding thread and an encoder for each virtual desktop block according to the divided virtual desktop blocks; and a compression encoding unit configured to be utilized as the virtual desktop block The compressed coding thread and the encoder are created, and a small desktop frame corresponding to the virtual desktop block is compression-encoded.
  • the client 202 includes: a creating and buffering module 2021, configured to create a decoding display thread and a decoder for the small stream frame according to the received small stream frame, and send the received small stream frame into the small stream frame. Going to the corresponding cache queue; the processing and display module 2022 is configured to decompress, integrate, and display the small stream frame in the corresponding cache queue by using the decoded display thread and the decoder created for the small stream frame. Reproduces each frame of the virtual desktop screen of the server.
  • a creating and buffering module 2021 configured to create a decoding display thread and a decoder for the small stream frame according to the received small stream frame, and send the received small stream frame into the small stream frame. Going to the corresponding cache queue; the processing and display module 2022 is configured to decompress, integrate, and display the small stream frame in the corresponding cache queue by using the decoded display thread and the decoder created for the small stream frame. Reproduces each frame of the virtual desktop screen of the server.
  • the processing and display module 2022 includes: a processing unit, configured to use the decoding display thread created for the small stream frame to take the small stream frame from a corresponding cache queue, and use the decoder to pair the small stream frame Performing a decompression process to obtain desktop transcoding data corresponding to the small stream frame; the display unit is configured to integrate all the desktop transcoding data, obtain the entire desktop transcoding data, and transcode the obtained entire desktop The data is displayed and processed.
  • the server includes a server and a client.
  • the server includes a flow message generating module and a stream data generating module.
  • the flow message receiving processing module, the stream data receiving module and the decoding display module are included.
  • the flow message generating module is responsible for forming a control message related to the desktop stream, and mainly refers to a flow creation message and a flow destruction message.
  • Each block of equal size corresponds to a small stream, and each stream has its own stream creation and destruction message.
  • the stream creation message includes the following information: stream ID that is different from other streams; area coordinates corresponding to the stream; picture pixel size; number of blocks of the entire desktop; codec mode.
  • the stream destruction message indicates the corresponding small stream ID.
  • the stream data generating module is responsible for dividing the desktop into blocks, and then compressing and encoding the pictures of the blocks in a multi-threaded manner. Then, the encapsulated data stream will contain the following information: small desktop frame data; stream ID; the serial number of the current desktop frame, and the number of small streams in which the current frame changes.
  • the flow message receiving and processing module is responsible for receiving and processing a control message related to the server flow.
  • Receive stream creation messages create their own stream objects, each stream object contains its own area information, decoder, data receiving and forwarding thread, and allocate a buffer space to store the entire desktop transcoded data, where each stream is in turn Take up one of the space segments. After all the changed small stream frames have been transcoded, the data of the entire cache space is displayed.
  • Receive stream destruction messages destroy small streams, decoders, threads and other resources.
  • the stream data receiving module is configured to receive the small stream frames sequentially sent by the server end and send them to the corresponding small stream frame buffer queue in real time.
  • the decoding display module includes a decoding display thread corresponding to each small stream, and each thread takes out a small stream frame from the corresponding frame buffer queue in real time, and sends it to the decoder for decoding and transcoding operations.
  • the transcoding buffer space is integrated to perform the upper screen.
  • FIG. 4 is a flowchart of generating server flow data according to an embodiment of the present invention, as shown in FIG. 4, including:
  • 401 Divide the entire desktop into a plurality of blocks of equal size according to the resolution of the desktop. For example, for a desktop of 1440*900, a total of 16 blocks of 2 rows and 8 columns may be divided.
  • Each block creates a separate compression coding thread and an encoder.
  • Each block build stream creates a message.
  • Each block encapsulation respective stream creation message is sent to the client, and the message content includes: a stream ID; an area coordinate corresponding to the stream; a picture pixel size; a block number of the entire desktop; a codec mode.
  • the thread corresponding to the change block compresses and encodes the picture of the block.
  • the thread encapsulates a small stream frame.
  • the content of the small stream frame format is: small desktop stream frame data; a stream ID; a sequence number of the current desktop frame, and a number of small streams in which the current frame changes.
  • the small stream frames are sequentially sent to the client in the order of the stream ID.
  • the overall encoding of the entire desktop is changed to a plurality of regions separately, which improves the coding efficiency.
  • the number of small streams that change and the ID of the small stream are marked in the small stream frame, which can prevent the client from being confused when processing the multi-stream data.
  • FIG. 5 is a flowchart of a multi-stream processing of a client provided by an embodiment of the present disclosure. As shown in FIG. 5, the method includes:
  • the client receives and parses the small stream creation message sent by the server. Parsing the codec format, the desktop pixel size, the number of partitions, the stream ID, and the location of the region corresponding to the block.
  • the client creates a small stream object according to the parsed related parameters.
  • the client receives a set of small stream frame data sent by the server.
  • the set of stream data belongs to the same desktop frame, and carries the frame sequence number, and at the same time, the stream data can know how many changed small streams of the desktop frame.
  • the small stream frame data in the stream data is placed in the corresponding buffer queue according to the stream ID.
  • the decoding display thread determines whether the stream has been destroyed, or the thread directly exits.
  • the decoding display thread determines whether there is data in the cache, and if not, sleeps for a certain time to perform step 505.
  • the decoding display thread determines, according to the sequence number of the previous frame of the mark, whether the previous frame has completed the upper screen, and if not, sleeps for a period of time to perform step 505. If the upper screen has been completed, proceed to the next step.
  • the decoding display thread extracts a small stream frame from the cache queue.
  • step 505 is performed. If it continues, it will continue to execute.
  • the validity criterion is that the frame sequence number is compared with the sequence number N of the frame on which the previous screen was completed, and is larger than N, otherwise it is invalid.
  • the decoding display thread sends the small stream frame to the decoder for decoding.
  • the decoder transcodes the small stream frame.
  • step 505 Determine whether each changed small stream frame has completed transcoding, and if it has been completed, execute downward, otherwise step 505 is performed.
  • the decoder marks the current frame sequence number as the frame sequence number of the previous upper screen, and acts on step 509.
  • the decoder transcodes the entire desktop onto the screen.
  • the decoding display thread marks the frame as the previous frame and acts on step 507.
  • the decoding display thread deletes the small stream frame buffer, releases the memory resource, and skips to step 505.
  • the client receives the small stream destruction message
  • the corresponding decoding display thread will exit, the decoder will be destroyed, the cache queue will be released, and the small stream object will also be deleted.
  • Embodiments of the present disclosure also provide a computer readable storage medium storing computer executable instructions that, when executed by a processor, implement the methods described above.
  • the beneficial effects of the disclosure are as follows: the high compression ratio encoding mode ensures that the desktop frame reduces the bandwidth resource consumption as much as possible in the network transmission process; under the multi-threading scheme multi-threading mechanism, the efficiency of the desktop frame encoding compression is greatly improved. Make full use of the server's CPU resources; at the same time, the client's multi-threading mechanism also improves the efficiency of the client's decoding and transcoding of the desktop frame; in addition, the multi-stream mechanism's small stream frame carries the desktop frame serial number information and changes. The number of streams and the stream ID, the client only needs to decode and transcode the stream frame, which relieves the pressure on the client. At the same time, it cooperates with the decoding logic of the decoding display thread to ensure that the desktop is displayed frame by frame and does not cause the screen to be disordered.
  • the server cuts the picture (desktop frame) of the virtual desktop into a plurality of equal-sized blocks according to the desktop resolution, and then dynamically compresses the pictures of each individual block to form a high-compression ratio code to form a
  • a small desktop data stream is sent to the client, and the client decodes and transcodes each small stream frame in a parallel manner and finally displays it on the display device.
  • the server compares the current desktop frame with the previous frame desktop, marks the block that changes from the previous frame, and only encodes the changed block image into a small stream frame and sends it to the client.
  • the changed small stream frame performs operations such as decoding, transcoding, and the like, and then merges the unchanged portions of the previous desktop frame for display, which can alleviate the pressure on the client.
  • the data processing of the blocks can be completed in a multi-thread parallel manner. Making full use of CPU resources can also reduce CPU usage and improve coding efficiency.
  • the client is also the same, and the disclosed scheme occupies less network bandwidth, so the fluency of the desktop operation is comprehensively improved.
  • computer storage medium includes volatile and nonvolatile, implemented in any method or technology for storing information, such as computer readable instructions, data structures, program modules or other data. Sex, removable and non-removable media.
  • Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridge, magnetic tape, magnetic disk storage or other magnetic storage device, or may Any other medium used to store the desired information and that can be accessed by the computer.
  • communication media typically includes computer readable instructions, data structures, program modules or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and can include any information delivery media. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Hardware Design (AREA)
  • Information Transfer Between Computers (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

提供了一种虚拟桌面数据显示的方法及系统，所述方法包括：服务端根据虚拟桌面分辨率，将所述虚拟桌面分割成大小相等的多个虚拟桌面块，使每个虚拟桌面块对应一条小桌面流和一幅幅小桌面帧（S101）；服务端通过对每个虚拟桌面块对应的一幅幅小桌面帧进行压缩编码，封装成对应于所述小桌面流的小流帧（S102）；服务端将所封装的小流帧发送给客户端，以便客户端通过对收到的所述小流帧进行解压缩、整合和显示处理，再现服务端的每帧虚拟桌面画面（S103）。

Description

一种虚拟桌面数据显示的方法及系统 技术领域
本公开涉及无线通讯技术领域，特别涉及一种虚拟桌面数据显示的方法及系统。
背景技术
云桌面系统，是云计算技术越来越广泛的应用于政企、教育等领域的一个重要的成果。而云桌面操作的流畅性，将直接影响到用户体验，同时也是云桌面系统最基本、最重要的一项性能指标。每个云客户端用户各自登陆自己的云桌面，要保证桌面操作的流畅性。而其流畅性受两个最基本条件的限制：网络带宽以及服务端的硬件性能尤其是CPU(Central Processing Unit，中央处理器)的处理能力。
目前市场上主流的桌面数据显示方案有如下几种：
第一种是将桌面数据不经过任何处理或者经过简单的压缩编码处理(如Mjpeg)后发送给云客户端。因为没有经过压缩或者压缩率比较低，所以服务端的CPU压力比较小，在网络带宽足够的情况下，桌面数据能很快发送给客户端进行处理，同时客户端也能比较轻松的对桌面数据进行处理(解压解码比较快或者根本无需进行)。其缺点也同样比较明显，即对网络带宽要求太高，因为传输的桌面数据量相对比较大。另外，一个服务端主机同时将映射出好几个虚拟云桌面，在多个用户同时登陆云桌面的情况下，势必抢占网络带宽，导致桌面数据传输速率受到制约，出现桌面操作延时卡顿等问题。
第二种方案是对整张桌面数据的画面(桌面帧)进行高压缩比编码(如H264/H265等)，再传送给客户端处理，因为经过高度压缩，桌面数据量会很明显的减少，能很明显的降低带宽占用率。其缺点是，高压缩比编码对服务端CPU是一种很大的负担，同时对客户端的解压缩解码的耗时能力和效率能力也是一个很大的挑战。
发明概述
以下是对本文详细描述的主题的概述。本概述并非是为了限制权利要求的保护范围。
针对服务端对整张桌面数据的画面(桌面帧)进行高压缩比编码而造成CPU负担很大、同时对客户端的解压缩解码能力要求也很高、效率也相对低下的问题，本公开实施例提供了一种虚拟桌面数据显示的方法及系统。
根据本公开实施例，提供了一种虚拟桌面数据显示的方法，包括：
服务端根据虚拟桌面分辨率，将所述虚拟桌面分割成大小相等的多个虚拟桌面块，使每个虚拟桌面块对应一条小桌面流和一幅幅小桌面帧；
服务端通过对每个虚拟桌面块对应的一幅幅小桌面帧进行压缩编码，封装成对应于所述小桌面流的小流帧；
服务端将所封装的小流帧发送给客户端，以便客户端通过对收到的所述小流帧进行解压缩、整合和显示处理，再现服务端的每帧虚拟桌面画面。
根据本公开实施例，提供了一种虚拟桌面数据显示的系统，包括服务端和客户端，其中服务端包括：
分割模块，设置为根据虚拟桌面分辨率，将所述虚拟桌面分割成大小相等的多个虚拟桌面块，使每个虚拟桌面块对应一条小桌面流和一幅幅小桌面帧；
压缩编码模块，设置为通过对每个虚拟桌面块对应的一幅幅小桌面帧进行压缩编码，封装成对应于所述小桌面流的小流帧；
发送模块，设置为将所封装的小流帧发送给客户端，以便客户端通过对收到的所述小流帧进行解压缩、整合和显示处理，再现服务端的每帧虚拟桌面画面。
根据本公开实施例，还提供了一种计算机可读存储介质，存储有计算机可执行指令，所述计算机可执行指令被处理器执行时实现以上描述的方法。
在阅读并理解了附图和详细描述后，可以明白其他方面。
附图概述
图1是本公开实施例提供的一种虚拟桌面数据显示的方法流程图；
图2是本公开实施例提供的一种虚拟桌面数据显示的系统示意图；
图3是本公开实施例提供的虚拟桌面数据显示系统结构图；
图4是本公开实施例提供的服务端流数据产生流程图；
图5是本公开实施例提供的客户端多流处理流程图。
详述
以下结合附图对本公开的示例性实施例进行详细说明，可以理解，以下所说明的示例性实施例仅用于说明和解释本公开，并不用于限定本公开。
图1是本公开实施例提供的一种虚拟桌面数据显示的方法流程图，如图1所示，包括：
步骤S101：服务端根据虚拟桌面分辨率，将所述虚拟桌面分割成大小相等的多个虚拟桌面块，使每个虚拟桌面块对应一条小桌面流和一幅幅小桌面帧；
每一个桌面块区域内的一幅桌面画面就是一个小桌面帧，不同桌面块区域内的小桌面帧分属不同的小桌面流。
步骤S102：服务端通过对每个虚拟桌面块对应的一幅幅小桌面帧进行压缩编码，封装成对应于所述小桌面流的小流帧；
每一个小流帧都包含所属小桌面流的相关信息，如流ID等。
步骤S103：服务端将所封装的小流帧发送给客户端，以便客户端通过对收到的所述小流帧进行解压缩、整合和显示处理，再现服务端的每帧虚拟桌面画面。
其中，所述服务端将所封装的小流帧发送给客户端之后，还包括：服务端通过对每个虚拟桌面块的当前帧桌面画面进行检测，判断所述虚拟桌面块的当前帧桌面画面与上一帧桌面画面相比是否发生变化；若判断所述虚拟桌面块的当前帧桌面画面与上一帧桌面画面相比发生了变化，则服务端对所述虚拟桌面块对应的一幅幅小桌面帧进行压缩编码，封装成对应于所述小桌面 流的小流帧，并将所述小流帧发送给客户端。
其中，所述服务端对所述虚拟桌面块对应的一幅幅小桌面帧进行压缩编码包括：服务端根据所分割的虚拟桌面块，为每个虚拟桌面块创建压缩编码线程和编码器；服务端利用为所述虚拟桌面块创建的压缩编码线程和编码器，对所述虚拟桌面块对应的一幅幅小桌面帧进行压缩编码。
其中，所述客户端通过对收到的所述小流帧进行解压缩、整合和显示处理，再现服务端的每帧虚拟桌面画面包括：客户端接收服务端发送的小流帧，并为所述小流帧创建解码显示线程和解码器，并将所接收的小流帧送入到对应的缓存队列中；客户端利用为所述小流帧创建的解码显示线程和解码器，将对应缓存队列中的所述小流帧进行解压缩、整合和显示处理，再现服务端的每帧虚拟桌面画面。
其中，所述客户端利用为所述小流帧创建的解码显示线程和解码器，将对应缓存队列中的所述小流帧进行解压缩、整合和显示处理包括：客户端利用为所述小流帧创建的解码显示线程从对应缓存队列中取出所述小流帧，并利用解码器对所述小流帧进行解压缩处理，得到对应所述小流帧的桌面转码数据；客户端将所有的桌面转码数据进行整合处理，得到整个桌面转码数据，并将所得到的整个桌面转码数据进行显示处理。
图2是本公开实施例提供的一种虚拟桌面数据显示的系统示意图，如图2所示，包括服务端201和客户端202，其中服务端201包括：分割模块2011，设置为根据虚拟桌面分辨率，将所述虚拟桌面分割成大小相等的多个虚拟桌面块，使每个虚拟桌面块对应一条小桌面流和一幅幅小桌面帧；压缩编码模块2012，设置为通过对每个虚拟桌面块对应的一幅幅小桌面帧进行压缩编码，封装成对应于所述小桌面流的小流帧；发送模块2013，设置为将所封装的小流帧发送给客户端，以便客户端通过对收到的所述小流帧进行解压缩、整合和显示处理，再现服务端的每帧虚拟桌面画面。
其中，所述发送模块2013包括：检测单元，设置为通过对每个虚拟桌面块的当前帧桌面画面进行检测，判断所述虚拟桌面块的当前帧桌面画面与上一帧桌面画面相比是否发生变化；发送单元，设置为当判断所述虚拟桌面块的当前帧桌面画面与上一帧桌面画面相比发生了变化，则服务端对所述虚 拟桌面块对应的一幅幅小桌面帧进行压缩编码，封装成对应于所述小桌面流的小流帧，并将所述小流帧发送给客户端。
其中，所述压缩编码模块2012包括：创建单元，设置为根据所分割的虚拟桌面块，为每个虚拟桌面块创建压缩编码线程和编码器；压缩编码单元，设置为利用为所述虚拟桌面块创建的压缩编码线程和编码器，对所述虚拟桌面块对应的一幅幅小桌面帧进行压缩编码。
其中，所述客户端202包括：创建及缓存模块2021，设置为根据接收到的小流帧，并为所述小流帧创建解码显示线程和解码器，并将所接收的小流帧送入到对应的缓存队列中；处理及显示模块2022，设置为利用为所述小流帧创建的解码显示线程和解码器，将对应缓存队列中的所述小流帧进行解压缩、整合和显示处理，再现服务端的每帧虚拟桌面画面。
其中，所述处理及显示模块2022包括：处理单元，设置为利用为所述小流帧创建的解码显示线程从对应缓存队列中取出所述小流帧，并利用解码器对所述小流帧进行解压缩处理，得到对应所述小流帧的桌面转码数据；显示单元，设置为将所有的桌面转码数据进行整合处理，得到整个桌面转码数据，并将所得到的整个桌面转码数据进行显示处理。
图3是本公开实施例提供的虚拟桌面数据显示系统结构图，如图3所示，包括服务端和客户端；其中所述服务端包括流消息发生模块和流数据产生模块；所述客户端包括流消息接收处理模块、流数据接收模块以及解码显示模块。
所述流消息发生模块负责形成与桌面流相关的控制消息，主要指流创建消息和流销毁消息。
大小相等的每一个块对应一个小流，每一个小流都会有各自的流创建和销毁消息。
流创建消息包含如下信息：区别于其他流的流ID；流对应的区域坐标；画面像素大小；整个桌面的分块数；编解码方式等。
流销毁消息标示对应的小流ID。
所述流数据产生模块负责将桌面进行分块，然后再用多线程的方式将各 个块的画面进行压缩编码。然后进行封装，封装后的数据流将包含以下信息：小桌面帧数据；流ID；当前桌面帧的序列号，当前帧发生变化的小流的数目。
所述流消息接收处理模块负责接收到服务端流相关的控制消息并处理。
接收流创建消息创建各自的流对象，每个小流对象包含各自的区域信息，解码器，数据接收转发线程，以及分配一段缓存空间来存放整个桌面转码后的数据，其中每个小流依次占用其中某一个空间段。等变化的小流帧全部完成转码后，再将整个缓存空间的数据进行显示。
接收流销毁消息，销毁小流，解码器，线程等资源。
所述流数据接收模块负责接收服务端依次传来的小流帧实时的送入到对应的小流帧缓存队列中。
所述解码显示模块包含每个小流对应的解码显示线程，每个线程实时从对应的帧缓存队列中取出小流帧，送入解码器进行解码、转码操作。当整个桌面所有的变化的小流帧都完成了解码、转码后，再整合转码缓存空间进行上屏。
图4是本发明实施例提供的服务端流数据产生流程图，如图4所示，包括：
401、根据桌面分辨率把整个桌面划分为大小相等个若干个块，比如对于1440*900的桌面，可以划分2行8列共16个块。
402、每个块创建独立的压缩编码线程以及编码器。
403、每个块构建流创建消息。
404、每个块封装各自的流创建消息发送至客户端，消息内容包括：流ID；流对应的区域坐标；画面像素大小；整个桌面的分块数；编解码方式等。
405、动态计算当前桌面画面相比上一帧桌面画面发生变化的区域，落在哪些块内。
406、变化块所对应的线程对块的画面进行压缩编码。
407、上述线程封装小流帧；小流帧格式包含的内容为：小桌面流帧数据； 流ID；当前桌面帧的序列号，当前帧发生变化的小流的数目。
408、小流帧按照流ID顺序依次发送至客户端。
依据上述流程，将整个桌面进行整体编码的方式改为若干个区域单独编码，提升了编码效率。同时小流帧中标记了发生变化的小流个数以及小流ID，能防止客户端处理多流数据时产生混乱。
图5是本公开实施例提供的客户端多流处理流程图，如图5所示，包括：
501、客户端接收并解析服务端发来的小流创建消息。解析出编解码格式、桌面像素大小、所分块的数目、所在流ID，以及块对应的区域位置等。
502、客户端根据解析出来的相关各参数，创建小流对象。
503、在流对象中创建对应的解码显示线程以及解码器，同时会创建一段缓存空间来存放整个桌面转码数据，其中每个小流依次占用其中某一个空间段。另外还要开辟存放小流帧数据的缓存队列。解码显示线程会在创建的同时运行，实时从对应的缓存队列中取出小流帧数据。
504、客户端接收到服务端发过来的一组小流帧数据。该组流数据属于同一个桌面帧，并携带了帧序列号，同时从流数据中能获知该桌面帧一共有多少个变化的小流。根据流ID将流数据中的小流帧数据放入对应的缓存队列。
505、解码显示线程判断流是否已经被销毁，否则线程直接退出。
506、解码显示线程判断缓存中是否有数据，如果没有则休眠一定时间执行步骤505。
507、解码显示线程根据标记的上一帧的序列号判断上一帧是否已经完成上屏，如果没有则休眠一段时间执行步骤505。如果已经完成上屏，则继续往下执行。
508、解码显示线程从缓存队列中取出小流帧。
509、根据帧序列号判断有效性。如果无效则直接丢弃然后执行步骤505。如果有效继续往下执行。其有效性判断依据是帧序列号与上一次完成上屏的帧的序列号N进行比较，比N大则为有效，否则为无效。
510、解码显示线程将小流帧送入解码器解码。
511、解码器将小流帧进行转码。
512、转码后放入桌面转码数据缓存空间的对应区间。
513、判断各个变化的小流帧是否都已经完成了转码，如果已经完成，则往下执行，否则执行步骤505。
514、解码器标记当前帧序列号为上一次完成上屏的帧序列号，作用于步骤509。
515、解码器将整个桌面转码数据上屏。
516、解码显示线程标记该帧为上一帧，作用于步骤507。
517、解码显示线程删除该小流帧缓存，释放内存资源，跳至步骤505。
518、当客户端收到小流销毁消息，对应的解码显示线程会退出，解码器会销毁，缓存队列会释放，小流对象也将删除。
本公开实施例还提供了一种计算机可读存储介质，存储有计算机可执行指令，所述计算机可执行指令被处理器执行时实现以上描述的方法。
本公开的有益效果为：高压缩比的编码方式保证了桌面帧在网络传输过程中尽可能的降低带宽资源的消耗；多流方案的多线程机制下，桌面帧编码压缩的效率得到大大提升，充分利用了服务端的CPU资源；同时客户端的多线程机制也提高了客户端对桌面帧的解码转码的效率；另外，多流机制的小流帧中携带了桌面帧序列号信息、发生变化的小流数目以及流ID，客户端只需解码和转码小流帧，缓解了客户端的压力，同时配合解码显示线程的保护逻辑，保证桌面是逐帧显示，不至于导致画面错乱。
根据本公开实施例提供的方案，服务端根据桌面分辨率将虚拟桌面的画面(桌面帧)先切割成若干大小相等的块，然后将每个单独块的画面进行动态高压缩比编码，形成一个个小型的桌面数据流，发给客户端，客户端使用并行的方式将各个小流帧各自解码、转码最后统一在显示设备上显示。同时服务端会将当前桌面帧与上一帧桌面进行对比，标记相对上一帧桌面发生变化的块，只将变化的块的画面编码成小流帧发送给客户端，如此，客户端只要将变化的小流帧进行解码、转码等操作，然后合并上一桌面帧的未发生变化 的部分进行显示，如此能减缓客户端的压力。
根据本公开实施例提供的方案，将桌面帧分成若干个块以后，这些块的数据处理可以通过多线程并行的方式完成。充分利用了CPU资源，也能降低CPU的使用率，提高了编码效率。同时客户端也是如此，加上本公开方案占用网络带宽少，所以综合提升桌面操作的流畅性。
本领域普通技术人员可以理解，上文中所公开方法中的全部或某些步骤、系统、装置中的功能模块/单元可以被实施为软件、固件、硬件及其适当的组合。在硬件实施方式中，在以上描述中提及的功能模块/单元之间的划分不一定对应于物理组件的划分；例如，一个物理组件可以具有多个功能，或者一个功能或步骤可以由若干物理组件合作执行。某些组件或所有组件可以被实施为由处理器，如数字信号处理器或微处理器执行的软件，或者被实施为硬件，或者被实施为集成电路，如专用集成电路。这样的软件可以分布在计算机可读介质上，计算机可读介质可以包括计算机存储介质(或非暂时性介质)和通信介质(或暂时性介质)。如本领域普通技术人员公知的，术语计算机存储介质包括在用于存储信息(诸如计算机可读指令、数据结构、程序模块或其他数据)的任何方法或技术中实施的易失性和非易失性、可移除和不可移除介质。计算机存储介质包括但不限于RAM、ROM、EEPROM、闪存或其他存储器技术、CD-ROM、数字多功能盘(DVD)或其他光盘存储、磁盒、磁带、磁盘存储或其他磁存储装置、或者可以用于存储期望的信息并且可以被计算机访问的任何其他的介质。此外，本领域普通技术人员公知的是，通信介质通常包含计算机可读指令、数据结构、程序模块或者诸如载波或其他传输机制之类的调制数据信号中的其他数据，并且可包括任何信息递送介质。
尽管上文对本公开进行了详细说明，但是本公开不限于此，本技术领域技术人员可以根据本公开的原理进行各种修改。因此，凡按照本公开原理所作的修改，都被理解为落入本公开的保护范围。

Claims (11)

  1. 一种虚拟桌面数据显示的方法，包括：
    服务端根据虚拟桌面分辨率，将所述虚拟桌面分割成大小相等的多个虚拟桌面块，使每个虚拟桌面块对应一条小桌面流和一幅幅小桌面帧(S101)；
    服务端通过对每个虚拟桌面块对应的一幅幅小桌面帧进行压缩编码，封装成对应于所述小桌面流的小流帧(S102)；
    服务端将所封装的小流帧发送给客户端，以便客户端通过对收到的所述小流帧进行解压缩、整合和显示处理，再现服务端的每帧虚拟桌面画面(S103)。
  2. 根据权利要求1所述的方法，所述服务端将所封装的小流帧发送给客户端之后，还包括：
    服务端通过对每个虚拟桌面块的当前帧桌面画面进行检测，判断所述虚拟桌面块的当前帧桌面画面与上一帧桌面画面相比是否发生变化；
    若判断所述虚拟桌面块的当前帧桌面画面与上一帧桌面画面相比发生了变化，则服务端对所述虚拟桌面块对应的一幅幅小桌面帧进行压缩编码，封装成对应于所述小桌面流的小流帧，并将所述小流帧发送给客户端。
  3. 根据权利要求1或2所述的方法，所述服务端对所述虚拟桌面块对应的一幅幅小桌面帧进行压缩编码包括：
    服务端根据所分割的虚拟桌面块，为每个虚拟桌面块创建压缩编码线程和编码器；
    服务端利用为所述虚拟桌面块创建的压缩编码线程和编码器，对所述虚拟桌面块对应的一幅幅小桌面帧进行压缩编码。
  4. 根据权利要求3所述的方法，所述客户端通过对收到的所述小流帧进行解压缩、整合和显示处理，再现服务端的每帧虚拟桌面画面包括：
    客户端接收服务端发送的小流帧，并为所述小流帧创建解码显示线程和解码器，并将所接收的小流帧送入到对应的缓存队列中；
    客户端利用为所述小流帧创建的解码显示线程和解码器，将对应缓存队 列中的所述小流帧进行解压缩、整合和显示处理，再现服务端的每帧虚拟桌面画面。
  5. 根据权利要求3所述的方法，所述客户端利用为所述小流帧创建的解码显示线程和解码器，将对应缓存队列中的所述小流帧进行解压缩、整合和显示处理包括：
    客户端利用为所述小流帧创建的解码显示线程从对应缓存队列中取出所述小流帧，并利用解码器对所述小流帧进行解压缩处理，得到对应所述小流帧的桌面转码数据；
    客户端将所有的桌面转码数据进行整合处理，得到整个桌面转码数据，并将所得到的整个桌面转码数据进行显示处理。
  6. 一种虚拟桌面数据显示的系统，包括服务端(201)和客户端(202)，其中服务端(201)包括：
    分割模块(2011)，设置为根据虚拟桌面分辨率，将所述虚拟桌面分割成大小相等的多个虚拟桌面块，使每个虚拟桌面块对应一条小桌面流和一幅幅小桌面帧；
    压缩编码模块(2012)，设置为通过对每个虚拟桌面块对应的一幅幅小桌面帧进行压缩编码，封装成对应于所述小桌面流的小流帧；
    发送模块(2013)，设置为将所封装的小流帧发送给客户端(202)，以便客户端(202)通过对收到的所述小流帧进行解压缩、整合和显示处理，再现服务端(201)的每帧虚拟桌面画面。
  7. 根据权利要求6所述的系统，所述发送模块(2013)包括：
    检测单元，设置为通过对每个虚拟桌面块的当前帧桌面画面进行检测，判断所述虚拟桌面块的当前帧桌面画面与上一帧桌面画面相比是否发生变化；
    发送单元，设置为当判断所述虚拟桌面块的当前帧桌面画面与上一帧桌面画面相比发生了变化，则服务端(201)对所述虚拟桌面块对应的一幅幅小桌面帧进行压缩编码，封装成对应于所述小桌面流的小流帧，并将所述小流帧发送给客户端(202)。
  8. 根据权利要求6或7所述的系统，所述压缩编码模块(2012)包括：
    创建单元，设置为根据所分割的虚拟桌面块，为每个虚拟桌面块创建压缩编码线程和编码器；
    压缩编码单元，设置为利用为所述虚拟桌面块创建的压缩编码线程和编码器，对所述虚拟桌面块对应的一幅幅小桌面帧进行压缩编码。
  9. 根据权利要求8所述的系统，所述客户端(202)包括：
    创建及缓存模块(2021)，设置为根据接收到的小流帧，并为所述小流帧创建解码显示线程和解码器，并将所接收的小流帧送入到对应的缓存队列中；
    处理及显示模块(2022)，设置为利用为所述小流帧创建的解码显示线程和解码器，将对应缓存队列中的所述小流帧进行解压缩、整合和显示处理，再现服务端(201)的每帧虚拟桌面画面。
  10. 根据权利要求9所述的系统，所述处理及显示模块(2022)包括：
    处理单元，设置为利用为所述小流帧创建的解码显示线程从对应缓存队列中取出所述小流帧，并利用解码器对所述小流帧进行解压缩处理，得到对应所述小流帧的桌面转码数据；
    显示单元，设置为将所有的桌面转码数据进行整合处理，得到整个桌面转码数据，并将所得到的整个桌面转码数据进行显示处理。
  11. 一种计算机可读存储介质，存储有计算机可执行指令，所述计算机可执行指令被处理器执行时实现权利要求1-5中任一项所述的方法。
PCT/CN2018/095764 2017-07-17 2018-07-16 一种虚拟桌面数据显示的方法及系统 WO2019015543A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP18835791.7A EP3657316A4 (en) 2017-07-17 2018-07-16 VIRTUAL DESKTOP DATA DISPLAY METHOD AND SYSTEM
RU2019143752A RU2742860C1 (ru) 2017-07-17 2018-07-16 Способ и система для отображения данных виртуального рабочего стола

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710581019.3 2017-07-17
CN201710581019.3A CN109271218A (zh) 2017-07-17 2017-07-17 一种虚拟桌面数据显示的方法及系统

Publications (1)

Publication Number Publication Date
WO2019015543A1 true WO2019015543A1 (zh) 2019-01-24

Family

ID=65016515

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/095764 WO2019015543A1 (zh) 2017-07-17 2018-07-16 一种虚拟桌面数据显示的方法及系统

Country Status (4)

Country Link
EP (1) EP3657316A4 (zh)
CN (1) CN109271218A (zh)
RU (1) RU2742860C1 (zh)
WO (1) WO2019015543A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113365065A (zh) * 2021-06-09 2021-09-07 湖南大学 用于rpa机器人屏幕录制的无损视频编码方法和解码方法
CN114374726A (zh) * 2022-01-07 2022-04-19 阿里巴巴（中国）有限公司 一种云桌面处理方法及系统

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109876434A (zh) * 2019-02-20 2019-06-14 深圳威尔视觉传媒有限公司 一种云游戏图像编码方法和相关装置
CN111405071A (zh) * 2020-06-02 2020-07-10 广东睿江云计算股份有限公司 一种基于云桌面的画面传输方法及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101835043A (zh) * 2010-03-23 2010-09-15 熔点网讯(北京)科技有限公司 基于块编码的带宽自适应桌面共享方法
CN105224390A (zh) * 2015-09-22 2016-01-06 宁夏大学 一种虚拟桌面压缩的方法
CN106201401A (zh) * 2016-07-08 2016-12-07 中霆云计算科技（上海）有限公司 用于虚拟桌面呈现的3d图像加速显示方法
CN106797398A (zh) * 2014-10-16 2017-05-31 华为技术有限公司 用于向客户端提供虚拟桌面服务的方法和系统

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FI950917A (fi) * 1995-02-28 1996-08-29 Nokia Telecommunications Oy Puhekoodausparametrien käsittely tietoliikennejärjestelmässä
IL117133A (en) * 1996-02-14 1999-07-14 Olivr Corp Ltd Method and system for providing on-line virtual reality movies
US8711923B2 (en) * 2002-12-10 2014-04-29 Ol2, Inc. System and method for selecting a video encoding format based on feedback data
US8179960B2 (en) * 2008-11-12 2012-05-15 Alcatel Lucent Method and apparatus for performing video coding and decoding with use of virtual reference data
US20100225655A1 (en) * 2009-03-06 2010-09-09 Microsoft Corporation Concurrent Encoding/Decoding of Tiled Data
US8441494B2 (en) * 2009-04-23 2013-05-14 Vmware, Inc. Method and system for copying a framebuffer for transmission to a remote display
US9412332B2 (en) * 2013-12-20 2016-08-09 Blackberry Limited Method for wirelessly transmitting content from a source device to a sink device
CN106210443B (zh) * 2016-08-31 2019-05-24 广东省广告集团股份有限公司 网络控制显示内容同步的方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101835043A (zh) * 2010-03-23 2010-09-15 熔点网讯(北京)科技有限公司 基于块编码的带宽自适应桌面共享方法
CN106797398A (zh) * 2014-10-16 2017-05-31 华为技术有限公司 用于向客户端提供虚拟桌面服务的方法和系统
CN105224390A (zh) * 2015-09-22 2016-01-06 宁夏大学 一种虚拟桌面压缩的方法
CN106201401A (zh) * 2016-07-08 2016-12-07 中霆云计算科技（上海）有限公司 用于虚拟桌面呈现的3d图像加速显示方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3657316A4 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113365065A (zh) * 2021-06-09 2021-09-07 湖南大学 用于rpa机器人屏幕录制的无损视频编码方法和解码方法
CN113365065B (zh) * 2021-06-09 2024-04-26 湖南大学 用于rpa机器人屏幕录制的无损视频编码方法和解码方法
CN114374726A (zh) * 2022-01-07 2022-04-19 阿里巴巴（中国）有限公司 一种云桌面处理方法及系统

Also Published As

Publication number Publication date
EP3657316A4 (en) 2021-03-31
EP3657316A1 (en) 2020-05-27
RU2742860C1 (ru) 2021-02-11
CN109271218A (zh) 2019-01-25

Similar Documents

Publication Publication Date Title
WO2019015543A1 (zh) 一种虚拟桌面数据显示的方法及系统
US11871014B2 (en) Method for signaling a step-wise temporal sub-layer access sample
WO2019024919A1 (zh) 视频转码方法及其装置、服务器、可读存储介质
TW201837866A (zh) 圖片檔處理方法、設備及儲存介質
US11700419B2 (en) Re-encoding predicted picture frames in live video stream applications
US9426476B2 (en) Video stream
CN109155875A (zh) 用于对定时媒体数据进行封装和解析的方法、装置和计算机程序
US10976986B2 (en) System and method for forwarding an application user interface
US10200707B2 (en) Video bit stream decoding
US10582232B1 (en) Transcoding frame-synchronous metadata for segmented video delivery
CN111885346B (zh) 一种画面码流合成方法、终端、电子设备和存储介质
WO2022111554A1 (zh) 一种视角切换方法及装置
WO2019128668A1 (zh) 视频码流处理方法、装置、网络设备和可读存储介质
EP2804143A1 (en) System and method for forwarding a graphics command stream
CN112165653B (zh) 一种视频播放方法、装置及设备
US11671610B2 (en) Methods, apparatuses, computer programs and computer-readable media for scalable image coding
US10609383B2 (en) Video compression using down-sampling patterns in two phases
KR102238091B1 (ko) 3d 모델 압축 및 압축해제 시스템 및 방법
US10115174B2 (en) System and method for forwarding an application user interface
US20220124135A1 (en) Systems and methods of server-side streaming adaptation in adaptive media streaming systems
CN110798715A (zh) 基于图像串的视频播放方法及系统
Kammachi‐Sreedhar et al. Omnidirectional video delivery with decoder instance reduction
US20240022743A1 (en) Decoding a video stream on a client device
CN113973224B (zh) 媒体信息的传输方法、计算设备及存储介质
WO2023040825A1 (zh) 媒体信息的传输方法、计算设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18835791

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2018835791

Country of ref document: EP

Effective date: 20200217