WO2014121477A1 - Video redirection method, device and system, and computer readable medium - Google Patents

Video redirection method, device and system, and computer readable medium Download PDF

Info

Publication number
WO2014121477A1
WO2014121477A1 PCT/CN2013/071493 CN2013071493W WO2014121477A1 WO 2014121477 A1 WO2014121477 A1 WO 2014121477A1 CN 2013071493 W CN2013071493 W CN 2013071493W WO 2014121477 A1 WO2014121477 A1 WO 2014121477A1
Authority
WO
WIPO (PCT)
Prior art keywords
data frame
video data
video
client
processing instruction
Prior art date
Application number
PCT/CN2013/071493
Other languages
French (fr)
Chinese (zh)
Inventor
陈普
许天锡
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to PCT/CN2013/071493 priority Critical patent/WO2014121477A1/en
Priority to CN201380000046.7A priority patent/CN104067627B/en
Publication of WO2014121477A1 publication Critical patent/WO2014121477A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements

Definitions

  • the present invention relates to the field of information technology, and in particular, to a video redirection method, apparatus, system, and computer readable medium.
  • Remote desktop With the continuous development of computer technology, remote desktop has become a popular technology. Remote desktop technology transfers the content displayed on the remote server screen to the local client for display by logging in to the remote server. Remote desktop applications not only refer to remote screen displays, but also remote video delivery.
  • a multimedia player based on the DirectShow® framework introduced by Microsoft® can be used to load a video file through the video player on a remote server.
  • the Filter in DirectShow® intercepts the undecoded video data, inserts the Filter in the client's DirectShow® framework, receives the video data sent by the remote server for decoding and display, and sends the video data display information to the client, but the video is redirected.
  • the solution relies on Microsoft® Media Player® video player and does not post-process video data, affecting the user experience.
  • the embodiment of the invention discloses a method, a device, a system and a computer readable medium for video redirection to implement post processing of video data frames.
  • the embodiment of the present invention discloses a video redirection method, including: intercepting a first video data frame and a first video data frame processing instruction sent by a video player to a virtual display driver;
  • the video data frame processing instruction processes the second video data frame
  • the first video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction
  • the second video The data frame is the first video data frame or a video data frame obtained by converting the first video data frame by a video encoding format; when the client does not support video data frame decoding, sending the data to the client The second video data frame and the second video data frame processing instruction, so that the client processes the second video data frame according to the second video data frame processing instruction, where the second video data frame processing The instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is a decoded video data frame, the second video data frame is the first video data frame, The second video data frame processing instruction is the first video data frame processing instruction; when the first video data frame is
  • the method further includes: when the client supports video data frame decoding, and the client does not support the video player to the virtual
  • the first video data frame is subjected to video encoding format conversion to obtain the second video data frame; wherein, the second video data frame is
  • the video encoding format is a video encoding format supported by the client.
  • the first video data frame is subjected to video encoding format conversion to obtain the second video data frame, Specifically, the virtual display driver performs video encoding format conversion on the first video data frame to obtain the second video data frame.
  • the method includes: when the client does not support video data frame decoding, and when the first video data frame is an undecoded video data frame, sending the second video data frame and the second video data frame processing to the client Before the instruction, the first video data frame is decoded according to the video data frame decoding instruction in the first video data frame processing instruction to obtain the second video data frame.
  • the method further includes: when the client does not support video data frame decoding, Before the client sends the second video data frame and the second video data frame processing instruction, compressing the second video data frame; sending a second video data frame to the client, specifically to the The client sends the compressed second video data frame.
  • the method further includes: sending, by the intercepting video player, a virtual display driver Obtaining a video data frame processing capability of the client before a video data frame and a first video data frame processing instruction; the video data frame processing capability of the client includes a video data frame decoding capability, according to the video data frame decoding capability And determining whether the client supports video data frame decoding.
  • the method further includes: determining, according to the video data frame processing capability of the client When the client supports the decoding of the video data frame, before sending the second video data frame and the first video data frame processing instruction to the client, acquiring video encoding format information of the video data frame supported by the client, according to The video encoding format information determines whether the client supports a video encoding format of the first video data frame.
  • the intercepting the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver specifically, the virtual display driver intercepts the first video data frame And the first video processing instruction.
  • an embodiment of the present invention discloses a server, including:
  • An intercepting unit configured to intercept a first video data frame and a first video data frame processing instruction sent by the video player to the virtual display driver;
  • a sending unit configured to send, when the client supports video data frame decoding, a second video data frame and the first video data frame processing instruction to the client, so that the client is configured according to the first video data
  • the frame processing instruction processes the second video data frame, where the first video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, and the second video data frame a video data frame obtained by converting the first video data frame or the first video data frame by a video encoding format;
  • the client When the client does not support video data frame decoding, sending a second video data frame and a second video data frame processing instruction to the client, so that the client processes the instruction according to the second video data frame.
  • the second video data frame processing is performed, where the second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is a decoded video data frame
  • the second video data frame is the first video data frame
  • the second video data frame processing instruction is the first video data frame processing instruction
  • the first video data frame is undecoded In the video data frame
  • the second video data frame is a decoded first video data frame
  • the first video data frame processing instruction includes a video data frame decoding instruction, the video data frame post-processing instruction, and the video.
  • the data frame presents instructions.
  • the method further includes a converting unit, where the converting unit is configured to support video data frame decoding when the client supports When the client does not support the video encoding format of the first video data frame sent by the video player to the virtual display driver, the first video data frame is converted into a video encoding format to obtain the first And a video encoding format of the second video data frame is a video encoding format supported by the client.
  • the converting unit is specifically the virtual display driving, and the video data of the first video data frame is performed.
  • the encoding format conversion, obtaining the second video data frame specifically includes: the virtual display driver performs video encoding format conversion on the first video data frame to obtain the second video data frame.
  • the method further includes: a decoding unit, when the client does not support video data frame decoding, and when the first video data frame is The undecoded video data frame, before the second video data frame and the second video data frame processing instruction are sent to the client, according to the video data frame decoding instruction in the first video data frame processing instruction The first video data frame is decoded to obtain the second video data frame.
  • the method further includes a compression unit, where the compression unit is configured to: when the client does not support video data frames Decoding, before transmitting the second video data frame and the second video data frame processing instruction to the client, compressing the second video data frame; sending a second video data frame to the client, Specifically, the compressed second video data frame is sent to the client.
  • the method further includes an acquiring unit, where the acquiring unit is used by the acquiring unit Obtaining before acquiring the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver
  • the video data frame processing capability of the client; the video data frame processing capability of the client includes a video data frame decoding capability, to determine whether the client supports video data frame decoding according to the video data frame decoding capability.
  • the acquiring unit is further configured to: according to the video data frame processing capability of the client And determining, before the client supports video data frame decoding, acquiring a video encoding format of the video data frame supported by the client, before sending the second video data frame and the first video data frame processing instruction to the client And determining information, according to the video encoding format information, whether the client supports a video encoding format of the first video data frame.
  • the intercepting unit is specifically a virtual display driver, and the virtual display driver Intercepting the first video data frame and the first video processing instruction.
  • an embodiment of the present invention discloses a server, including a memory and a central processing unit, where the memory and the central processing unit are connected by a system bus, where the memory stores computer executable instructions when the server is running.
  • the central processor executes the computer-executable instructions stored in the memory, so that the server performs the following steps: intercepting a first video data frame and a first video data frame processing instruction sent by the video player to the virtual display driver;
  • the second video data frame instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, where the second video data frame is the first Video data frame or a video data frame obtained by converting the first video data frame by a video encoding format;
  • the client does not support video data frame decoding, sending the second video data frame and the second video data frame to the client Directing, by the client, to process the second video data frame according to the second video data frame processing instruction, where the second video data frame processing instruction includes a video data frame post-processing instruction and video data a frame presentation instruction;
  • the first video data frame is a decoded video data frame
  • the second video data frame is the first video data frame
  • the second video data frame processing instruction is the a video data frame processing instruction
  • the first video data frame is an undecoded video data
  • the method further includes: when the client supports video data frame decoding, and the client does not support the video player to the And performing a video encoding format conversion on the first video data frame to obtain the second video data frame, where the second video data frame is obtained.
  • the video encoding format is the video encoding format supported by the client.
  • the first video data frame is subjected to video encoding format conversion to obtain the second video data frame, Specifically, the virtual display driver performs video encoding format conversion on the first video data frame to obtain the second video data frame.
  • the method further includes: when the client does not support video data frame decoding, and when the first video data frame is undecoded a video data frame, before the second video data frame and the second video data frame processing instruction are sent to the client, processed according to the first video data frame
  • the video data frame decoding instruction in the instruction decodes the first video data frame to obtain the second video data frame.
  • the method further includes: when the client does not support video data frame decoding, to the client Before transmitting the second video data frame and the second video data frame processing instruction, compressing the second video data frame; sending the second video data frame to the client, specifically to the client The terminal sends the compressed second video data frame.
  • the method further includes: sending, by the intercepted video player, a virtual display driver Obtaining a video data frame processing capability of the client before the first video data frame and the first video data frame processing instruction; the video data frame processing capability of the client includes a video data frame decoding capability, to decode according to the video data frame Capability, determining whether the client supports video data frame decoding.
  • the method further includes: determining the client according to the video data frame processing capability of the client When the video data frame is decoded, the video encoding format information of the video data frame supported by the client is acquired before the second video data frame and the first video data frame processing instruction are sent to the client, according to the The video encoding format information determines whether the client supports the video encoding format of the first video data frame.
  • the intercepting video player sends the first video to the virtual display driver
  • the data frame and the first video data frame processing instruction in particular, the virtual display driver intercepts the first video data frame and the first video processing instruction.
  • an embodiment of the present invention discloses a video redirection system, including a client. And the server according to any one of the second aspects of the embodiments of the present invention.
  • the embodiment of the present invention discloses a video redirection system, including a client and a server according to any one of the third aspects of the embodiments of the present invention.
  • an embodiment of the present invention discloses a computer readable storage medium.
  • the computer executes the computer readable storage medium, the computer performs the following steps: intercepting a first video sent by the video player to the virtual display driver. a data frame and a first video data frame processing instruction;
  • the second video data frame instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, where the second video data frame is the first a video data frame or a video data frame obtained by converting the first video data frame by a video encoding format; when the client does not support video data frame decoding, sending the second video data frame and the second to the client The video data frame processing instruction, so that the client processes the second video data frame according to the second video data frame processing instruction, where the second video data frame processing instruction includes video data frame post processing Command and video data frame presentation instructions; when the first video data frame is a decoded video data frame, the second video data The frame is the first video data frame, the second video data frame processing instruction is the first video data frame processing instruction; when the first video data frame is an undecoded video data frame, the The second
  • the method further includes: when the client supports video data frame decoding, and the client does not support the video
  • the client when the player is in the video encoding format of the first video data frame sent by the virtual display driver, the first video data frame is subjected to video encoding format conversion to obtain the second video data frame;
  • the video encoding format of the second video data frame is a video encoding format supported by the client.
  • the first video data frame is subjected to video encoding format conversion to obtain the second video data frame, Specifically, the virtual display driver performs video encoding format conversion on the first video data frame to obtain the second video data frame.
  • the method further includes: when the client does not support video data frame decoding, and when the first video data frame is an undecoded video data frame, Decoding the first video data frame according to the video data frame decoding instruction in the first video data frame processing instruction before sending the second video data frame and the second video data frame processing instruction to the client Obtaining the second video data frame.
  • the method further includes: when the client does not support video data frame decoding, to the client Before transmitting the second video data frame and the second video data frame processing instruction, compressing the second video data frame; sending a second video data frame to the client, specifically sending to the client The compressed second video data frame.
  • the method further includes: sending, by the intercepted video player, a virtual display driver Obtaining a video data frame processing capability of the client before the first video data frame and the first video data frame processing instruction; the video data frame processing capability of the client includes a video data frame decoding capability, to decode according to the video data frame Capability, determining whether the client supports video data frame decoding.
  • the method further includes: determining, according to the video data frame processing capability of the client, the client When the video data frame is decoded, the video encoding format information of the video data frame supported by the client is acquired before the second video data frame and the first video data frame processing instruction are sent to the client, according to the The video encoding format information determines whether the client supports the video encoding format of the first video data frame.
  • the intercepting video player sends the first to the virtual display driver a video data frame and a first video data frame processing instruction, specifically, the virtual display driver intercepts the first video data frame and the first video processing instruction.
  • the embodiment of the present invention intercepts the video data frame and the video data frame processing instruction sent by the video player to the virtual display driver, and according to whether the client supports video data frame decoding, the video data frame, the video data frame decoding instruction, and the video data frame.
  • the post-processing instruction and the video data frame presentation instruction are sent to the client for processing, or the decoded video data frame, the video data frame post-processing instruction, and the video data frame presentation instruction are sent to the client for processing, thereby being implemented on the client side.
  • the video data frame is post-processed to ensure the user's video experience.
  • the present invention can be implemented without relying on a specific video player.
  • FIG. 1 is a schematic diagram of an application scenario system according to an embodiment of the present invention.
  • FIG. 2 is a schematic structural diagram of a remote desktop server according to an embodiment of the present invention.
  • FIG. 3 is a flowchart of post processing of a video data frame according to an embodiment of the present invention.
  • FIG. 4a and FIG. 4b are diagrams showing a post-processing effect of a video data frame according to an embodiment of the present invention
  • FIG. 5 is a diagram showing a rendering effect of a video data frame according to an embodiment of the present invention
  • FIG. 6a, 6b, 6c, and 6d are schematic diagrams of a virtual desktop structure according to an embodiment of the present invention
  • FIG. 7 is a flowchart of video redirection according to an embodiment of the present invention
  • FIG. 8 is a schematic structural diagram of a server according to an embodiment of the present invention.
  • FIG. 9 is a schematic structural diagram of a system according to an embodiment of the present invention.
  • the client initiates a video play request, and the server loads the video file through the video player according to the video play request of the client, and transmits the video stream of the video file to the client.
  • This process is called video redirection.
  • the server in the embodiment of the present invention may be a general personal computer or other terminal, or may be a commercial server, etc., and is not specifically limited herein.
  • the embodiment of the present invention provides a remote desktop service based on the Windows® operating system platform to the client.
  • the invention is described in detail by way of example. However, the solution of the present invention is not limited to the Windows® operating system. According to the solution of the present invention, it can be extended to other common system platforms, such as the Linux® operating system.
  • the embodiment of the invention discloses a method for video redirection, which is applied to a remote desktop scenario.
  • Remote desktop scenarios typically consist of a local computer and a remote computer.
  • Local computer pass Access the remote computer and transfer the remote computer desktop to the local computer.
  • the local computer that is, the client in the embodiment of the present invention
  • the remote computer that is, the server in the embodiment of the present invention.
  • the video redirection in the embodiment of the present invention refers to transmitting the video data stream of the video file on the server to the client.
  • One of the remote desktop scenarios is a virtual desktop infrastructure scenario, but the present invention is not limited to the architecture shown in FIG. 1.
  • the solution of the present invention can be applied to other scenarios for transmitting remote video data to a client.
  • the virtual desktop infrastructure scenario is shown in FIG. 1.
  • FIG. 1 is not specifically limited to the application of the present invention.
  • the left side of FIG. 1 is a client of a virtual desktop, and the device form of the client may be a general computer 101a, a tablet 101b, a smart phone 101c, and the like. They access the remote desktop service over the network 102 using the Remote Desktop Protocol 103.
  • the servers 204a....204n provide a carrier for the remote desktop, and the user's virtual desktop (Virtual Desktop) runs on the server in the form of virtual machines 205a, 205b-205n.
  • the virtual desktop management system 106 is configured to provide functions such as mapping of a client's client to a virtual machine.
  • the client first connects to the virtual desktop management system 106, obtains the virtual machine address of the user, and then connects to the virtual machine.
  • the virtual desktop management system 106 can be a server or a general personal computer. The present invention does not specifically limit this.
  • the user accesses the virtual desktop (ie, virtual machine) assigned to the user through the client, and the virtual desktop transmits the content accessed by the user to the user's client for display.
  • a typical structure in the server 204a includes three parts, a hardware platform 300, a virtualization platform (also referred to as a hypervisor) 301, and a virtual machine execution space 302.
  • the hard disk 310, the central processing unit 311, the network interface card 312, and the memory 313 in the physical structure of the server may be included in the hardware platform 300.
  • Hypervisor 301 and virtual machine execution space 302 provide virtual machine respectively Hardware platform and software platform.
  • the Hypervisor includes virtual machine monitors 320a, 320b-320n.
  • Hypervisor is the middle layer between physical hardware and virtual machines, and is responsible for coordinating the access of each virtual machine to the hardware platform.
  • the virtual machine monitor includes a virtual hardware platform that implements the virtual machine.
  • the virtual machine monitor 320a includes a virtual hardware platform 321a.
  • the virtual hardware platform includes a virtual memory 322, a central processing unit 323, a hard disk 324, a graphics card 325, a network interface card 326, and the like.
  • Running in a virtual machine execution space 302 multiple virtual machines 205a, 205b ⁇ 205 ⁇ ⁇ 205a of the virtual machine software system running on a virtual machine monitor 320a provides a virtual hardware platform 321a, other virtual machine software system is also running in the corresponding Virtual machine monitor on the virtual hardware platform.
  • the guest operating system 330 runs in the virtual machine 205a.
  • the guest operating system 330 includes a device driver layer 331, and the device driver layer 331 is installed with drivers such as a virtual display driver 334 and a network interface card driver 336.
  • a video player 332 is included in the virtual machine 205a.
  • the guest operating system 330 can be a Windows® operating system platform provided by Microsoft®.
  • the video file is loaded by the video player 332.
  • the video data frame processing capability of the client 101a is provided to the virtual machine 205a, wherein the video data frame processing capability includes the video data frame decoding capability, and the client 101a is also provided to the virtual machine 205a.
  • Video encoding format information of supported video data frames such as video files supporting video encoding formats such as H.264, AVI, and FLV.
  • a local video is taken as an example, and a video play request from the client 101a is received by the desktop protocol server 335, and the video player 332 loads the video file according to the video play request.
  • the video player 332 demultiplexes the loaded video file to obtain a video data frame.
  • the video player 332 needs to determine whether the virtual machine 205a supports the video hardware acceleration function during initialization, and the video player 332 sends the video hardware plus to the virtual display driver 334.
  • Speed capability query request Video hardware acceleration refers to processing a video data frame by using a computer software instead of a software algorithm of a video player, and specifically includes decoding a video data frame and post-processing a video data frame.
  • DXVA Mircosoft® video hardware acceleration
  • DXVA DirectX® Video Acceleration
  • the video data frame is post-processed and the video data frame is post-processed.
  • Presentation is described.
  • DXVA not only provides video data frame decoding processing, but also video data frame post-processing, such as image scaling, color space conversion and de-interlacing.
  • DXVA is a post-processing flowchart for video data frames. DXVA can process the main data stream and the 15-way sub-data stream.
  • the main data stream is subjected to noise filtering, de-interlacing processing, detail filtering and color space conversion, it is mixed with the color space-converted sub-data stream and mixed.
  • the subsequent video data stream is subjected to image adjustment, wherein the image adjustment includes brightness, contrast, saturation adjustment, and the like.
  • the image-scaled video stream is image scaled to adjust the image size.
  • the image-scaled data is subjected to color space conversion, so that the video data stream is subjected to video data frame presentation.
  • color space conversion including conversion from RGB color space to YUV color space, or conversion from YUV color space to RGB color space, where RGB represents red, green and blue, respectively, YUV represents brightness, chromaticity and concentration, respectively.
  • FIG 3 depicts the process of video post processing. Not all functions in the actual application will be used. In actual tests, most players such as Microsoft® Media Player, Storm Video 8, QQ® Video Player, etc. only use image scaling by default to scale the decoded video data frame from the original size to the playback window size.
  • the video data frame post-processing is specifically performed by taking image scaling as an example, and the decoded video data frame is subjected to image scaling, and the original image size is adjusted to be suitable for presentation on the display. The specific effect is shown in Figure 4a.
  • Video data frame post-processing can also include transparency blending, as shown 4b is shown.
  • the video data frame post-processing in the embodiment of the present invention may further include brightness, contrast, and saturation adjustment. This allows the client to present a better video experience to the user.
  • the decoded video data frame is subjected to video data frame post-processing, and the video data frame post-processed by the video data frame is sent to the graphics card according to the video data frame presentation instruction, and is presented by the display.
  • the video data frame presentation instruction carries a presentation parameter, and the presentation parameter includes a video data frame to be presented and a presentation area of the video data frame.
  • an image formed after post-processing of a video data frame includes four regions to be presented, denoted as 1, 2, 3, and 4, respectively, and a desktop image, which is presented on a display, wherein the desktop image has The area is used to present the user with the "display settings, audio settings, and subtitle settings" options.
  • the "display settings, audio settings, and subtitle settings” option areas need to be presented, and the "display”
  • the setting, audio setting, and subtitle setting "option area corresponds to the intermediate area enclosed by the four presentation areas 1, 2, 3, and 4. Therefore, the video data frames corresponding to the intermediate areas surrounded by the four presentation areas 1, 2, 3, and 4 do not need to be presented, and the video data frames constituting the intermediate areas surrounded by the four presentation areas 1, 2, 3, and 4 are not presented.
  • the video data frame This allows the video picture to be integrated with the desktop, providing users with a better video viewing experience.
  • the principle of the video data frame post-processing performed by the client 101a in the embodiment of the present invention is the same as that of the present invention and will not be described again.
  • the hardware of the virtual machine 205a is not the real physical hardware of the virtual machine 205a, but the virtual hardware provided by the virtual machine monitor 320a.
  • the video player 332 queries the virtual display driver 334 for the video hardware acceleration capability of the virtual machine 205a, and can be queried by calling the GetCaps function.
  • the hardware acceleration device can be created by calling the CreateDevice or CreateDecodeDevice function to determine whether the virtual machine 205a supports video hardware acceleration.
  • the virtual display driver 334 When the video player 332 initiates a video querying the virtual display driver 334 for the virtual machine 205a In the hardware acceleration capability, the virtual display driver 334 returns a query request response supporting the video hardware acceleration capability to the video player 332, and the video player 332 determines the video hardware acceleration capability supported by the virtual machine 205a according to the query request response, wherein the video hardware acceleration Capabilities include video data frame decoding capabilities and video data frame post-processing capabilities.
  • the video player 332 sends the video data frame and the video data frame processing instruction to the virtual display driver 334.
  • an implementation manner is that the virtual display driver 334 intercepts the video data frame and the video data frame processed by the video player 332. instruction.
  • the virtual display driver 334 intercepts the video data frame and video data frame processing instructions is specifically described below.
  • the video player 332 in the virtual machine 205a according to the video play request sent by the corresponding client 101a, after the video player 332 loads the video file, starts the video hardware acceleration, and sends the video data frame and the video data to the virtual display driver 334. Frame processing instructions.
  • the loaded video file that is, the video data stream is demultiplexed, to obtain a video data frame therein.
  • a video hardware acceleration interface is defined in the DirectX® runtime library 333, and the video hardware acceleration is initiated by the video player 332 calling the DirectX® runtime library 333, informing the virtual display driver 334 to process the instruction according to the video data frame.
  • the video data frame transmitted by the video player 332 is processed.
  • the virtual display driver 334 intercepts the video data frame and video data frame processing instructions.
  • the virtual display driver is Windows® Display Driver Model (referred to as WDDM display driver model) or Windows® XP Display Driver Model (referred to as XPDM display driver model).
  • the WDDM or XPDM display driver models include the Display Driver and Miniport Driver modules.
  • an implementation scheme is that the display driver module of the virtual display driver intercepts the video data frame and the video data frame processing instruction.
  • an implementation scheme is a WDDM or XPDM display driver model.
  • the video hardware acceleration interface is defined in the Display Driver module. When the video data frame and the video data frame processing instruction pass through the interface, the video data frame and the video data frame processing instruction are intercepted.
  • the embodiment of the present invention intercepts another embodiment of the video data frame and the video data frame processing instruction.
  • the video player 332 in the virtual machine 205a performs video playback according to the video playback request sent by the corresponding client 101a.
  • the video file is loaded by the device 332
  • the video hardware acceleration is turned on, and the video data frame and the video data frame processing instruction are transmitted to the virtual display driver 334.
  • the video player 332 loads the video file, the loaded video file, that is, the video data stream is demultiplexed, to obtain a video data frame therein.
  • a video hardware acceleration interface is defined in the DirectX® runtime library 333, and the video hardware acceleration is initiated by the video player 332 calling the DirectX® runtime library 333, informing the virtual display driver 334 to process the instruction according to the video data frame.
  • the video data frame transmitted by the video player 332 is processed.
  • the DirectX® runtime library 333 invokes the virtual display driver 334 to process the video data frame, ie, after the video data frame and video data frame processing instructions pass through the DirectX® runtime 333, before the virtual display driver 334 is reached, the dynamic link
  • the library intercepts the video data frame and video data frame processing instructions.
  • one of the dynamic link library implementations is to use the Hook technology to intercept the call of the virtual display driver 334 by the DirectX® runtime library 333, and implement a dynamic link library file (dll file) for the virtual display driver 334 Hook, and It is set to be loaded first in the registry to intercept the call of the DirectX® runtime 333 to the virtual display driver 334 to enable interception of video data frames and video data frame processing instructions.
  • dll file dynamic link library file
  • the embodiment of the present invention intercepts another embodiment of the video data frame and the video data frame processing instruction.
  • the video player 332 in the virtual machine 205a performs video playback according to the video playback request sent by the corresponding client 101a.
  • the video hardware acceleration is turned on, and the video data frame and the video are transmitted to the virtual display driver 334. Frequency data frame processing instructions.
  • the video player 332 demultiplexes the loaded video file, that is, the video data stream, to obtain a video data frame therein.
  • a video hardware acceleration interface is defined in the DirectX® runtime 333, and the DirectX® runtime 333 is called by the video player 332 to send video data frames and video data frame processing instructions to the virtual display driver 334.
  • the video hardware acceleration is initiated, and the virtual display driver 334 is notified to process the video data frame sent by the video player 332.
  • the DirectX® runtime intercepts the video data frame and video data frame processing instructions.
  • the embodiment of the present invention intercepts another embodiment of the video data frame and the video data frame processing instruction.
  • the video player 332 in the virtual machine 205a performs video playback according to the video playback request sent by the corresponding client 101a.
  • the video file is loaded by the device 332
  • the video hardware acceleration is turned on, and the video data frame and the video data frame processing instruction are transmitted to the virtual display driver 334.
  • the video player 332 loads the video file, the loaded video file, that is, the video data stream is demultiplexed, to obtain a video data frame therein.
  • a video hardware acceleration interface is defined in the DirectX® runtime library 333, and the video player 332 calls the DirectX® runtime library 333 to initiate video hardware acceleration, when transmitting video data frames and video data frame processing instructions. That is, before the video data frame and video data frame processing instructions arrive at the DirectX® runtime 333, the dynamic link library intercepts the video data frame and video data frame processing instructions. To do this, you need to implement a dynamic link library file (dll file) for the DirectX® runtime 333 Hook, and set it to be loaded first in the registry to intercept the video player 332 to the DirectX® runtime 333. Called to achieve interception of video data frames and video data frame processing instructions.
  • dll file dynamic link library file
  • the video player 332 queries the virtual display driver 334.
  • Know the virtual machine 205a support
  • the video data frame is decoded and the video data frame is post-processed, and the video data frame and the video data frame are captured when the client 101a supports the video data frame decoding and the video encoding format of the video data frame sent by the video player 332 to the virtual display driver 334.
  • the processing instruction sends the intercepted video data frame and the video data frame to the desktop protocol server 335.
  • the desktop protocol server 335 transmits the intercepted video data frame and video data frame processing instructions to the client 101a.
  • the video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction.
  • This embodiment is applicable to the following scenarios: intercepting a first video data frame and a first video data frame processing instruction sent by the video player 332 to the virtual display driver 334; and when the client 101a supports video data frame decoding, the client is 101a transmitting a second video data frame and the first video data frame processing instruction, so that the client 101a processes the second video data frame according to the first video data frame processing instruction, where the first The video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, and the second video data frame is the first video data frame.
  • the video player 332 queries the virtual display driver 334. It is learned that the virtual machine 205a supports video data frame decoding and video data frame post-processing, and the client 101a supports video data frame decoding, but does not support the video encoding format of the video data frame transmitted by the video player 332 to the virtual display driver 334.
  • the video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction
  • the desktop protocol server 335 receives the intercepted video.
  • the data frame, the video codec module of the desktop protocol server 335 decodes the video data frame according to the video data frame decoding instruction, and performs the decoded video data frame according to the video encoding format of the video data frame supported by the client 101a.
  • the video data frame processing instruction includes, a video data frame post-processing instruction, a video data frame presentation instruction, and a video data frame decoding instruction, where the video The data frame decoding instruction includes video encoding format parameters of the re-encoded video data frame.
  • This embodiment is applicable to the following scenarios: intercepting a first video data frame and a first video data frame processing instruction sent by the video player 332 to the virtual display driver 334; and when the client 101a supports video data frame decoding, the client is 101a transmitting a second video data frame and the first video data frame processing instruction, so that the client 101a processes the second video data frame according to the first video data frame processing instruction, where the first The video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, where the second video data frame is a video data frame obtained by converting the first video data frame by a video encoding format. .
  • the video player 332 is virtualized in a scheme in which the virtual display driver 334 as shown in FIG. 6a intercepts the video data frame and the video data frame processing instructions transmitted by the video player 332 to the virtual display driver 334.
  • the display driver 334 queries the learned virtual machine 205a to support video data frame decoding and video data frame post-processing, and the client 101a supports video data frame decoding, but does not support the video player 332 to virtual display.
  • the video codec module in the virtual display driver 334 decodes the captured video data frame according to the intercepted video data frame decoding instruction, according to the support of the client 101a.
  • the video encoding format of the video data frame encodes the decoded video data frame to generate a new video data frame decoding instruction, where the new video data frame decoding instruction includes a video encoding format parameter of the re-encoded video data frame
  • the video data frame processing instruction includes a video data frame post-processing instruction, a video data frame presentation instruction, and a video data frame decoding instruction, where the video data frame decoding instruction includes a video encoding format parameter of the re-encoded video data frame.
  • the desktop protocol server 335 sends the re-encoded video data frame and the video data frame processing instruction to the client 101a, where the video data frame processing instruction includes, the video data frame post-processing instruction, the video data frame presentation instruction, and the video data frame decoding.
  • the virtual machine 205a on the server 204a does not process the video data frame according to the video data frame post-processing instruction, but is processed by the client, which fully utilizes the computing resources of the client, and saves the computing resources of the server.
  • This embodiment is applicable to the following scenarios: intercepting a first video data frame and a first video data frame processing instruction sent by the video player 332 to the virtual display driver 334; and when the client 101a supports video data frame decoding, the client is 101a transmitting a second video data frame and the first video data frame processing instruction, so that the client 101a processes the second video data frame according to the first video data frame processing instruction, where the first
  • the video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, where the second video data frame is a video obtained by converting the first video data frame by a video encoding format.
  • the video player 332 decodes the video data frame to obtain a decoded video data frame, and the video player 332 sends the decoded video data frame and the video data frame processing instruction to the virtual display driver 334; wherein the video data video
  • the desktop protocol server 335 sends the decoded video data frame and the video data frame processing instruction to the client 101a.
  • the decoded video data frame may be compressed, which may be implemented by a compression module in the desktop protocol server 335.
  • This embodiment is applicable to: capturing the first video data frame and the first video data frame processing instruction sent by the video player 332 to the virtual display driver 334; when the client 101a does not support video data frame decoding, The client 101a sends a second video data frame and a second video data frame processing instruction, so that the client 101a processes the second video data frame according to the second video data frame processing instruction, where
  • the second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is a decoded video data frame, the second video data frame is the first Video data frame, The second video data frame processing instruction is the first video data frame processing instruction.
  • the virtual display driver 334 as shown in FIG. 6a intercepts the video data frame and the video data frame processing instruction sent by the video player 332 to the virtual display driver 334, when the client 101a does not support The video data frame is decoded, and the video player 332 queries the virtual display driver 334 to learn that the virtual machine 205a supports the video data frame post-processing but does not support the video data frame decoding. After the video player 332 loads the video file, the video file is demultiplexed.
  • the video player 332 decodes the video data frame to obtain a decoded video data frame, and the video player 332 sends the decoded video data frame and the video data frame processing instruction to the virtual display driver 334; Present the instruction.
  • the decoded video data frame and the video data frame processing instruction sent by the video player 332 are intercepted, and the desktop protocol server 335 transmits the decoded video data frame and the video data frame processing instruction to the client 101a.
  • the virtual display driver 334 intercepts the decoded video data frame and the video data frame processing instruction, and can compress the decoded video data frame, which can be implemented by the compression module in the virtual display driver 334.
  • This embodiment is applicable to: capturing the first video data frame and the first video data frame processing instruction sent by the video player 332 to the virtual display driver 334; when the client 101a does not support video data frame decoding, The client 101a sends a second video data frame and a second video data frame processing instruction, so that the client 101a processes the second video data frame according to the second video data frame processing instruction, where
  • the second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is a decoded video data frame, the second video data frame is the first Video data frame, The second video data frame processing instruction is the first video data frame processing instruction.
  • FIG. 6a, FIG. 6b, FIG. 6c and FIG. 6d in the scheme of intercepting the video data frame and the video data frame processing instruction sent by the video player 332 to the virtual display driver 334, when the client 101a does not support the video data frame decoding, And the video player 332 queries the virtual display driver 334 to learn that the virtual machine 205a supports video data frame decoding and video data frame post-processing. After the video player 332 loads the video file, the video file is demultiplexed to obtain a video data frame and a video.
  • the player 332 transmits a video data frame and a video data frame processing instruction to the virtual display driver 334; wherein the video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction.
  • the video data frame and the video data frame processing instruction sent by the video player 332 are intercepted, and the desktop protocol server 335 decodes the video data frame according to the video data frame decoding instruction in the video data frame processing instruction, and the desktop protocol server 335 sends the decoded data frame.
  • the video data frame and video data frame processing instructions are sent to the client 101a, wherein the video data frame processing instructions include a video data frame post-processing instruction and a video data frame presentation instruction.
  • the desktop protocol server 335 can compress the decoded video data frame, which can be implemented by a compression module in the desktop protocol server 335, or can be implemented by a compression module in the virtual display driver 334, and then The compressed decoded video data frame and video data frame processing instructions are transmitted by the desktop protocol server 335 to the client 101a. Since the virtual machine 205a in the server 204a does not process the video data frame according to the video data frame post-processing instruction, the computing resources and network transmission bandwidth of the server are saved, and the computing resources of the client are fully utilized.
  • This embodiment is applicable to: capturing the first video data frame and the first video data frame processing instruction sent by the video player 332 to the virtual display driver 334; when the client 101a does not support video data frame decoding, The client 101a sends a second video data frame and a second video data frame processing instruction, so that the client 101a processes the second video data frame according to the second video data frame processing instruction.
  • the second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction.
  • the first video data frame is an undecoded video data frame
  • the second video data frame is The decoded first video data frame
  • the first video data frame processing instruction includes a video data frame decoding instruction, the video data frame post-processing instruction, and the video data frame rendering instruction.
  • the video player 332 when the client 101a does not support video data frame decoding, and the video player 332 queries the virtual display driver 334 to learn that the virtual machine 205a supports video data frame decoding and After the video data frame is post-processed, after the video player 332 loads the video file, the video file is demultiplexed to obtain a video data frame, and the video player 332 sends a video data frame and a video data frame processing instruction to the virtual display driver 334;
  • the data frame processing instructions include a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction.
  • the virtual display driver 334 intercepts the video data frame and the video data frame processing instruction sent by the video player 332, decodes the video data frame according to the video data frame decoding instruction in the video data frame processing instruction, and sends the decoded data by the desktop protocol server 335.
  • the video data frame and video data frame processing instructions are to the client 101a, wherein the video data frame processing instructions include a video data frame post-processing instruction and a video data frame presentation instruction.
  • the virtual display driver 334 may decode the decoded video data frame, and may be implemented by the compression module in the virtual display driver 334, or may be implemented by the desktop protocol server 335.
  • the compression module is implemented, and then the compressed decoded video data frame and the video data frame processing instruction are sent by the desktop protocol server 335 to the client 101a. Since the server 204a processes the data, the computing resources and network transmission bandwidth of the server are saved, and the computing resources of the client are fully utilized.
  • This embodiment is applicable to the following scenario: intercepting the first video data frame and the first video data frame processing finger sent by the video player 332 to the virtual display driver 334 When the client 101a does not support video data frame decoding, the second video data frame and the second video data frame processing instruction are sent to the client 101a, so that the client 101a is based on the second video.
  • the data frame processing instruction processes the second video data frame, where the second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is not The second video data frame is the decoded first video data frame, and the first video data frame processing instruction includes a video data frame decoding instruction and the video data frame post-processing instruction. And presenting instructions to the video data frame.
  • the embodiment of the present invention intercepts the video data frame and the video data frame processing instruction sent by the video player to the virtual display driver, and according to whether the client supports video data frame decoding, the video data frame, the video data frame decoding instruction, and the video data frame.
  • the post-processing instruction and the video data frame presentation instruction are sent to the client for processing, or the decoded video data frame, the video data frame post-processing instruction, and the video data frame presentation instruction are sent to the client for processing, thereby being implemented on the client side.
  • the video data frame is post-processed to ensure the user's video experience.
  • the present invention can be implemented without relying on a specific video player.
  • FIG. 7 A specific exemplary flowchart of the embodiment of the present invention is shown in FIG. 7 , and the specific steps are as follows:
  • Step 701 Determine whether the client has a video data frame decoding capability, and whether the virtual machine supports video data frame decoding and video data frame post-processing.
  • step 702a is performed;
  • the virtual machine does not support the video data frame decoding but supports the video data frame post-processing, and the step 702b is performed;
  • step 702c is performed.
  • Step 702a intercepting a video data frame and a video data frame processing instruction sent by the video player to the virtual display driver;
  • the video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction;
  • Step 703a Determine whether the client supports the video encoding format of the video data frame sent by the video player.
  • step 704a When the client supports the video encoding format of the video data frame sent by the video player, step 704a is performed; when the client does not support the video encoding format of the video data frame sent by the video player, step 704a' is performed.
  • Step 704a Send the intercepted video data frame and the video data frame processing instruction to the client;
  • Step 704a' Convert the video data frame into a video data frame of a video encoding format supported by the client.
  • Step 702b intercepting the decoded video data frame, the video data frame post-processing instruction, and the video data frame presentation instruction sent by the video player to the virtual display driver;
  • Step 703B Optionally, after step 702b, compressing the intercepted decoded video data frame;
  • Step 704C Send the decoded video data frame and the video data frame processing instruction to the client;
  • the video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction.
  • Step 702c intercepting a video data frame and a video data frame processing instruction sent by the video player;
  • the video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction;
  • Step 703c Decode the video data frame according to a video data frame decoding instruction.
  • steps 703B and 704C may also be continued.
  • the embodiment of the present invention provides a server 800, as shown in FIG. 8, including an intercepting unit 801 and a sending unit 802.
  • the intercepting unit 801 is configured to intercept the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver;
  • the sending unit 802 is configured to: when the client supports decoding of the video data frame, send the second video data frame and the first video data frame processing instruction to the client, so that the client is configured according to the first video
  • the data frame processing instruction processes the second video data frame, where the first video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, the second video data
  • the frame is the first video data frame or a video data frame obtained by converting the first video data frame by a video encoding format;
  • the sending unit 802 is configured to: when the client does not support video data frame decoding, send a second video data frame and a second video data frame processing instruction to the client, so that the client is according to the second The video data frame processing instruction processes the second video data frame, where the second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is The second video data frame is the first video data frame, and the second video data frame processing instruction is the first video data frame processing instruction; when the first video is When the data frame is an undecoded video data frame, the second video data frame is a decoded first video data frame, and the first video data frame processing instruction includes a video data frame decoding instruction, and the video data frame is Processing instructions and the video data frame presentation instructions.
  • the embodiment of the present invention intercepts a video data frame and a video data frame processing instruction sent by the video player to the virtual display driver, and according to whether the client supports the video data frame.
  • Decoding transmitting a video data frame, a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction to a client for processing, or decoding the decoded video data frame, a video data frame post-processing instruction, and video data
  • the frame rendering instruction is sent to the client for processing, so that the video data frame post-processing can be implemented on the client to ensure the user's video experience.
  • the present invention can be implemented without relying on a specific video player.
  • a conversion unit 803 is further included, where the conversion unit 803 is configured to: when the client supports video data frame decoding, and the client does not support the video player to send to the virtual display driver The video encoding format of the first video data frame, the first video data frame is subjected to a video encoding format conversion to obtain the second video data frame; wherein the video encoding format of the second video data frame is the The video encoding format supported by the client.
  • the converting unit 803 is specifically the virtual display driver, and the video data format conversion is performed on the first video data frame to obtain the second video data frame, which specifically includes: The virtual display driver performs video encoding format conversion on the first video data frame to obtain the second video data frame.
  • the server 800 provided by the embodiment of the present invention includes a decoding unit 801 and a sending unit 802, and a decoding unit 804, configured to: when the client does not support video data frame decoding, and when the first video data frame is not Decoding the video data frame, before transmitting the second video data frame and the second video data frame processing instruction to the client, according to the video data frame decoding instruction in the first video data frame processing instruction The first video data frame is decoded to obtain the second video data frame.
  • the server 800 provided by the embodiment of the present invention includes, in addition to the intercepting unit 801 and the sending unit 802, or the server 800, in addition to the intercepting unit 801, the sending unit 802, and the decoding unit 803, a compression unit 805, where the compression unit 805 is used.
  • the second video data frame is compressed before the second video data frame and the second video data frame processing instruction are sent to the client, and then sent to the client.
  • the second video data frame is specifically configured to send the compressed second video data frame to the client, thereby saving network transmission bandwidth.
  • the server 800 of the foregoing embodiment of the present invention further includes an obtaining unit 806, where the acquiring unit 806 is configured to intercept the first video data frame sent by the video player to the virtual display driver. Obtaining a video data frame processing capability of the client before the first video data frame processing instruction; the video data frame processing capability of the client includes a video data frame decoding capability, to determine a location according to the video data frame decoding capability Whether the client supports video data frame decoding. Further, the obtaining unit 806 further determines, according to the video data frame processing capability of the client, that the client supports the video data frame decoding, and sends the second video data frame and the first video data to the client. Before the frame processing instruction, the video encoding format information of the video data frame supported by the client is obtained, to determine, according to the video encoding format information, whether the client supports the video encoding format of the first video data frame.
  • the intercepting unit is specifically a virtual display driver, and the virtual display driver intercepts the first video data frame and the first video processing instruction.
  • the embodiment of the present invention further provides a video redirection system 900, including a server 800 and a client 1000.
  • the server 800 is configured to intercept the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver;
  • the second video data frame and the first video data frame processing instruction are sent to the client 1000, so that the client 1000 processes according to the first video data frame.
  • the instruction processes the second video data frame, where the first video data frame processing instruction includes a video data frame decoding instruction and a video a data frame post-processing instruction and a video data frame presentation instruction, where the second video data frame is the first video data frame or a video data frame obtained by converting the first video data frame by a video encoding format;
  • the processing instruction is configured to process the second video data frame, where the second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is decoded
  • the second video data frame is the first video data frame
  • the second video data frame processing instruction is the first video data frame processing instruction
  • the first video data frame is In the case of an undecoded video data frame
  • the second video data frame is the decoded first video data frame
  • the first video data frame processing instruction includes a video data frame decoding instruction, and the video data frame post processing The instructions and the video data frame present instructions.
  • the client 1000 is configured to process the video data frame according to the video data frame processing instruction sent by the server 800.
  • the embodiment of the present invention intercepts the video data frame and the video data frame processing instruction sent by the video player to the virtual display driver, and according to whether the client supports video data frame decoding, the video data frame, the video data frame decoding instruction, and the video data frame.
  • the post-processing instruction and the video data frame presentation instruction are sent to the client for processing, or the decoded video data frame, the video data frame post-processing instruction, and the video data frame presentation instruction are sent to the client for processing, thereby being implemented on the client side.
  • the video data frame is post-processed to ensure the user's video experience.
  • the present invention can be implemented without relying on a specific video player.
  • the server 800 and the client 1000 provided in the embodiment of the present invention may be a general computer, a mobile terminal, a workstation, or a dedicated server, etc., which is not specifically limited by the present invention, and has a memory and at least one central processor, and the memory and the At least one central processing unit is coupled by a bus, the memory stores computer instructions, and the at least one central processing unit is configured to execute computer instructions in the memory, and also includes a common external interface and the like to implement some necessary components of the server function.
  • the algorithm for the computer instructions stored in the memory may refer to the description of the flowchart shown in FIG. 7.
  • the virtual desktop-based remote desktop architecture provided by the embodiment of the present invention, those skilled in the art may appreciate that the present invention is not limited to the described virtual desktop architecture of the present embodiment, and the remote desktop invention based on other types of virtual desktops The same applies.
  • the present invention is not only applicable to a virtual desktop-based remote desktop architecture, but also to other application scenarios that provide video data to remote clients.
  • the disclosed systems, devices, and methods may be implemented in other ways.
  • the device embodiments described above are merely illustrative, for example, the division of the unit is merely a logical function. Dividing, there may be another way of dividing in actual implementation, for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be electrical, mechanical or otherwise.
  • the components displayed as the unit may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the functions may be stored in a computer readable storage medium if implemented in the form of a software functional unit and sold or used as a standalone product.
  • the technical solution of the present invention which is essential or contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product, which is stored in a storage medium, including
  • the instructions are used to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes: NAS (Network Attached Storage), U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or A variety of media such as optical discs that can store program code.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

By intercepting video data frames and a video data frame processing instruction transmitted from a video player to a virtual display driver, according to whether a client supports video data frame decoding, the embodiments of the present invention transmit to the client the video data frames, a video data frame decoding instruction, a video data frame post-processing instruction and a video data frame presentation instruction for processing, or transmit to the client the decoded video data frames, the video data frame post-processing instruction and the video data frame presentation instruction for processing, and thus can realize video data frame post-processing at the client, ensuring the video experience of users. At the same time, provided that a server-side video player supports the video redirection function, the present invention can be realized, without relying on a certain specific video player.

Description

一种视频重定向的方法、 装置、 系统及计算机可读介质 技术领域  Method, device, system and computer readable medium for video redirection
本发明涉及信息技术领域, 特别涉及一种视频重定向的方法、 装 置、 系统及计算机可读介质。  The present invention relates to the field of information technology, and in particular, to a video redirection method, apparatus, system, and computer readable medium.
背景技术 Background technique
随着计算机技术的不断发展, 远程桌面成为一种流行技术。 远程 桌面技术通过登陆到远程服务器,将该远程服务器屏幕显示的内容传 递到本地客户端进行显示。 远程桌面应用并不仅指远程的屏幕显示, 还可以包含远程视频传送。  With the continuous development of computer technology, remote desktop has become a popular technology. Remote desktop technology transfers the content displayed on the remote server screen to the local client for display by logging in to the remote server. Remote desktop applications not only refer to remote screen displays, but also remote video delivery.
现有技术中, 在使用远程桌面中的视频播放器播放视频文件时, 使用基于 Microsoft®公司推出的 DirectShow®框架的多媒体播放器, 在远程服务器上通过该视频播放器加载视频文件后, 可以从 DirectShow®中的 Filter 截获未解码的视频数据, 在客户端的 DirectShow®框架中也插入 Filter, 接收远程服务器发送的视频数据进 行解码显示, 同时将视频数据显示信息发送到客户端, 但是该视频重 定向方案需要依赖 Microsoft®公司的 Media Player®视频播放器,并且 不能对视频数据进行后处理, 从而影响用户体验。  In the prior art, when playing a video file using a video player in a remote desktop, a multimedia player based on the DirectShow® framework introduced by Microsoft® can be used to load a video file through the video player on a remote server. The Filter in DirectShow® intercepts the undecoded video data, inserts the Filter in the client's DirectShow® framework, receives the video data sent by the remote server for decoding and display, and sends the video data display information to the client, but the video is redirected. The solution relies on Microsoft® Media Player® video player and does not post-process video data, affecting the user experience.
发明内容  Summary of the invention
本发明实施例公开了一种视频重定向的方法、 装置、 系统及计算 机可读介质, 以实现视频数据帧后处理。  The embodiment of the invention discloses a method, a device, a system and a computer readable medium for video redirection to implement post processing of video data frames.
第一方面, 本发明实施例公开了一种视频重定向的方法, 包括: 截获视频播放器向虚拟显示驱动发送的第一视频数据帧和第一 视频数据帧处理指令;  In a first aspect, the embodiment of the present invention discloses a video redirection method, including: intercepting a first video data frame and a first video data frame processing instruction sent by a video player to a virtual display driver;
当客户端支持视频数据帧解码时,向所述客户端发送第二视频数 据帧和所述第一视频数据帧处理指令,以使所述客户端根据所述第一 视频数据帧处理指令对所述第二视频数据帧进行处理,所述第一视频 数据帧处理指令包括视频数据帧解码指令、视频数据帧后处理指令和 视频数据帧呈现指令,所述第二视频数据帧为所述第一视频数据帧或 为所述第一视频数据帧经过视频编码格式转换获得的视频数据帧; 当所述客户端不支持视频数据帧解码时,向所述客户端发送第二 视频数据帧和第二视频数据帧处理指令,以使所述客户端根据所述第 二视频数据帧处理指令对所述第二视频数据帧进行处理, 其中, 所述 第二视频数据帧处理指令包括视频数据帧后处理指令和视频数据帧 呈现指令; 当所述第一视频数据帧为解码后的视频数据帧时, 所述第 二视频数据帧为所述第一视频数据帧,所述第二视频数据帧处理指令 为所述第一视频数据帧处理指令; 当所述第一视频数据帧为未解码的 视频数据帧时, 所述第二视频数据帧为解码后的第一视频数据帧, 所 述第一视频数据帧处理指令包括视频数据帧解码指令、所述视频数据 帧后处理指令和所述视频数据帧呈现指令。 Transmitting, by the client, a second video data frame and the first video data frame processing instruction to the client, so that the client is configured according to the first The video data frame processing instruction processes the second video data frame, where the first video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, the second video The data frame is the first video data frame or a video data frame obtained by converting the first video data frame by a video encoding format; when the client does not support video data frame decoding, sending the data to the client The second video data frame and the second video data frame processing instruction, so that the client processes the second video data frame according to the second video data frame processing instruction, where the second video data frame processing The instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is a decoded video data frame, the second video data frame is the first video data frame, The second video data frame processing instruction is the first video data frame processing instruction; when the first video data frame is an undecoded video data frame The second video data frame is a decoded first video data frame, and the first video data frame processing instruction includes a video data frame decoding instruction, the video data frame post-processing instruction, and the video data frame presentation. instruction.
结合本发明第一方面实施例, 在第一种可能的实施方式中, 还包 括: 当所述客户端支持视频数据帧解码时, 并且所述客户端不支持所 述视频播放器向所述虚拟显示驱动发送的所述第一视频数据帧的视 频编码格式时, 将所述第一视频数据帧进行视频编码格式转换, 获 得所述第二视频数据帧; 其中, 所述第二视频数据帧的视频编码格式 为所述客户端支持的视频编码格式。  With reference to the first aspect of the present invention, in a first possible implementation manner, the method further includes: when the client supports video data frame decoding, and the client does not support the video player to the virtual When the video encoding format of the first video data frame that is sent by the driver is displayed, the first video data frame is subjected to video encoding format conversion to obtain the second video data frame; wherein, the second video data frame is The video encoding format is a video encoding format supported by the client.
结合本发明第一方面的第一种可能的实施方式,在第二种可能的 实施方式中, 所述将所述第一视频数据帧进行视频编码格式转换, 获 得所述第二视频数据帧, 具体包括: 所述虚拟显示驱动将所述第一视 频数据帧进行视频编码格式转换, 获得所述第二视频数据帧。  With reference to the first possible implementation manner of the first aspect of the present invention, in a second possible implementation manner, the first video data frame is subjected to video encoding format conversion to obtain the second video data frame, Specifically, the virtual display driver performs video encoding format conversion on the first video data frame to obtain the second video data frame.
结合本发明第一方面的实施例, 在第三种可能的实施方式中, 还 包括当所述客户端不支持视频数据帧解码时,并且当所述第一视频数 据帧为未解码的视频数据帧时, 向所述客户端发送第二视频数据帧 和第二视频数据帧处理指令之前,根据所述第一视频数据帧处理指令 中的所述视频数据帧解码指令对所述第一视频数据帧进行解码,以获 得所述第二视频数据帧。 In conjunction with the embodiment of the first aspect of the present invention, in a third possible implementation, The method includes: when the client does not support video data frame decoding, and when the first video data frame is an undecoded video data frame, sending the second video data frame and the second video data frame processing to the client Before the instruction, the first video data frame is decoded according to the video data frame decoding instruction in the first video data frame processing instruction to obtain the second video data frame.
结合本发明第一方面实施例,或第一方面的第三种可能的实施方 式, 在第四种可能的实施方式中, 还包括: 当所述客户端不支持视频 数据帧解码时,向所述客户端发送所述第二视频数据帧和所述第二视 频数据帧处理指令之前, 压缩所述第二视频数据帧; 则向所述客户端 发送第二视频数据帧,具体为向所述客户端发送压缩后的第二视频数 据帧。  With reference to the first aspect of the present invention, or the third possible implementation manner of the first aspect, in a fourth possible implementation, the method further includes: when the client does not support video data frame decoding, Before the client sends the second video data frame and the second video data frame processing instruction, compressing the second video data frame; sending a second video data frame to the client, specifically to the The client sends the compressed second video data frame.
结合本发明第一方面或第一方面的第一至第四种任一可能的实 施方式, 在第五种可能的实施方式中, 还包括: 所述截获视频播放器 向虚拟显示驱动发送的第一视频数据帧和第一视频数据帧处理指令 之前, 获取所述客户端的视频数据帧处理能力; 所述客户端的视频数 据帧处理能力包括视频数据帧解码能力,以根据所述视频数据帧解码 能力, 确定所述客户端是否支持视频数据帧解码。  With reference to the first aspect of the present invention or any one of the first to fourth possible embodiments of the first aspect, in a fifth possible implementation, the method further includes: sending, by the intercepting video player, a virtual display driver Obtaining a video data frame processing capability of the client before a video data frame and a first video data frame processing instruction; the video data frame processing capability of the client includes a video data frame decoding capability, according to the video data frame decoding capability And determining whether the client supports video data frame decoding.
结合本发明第一方面的第一种、第二种或第五种可能的实施方式, 在第六种可能的实施方式中, 还包括: 根据所述客户端的视频数据帧 处理能力, 确定所述客户端支持视频数据帧解码时, 向所述客户端发 送第二视频数据帧和所述第一视频数据帧处理指令之前,获取所述客 户端支持的视频数据帧的视频编码格式信息,以根据所述视频编码格 式信息 ,确定所述客户端是否支持所述第一视频数据帧的视频编码格 式。 结合本发明第一方面, 或第一至第六种任一可能的实施方式, 在 第七种可能的实施方式中,所述截获视频播放器向虚拟显示驱动发送 的第一视频数据帧和第一视频数据帧处理指令,具体为所述虚拟显示 驱动截获所述第一视频数据帧和所述第一视频处理指令。 With reference to the first, second or fifth possible implementation manner of the first aspect of the present invention, in a sixth possible implementation, the method further includes: determining, according to the video data frame processing capability of the client When the client supports the decoding of the video data frame, before sending the second video data frame and the first video data frame processing instruction to the client, acquiring video encoding format information of the video data frame supported by the client, according to The video encoding format information determines whether the client supports a video encoding format of the first video data frame. In combination with the first aspect of the invention, or any of the first to sixth possible embodiments, In a seventh possible implementation manner, the intercepting the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver, specifically, the virtual display driver intercepts the first video data frame And the first video processing instruction.
第二方面, 本发明实施例公开了一种服务器, 包括:  In a second aspect, an embodiment of the present invention discloses a server, including:
截获单元,用于截获视频播放器向虚拟显示驱动发送的第一视频 数据帧和第一视频数据帧处理指令;  An intercepting unit, configured to intercept a first video data frame and a first video data frame processing instruction sent by the video player to the virtual display driver;
发送单元, 用于当客户端支持视频数据帧解码时, 向所述客户端 发送第二视频数据帧和所述第一视频数据帧处理指令,以使所述客户 端根据所述第一视频数据帧处理指令对所述第二视频数据帧进行处 理, 所述第一视频数据帧处理指令包括视频数据帧解码指令、视频数 据帧后处理指令和视频数据帧呈现指令,所述第二视频数据帧为所述 第一视频数据帧或为所述第一视频数据帧经过视频编码格式转换获 得的视频数据帧;  a sending unit, configured to send, when the client supports video data frame decoding, a second video data frame and the first video data frame processing instruction to the client, so that the client is configured according to the first video data The frame processing instruction processes the second video data frame, where the first video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, and the second video data frame a video data frame obtained by converting the first video data frame or the first video data frame by a video encoding format;
当所述客户端不支持视频数据帧解码时,向所述客户端发送第二 视频数据帧和第二视频数据帧处理指令,以使所述客户端根据所述第 二视频数据帧处理指令对所述第二视频数据帧进行处理, 其中, 所述 第二视频数据帧处理指令包括视频数据帧后处理指令和视频数据帧 呈现指令; 当所述第一视频数据帧为解码后的视频数据帧时, 所述第 二视频数据帧为所述第一视频数据帧,所述第二视频数据帧处理指令 为所述第一视频数据帧处理指令; 当所述第一视频数据帧为未解码的 视频数据帧时, 所述第二视频数据帧为解码后的第一视频数据帧, 所 述第一视频数据帧处理指令包括视频数据帧解码指令、所述视频数据 帧后处理指令和所述视频数据帧呈现指令。  When the client does not support video data frame decoding, sending a second video data frame and a second video data frame processing instruction to the client, so that the client processes the instruction according to the second video data frame. The second video data frame processing is performed, where the second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is a decoded video data frame The second video data frame is the first video data frame, the second video data frame processing instruction is the first video data frame processing instruction; when the first video data frame is undecoded In the video data frame, the second video data frame is a decoded first video data frame, and the first video data frame processing instruction includes a video data frame decoding instruction, the video data frame post-processing instruction, and the video. The data frame presents instructions.
结合本发明第二方面, 在第一种可能的实施方式中, 还包括转换 单元, 所述转换单元用于当所述客户端支持视频数据帧解码时, 并且 所述客户端不支持所述视频播放器向所述虚拟显示驱动发送的所述 第一视频数据帧的视频编码格式时, 将所述第一视频数据帧进行视 频编码格式转换, 获得所述第二视频数据帧; 其中, 所述第二视频数 据帧的视频编码格式为所述客户端支持的视频编码格式。 With reference to the second aspect of the present invention, in a first possible implementation, the method further includes a converting unit, where the converting unit is configured to support video data frame decoding when the client supports When the client does not support the video encoding format of the first video data frame sent by the video player to the virtual display driver, the first video data frame is converted into a video encoding format to obtain the first And a video encoding format of the second video data frame is a video encoding format supported by the client.
结合本发明第二方面的第一种可能的实施方式,在第二种可能的 实施方式中, 所述转换单元具体为所述虚拟显示驱动, 则所述将所述 第一视频数据帧进行视频编码格式转换, 获得所述第二视频数据帧, 具体包括:所述虚拟显示驱动将所述第一视频数据帧进行视频编码格 式转换, 获得所述第二视频数据帧。  With reference to the first possible implementation manner of the second aspect of the present invention, in a second possible implementation manner, the converting unit is specifically the virtual display driving, and the video data of the first video data frame is performed. The encoding format conversion, obtaining the second video data frame, specifically includes: the virtual display driver performs video encoding format conversion on the first video data frame to obtain the second video data frame.
结合本发明第二方面的实施例, 在第三种可能的实施方式中, 还 包括解码单元, 用于当所述客户端不支持视频数据帧解码时, 并且当 所述第一视频数据帧为未解码的视频数据帧时 ,向所述客户端发送第 二视频数据帧和第二视频数据帧处理指令之前,根据所述第一视频数 据帧处理指令中的所述视频数据帧解码指令对所述第一视频数据帧 进行解码, 以获得所述第二视频数据帧。  With reference to the embodiment of the second aspect of the present invention, in a third possible implementation, the method further includes: a decoding unit, when the client does not support video data frame decoding, and when the first video data frame is The undecoded video data frame, before the second video data frame and the second video data frame processing instruction are sent to the client, according to the video data frame decoding instruction in the first video data frame processing instruction The first video data frame is decoded to obtain the second video data frame.
结合本发明第二方面或第二方面的第三种可能的实施方式,在第 四种可能的实施方式中, 还包括压缩单元, 所述压缩单元用于当所述 客户端不支持视频数据帧解码时,向所述客户端发送所述第二视频数 据帧和所述第二视频数据帧处理指令之前,压缩所述第二视频数据帧; 则向所述客户端发送第二视频数据帧,具体为向所述客户端发送压缩 后的第二视频数据帧。  With reference to the second aspect of the present invention or the third possible implementation manner of the second aspect, in a fourth possible implementation, the method further includes a compression unit, where the compression unit is configured to: when the client does not support video data frames Decoding, before transmitting the second video data frame and the second video data frame processing instruction to the client, compressing the second video data frame; sending a second video data frame to the client, Specifically, the compressed second video data frame is sent to the client.
结合本发明第二方面,或者第二方面的第一至第四种任一可能的 实施方式, 在第五种可能的实施方式中, 还包括获取单元, 所述获取 单元用于所述截获单元截获所述视频播放器向所述虚拟显示驱动发 送的所述第一视频数据帧和所述第一视频数据帧处理指令之前 ,获取 所述客户端的视频数据帧处理能力;所述客户端的视频数据帧处理能 力包括视频数据帧解码能力, 以根据所述视频数据帧解码能力, 确定 所述客户端是否支持视频数据帧解码。 With reference to the second aspect of the present invention, or any one of the first to fourth possible embodiments of the second aspect, in a fifth possible implementation, the method further includes an acquiring unit, where the acquiring unit is used by the acquiring unit Obtaining before acquiring the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver The video data frame processing capability of the client; the video data frame processing capability of the client includes a video data frame decoding capability, to determine whether the client supports video data frame decoding according to the video data frame decoding capability.
结合本发明第二方面的第一种、第二种或第五种可能的实施方式, 在第六种可能的实施方式中,所述获取单元还用于根据所述客户端的 视频数据帧处理能力, 确定所述客户端支持视频数据帧解码时, 向所 述客户端发送第二视频数据帧和所述第一视频数据帧处理指令之前, 获取所述客户端支持的视频数据帧的视频编码格式信息,以根据所述 视频编码格式信息,确定所述客户端是否支持所述第一视频数据帧的 视频编码格式。  With reference to the first, second or fifth possible implementation manner of the second aspect of the present invention, in a sixth possible implementation, the acquiring unit is further configured to: according to the video data frame processing capability of the client And determining, before the client supports video data frame decoding, acquiring a video encoding format of the video data frame supported by the client, before sending the second video data frame and the first video data frame processing instruction to the client And determining information, according to the video encoding format information, whether the client supports a video encoding format of the first video data frame.
结合本发明第二方面,或者第二方面的第一至第六种任一可能的 实施方式, 在第七种可能的实施方式中, 所述截获单元具体为虚拟显 示驱动,所述虚拟显示驱动截获所述第一视频数据帧和所述第一视频 处理指令。  With reference to the second aspect of the present invention, or any of the first to sixth possible implementation manners of the second aspect, in the seventh possible implementation, the intercepting unit is specifically a virtual display driver, and the virtual display driver Intercepting the first video data frame and the first video processing instruction.
第三方面, 本发明实施例公开了一种服务器, 包括存储器和中央 处理器, 所述存储器和所述中央处理器通过系统总线相连, 所述存储 器存储计算机可执行指令, 当所述服务器运行时, 所述中央处理器执 行所述存储器存储的计算机可执行指令,使所述服务器执行如下步骤: 截获视频播放器向虚拟显示驱动发送的第一视频数据帧和第一 视频数据帧处理指令;  In a third aspect, an embodiment of the present invention discloses a server, including a memory and a central processing unit, where the memory and the central processing unit are connected by a system bus, where the memory stores computer executable instructions when the server is running. The central processor executes the computer-executable instructions stored in the memory, so that the server performs the following steps: intercepting a first video data frame and a first video data frame processing instruction sent by the video player to the virtual display driver;
当客户端支持视频数据帧解码时,向所述客户端发送第二视频数 据帧和所述第一视频数据帧处理指令,以使所述客户端根据所述第一 视频数据帧处理指令对所述第二视频数据帧进行处理,所述第一视频 数据帧处理指令包括视频数据帧解码指令、视频数据帧后处理指令和 视频数据帧呈现指令,所述第二视频数据帧为所述第一视频数据帧或 为所述第一视频数据帧经过视频编码格式转换获得的视频数据帧; 当所述客户端不支持视频数据帧解码时,向所述客户端发送第二 视频数据帧和第二视频数据帧处理指令,以使所述客户端根据所述第 二视频数据帧处理指令对所述第二视频数据帧进行处理, 其中, 所述 第二视频数据帧处理指令包括视频数据帧后处理指令和视频数据帧 呈现指令; 当所述第一视频数据帧为解码后的视频数据帧时, 所述第 二视频数据帧为所述第一视频数据帧,所述第二视频数据帧处理指令 为所述第一视频数据帧处理指令; 当所述第一视频数据帧为未解码的 视频数据帧时, 所述第二视频数据帧为解码后的第一视频数据帧, 所 述第一视频数据帧处理指令包括视频数据帧解码指令、所述视频数据 帧后处理指令和所述视频数据帧呈现指令。 When the client supports video data frame decoding, sending a second video data frame and the first video data frame processing instruction to the client, so that the client processes the instruction according to the first video data frame Processing, the second video data frame instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, where the second video data frame is the first Video data frame or a video data frame obtained by converting the first video data frame by a video encoding format; when the client does not support video data frame decoding, sending the second video data frame and the second video data frame to the client Directing, by the client, to process the second video data frame according to the second video data frame processing instruction, where the second video data frame processing instruction includes a video data frame post-processing instruction and video data a frame presentation instruction; when the first video data frame is a decoded video data frame, the second video data frame is the first video data frame, and the second video data frame processing instruction is the a video data frame processing instruction; when the first video data frame is an undecoded video data frame, the second video data frame is a decoded first video data frame, and the first video data frame processing instruction A video data frame decoding instruction, the video data frame post-processing instruction, and the video data frame rendering instruction are included.
结合本发明第三方面的实施例, 在第一种可能的实施方式中, 还 包括: 当所述客户端支持视频数据帧解码时, 并且所述客户端不支持 所述视频播放器向所述虚拟显示驱动发送的所述第一视频数据帧的 视频编码格式时, 将所述第一视频数据帧进行视频编码格式转换, 获 得所述第二视频数据帧; 其中, 所述第二视频数据帧的视频编码格式 为所述客户端支持的视频编码格式。  With reference to the embodiment of the third aspect of the present invention, in a first possible implementation, the method further includes: when the client supports video data frame decoding, and the client does not support the video player to the And performing a video encoding format conversion on the first video data frame to obtain the second video data frame, where the second video data frame is obtained. The video encoding format is the video encoding format supported by the client.
结合本发明第三方面的第一种可能的实施方式,在第二种可能的 实施方式中, 所述将所述第一视频数据帧进行视频编码格式转换, 获 得所述第二视频数据帧, 具体包括: 所述虚拟显示驱动将所述第一视 频数据帧进行视频编码格式转换, 获得所述第二视频数据帧。  With reference to the first possible implementation manner of the third aspect of the present invention, in a second possible implementation manner, the first video data frame is subjected to video encoding format conversion to obtain the second video data frame, Specifically, the virtual display driver performs video encoding format conversion on the first video data frame to obtain the second video data frame.
结合本发明第三方面的实施例, 在第三种可能的实施方式中, 还 包括所述当所述客户端不支持视频数据帧解码时,并且当所述第一视 频数据帧为未解码的视频数据帧时, 向所述客户端发送第二视频数 据帧和第二视频数据帧处理指令之前,根据所述第一视频数据帧处理 指令中的所述视频数据帧解码指令对所述第一视频数据帧进行解码, 以获得所述第二视频数据帧。 With reference to the embodiment of the third aspect of the present invention, in a third possible implementation, the method further includes: when the client does not support video data frame decoding, and when the first video data frame is undecoded a video data frame, before the second video data frame and the second video data frame processing instruction are sent to the client, processed according to the first video data frame The video data frame decoding instruction in the instruction decodes the first video data frame to obtain the second video data frame.
结合本发明第三方面或第三方面的第三种可能的实施方式,在第 四种可能的实施方式中, 还包括: 当所述客户端不支持视频数据帧解 码时,向所述客户端发送所述第二视频数据帧和所述第二视频数据帧 处理指令之前, 压缩所述第二视频数据帧; 则向所述客户端发送所述 第二视频数据帧 ,具体为向所述客户端发送压缩后的第二视频数据帧。  With reference to the third aspect of the present invention or the third possible implementation manner of the third aspect, in a fourth possible implementation, the method further includes: when the client does not support video data frame decoding, to the client Before transmitting the second video data frame and the second video data frame processing instruction, compressing the second video data frame; sending the second video data frame to the client, specifically to the client The terminal sends the compressed second video data frame.
结合本发明第三方面,或者第三方面的第一至第四种任一可能的 实施方式, 在第五种可能的实施方式中, 还包括: 所述截获视频播放 器向虚拟显示驱动发送的第一视频数据帧和第一视频数据帧处理指 令之前, 获取所述客户端的视频数据帧处理能力; 所述客户端的视频 数据帧处理能力包括视频数据帧解码能力,以根据所述视频数据帧解 码能力, 确定所述客户端是否支持视频数据帧解码。  With reference to the third aspect of the present invention, or any one of the first to fourth possible embodiments of the third aspect, in a fifth possible implementation, the method further includes: sending, by the intercepted video player, a virtual display driver Obtaining a video data frame processing capability of the client before the first video data frame and the first video data frame processing instruction; the video data frame processing capability of the client includes a video data frame decoding capability, to decode according to the video data frame Capability, determining whether the client supports video data frame decoding.
结合本发明第三方面的第一、 第二或第五种可能的实施方式, 在 第六种可能的实施方式中, 还包括: 根据所述客户端的视频数据帧处 理能力, 确定所述客户端支持视频数据帧解码时, 向所述客户端发送 第二视频数据帧和所述第一视频数据帧处理指令之前,获取所述客户 端支持的视频数据帧的视频编码格式信息,以根据所述视频编码格式 信息 ,确定所述客户端是否支持所述第一视频数据帧的视频编码格式。  With reference to the first, second or fifth possible implementation manner of the third aspect of the present invention, in a sixth possible implementation, the method further includes: determining the client according to the video data frame processing capability of the client When the video data frame is decoded, the video encoding format information of the video data frame supported by the client is acquired before the second video data frame and the first video data frame processing instruction are sent to the client, according to the The video encoding format information determines whether the client supports the video encoding format of the first video data frame.
结合本发明第三方面,或者第三方面的第一至第六种任一可能的 实施方式, 在第七种可能的实施方式中, 所述截获视频播放器向虚拟 显示驱动发送的第一视频数据帧和第一视频数据帧处理指令,具体为 所述虚拟显示驱动截获所述第一视频数据帧和所述第一视频处理指 令。  With reference to the third aspect of the present invention, or any of the first to sixth possible implementation manners of the third aspect, in the seventh possible implementation manner, the intercepting video player sends the first video to the virtual display driver The data frame and the first video data frame processing instruction, in particular, the virtual display driver intercepts the first video data frame and the first video processing instruction.
第四方面, 本发明实施例公开了一种视频重定向系统, 包括客户 端和本发明实施例第二方面任一所述的服务器。 In a fourth aspect, an embodiment of the present invention discloses a video redirection system, including a client. And the server according to any one of the second aspects of the embodiments of the present invention.
第五方面, 本发明实施例公开了一种视频重定向系统, 包括客户 端和本发明实施例第三方面任一所述的服务器。  In a fifth aspect, the embodiment of the present invention discloses a video redirection system, including a client and a server according to any one of the third aspects of the embodiments of the present invention.
第六方面, 本发明实施例公开了一种计算机可读存储介质, 当计 算机执行所述计算机可读存储介质时, 所述计算机执行如下步骤: 截 获视频播放器向虚拟显示驱动发送的第一视频数据帧和第一视频数 据帧处理指令;  In a sixth aspect, an embodiment of the present invention discloses a computer readable storage medium. When the computer executes the computer readable storage medium, the computer performs the following steps: intercepting a first video sent by the video player to the virtual display driver. a data frame and a first video data frame processing instruction;
当客户端支持视频数据帧解码时,向所述客户端发送第二视频数 据帧和所述第一视频数据帧处理指令,以使所述客户端根据所述第一 视频数据帧处理指令对所述第二视频数据帧进行处理,所述第一视频 数据帧处理指令包括视频数据帧解码指令、视频数据帧后处理指令和 视频数据帧呈现指令,所述第二视频数据帧为所述第一视频数据帧或 为所述第一视频数据帧经过视频编码格式转换获得的视频数据帧; 当所述客户端不支持视频数据帧解码时,向所述客户端发送第二 视频数据帧和第二视频数据帧处理指令,以使所述客户端根据所述第 二视频数据帧处理指令对所述第二视频数据帧进行处理, 其中, 所述 第二视频数据帧处理指令包括视频数据帧后处理指令和视频数据帧 呈现指令; 当所述第一视频数据帧为解码后的视频数据帧时, 所述第 二视频数据帧为所述第一视频数据帧,所述第二视频数据帧处理指令 为所述第一视频数据帧处理指令; 当所述第一视频数据帧为未解码的 视频数据帧时, 所述第二视频数据帧为解码后的第一视频数据帧, 所 述第一视频数据帧处理指令包括视频数据帧解码指令、所述视频数据 帧后处理指令和所述视频数据帧呈现指令。  When the client supports video data frame decoding, sending a second video data frame and the first video data frame processing instruction to the client, so that the client processes the instruction according to the first video data frame Processing, the second video data frame instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, where the second video data frame is the first a video data frame or a video data frame obtained by converting the first video data frame by a video encoding format; when the client does not support video data frame decoding, sending the second video data frame and the second to the client The video data frame processing instruction, so that the client processes the second video data frame according to the second video data frame processing instruction, where the second video data frame processing instruction includes video data frame post processing Command and video data frame presentation instructions; when the first video data frame is a decoded video data frame, the second video data The frame is the first video data frame, the second video data frame processing instruction is the first video data frame processing instruction; when the first video data frame is an undecoded video data frame, the The second video data frame is a decoded first video data frame, and the first video data frame processing instruction includes a video data frame decoding instruction, the video data frame post-processing instruction, and the video data frame presentation instruction.
结合本发明第六方面, 在第一种可能的实施方式中, 还包括: 当 所述客户端支持视频数据帧解码时,并且所述客户端不支持所述视频 播放器向所述虚拟显示驱动发送的所述第一视频数据帧的视频编码 格式时, 将所述第一视频数据帧进行视频编码格式转换, 获得所述第 二视频数据帧; 其中, 所述第二视频数据帧的视频编码格式为所述客 户端支持的视频编码格式。 With reference to the sixth aspect of the present invention, in a first possible implementation, the method further includes: when the client supports video data frame decoding, and the client does not support the video When the player is in the video encoding format of the first video data frame sent by the virtual display driver, the first video data frame is subjected to video encoding format conversion to obtain the second video data frame; The video encoding format of the second video data frame is a video encoding format supported by the client.
结合本发明第六方面的第一种可能的实施方式,在第二种可能的 实施方式中, 所述将所述第一视频数据帧进行视频编码格式转换, 获 得所述第二视频数据帧, 具体包括: 所述虚拟显示驱动将所述第一视 频数据帧进行视频编码格式转换, 获得所述第二视频数据帧。  With reference to the first possible implementation manner of the sixth aspect of the present invention, in a second possible implementation manner, the first video data frame is subjected to video encoding format conversion to obtain the second video data frame, Specifically, the virtual display driver performs video encoding format conversion on the first video data frame to obtain the second video data frame.
结合本发明第六方面, 在第三种可能的实施方式中, 还包括当所 述客户端不支持视频数据帧解码时,并且当所述第一视频数据帧为未 解码的视频数据帧时, 向所述客户端发送第二视频数据帧和第二视 频数据帧处理指令之前,根据所述第一视频数据帧处理指令中的所述 视频数据帧解码指令对所述第一视频数据帧进行解码,以获得所述第 二视频数据帧。  With reference to the sixth aspect of the present invention, in a third possible implementation, the method further includes: when the client does not support video data frame decoding, and when the first video data frame is an undecoded video data frame, Decoding the first video data frame according to the video data frame decoding instruction in the first video data frame processing instruction before sending the second video data frame and the second video data frame processing instruction to the client Obtaining the second video data frame.
结合本发明第六方面或第六方面的第三种可能的实施方式,在第 四种可能的实施方式中, 还包括: 当所述客户端不支持视频数据帧解 码时,向所述客户端发送所述第二视频数据帧和所述第二视频数据帧 处理指令之前, 压缩所述第二视频数据帧; 则向所述客户端发送第二 视频数据帧, 具体为向所述客户端发送压缩后的第二视频数据帧。  With reference to the sixth aspect of the present invention or the third possible implementation manner of the sixth aspect, in a fourth possible implementation, the method further includes: when the client does not support video data frame decoding, to the client Before transmitting the second video data frame and the second video data frame processing instruction, compressing the second video data frame; sending a second video data frame to the client, specifically sending to the client The compressed second video data frame.
结合本发明第六方面,或者第六方面的第一至第四种任一可能的 实施方式, 在第五种可能的实施方式中, 还包括: 所述截获视频播放 器向虚拟显示驱动发送的第一视频数据帧和第一视频数据帧处理指 令之前, 获取所述客户端的视频数据帧处理能力; 所述客户端的视频 数据帧处理能力包括视频数据帧解码能力,以根据所述视频数据帧解 码能力, 确定所述客户端是否支持视频数据帧解码。 结合本发明第六方面的第一、 第二或第五种可能的实施方式, 在 第六种可能的实施方式中, 还包括: 根据所述客户端的视频数据帧处 理能力, 确定所述客户端支持视频数据帧解码时, 向所述客户端发送 第二视频数据帧和所述第一视频数据帧处理指令之前,获取所述客户 端支持的视频数据帧的视频编码格式信息,以根据所述视频编码格式 信息 ,确定所述客户端是否支持所述第一视频数据帧的视频编码格式。 With reference to the sixth aspect of the present invention, or any one of the first to fourth possible embodiments of the sixth aspect, in a fifth possible implementation, the method further includes: sending, by the intercepted video player, a virtual display driver Obtaining a video data frame processing capability of the client before the first video data frame and the first video data frame processing instruction; the video data frame processing capability of the client includes a video data frame decoding capability, to decode according to the video data frame Capability, determining whether the client supports video data frame decoding. With reference to the first, second or fifth possible implementation manner of the sixth aspect of the present invention, in a sixth possible implementation, the method further includes: determining, according to the video data frame processing capability of the client, the client When the video data frame is decoded, the video encoding format information of the video data frame supported by the client is acquired before the second video data frame and the first video data frame processing instruction are sent to the client, according to the The video encoding format information determines whether the client supports the video encoding format of the first video data frame.
结合本发明第六方面,或者本发明第六方面的第一至第六种任一 可能的实施方式, 在第七种可能的实施方式中, 所述截获视频播放器 向虚拟显示驱动发送的第一视频数据帧和第一视频数据帧处理指令, 具体为所述虚拟显示驱动截获所述第一视频数据帧和所述第一视频 处理指令。  With reference to the sixth aspect of the present invention, or any of the first to sixth possible implementation manners of the sixth aspect of the present invention, in the seventh possible implementation manner, the intercepting video player sends the first to the virtual display driver a video data frame and a first video data frame processing instruction, specifically, the virtual display driver intercepts the first video data frame and the first video processing instruction.
本发明实施例通过截获视频播放器向虚拟显示驱动发送的视频 数据帧和视频数据帧处理指令,并且根据客户端是否支持视频数据帧 解码, 将视频数据帧、 视频数据帧解码指令、 视频数据帧后处理指令 和视频数据帧呈现指令发送到客户端进行处理,或者将解码后的视频 数据帧、视频数据帧后处理指令和视频数据帧呈现指令发送到客户端 进行处理, 从而可以在客户端实现视频数据帧后处理, 保证用户视频 体验。同时只要服务器侧视频播放器支持视频重定向功能即可实现本 发明, 不需要依赖某一种特定的视频播放器。  The embodiment of the present invention intercepts the video data frame and the video data frame processing instruction sent by the video player to the virtual display driver, and according to whether the client supports video data frame decoding, the video data frame, the video data frame decoding instruction, and the video data frame. The post-processing instruction and the video data frame presentation instruction are sent to the client for processing, or the decoded video data frame, the video data frame post-processing instruction, and the video data frame presentation instruction are sent to the client for processing, thereby being implemented on the client side. The video data frame is post-processed to ensure the user's video experience. At the same time, as long as the server side video player supports the video redirection function, the present invention can be implemented without relying on a specific video player.
附图说明  DRAWINGS
为了更清楚地说明本发明实施例的技术方案,下面将对实施例或 现有技术描述中所需要使用的附图作简单地介绍, 显而易见地, 下面 描述中的附图仅仅是本发明的一些实施例,对于本领域普通技术人员 来讲, 在不付出创造性劳动的前提下, 还可以根据这些附图获得其他 的附图。 图 1为本发明实施例应用场景系统示意图; In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description are only some of the present invention. For the embodiments, those skilled in the art can obtain other drawings according to the drawings without any creative work. FIG. 1 is a schematic diagram of an application scenario system according to an embodiment of the present invention;
图 2为本发明实施例远程桌面服务器结构示意图;  2 is a schematic structural diagram of a remote desktop server according to an embodiment of the present invention;
图 3为本发明实施例视频数据帧后处理流程图;  3 is a flowchart of post processing of a video data frame according to an embodiment of the present invention;
图 4a和图 4b为本发明实施例视频数据帧后处理效果图; 图 5为本发明实施例视频数据帧呈现效果图;  4a and FIG. 4b are diagrams showing a post-processing effect of a video data frame according to an embodiment of the present invention; FIG. 5 is a diagram showing a rendering effect of a video data frame according to an embodiment of the present invention;
图 6a、 6b、 6c、 6d为本发明实施例虚拟桌面结构示意图; 图 7为本发明实施例视频重定向流程图;  6a, 6b, 6c, and 6d are schematic diagrams of a virtual desktop structure according to an embodiment of the present invention; FIG. 7 is a flowchart of video redirection according to an embodiment of the present invention;
图 8为本发明实施例服务器结构示意图;  8 is a schematic structural diagram of a server according to an embodiment of the present invention;
图 9为本发明实施例系统结构示意图。  FIG. 9 is a schematic structural diagram of a system according to an embodiment of the present invention.
具体实施方式  detailed description
下面将结合本发明实施例中的附图,对本发明实施例中的技术方 案进行清楚、 完整地描述, 显然, 所描述的实施例是本发明一部分实 施例, 而不是全部的实施例。 基于本发明中的实施例, 本领域普通技 术人员在没有作出创造性劳动前提下所获得的所有其他实施例 ,都属 于本发明保护的范围。  The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.
远程桌面环境中, 客户端发起视频播放请求, 服务器根据客户端 的视频播放请求, 通过视频播放器加载视频文件, 将视频文件的视频 流传输到客户端, 这一过程称为视频重定向。 本发明实施例中的服务 器可以为普通个人计算机或其他终端, 也可以为商业服务器等, 在这 里并不具体限定, 本发明实施例以向客户端提供基于 Windows®操作 系统平台的远程桌面服务为例对发明方案进行详细描述。但是本发明 方案并不仅限于 Windows®操作系统, 根据本发明方案, 同样可以扩 展到其他常见的系统平台, 如 Linux®操作系统等。  In the remote desktop environment, the client initiates a video play request, and the server loads the video file through the video player according to the video play request of the client, and transmits the video stream of the video file to the client. This process is called video redirection. The server in the embodiment of the present invention may be a general personal computer or other terminal, or may be a commercial server, etc., and is not specifically limited herein. The embodiment of the present invention provides a remote desktop service based on the Windows® operating system platform to the client. The invention is described in detail by way of example. However, the solution of the present invention is not limited to the Windows® operating system. According to the solution of the present invention, it can be extended to other common system platforms, such as the Linux® operating system.
本发明实施例公开了视频重定向的方法, 应用于远程桌面场景。 远程桌面场景通常由本地计算机和远程计算机组成。本地计算机通过 访问远程计算机,将远程计算机桌面传送到本地计算机。本地计算机, 即本发明实施例中的客户端; 远程计算机, 即本发明实施例中的服务 器。本发明实施例中的视频重定向是指将服务器上的视频文件的视频 数据流传送到客户端。其中一种远程桌面场景为虚拟桌面基础架构场 景, 但是本发明并不局限于图 1所示架构, 本发明方案可以应用于其 他用于传送远程视频数据到客户端的场景。虚拟桌面基础架构场景如 图 1所示, 本实施场景仅是一种示例性说明, 只是给出了与本发明实 施例相关的组成部分, 但是图 1并不是对本发明应用的具体限定。 图 1中左侧是虚拟桌面的客户端, 客户端的设备形式可以是普通计算机 101a, 平板电脑 101b、 智能手机 101c等。 它们通过网络 102使用远 程桌面协议 103访问远程桌面服务。 服务器 204a....204η提供了远程 桌面的载体, 用户的虚拟桌面(Virtual Desktop)以虚拟机 205a、 205b—205n的形式运行于服务器上。 虚拟桌面管理系统 106, 用于提 供用户的客户端与虚拟机的映射等功能。客户端首先连接到虚拟桌面 管理系统 106, 获取用户的虚拟机地址, 进而连接到虚拟机, 虚拟桌 面管理系统 106可以为服务器, 也可以为普通个人计算机等, 本发明 对此不作具体限定。用户通过客户端访问服务器上分配给该用户的虚 拟桌面 (即虚拟机), 该虚拟桌面将用户访问的内容传输到用户的客 户端进行显示。 The embodiment of the invention discloses a method for video redirection, which is applied to a remote desktop scenario. Remote desktop scenarios typically consist of a local computer and a remote computer. Local computer pass Access the remote computer and transfer the remote computer desktop to the local computer. The local computer, that is, the client in the embodiment of the present invention; the remote computer, that is, the server in the embodiment of the present invention. The video redirection in the embodiment of the present invention refers to transmitting the video data stream of the video file on the server to the client. One of the remote desktop scenarios is a virtual desktop infrastructure scenario, but the present invention is not limited to the architecture shown in FIG. 1. The solution of the present invention can be applied to other scenarios for transmitting remote video data to a client. The virtual desktop infrastructure scenario is shown in FIG. 1. This embodiment is merely an exemplary description, and only the components related to the embodiment of the present invention are given, but FIG. 1 is not specifically limited to the application of the present invention. The left side of FIG. 1 is a client of a virtual desktop, and the device form of the client may be a general computer 101a, a tablet 101b, a smart phone 101c, and the like. They access the remote desktop service over the network 102 using the Remote Desktop Protocol 103. The servers 204a....204n provide a carrier for the remote desktop, and the user's virtual desktop (Virtual Desktop) runs on the server in the form of virtual machines 205a, 205b-205n. The virtual desktop management system 106 is configured to provide functions such as mapping of a client's client to a virtual machine. The client first connects to the virtual desktop management system 106, obtains the virtual machine address of the user, and then connects to the virtual machine. The virtual desktop management system 106 can be a server or a general personal computer. The present invention does not specifically limit this. The user accesses the virtual desktop (ie, virtual machine) assigned to the user through the client, and the virtual desktop transmits the content accessed by the user to the user's client for display.
以服务器 204a为例, 硬件结构和软件结构是虚拟桌面基础架构 中最重要的组件, 它的典型结构如图 2所示。 服务器 204a中的典型 结构包括 3个部分, 分别为硬件平台 300、 虚拟化平台 (也可以称为 Hypervisor ) 301和虚拟机执行空间 302。硬件平台 300中可以包括服 务器物理结构中的硬盘 310、 中央处理器 311、 网络接口卡 312和存 储器 313。 Hypervisor301和虚拟机执行空间 302分别提供了虚拟机的 硬件平台和软件平台。 Hypervisor 包含了虚拟机监视器 320a、 320b—320n。 Hypervisor是物理硬件和虚拟机之间的中间层, 负责协 调各个虚拟机对硬件平台的访问。虚拟机监视器包括实现虚拟机的虚 拟硬件平台。图 2中,虚拟机监视器 320a中包含虚拟硬件平台 321a, 虚拟硬件平台中包含了虚拟出来的存储器 322, 中央处理器 323 , 硬 盘 324, 显卡 325和网络接口卡 326等。 虚拟机执行空间 302中运行 了多个虚拟机 205a、 205b ·205ηο 虚拟机 205a的软件系统运行在虚 拟机监视器 320a提供的虚拟硬件平台 321a上,其他虚拟机的软件系 统也运行在对应的虚拟机监视器的虚拟硬件平台上。 客户操作系统 330运行在虚拟机 205a中, 客户操作系统 330包含了一个设备驱动 层 331 , 设备驱动层 331安装了虚拟显示驱动 334、 网络接口卡驱动 336等驱动程序。 虚拟机 205a中包括视频播放器 332。 其中, 客户操 作系统 330可以为微软 ®公司提供的 Windows®操作系统平台。 Taking the server 204a as an example, the hardware structure and the software structure are the most important components in the virtual desktop infrastructure, and its typical structure is shown in FIG. 2. A typical structure in the server 204a includes three parts, a hardware platform 300, a virtualization platform (also referred to as a hypervisor) 301, and a virtual machine execution space 302. The hard disk 310, the central processing unit 311, the network interface card 312, and the memory 313 in the physical structure of the server may be included in the hardware platform 300. Hypervisor 301 and virtual machine execution space 302 provide virtual machine respectively Hardware platform and software platform. The Hypervisor includes virtual machine monitors 320a, 320b-320n. Hypervisor is the middle layer between physical hardware and virtual machines, and is responsible for coordinating the access of each virtual machine to the hardware platform. The virtual machine monitor includes a virtual hardware platform that implements the virtual machine. In FIG. 2, the virtual machine monitor 320a includes a virtual hardware platform 321a. The virtual hardware platform includes a virtual memory 322, a central processing unit 323, a hard disk 324, a graphics card 325, a network interface card 326, and the like. Running in a virtual machine execution space 302 multiple virtual machines 205a, 205b · 205η ο 205a of the virtual machine software system running on a virtual machine monitor 320a provides a virtual hardware platform 321a, other virtual machine software system is also running in the corresponding Virtual machine monitor on the virtual hardware platform. The guest operating system 330 runs in the virtual machine 205a. The guest operating system 330 includes a device driver layer 331, and the device driver layer 331 is installed with drivers such as a virtual display driver 334 and a network interface card driver 336. A video player 332 is included in the virtual machine 205a. Among them, the guest operating system 330 can be a Windows® operating system platform provided by Microsoft®.
如图 2所示, 用户使用客户端 101a登陆到虚拟机 205a观看视频 时, 比如观看优酷 ®网络视频, 或者虚拟机 205a本地视频文件时, 通过视频播放器 332, 加载视频文件。 用户使用客户端 101a登陆到 虚拟机 205a时, 向虚拟机 205a提供客户端 101a的视频数据帧处理 能力, 其中, 视频数据帧处理能力包括视频数据帧解码能力, 还向虚 拟机 205a提供客户端 101a支持的视频数据帧的视频编码格式信息, 比如支持 H.264、 AVI和 FLV等视频编码格式的视频文件。 本发明实 施例以观看本地视频为例 ,通过桌面协议服务端 335接收到来自客户 端 101a的视频播放请求, 视频播放器 332根据该视频播放请求, 加 载视频文件。视频播放器 332对加载的视频文件解复用获得视频数据 帧。 视频播放器 332在初始化时需要确定虚拟机 205a是否支持视频 硬件加速功能,视频播放器 332向虚拟显示驱动 334发送视频硬件加 速能力查询请求。视频硬件加速是指利用计算机硬件代替视频播放器 的软件算法对视频数据帧进行处理,具体可以包括对视频数据帧进行 解码和视频数据帧后处理。 As shown in FIG. 2, when the user logs in to the virtual machine 205a to view the video using the client 101a, such as watching Youku® network video, or the virtual machine 205a local video file, the video file is loaded by the video player 332. When the user logs in to the virtual machine 205a using the client 101a, the video data frame processing capability of the client 101a is provided to the virtual machine 205a, wherein the video data frame processing capability includes the video data frame decoding capability, and the client 101a is also provided to the virtual machine 205a. Video encoding format information of supported video data frames, such as video files supporting video encoding formats such as H.264, AVI, and FLV. In the embodiment of the present invention, a local video is taken as an example, and a video play request from the client 101a is received by the desktop protocol server 335, and the video player 332 loads the video file according to the video play request. The video player 332 demultiplexes the loaded video file to obtain a video data frame. The video player 332 needs to determine whether the virtual machine 205a supports the video hardware acceleration function during initialization, and the video player 332 sends the video hardware plus to the virtual display driver 334. Speed capability query request. Video hardware acceleration refers to processing a video data frame by using a computer software instead of a software algorithm of a video player, and specifically includes decoding a video data frame and post-processing a video data frame.
其中关于视频数据帧后处理,本实施例中以 Mircosoft®的视频硬 件加速( DirectX® Video Acceleration, 简称为 DXVA )为例, 对视频 数据帧后处理及经视频数据帧后处理后的视频数据帧呈现进行描述。 DXVA不仅仅提供视频数据帧解码处理, 还包括视频数据帧后处理, 比如图像缩放、 色彩空间转换和反交错等。 如图 3所示, DXVA为视 频数据帧后处理流程图。 DXVA可以对主数据流和 15路分数据流进 行处理, 将主数据流进行噪声过滤、 反交错处理、 细节过滤和色彩空 间转换后, 与经过色彩空间转换后的分数据流进行混合, 将混合后的 视频数据流进行图像调整, 其中图像调整包括亮度、 对比度、 饱和度 调整等。 将经过图像调整后的视频数据流进行图像缩放, 从而可以调 整图像大小。 将经图像缩放处理后的数据进行色彩空间转换, 从而使 视频数据流进行视频数据帧呈现。其中,颜色空间转换, 包括由 RGB 色彩空间转换至 YUV色彩空间, 或者由 YUV色彩空间转换至 RGB 色彩空间,其中 RGB分别表示红、绿和蓝色, YUV分别表示明亮度、 色度和浓度, 本发明实施例在此不再赘述。  For video data frame post-processing, in this embodiment, Mircosoft® video hardware acceleration (DirectX® Video Acceleration, referred to as DXVA) is taken as an example, and the video data frame is post-processed and the video data frame is post-processed. Presentation is described. DXVA not only provides video data frame decoding processing, but also video data frame post-processing, such as image scaling, color space conversion and de-interlacing. As shown in Figure 3, DXVA is a post-processing flowchart for video data frames. DXVA can process the main data stream and the 15-way sub-data stream. After the main data stream is subjected to noise filtering, de-interlacing processing, detail filtering and color space conversion, it is mixed with the color space-converted sub-data stream and mixed. The subsequent video data stream is subjected to image adjustment, wherein the image adjustment includes brightness, contrast, saturation adjustment, and the like. The image-scaled video stream is image scaled to adjust the image size. The image-scaled data is subjected to color space conversion, so that the video data stream is subjected to video data frame presentation. Among them, color space conversion, including conversion from RGB color space to YUV color space, or conversion from YUV color space to RGB color space, where RGB represents red, green and blue, respectively, YUV represents brightness, chromaticity and concentration, respectively. The embodiments of the present invention are not described herein again.
图 3描述了视频后处理的过程。实际应用中不一定所有的功能都 会用到。 实际测试中发现, 多数播放器如 Microsoft® Media Player, 暴风影音⑧, QQ®影音播放器等默认只使用了图像缩放功能, 用以将 解码后的视频数据帧从原始大小缩放到播放窗口大小。本发明实施例 中进行视频数据帧后处理具体以图像缩放为例,将解码后的视频数据 帧进行图像缩放, 将原始图像大小调整以适合在显示器上呈现。 具体 效果如图 4a所示。 视频数据帧后处理还可以包括透明度混合, 如图 4b 所示。 本发明实施例中视频数据帧后处理还可以包括亮度、 对比 度和饱和度调整。 从而使客户端呈现给用户更好的视频体验效果。 Figure 3 depicts the process of video post processing. Not all functions in the actual application will be used. In actual tests, most players such as Microsoft® Media Player, Storm Video 8, QQ® Video Player, etc. only use image scaling by default to scale the decoded video data frame from the original size to the playback window size. In the embodiment of the present invention, the video data frame post-processing is specifically performed by taking image scaling as an example, and the decoded video data frame is subjected to image scaling, and the original image size is adjusted to be suitable for presentation on the display. The specific effect is shown in Figure 4a. Video data frame post-processing can also include transparency blending, as shown 4b is shown. The video data frame post-processing in the embodiment of the present invention may further include brightness, contrast, and saturation adjustment. This allows the client to present a better video experience to the user.
将解码后的视频数据帧进行视频数据帧后处理,根据视频数据帧 呈现指令, 将经视频数据帧后处理的视频数据帧发送到显卡, 通过显 示器进行呈现。 其中, 视频数据帧呈现指令携带呈现参数, 呈现参数 包括要呈现的视频数据帧和视频数据帧的呈现区域。 如图 5所示, 经 过视频数据帧后处理之后形成的图像, 包括需要呈现的 4个区域, 分 别表示为①、 ②、 ③和④, 以及桌面图像, 在显示器上呈现, 其中, 桌面图像有一处区域用于对用户呈现 "显示设定、 音频设定和字幕设 定" 选项, 在呈现视频时, 同时需要呈现 "显示设定、 音频设定和字 幕设定" 选项区域, 而该 "显示设定、 音频设定和字幕设定" 选项区 域与①、 ②、 ③和④四个呈现区域围成的中间区域对应。 因此, ①、 ②、③和④四个呈现区域围成的中间区域对应的视频数据帧不需要呈 现, 构成①、 ②、 ③和④四个呈现区域围成的中间区域的视频数据帧 不呈现该视频数据帧。从而使视频画面与桌面融合为一体, 为用户提 供更好的视频观看体验。 本发明实施例中客户端 101a进行视频数据 帧后处理原理与此相同, 不再赘述。  The decoded video data frame is subjected to video data frame post-processing, and the video data frame post-processed by the video data frame is sent to the graphics card according to the video data frame presentation instruction, and is presented by the display. The video data frame presentation instruction carries a presentation parameter, and the presentation parameter includes a video data frame to be presented and a presentation area of the video data frame. As shown in FIG. 5, an image formed after post-processing of a video data frame includes four regions to be presented, denoted as 1, 2, 3, and 4, respectively, and a desktop image, which is presented on a display, wherein the desktop image has The area is used to present the user with the "display settings, audio settings, and subtitle settings" options. When the video is presented, the "display settings, audio settings, and subtitle settings" option areas need to be presented, and the "display" The setting, audio setting, and subtitle setting "option area corresponds to the intermediate area enclosed by the four presentation areas 1, 2, 3, and 4. Therefore, the video data frames corresponding to the intermediate areas surrounded by the four presentation areas 1, 2, 3, and 4 do not need to be presented, and the video data frames constituting the intermediate areas surrounded by the four presentation areas 1, 2, 3, and 4 are not presented. The video data frame. This allows the video picture to be integrated with the desktop, providing users with a better video viewing experience. The principle of the video data frame post-processing performed by the client 101a in the embodiment of the present invention is the same as that of the present invention and will not be described again.
在本发明实施例中, 这里虚拟机 205a的硬件, 本领域技术人员 可以理解, 并不是虚拟机 205a的真实物理硬件, 而是通过虚拟机监 视器 320a提供的虚拟硬件。 视频播放器 332向虚拟显示驱动 334查 询虚拟机 205a的视频硬件加速能力, 可以通过调用 GetCaps函数进 行查询, 或者, 可以通过调用 CreateDevice或 CreateDecodeDevice函 数创建硬件加速设备,确定虚拟机 205a是否支持视频硬件加速功能。 当视频播放器 332启动向虚拟显示驱动 334查询虚拟机 205a的视频 硬件加速能力时,虚拟显示驱动 334向视频播放器 332返回支持视频 硬件加速能力的查询请求响应,视频播放器 332根据该查询请求响应 确定虚拟机 205a支持的视频硬件加速能力, 其中, 视频硬件加速能 力包括视频数据帧解码能力和视频数据帧后处理能力。 In the embodiment of the present invention, the hardware of the virtual machine 205a, as understood by those skilled in the art, is not the real physical hardware of the virtual machine 205a, but the virtual hardware provided by the virtual machine monitor 320a. The video player 332 queries the virtual display driver 334 for the video hardware acceleration capability of the virtual machine 205a, and can be queried by calling the GetCaps function. Alternatively, the hardware acceleration device can be created by calling the CreateDevice or CreateDecodeDevice function to determine whether the virtual machine 205a supports video hardware acceleration. Features. When the video player 332 initiates a video querying the virtual display driver 334 for the virtual machine 205a In the hardware acceleration capability, the virtual display driver 334 returns a query request response supporting the video hardware acceleration capability to the video player 332, and the video player 332 determines the video hardware acceleration capability supported by the virtual machine 205a according to the query request response, wherein the video hardware acceleration Capabilities include video data frame decoding capabilities and video data frame post-processing capabilities.
视频播放器 332向虚拟显示驱动 334发送视频数据帧和视频数据 帧处理指令, 如图 6a所示, 一种实现方式为虚拟显示驱动 334截获 视频播放器 332发送的视频数据帧和视频数据帧处理指令。  The video player 332 sends the video data frame and the video data frame processing instruction to the virtual display driver 334. As shown in FIG. 6a, an implementation manner is that the virtual display driver 334 intercepts the video data frame and the video data frame processed by the video player 332. instruction.
下面具体描述虚拟显示驱动 334 截获视频数据帧和视频数据帧 处理指令的过程。 具体地, 虚拟机 205a中的视频播放器 332根据对 应的客户端 101a发送的视频播放请求, 视频播放器 332加载视频文 件之后, 开启视频硬件加速, 向虚拟显示驱动 334发送视频数据帧和 视频数据帧处理指令。 加载视频文件后, 对加载后的视频文件, 即对 视频数据流解复用, 获得其中的视频数据帧。 具体地, 就视频播放而 言, 在 DirectX®运行库 333 中定义了视频硬件加速接口, 通过视频 播放器 332调用 DirectX®运行库 333启动视频硬件加速, 通知虚拟 显示驱动 334根据视频数据帧处理指令对视频播放器 332发送的视频 数据帧进行处理。 当视频数据帧和视频数据帧处理指令发送至虚拟显 示驱动 334时,虚拟显示驱动 334截获视频数据帧和视频数据帧处理 指令。 如图 6a所示, 在 Windows®操作系统平台, 虚拟显示驱动为 Windows® Display Driver Model (简称 WDDM显示驱动模型)或者为 Windows® XP Display Driver Model (简称 XPDM显示驱动模型)。 具 体地, WDDM 或 XPDM 显示驱动模型均包括 Display Driver 和 Miniport Driver两部分模块。 本发明实施例中, 一种实现方案为虚拟 显示驱动的 Display Driver模块截获视频数据帧和视频数据帧处理指 令。 具体地, 一种实现方案为 WDDM 或 XPDM显示驱动模型在 Display Driver模块中定义视频硬件加速接口, 当视频数据帧和视频 数据帧处理指令通过该接口时,截获该视频数据帧和视频数据帧处理 指令。 The process by which the virtual display driver 334 intercepts the video data frame and video data frame processing instructions is specifically described below. Specifically, the video player 332 in the virtual machine 205a, according to the video play request sent by the corresponding client 101a, after the video player 332 loads the video file, starts the video hardware acceleration, and sends the video data frame and the video data to the virtual display driver 334. Frame processing instructions. After the video file is loaded, the loaded video file, that is, the video data stream is demultiplexed, to obtain a video data frame therein. Specifically, in terms of video playback, a video hardware acceleration interface is defined in the DirectX® runtime library 333, and the video hardware acceleration is initiated by the video player 332 calling the DirectX® runtime library 333, informing the virtual display driver 334 to process the instruction according to the video data frame. The video data frame transmitted by the video player 332 is processed. When the video data frame and video data frame processing instructions are sent to the virtual display driver 334, the virtual display driver 334 intercepts the video data frame and video data frame processing instructions. As shown in Figure 6a, on the Windows® operating system platform, the virtual display driver is Windows® Display Driver Model (referred to as WDDM display driver model) or Windows® XP Display Driver Model (referred to as XPDM display driver model). Specifically, the WDDM or XPDM display driver models include the Display Driver and Miniport Driver modules. In the embodiment of the present invention, an implementation scheme is that the display driver module of the virtual display driver intercepts the video data frame and the video data frame processing instruction. Specifically, an implementation scheme is a WDDM or XPDM display driver model. The video hardware acceleration interface is defined in the Display Driver module. When the video data frame and the video data frame processing instruction pass through the interface, the video data frame and the video data frame processing instruction are intercepted.
本发明实施例截获视频数据帧和视频数据帧处理指令的另一种 实施方式, 如图 6b所示, 虚拟机 205a中的视频播放器 332根据对应 的客户端 101a发送的视频播放请求, 视频播放器 332加载视频文件 之后, 开启视频硬件加速, 向虚拟显示驱动 334发送视频数据帧和视 频数据帧处理指令。视频播放器 332加载视频文件后, 对加载后的视 频文件, 即对视频数据流解复用, 获得其中的视频数据帧。 具体地, 就视频播放而言, 在 DirectX®运行库 333 中定义了视频硬件加速接 口, 通过视频播放器 332调用 DirectX®运行库 333启动视频硬件加 速, 通知虚拟显示驱动 334 根据视频数据帧处理指令对视频播放器 332发送的视频数据帧进行处理。 DirectX®运行库 333调用所述虚拟 显示驱动 334对所述视频数据帧进行处理时,即在视频数据帧和视频 数据帧处理指令通过 DirectX®运行库 333后,到达虚拟显示驱动 334 之前,动态链接库截获该视频数据帧和视频数据帧处理指令。具体地, 其中一种动态链接库实现方式为釆用 Hook技术截获 DirectX®运行库 333对虚拟显示驱动 334的调用, 需要实现一个对虚拟显示驱动 334 Hook的动态链接库文件 (dll文件),并在注册表中将其设为最先加载, 以此拦截 DirectX®运行库 333对虚拟显示驱动 334的调用 , 从而实 现视频数据帧和视频数据帧处理指令的截获。  The embodiment of the present invention intercepts another embodiment of the video data frame and the video data frame processing instruction. As shown in FIG. 6b, the video player 332 in the virtual machine 205a performs video playback according to the video playback request sent by the corresponding client 101a. After the video file is loaded by the device 332, the video hardware acceleration is turned on, and the video data frame and the video data frame processing instruction are transmitted to the virtual display driver 334. After the video player 332 loads the video file, the loaded video file, that is, the video data stream is demultiplexed, to obtain a video data frame therein. Specifically, in terms of video playback, a video hardware acceleration interface is defined in the DirectX® runtime library 333, and the video hardware acceleration is initiated by the video player 332 calling the DirectX® runtime library 333, informing the virtual display driver 334 to process the instruction according to the video data frame. The video data frame transmitted by the video player 332 is processed. The DirectX® runtime library 333 invokes the virtual display driver 334 to process the video data frame, ie, after the video data frame and video data frame processing instructions pass through the DirectX® runtime 333, before the virtual display driver 334 is reached, the dynamic link The library intercepts the video data frame and video data frame processing instructions. Specifically, one of the dynamic link library implementations is to use the Hook technology to intercept the call of the virtual display driver 334 by the DirectX® runtime library 333, and implement a dynamic link library file (dll file) for the virtual display driver 334 Hook, and It is set to be loaded first in the registry to intercept the call of the DirectX® runtime 333 to the virtual display driver 334 to enable interception of video data frames and video data frame processing instructions.
本发明实施例截获视频数据帧和视频数据帧处理指令的另一种 实施方式, 如图 6c所示, 虚拟机 205a中的视频播放器 332根据对应 的客户端 101a发送的视频播放请求, 视频播放器 332加载视频文件 之后, 开启视频硬件加速, 向虚拟显示驱动 334发送视频数据帧和视 频数据帧处理指令。视频播放器 332加载视频文件后, 对加载后的视 频文件, 即对视频数据流解复用, 获得其中的视频数据帧。 具体地, 就视频播放而言, 在 DirectX®运行库 333 中定义了视频硬件加速接 口, 通过视频播放器 332调用 DirectX®运行库 333向虚拟显示驱动 334发送视频数据帧和视频数据帧处理指令, 启动视频硬件加速, 通 知虚拟显示驱动 334对视频播放器 332发送的视频数据帧进行处理。 在视频数据帧和视频数据帧处理指令通过 DirectX®运行库 333 时, DirectX®运行库截获该视频数据帧和视频数据帧处理指令。 The embodiment of the present invention intercepts another embodiment of the video data frame and the video data frame processing instruction. As shown in FIG. 6c, the video player 332 in the virtual machine 205a performs video playback according to the video playback request sent by the corresponding client 101a. After the video file is loaded by the device 332, the video hardware acceleration is turned on, and the video data frame and the video are transmitted to the virtual display driver 334. Frequency data frame processing instructions. After loading the video file, the video player 332 demultiplexes the loaded video file, that is, the video data stream, to obtain a video data frame therein. Specifically, in terms of video playback, a video hardware acceleration interface is defined in the DirectX® runtime 333, and the DirectX® runtime 333 is called by the video player 332 to send video data frames and video data frame processing instructions to the virtual display driver 334. The video hardware acceleration is initiated, and the virtual display driver 334 is notified to process the video data frame sent by the video player 332. When the video data frame and video data frame processing instructions pass through the DirectX® runtime library 333, the DirectX® runtime intercepts the video data frame and video data frame processing instructions.
本发明实施例截获视频数据帧和视频数据帧处理指令的另一种 实施方式, 如图 6d所示, 虚拟机 205a中的视频播放器 332根据对应 的客户端 101a发送的视频播放请求, 视频播放器 332加载视频文件 之后, 开启视频硬件加速, 向虚拟显示驱动 334发送视频数据帧和视 频数据帧处理指令。视频播放器 332加载视频文件后, 对加载后的视 频文件, 即对视频数据流解复用, 获得其中的视频数据帧。 具体地, 就视频播放而言, 在 DirectX®运行库 333 中定义了视频硬件加速接 口,视频播放器 332调用 DirectX®运行库 333启动视频硬件加速, 发 送视频数据帧和视频数据帧处理指令时,即视频数据帧和视频数据帧 处理指令到达 DirectX®运行库 333之前, 动态链接库截获视频数据 帧和视频数据帧处理指令。 为此, 需要实现一个对 DirectX®运行库 333 Hook的动态链接库文件( dll文件), 并在注册表中将其设为最先 加载, 以此截获视频播放器 332对 DirectX®运行库 333的调用, 从 而实现视频数据帧和视频数据帧处理指令的截获。  The embodiment of the present invention intercepts another embodiment of the video data frame and the video data frame processing instruction. As shown in FIG. 6d, the video player 332 in the virtual machine 205a performs video playback according to the video playback request sent by the corresponding client 101a. After the video file is loaded by the device 332, the video hardware acceleration is turned on, and the video data frame and the video data frame processing instruction are transmitted to the virtual display driver 334. After the video player 332 loads the video file, the loaded video file, that is, the video data stream is demultiplexed, to obtain a video data frame therein. Specifically, in terms of video playback, a video hardware acceleration interface is defined in the DirectX® runtime library 333, and the video player 332 calls the DirectX® runtime library 333 to initiate video hardware acceleration, when transmitting video data frames and video data frame processing instructions. That is, before the video data frame and video data frame processing instructions arrive at the DirectX® runtime 333, the dynamic link library intercepts the video data frame and video data frame processing instructions. To do this, you need to implement a dynamic link library file (dll file) for the DirectX® runtime 333 Hook, and set it to be loaded first in the registry to intercept the video player 332 to the DirectX® runtime 333. Called to achieve interception of video data frames and video data frame processing instructions.
在上述图 6a、 图 6b、 图 6c和图 6d四种截获视频播放器 332向 虚拟显示驱动 334 发送的视频数据帧和视频数据帧处理指令的方案 中, 视频播放器 332向虚拟显示驱动 334查询获知虚拟机 205a支持 视频数据帧解码和视频数据帧后处理, 并且获取客户端 101a支持视 频数据帧解码和视频播放器 332向虚拟显示驱动 334发送的视频数据 帧的视频编码格式时, 截获视频数据帧和视频数据帧处理指令, 将截 获的视频数据帧和视频数据帧发送到桌面协议服务端 335。 桌面协议 服务端 335 将截获的视频数据帧和视频数据帧处理指令发送到客户 端 101a。 其中, 视频数据帧处理指令包括视频数据帧解码指令、 视 频数据帧后处理指令和视频数据帧呈现指令。 由于服务器 204a中的 因此, 节约了服务器的计算资源和网络传输带宽, 充分利用了客户端 的计算资源。 本实施例适用于如下场景: 截获视频播放器 332向虚拟 显示驱动 334发送的第一视频数据帧和第一视频数据帧处理指令; 当 客户端 101a支持视频数据帧解码时, 向所述客户端 101a发送第二视 频数据帧和所述第一视频数据帧处理指令,以使所述客户端 101a根据 所述第一视频数据帧处理指令对所述第二视频数据帧进行处理,所述 第一视频数据帧处理指令包括视频数据帧解码指令、视频数据帧后处 理指令和视频数据帧呈现指令,所述第二视频数据帧为所述第一视频 数据帧。 In the above-described schemes of capturing video data frames and video data frame processing instructions sent by the video player 332 to the virtual display driver 334 in FIGS. 6a, 6b, 6c, and 6d, the video player 332 queries the virtual display driver 334. Know the virtual machine 205a support The video data frame is decoded and the video data frame is post-processed, and the video data frame and the video data frame are captured when the client 101a supports the video data frame decoding and the video encoding format of the video data frame sent by the video player 332 to the virtual display driver 334. The processing instruction sends the intercepted video data frame and the video data frame to the desktop protocol server 335. The desktop protocol server 335 transmits the intercepted video data frame and video data frame processing instructions to the client 101a. The video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction. As a result of the server 204a, the computing resources and network transmission bandwidth of the server are saved, and the computing resources of the client are fully utilized. This embodiment is applicable to the following scenarios: intercepting a first video data frame and a first video data frame processing instruction sent by the video player 332 to the virtual display driver 334; and when the client 101a supports video data frame decoding, the client is 101a transmitting a second video data frame and the first video data frame processing instruction, so that the client 101a processes the second video data frame according to the first video data frame processing instruction, where the first The video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, and the second video data frame is the first video data frame.
在上述图 6a、 图 6b、 图 6c和图 6d四种截获视频播放器 332向 虚拟显示驱动 334 发送的视频数据帧和视频数据帧处理指令的方案 中, 视频播放器 332向虚拟显示驱动 334查询获知虚拟机 205a支持 视频数据帧解码和视频数据帧后处理, 并且客户端 101a支持视频数 据帧解码,但是不支持视频播放器 332向虚拟显示驱动 334发送的视 频数据帧的视频编码格式时,截获视频数据帧和视频数据帧处理指令, 其中视频数据帧处理指令包括视频数据帧解码指令、视频数据帧后处 理指令和视频数据帧呈现指令,桌面协议服务端 335接收截获的视频 数据帧,桌面协议服务端 335的视频编解码模块根据视频数据帧解码 指令对所述视频数据帧进行解码, 根据客户端 101a支持的视频数据 帧的视频编码格式对所述解码的视频数据帧进行编码,生成新的视频 数据帧解码指令,所述新的视频数据帧解码指令包含重新编码后的视 频数据帧的视频编码格式参数。发送重新编码后的视频数据帧和视频 数据帧处理指令到客户端 101a, 所述视频数据帧处理指令包括、 视 频数据帧后处理指令、 视频数据帧呈现指令和视频数据帧解码指令, 其中,视频数据帧解码指令包含重新编码后的视频数据帧的视频编码 格式参数。从而使得客户端能够根据自身的视频编码格式处理能力对 所述重新编码后的视频数据帧的进行解码。 由于服务器 204a中的虚 因此节约了服务器的计算资源和网络传输带宽,充分利用了客户端的 计算资源。 本实施例适用于如下场景: 截获视频播放器 332向虚拟显 示驱动 334发送的第一视频数据帧和第一视频数据帧处理指令; 当客 户端 101a支持视频数据帧解码时, 向所述客户端 101a发送第二视频 数据帧和所述第一视频数据帧处理指令,以使所述客户端 101a根据所 述第一视频数据帧处理指令对所述第二视频数据帧进行处理,所述第 一视频数据帧处理指令包括视频数据帧解码指令、视频数据帧后处理 指令和视频数据帧呈现指令,所述第二视频数据帧为所述第一视频数 据帧经过视频编码格式转换获得的视频数据帧。 In the above-described schemes of capturing video data frames and video data frame processing instructions sent by the video player 332 to the virtual display driver 334 in FIGS. 6a, 6b, 6c, and 6d, the video player 332 queries the virtual display driver 334. It is learned that the virtual machine 205a supports video data frame decoding and video data frame post-processing, and the client 101a supports video data frame decoding, but does not support the video encoding format of the video data frame transmitted by the video player 332 to the virtual display driver 334. a video data frame and a video data frame processing instruction, where the video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, and the desktop protocol server 335 receives the intercepted video. The data frame, the video codec module of the desktop protocol server 335 decodes the video data frame according to the video data frame decoding instruction, and performs the decoded video data frame according to the video encoding format of the video data frame supported by the client 101a. Encoding, generating a new video data frame decoding instruction, the new video data frame decoding instruction including a video encoding format parameter of the re-encoded video data frame. Transmitting the re-encoded video data frame and the video data frame processing instruction to the client 101a, where the video data frame processing instruction includes, a video data frame post-processing instruction, a video data frame presentation instruction, and a video data frame decoding instruction, where the video The data frame decoding instruction includes video encoding format parameters of the re-encoded video data frame. Thereby enabling the client to decode the re-encoded video data frame according to its own video encoding format processing capability. Due to the virtuality in the server 204a, the computing resources and network transmission bandwidth of the server are saved, and the computing resources of the client are fully utilized. This embodiment is applicable to the following scenarios: intercepting a first video data frame and a first video data frame processing instruction sent by the video player 332 to the virtual display driver 334; and when the client 101a supports video data frame decoding, the client is 101a transmitting a second video data frame and the first video data frame processing instruction, so that the client 101a processes the second video data frame according to the first video data frame processing instruction, where the first The video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, where the second video data frame is a video data frame obtained by converting the first video data frame by a video encoding format. .
在另一种实施方式中, 在如图 6a所示的虚拟显示驱动 334截获 视频播放器 332向虚拟显示驱动 334发送的视频数据帧和视频数据帧 处理指令的方案中,视频播放器 332向虚拟显示驱动 334查询获知虚 拟机 205a 支持视频数据帧解码和视频数据帧后处理, 并且客户端 101a支持视频数据帧解码, 但是不支持视频播放器 332向虚拟显示 驱动 334 发送的视频数据帧的视频编码格式时, 所述虚拟显示驱动 334中的视频编解码模块根据截获的视频数据帧解码指令对所述截获 的视频数据帧进行解码, 根据客户端 101a支持的视频数据帧的视频 编码格式对所述解码的视频数据帧进行编码,生成新的视频数据帧解 码指令,所述新的视频数据帧解码指令包含重新编码后的视频数据帧 的视频编码格式参数, 所述视频数据帧处理指令包括、视频数据帧后 处理指令、 视频数据帧呈现指令和视频数据帧解码指令, 其中, 视频 数据帧解码指令包含重新编码后的视频数据帧的视频编码格式参数。 从而使得客户端能够根据自身的视频编码格式处理能力对所述重新 编码后的视频数据帧的进行解码。桌面协议服务端 335发送重新编码 后的视频数据帧和视频数据帧处理指令到客户端 101a, 所述视频数 据帧处理指令包括、视频数据帧后处理指令、视频数据帧呈现指令和 视频数据帧解码指令, 其中, 视频数据帧解码指令包含重新编码后的 视频数据帧的视频编码格式参数。从而使得客户端能够根据自身的视 频编码格式处理能力对所述重新编码后的视频数据帧的进行解码。同 时服务器 204a上的虚拟机 205a没有根据视频数据帧后处理指令对视 频数据帧进行处理, 而是由客户端处理, 充分利用了客户端的计算资 源, 同时节省了服务器的计算资源。 本实施例适用于如下场景: 截获 视频播放器 332向虚拟显示驱动 334发送的第一视频数据帧和第一视 频数据帧处理指令; 当客户端 101a支持视频数据帧解码时, 向所述 客户端 101a发送第二视频数据帧和所述第一视频数据帧处理指令,以 使所述客户端 101a根据所述第一视频数据帧处理指令对所述第二视 频数据帧进行处理,所述第一视频数据帧处理指令包括视频数据帧解 码指令、视频数据帧后处理指令和视频数据帧呈现指令, 所述第二视 频数据帧为所述第一视频数据帧经过视频编码格式转换获得的视频 数据帧。 In another embodiment, the video player 332 is virtualized in a scheme in which the virtual display driver 334 as shown in FIG. 6a intercepts the video data frame and the video data frame processing instructions transmitted by the video player 332 to the virtual display driver 334. The display driver 334 queries the learned virtual machine 205a to support video data frame decoding and video data frame post-processing, and the client 101a supports video data frame decoding, but does not support the video player 332 to virtual display. When the video encoding format of the video data frame transmitted by 334 is driven, the video codec module in the virtual display driver 334 decodes the captured video data frame according to the intercepted video data frame decoding instruction, according to the support of the client 101a. The video encoding format of the video data frame encodes the decoded video data frame to generate a new video data frame decoding instruction, where the new video data frame decoding instruction includes a video encoding format parameter of the re-encoded video data frame, The video data frame processing instruction includes a video data frame post-processing instruction, a video data frame presentation instruction, and a video data frame decoding instruction, where the video data frame decoding instruction includes a video encoding format parameter of the re-encoded video data frame. Thereby enabling the client to decode the re-encoded video data frame according to its own video encoding format processing capability. The desktop protocol server 335 sends the re-encoded video data frame and the video data frame processing instruction to the client 101a, where the video data frame processing instruction includes, the video data frame post-processing instruction, the video data frame presentation instruction, and the video data frame decoding. The instruction, wherein the video data frame decoding instruction includes a video encoding format parameter of the re-encoded video data frame. Thereby enabling the client to decode the re-encoded video data frame according to its own video encoding format processing capability. At the same time, the virtual machine 205a on the server 204a does not process the video data frame according to the video data frame post-processing instruction, but is processed by the client, which fully utilizes the computing resources of the client, and saves the computing resources of the server. This embodiment is applicable to the following scenarios: intercepting a first video data frame and a first video data frame processing instruction sent by the video player 332 to the virtual display driver 334; and when the client 101a supports video data frame decoding, the client is 101a transmitting a second video data frame and the first video data frame processing instruction, so that the client 101a processes the second video data frame according to the first video data frame processing instruction, where the first The video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, where the second video data frame is a video obtained by converting the first video data frame by a video encoding format. Data Frame.
在上述图 6a、 图 6b、 图 6c和图 6d四种截获视频播放器 332向 虚拟显示驱动 334 发送的视频数据帧和视频数据帧处理指令的方案 中, 当客户端 101a不支持视频数据帧解码, 并且视频播放器 332向 虚拟显示驱动 334查询获知虚拟机 205a支持视频数据帧后处理但不 支持视频数据帧解码时, 视频播放器 332加载视频文件后, 对视频文 件解复用, 获取视频数据帧, 视频播放器 332对所述视频数据帧进行 解码, 获得解码后的视频数据帧, 视频播放器 332 向虚拟显示驱动 334发送解码后的视频数据帧和视频数据帧处理指令; 其中视频数据 视频播放器 332发送的解码后的视频数据帧和视频数据帧处理指令, 桌面协议服务端 335 发送解码后的视频数据帧和视频数据帧处理指 令至客户端 101a。 为减少网络传输带宽, 桌面协议服务端 335发送 解码后的视频数据帧和视频数据帧处理指令之前,可以压缩解码后的 视频数据帧, 具体可以由桌面协议服务端 335中的压缩模块来实现。 由于服务器 204a中的虚拟机 205a并没有根据视频数据帧后处理指令 对视频数据帧进行处理, 因此节约了服务器的计算资源和网络传输带 宽, 充分利用了客户端的计算资源。 本实施例适用如下场景: 截获视 频播放器 332向虚拟显示驱动 334发送的第一视频数据帧和第一视频 数据帧处理指令; 当所述客户端 101a不支持视频数据帧解码时, 向 所述客户端 101a发送第二视频数据帧和第二视频数据帧处理指令, 以使所述客户端 101a根据所述第二视频数据帧处理指令对所述第二 视频数据帧进行处理, 其中, 所述第二视频数据帧处理指令包括视频 数据帧后处理指令和视频数据帧呈现指令; 当所述第一视频数据帧为 解码后的视频数据帧时 ,所述第二视频数据帧为所述第一视频数据帧 , 所述第二视频数据帧处理指令为所述第一视频数据帧处理指令。 In the above-mentioned schemes of capturing video data frames and video data frame processing instructions sent by the video player 332 to the virtual display driver 334 in FIGS. 6a, 6b, 6c, and 6d, when the client 101a does not support video data frame decoding. And the video player 332 queries the virtual display driver 334 to learn that the virtual machine 205a supports the video data frame post-processing but does not support the video data frame decoding. After the video player 332 loads the video file, the video file is demultiplexed to obtain the video data. a frame, the video player 332 decodes the video data frame to obtain a decoded video data frame, and the video player 332 sends the decoded video data frame and the video data frame processing instruction to the virtual display driver 334; wherein the video data video The decoded video data frame and the video data frame processing instruction sent by the player 332, the desktop protocol server 335 sends the decoded video data frame and the video data frame processing instruction to the client 101a. To reduce the network transmission bandwidth, before the desktop protocol server 335 sends the decoded video data frame and the video data frame processing instruction, the decoded video data frame may be compressed, which may be implemented by a compression module in the desktop protocol server 335. Since the virtual machine 205a in the server 204a does not process the video data frame according to the video data frame post-processing instruction, the computing resources and network transmission bandwidth of the server are saved, and the computing resources of the client are fully utilized. This embodiment is applicable to: capturing the first video data frame and the first video data frame processing instruction sent by the video player 332 to the virtual display driver 334; when the client 101a does not support video data frame decoding, The client 101a sends a second video data frame and a second video data frame processing instruction, so that the client 101a processes the second video data frame according to the second video data frame processing instruction, where The second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is a decoded video data frame, the second video data frame is the first Video data frame, The second video data frame processing instruction is the first video data frame processing instruction.
在另一种实施方式中, 在如图 6a所示的虚拟显示驱动 334截获 视频播放器 332向虚拟显示驱动 334发送的视频数据帧和视频数据帧 处理指令的方案中, 当客户端 101a不支持视频数据帧解码, 并且视 频播放器 332向虚拟显示驱动 334查询获知虚拟机 205a支持视频数 据帧后处理但不支持视频数据帧解码时,视频播放器 332加载视频文 件后, 对视频文件解复用, 获取视频数据帧, 视频播放器 332对所述 视频数据帧进行解码, 获得解码后的视频数据帧, 视频播放器 332向 虚拟显示驱动 334发送解码后的视频数据帧和视频数据帧处理指令; 呈现指令。截获视频播放器 332发送的解码后的视频数据帧和视频数 据帧处理指令,桌面协议服务端 335发送解码后的视频数据帧和视频 数据帧处理指令至客户端 101a。 为减少网络传输带宽, 虚拟显示驱 动 334截获解码后的视频数据帧和视频数据帧处理指令,可以压缩解 码后的视频数据帧,具体可以由虚拟显示驱动 334中的压缩模块来实 现。 由于服务器 204a中的虚拟机 205a并没有根据视频数据帧后处理 指令对视频数据帧进行处理,因此节约了服务器的计算资源和网络传 输带宽, 充分利用了客户端的计算资源。 本实施例适用如下场景: 截 获视频播放器 332向虚拟显示驱动 334发送的第一视频数据帧和第一 视频数据帧处理指令; 当所述客户端 101a不支持视频数据帧解码时, 向所述客户端 101a发送第二视频数据帧和第二视频数据帧处理指令, 以使所述客户端 101a根据所述第二视频数据帧处理指令对所述第二 视频数据帧进行处理, 其中, 所述第二视频数据帧处理指令包括视频 数据帧后处理指令和视频数据帧呈现指令; 当所述第一视频数据帧为 解码后的视频数据帧时 ,所述第二视频数据帧为所述第一视频数据帧 , 所述第二视频数据帧处理指令为所述第一视频数据帧处理指令。 In another embodiment, in the scheme in which the virtual display driver 334 as shown in FIG. 6a intercepts the video data frame and the video data frame processing instruction sent by the video player 332 to the virtual display driver 334, when the client 101a does not support The video data frame is decoded, and the video player 332 queries the virtual display driver 334 to learn that the virtual machine 205a supports the video data frame post-processing but does not support the video data frame decoding. After the video player 332 loads the video file, the video file is demultiplexed. Obtaining a video data frame, the video player 332 decodes the video data frame to obtain a decoded video data frame, and the video player 332 sends the decoded video data frame and the video data frame processing instruction to the virtual display driver 334; Present the instruction. The decoded video data frame and the video data frame processing instruction sent by the video player 332 are intercepted, and the desktop protocol server 335 transmits the decoded video data frame and the video data frame processing instruction to the client 101a. To reduce the network transmission bandwidth, the virtual display driver 334 intercepts the decoded video data frame and the video data frame processing instruction, and can compress the decoded video data frame, which can be implemented by the compression module in the virtual display driver 334. Since the virtual machine 205a in the server 204a does not process the video data frame according to the video data frame post-processing instruction, the computing resources and network transmission bandwidth of the server are saved, and the computing resources of the client are fully utilized. This embodiment is applicable to: capturing the first video data frame and the first video data frame processing instruction sent by the video player 332 to the virtual display driver 334; when the client 101a does not support video data frame decoding, The client 101a sends a second video data frame and a second video data frame processing instruction, so that the client 101a processes the second video data frame according to the second video data frame processing instruction, where The second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is a decoded video data frame, the second video data frame is the first Video data frame, The second video data frame processing instruction is the first video data frame processing instruction.
在上述图 6a 图 6b、 图 6c和图 6d四种截获视频播放器 332向虚 拟显示驱动 334发送的视频数据帧和视频数据帧处理指令的方案中, 当客户端 101a不支持视频数据帧解码, 并且视频播放器 332向虚拟 显示驱动 334查询获知虚拟机 205a支持视频数据帧解码和视频数据 帧后处理时, 视频播放器 332加载视频文件后, 对视频文件解复用, 获取视频数据帧,视频播放器 332向虚拟显示驱动 334发送视频数据 帧和视频数据帧处理指令;其中视频数据帧处理指令包括视频数据帧 解码指令、视频数据帧后处理指令和视频数据帧呈现指令。 截获视频 播放器 332发送的视频数据帧和视频数据帧处理指令,桌面协议服务 端 335 根据视频数据帧处理指令中的视频数据帧解码指令对视频数 据帧进行解码,桌面协议服务端 335发送解码后的视频数据帧和视频 数据帧处理指令至客户端 101a, 其中视频数据帧处理指令包括视频 数据帧后处理指令和视频数据帧呈现指令。 为减少网络传输带宽, 桌 面协议服务端 335可以压缩解码后的视频数据帧,具体可以由桌面协 议服务端 335中的压缩模块来实现,也可以由虚拟显示驱动 334中的 压缩模块来实现,然后由桌面协议服务端 335发送压缩的解码后的视 频数据帧和视频数据帧处理指令至客户端 101a。 由于服务器 204a中 的虚拟机 205a并没有根据视频数据帧后处理指令对视频数据帧进行 处理, 因此节约了服务器的计算资源和网络传输带宽, 充分利用了客 户端的计算资源。 本实施例适用如下场景: 截获视频播放器 332向虚 拟显示驱动 334发送的第一视频数据帧和第一视频数据帧处理指令; 当所述客户端 101a不支持视频数据帧解码时, 向所述客户端 101a发 送第二视频数据帧和第二视频数据帧处理指令,以使所述客户端 101a 根据所述第二视频数据帧处理指令对所述第二视频数据帧进行处理 , 其中,所述第二视频数据帧处理指令包括视频数据帧后处理指令和视 频数据帧呈现指令; 当所述第一视频数据帧为未解码的视频数据帧时 , 所述第二视频数据帧为解码后的所述第一视频数据帧,所述第一视频 数据帧处理指令包括视频数据帧解码指令、所述视频数据帧后处理指 令和所述视频数据帧呈现指令。 In the above-mentioned FIG. 6a, FIG. 6b, FIG. 6c and FIG. 6d, in the scheme of intercepting the video data frame and the video data frame processing instruction sent by the video player 332 to the virtual display driver 334, when the client 101a does not support the video data frame decoding, And the video player 332 queries the virtual display driver 334 to learn that the virtual machine 205a supports video data frame decoding and video data frame post-processing. After the video player 332 loads the video file, the video file is demultiplexed to obtain a video data frame and a video. The player 332 transmits a video data frame and a video data frame processing instruction to the virtual display driver 334; wherein the video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction. The video data frame and the video data frame processing instruction sent by the video player 332 are intercepted, and the desktop protocol server 335 decodes the video data frame according to the video data frame decoding instruction in the video data frame processing instruction, and the desktop protocol server 335 sends the decoded data frame. The video data frame and video data frame processing instructions are sent to the client 101a, wherein the video data frame processing instructions include a video data frame post-processing instruction and a video data frame presentation instruction. To reduce the network transmission bandwidth, the desktop protocol server 335 can compress the decoded video data frame, which can be implemented by a compression module in the desktop protocol server 335, or can be implemented by a compression module in the virtual display driver 334, and then The compressed decoded video data frame and video data frame processing instructions are transmitted by the desktop protocol server 335 to the client 101a. Since the virtual machine 205a in the server 204a does not process the video data frame according to the video data frame post-processing instruction, the computing resources and network transmission bandwidth of the server are saved, and the computing resources of the client are fully utilized. This embodiment is applicable to: capturing the first video data frame and the first video data frame processing instruction sent by the video player 332 to the virtual display driver 334; when the client 101a does not support video data frame decoding, The client 101a sends a second video data frame and a second video data frame processing instruction, so that the client 101a processes the second video data frame according to the second video data frame processing instruction. The second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction. When the first video data frame is an undecoded video data frame, the second video data frame is The decoded first video data frame, the first video data frame processing instruction includes a video data frame decoding instruction, the video data frame post-processing instruction, and the video data frame rendering instruction.
在另一种实施方式中, 在如图 6a所示的方案中, 当客户端 101a 不支持视频数据帧解码,并且视频播放器 332向虚拟显示驱动 334查 询获知虚拟机 205a支持视频数据帧解码和视频数据帧后处理时, 视 频播放器 332加载视频文件后,对视频文件解复用,获取视频数据帧, 视频播放器 332向虚拟显示驱动 334发送视频数据帧和视频数据帧处 理指令; 其中视频数据帧处理指令包括视频数据帧解码指令、视频数 据帧后处理指令和视频数据帧呈现指令。虚拟显示驱动 334截获视频 播放器 332发送的视频数据帧和视频数据帧处理指令,根据视频数据 帧处理指令中的视频数据帧解码指令对视频数据帧进行解码,桌面协 议服务端 335 发送解码后的视频数据帧和视频数据帧处理指令至客 户端 101a, 其中视频数据帧处理指令包括视频数据帧后处理指令和 视频数据帧呈现指令。 为减少网络传输带宽, 虚拟显示驱动 334将截 获视频数据帧解码后, 可以压缩解码后的视频数据帧, 具体可以由虚 拟显示驱动 334中的压缩模块来实现, 也可以由桌面协议服务端 335 中的压缩模块来实现,然后由桌面协议服务端 335发送压缩的解码后 的视频数据帧和视频数据帧处理指令至客户端 101a。由于服务器 204a 行处理, 因此节约了服务器的计算资源和网络传输带宽, 充分利用了 客户端的计算资源。 本实施例适用如下场景: 截获视频播放器 332向 虚拟显示驱动 334 发送的第一视频数据帧和第一视频数据帧处理指 令;当所述客户端 101a不支持视频数据帧解码时,向所述客户端 101a 发送第二视频数据帧和第二视频数据帧处理指令, 以使所述客户端 101a根据所述第二视频数据帧处理指令对所述第二视频数据帧进行 处理, 其中, 所述第二视频数据帧处理指令包括视频数据帧后处理指 令和视频数据帧呈现指令; 当所述第一视频数据帧为未解码的视频数 据帧时, 所述第二视频数据帧为解码后的所述第一视频数据帧, 所述 第一视频数据帧处理指令包括视频数据帧解码指令、所述视频数据帧 后处理指令和所述视频数据帧呈现指令。 In another embodiment, in the scheme shown in FIG. 6a, when the client 101a does not support video data frame decoding, and the video player 332 queries the virtual display driver 334 to learn that the virtual machine 205a supports video data frame decoding and After the video data frame is post-processed, after the video player 332 loads the video file, the video file is demultiplexed to obtain a video data frame, and the video player 332 sends a video data frame and a video data frame processing instruction to the virtual display driver 334; The data frame processing instructions include a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction. The virtual display driver 334 intercepts the video data frame and the video data frame processing instruction sent by the video player 332, decodes the video data frame according to the video data frame decoding instruction in the video data frame processing instruction, and sends the decoded data by the desktop protocol server 335. The video data frame and video data frame processing instructions are to the client 101a, wherein the video data frame processing instructions include a video data frame post-processing instruction and a video data frame presentation instruction. To reduce the network transmission bandwidth, the virtual display driver 334 may decode the decoded video data frame, and may be implemented by the compression module in the virtual display driver 334, or may be implemented by the desktop protocol server 335. The compression module is implemented, and then the compressed decoded video data frame and the video data frame processing instruction are sent by the desktop protocol server 335 to the client 101a. Since the server 204a processes the data, the computing resources and network transmission bandwidth of the server are saved, and the computing resources of the client are fully utilized. This embodiment is applicable to the following scenario: intercepting the first video data frame and the first video data frame processing finger sent by the video player 332 to the virtual display driver 334 When the client 101a does not support video data frame decoding, the second video data frame and the second video data frame processing instruction are sent to the client 101a, so that the client 101a is based on the second video. The data frame processing instruction processes the second video data frame, where the second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is not The second video data frame is the decoded first video data frame, and the first video data frame processing instruction includes a video data frame decoding instruction and the video data frame post-processing instruction. And presenting instructions to the video data frame.
本发明实施例通过截获视频播放器向虚拟显示驱动发送的视频 数据帧和视频数据帧处理指令,并且根据客户端是否支持视频数据帧 解码, 将视频数据帧、 视频数据帧解码指令、 视频数据帧后处理指令 和视频数据帧呈现指令发送到客户端进行处理,或者将解码后的视频 数据帧、视频数据帧后处理指令和视频数据帧呈现指令发送到客户端 进行处理, 从而可以在客户端实现视频数据帧后处理, 保证用户视频 体验。同时只要服务器侧视频播放器支持视频重定向功能即可实现本 发明, 不需要依赖某一种特定的视频播放器。  The embodiment of the present invention intercepts the video data frame and the video data frame processing instruction sent by the video player to the virtual display driver, and according to whether the client supports video data frame decoding, the video data frame, the video data frame decoding instruction, and the video data frame. The post-processing instruction and the video data frame presentation instruction are sent to the client for processing, or the decoded video data frame, the video data frame post-processing instruction, and the video data frame presentation instruction are sent to the client for processing, thereby being implemented on the client side. The video data frame is post-processed to ensure the user's video experience. At the same time, as long as the server side video player supports the video redirection function, the present invention can be implemented without relying on a specific video player.
本发明实施例一种具体的示例性流程图, 如图 7所示, 具体步骤 下:  A specific exemplary flowchart of the embodiment of the present invention is shown in FIG. 7 , and the specific steps are as follows:
步骤 701 : 判断客户端是否具有视频数据帧解码能力, 并且虚拟 机是否支持视频数据帧解码和视频数据帧后处理。  Step 701: Determine whether the client has a video data frame decoding capability, and whether the virtual machine supports video data frame decoding and video data frame post-processing.
当客户端具有视频数据帧解码能力,并且虚拟机支持视频数据帧 解码和视频数据帧后处理时, 执行步骤 702a;  When the client has video data frame decoding capability, and the virtual machine supports video data frame decoding and video data frame post-processing, step 702a is performed;
当客户端不具有视频数据帧解码能力,虚拟机不支持视频数据帧 解码但支持视频数据帧后处理, 执行步骤 702b;  When the client does not have the video data frame decoding capability, the virtual machine does not support the video data frame decoding but supports the video data frame post-processing, and the step 702b is performed;
当客户端不具有视频数据帧解码能力,虚拟机支持视频数据帧解 码和视频数据帧后处理, 执行步骤 702c。 When the client does not have the video data frame decoding capability, the virtual machine supports the video data frame solution. After the code and video data frame is processed, step 702c is performed.
步骤 702a:截获视频播放器向虚拟显示驱动发送的视频数据帧和 视频数据帧处理指令;视频数据帧处理指令包括视频数据帧解码指令、 视频数据帧后处理指令和视频数据帧呈现指令;  Step 702a: intercepting a video data frame and a video data frame processing instruction sent by the video player to the virtual display driver; the video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction;
步骤 703a:判断客户端是否支持视频播放器发送的视频数据帧的 视频编码格式;  Step 703a: Determine whether the client supports the video encoding format of the video data frame sent by the video player.
当客户端支持视频播放器发送的视频数据帧的视频编码格式时, 执行步骤 704a; 当客户端不支持视频播放器发送的视频数据帧的视 频编码格式时, 执行步骤 704 a'。  When the client supports the video encoding format of the video data frame sent by the video player, step 704a is performed; when the client does not support the video encoding format of the video data frame sent by the video player, step 704a' is performed.
步骤 704a:发送截获的视频数据帧和视频数据帧处理指令到客户 端;  Step 704a: Send the intercepted video data frame and the video data frame processing instruction to the client;
步骤 704a' : 将所述视频数据帧转换为客户端支持的视频编码格 式的视频数据帧。  Step 704a': Convert the video data frame into a video data frame of a video encoding format supported by the client.
步骤 702b: 截获视频播放器向虚拟显示驱动发送的解码后的视 频数据帧、 视频数据帧后处理指令和视频数据帧呈现指令;  Step 702b: intercepting the decoded video data frame, the video data frame post-processing instruction, and the video data frame presentation instruction sent by the video player to the virtual display driver;
步骤 703B: 可选的, 步骤 702b之后, 压缩截获的解码后的视频 数据帧;  Step 703B: Optionally, after step 702b, compressing the intercepted decoded video data frame;
步骤 704C: 发送解码后的视频数据帧和视频数据帧处理指令到 客户端;视频数据帧处理指令包括视频数据帧后处理指令和视频数据 帧呈现指令。  Step 704C: Send the decoded video data frame and the video data frame processing instruction to the client; the video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction.
步骤 702c:截获视频播放器发送的视频数据帧和视频数据帧处理 指令; 视频数据帧处理指令包括视频数据帧解码指令、视频数据帧后 处理指令和视频数据帧呈现指令;  Step 702c: intercepting a video data frame and a video data frame processing instruction sent by the video player; the video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction;
步骤 703c:根据视频数据帧解码指令对所述视频数据帧进行解码。 可选地, 步骤 703c之后, 还可以继续执行步骤 703B和 704C。 图 Ί所示的流程图, 其具体实现可以参见各实施例基于图 6a至 图 6d的描述, 在此不再赘述。 Step 703c: Decode the video data frame according to a video data frame decoding instruction. Optionally, after step 703c, steps 703B and 704C may also be continued. For the specific implementation of the flowchart shown in FIG. 6a to FIG. 6d, details are not described herein.
与上述各实施例相对应, 本发明实施例提供了一种服务器 800 , 如图 8所示,包括截获单元 801和发送单元 802。其中,截获单元 801 , 用于截获视频播放器向虚拟显示驱动发送的第一视频数据帧和第一 视频数据帧处理指令;  Corresponding to the above embodiments, the embodiment of the present invention provides a server 800, as shown in FIG. 8, including an intercepting unit 801 and a sending unit 802. The intercepting unit 801 is configured to intercept the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver;
发送单元 802, 用于当客户端支持视频数据帧解码时, 向所述客 户端发送第二视频数据帧和所述第一视频数据帧处理指令,以使所述 客户端根据所述第一视频数据帧处理指令对所述第二视频数据帧进 行处理, 所述第一视频数据帧处理指令包括视频数据帧解码指令、视 频数据帧后处理指令和视频数据帧呈现指令,所述第二视频数据帧为 所述第一视频数据帧或为所述第一视频数据帧经过视频编码格式转 换获得的视频数据帧;  The sending unit 802 is configured to: when the client supports decoding of the video data frame, send the second video data frame and the first video data frame processing instruction to the client, so that the client is configured according to the first video The data frame processing instruction processes the second video data frame, where the first video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, the second video data The frame is the first video data frame or a video data frame obtained by converting the first video data frame by a video encoding format;
发送单元 802, 用于当所述客户端不支持视频数据帧解码时, 向 所述客户端发送第二视频数据帧和第二视频数据帧处理指令,以使所 述客户端根据所述第二视频数据帧处理指令对所述第二视频数据帧 进行处理, 其中, 所述第二视频数据帧处理指令包括视频数据帧后处 理指令和视频数据帧呈现指令; 当所述第一视频数据帧为解码后的视 频数据帧时, 所述第二视频数据帧为所述第一视频数据帧, 所述第二 视频数据帧处理指令为所述第一视频数据帧处理指令; 当所述第一视 频数据帧为未解码的视频数据帧时 ,所述第二视频数据帧为解码后的 第一视频数据帧,所述第一视频数据帧处理指令包括视频数据帧解码 指令、 所述视频数据帧后处理指令和所述视频数据帧呈现指令。  The sending unit 802 is configured to: when the client does not support video data frame decoding, send a second video data frame and a second video data frame processing instruction to the client, so that the client is according to the second The video data frame processing instruction processes the second video data frame, where the second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is The second video data frame is the first video data frame, and the second video data frame processing instruction is the first video data frame processing instruction; when the first video is When the data frame is an undecoded video data frame, the second video data frame is a decoded first video data frame, and the first video data frame processing instruction includes a video data frame decoding instruction, and the video data frame is Processing instructions and the video data frame presentation instructions.
本发明实施例通过截获视频播放器向虚拟显示驱动发送的视频 数据帧和视频数据帧处理指令,并且根据客户端是否支持视频数据帧 解码, 将视频数据帧、 视频数据帧解码指令、 视频数据帧后处理指令 和视频数据帧呈现指令发送到客户端进行处理,或者将解码后的视频 数据帧、视频数据帧后处理指令和视频数据帧呈现指令发送到客户端 进行处理, 从而可以在客户端实现视频数据帧后处理, 保证用户视频 体验。同时只要服务器侧视频播放器支持视频重定向功能即可实现本 发明, 不需要依赖某一种特定的视频播放器。 The embodiment of the present invention intercepts a video data frame and a video data frame processing instruction sent by the video player to the virtual display driver, and according to whether the client supports the video data frame. Decoding, transmitting a video data frame, a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction to a client for processing, or decoding the decoded video data frame, a video data frame post-processing instruction, and video data The frame rendering instruction is sent to the client for processing, so that the video data frame post-processing can be implemented on the client to ensure the user's video experience. At the same time, as long as the server side video player supports the video redirection function, the present invention can be implemented without relying on a specific video player.
进一步的, 还包括转换单元 803 , 所述转换单元 803用于当所述 客户端支持视频数据帧解码时,并且所述客户端不支持所述视频播放 器向所述虚拟显示驱动发送的所述第一视频数据帧的视频编码格式 时, 将所述第一视频数据帧进行视频编码格式转换, 获得所述第二 视频数据帧; 其中, 所述第二视频数据帧的视频编码格式为所述客户 端支持的视频编码格式。 一种可能的实施情况, 所述转换单元 803具 体为所述虚拟显示驱动,则所述将所述第一视频数据帧进行视频编码 格式转换, 获得所述第二视频数据帧, 具体包括: 所述虚拟显示驱动 将所述第一视频数据帧进行视频编码格式转换,获得所述第二视频数 据帧。  Further, a conversion unit 803 is further included, where the conversion unit 803 is configured to: when the client supports video data frame decoding, and the client does not support the video player to send to the virtual display driver The video encoding format of the first video data frame, the first video data frame is subjected to a video encoding format conversion to obtain the second video data frame; wherein the video encoding format of the second video data frame is the The video encoding format supported by the client. In a possible implementation, the converting unit 803 is specifically the virtual display driver, and the video data format conversion is performed on the first video data frame to obtain the second video data frame, which specifically includes: The virtual display driver performs video encoding format conversion on the first video data frame to obtain the second video data frame.
本发明实施例提供的服务器 800除包括截获单元 801和发送单元 802 , 还包括解码单元 804 , 用于当所述客户端不支持视频数据帧解 码时, 并且当所述第一视频数据帧为未解码的视频数据帧时, 向所述 客户端发送第二视频数据帧和第二视频数据帧处理指令之前,根据所 述第一视频数据帧处理指令中的所述视频数据帧解码指令对所述第 一视频数据帧进行解码, 以获得所述第二视频数据帧。  The server 800 provided by the embodiment of the present invention includes a decoding unit 801 and a sending unit 802, and a decoding unit 804, configured to: when the client does not support video data frame decoding, and when the first video data frame is not Decoding the video data frame, before transmitting the second video data frame and the second video data frame processing instruction to the client, according to the video data frame decoding instruction in the first video data frame processing instruction The first video data frame is decoded to obtain the second video data frame.
本发明实施例提供的服务器 800除包含截获单元 801和发送单元 802外, 或服务器 800除包含截获单元 801、 发送单元 802和解码单 元 803外, 还包括压缩单元 805 , 所述压缩单元 805用于当所述客户 端不支持视频数据帧解码时,向所述客户端发送所述第二视频数据帧 和所述第二视频数据帧处理指令之前, 压缩所述第二视频数据帧; 则 向所述客户端发送第二视频数据帧 ,具体为向所述客户端发送压缩后 的第二视频数据帧, 从而节省网络传输带宽。 The server 800 provided by the embodiment of the present invention includes, in addition to the intercepting unit 801 and the sending unit 802, or the server 800, in addition to the intercepting unit 801, the sending unit 802, and the decoding unit 803, a compression unit 805, where the compression unit 805 is used. When the customer When the video data frame decoding is not supported, the second video data frame is compressed before the second video data frame and the second video data frame processing instruction are sent to the client, and then sent to the client. The second video data frame is specifically configured to send the compressed second video data frame to the client, thereby saving network transmission bandwidth.
本发明上述实施例所述各服务器 800还包括获取单元 806 , 所述 获取单元 806用于所述截获单元 801截获所述视频播放器向所述虚拟 显示驱动发送的所述第一视频数据帧和所述第一视频数据帧处理指 令之前, 获取所述客户端的视频数据帧处理能力; 所述客户端的视频 数据帧处理能力包括视频数据帧解码能力,以根据所述视频数据帧解 码能力, 确定所述客户端是否支持视频数据帧解码。 进一步的, 所述 获取单元 806还根据所述客户端的视频数据帧处理能力,确定所述客 户端支持视频数据帧解码时,向所述客户端发送第二视频数据帧和所 述第一视频数据帧处理指令之前,获取所述客户端支持的视频数据帧 的视频编码格式信息, 以根据所述视频编码格式信息, 确定所述客户 端是否支持所述第一视频数据帧的视频编码格式。  The server 800 of the foregoing embodiment of the present invention further includes an obtaining unit 806, where the acquiring unit 806 is configured to intercept the first video data frame sent by the video player to the virtual display driver. Obtaining a video data frame processing capability of the client before the first video data frame processing instruction; the video data frame processing capability of the client includes a video data frame decoding capability, to determine a location according to the video data frame decoding capability Whether the client supports video data frame decoding. Further, the obtaining unit 806 further determines, according to the video data frame processing capability of the client, that the client supports the video data frame decoding, and sends the second video data frame and the first video data to the client. Before the frame processing instruction, the video encoding format information of the video data frame supported by the client is obtained, to determine, according to the video encoding format information, whether the client supports the video encoding format of the first video data frame.
本发明上述实施例所述服务器 800 , 所述截获单元具体为虚拟显 示驱动,所述虚拟显示驱动截获所述第一视频数据帧和所述第一视频 处理指令。  In the server 800 of the above embodiment, the intercepting unit is specifically a virtual display driver, and the virtual display driver intercepts the first video data frame and the first video processing instruction.
本发明实施例还提供了一种视频重定向系统 900 , 包括服务器 800和客户端 1000。 其中, 服务器 800用于截获视频播放器向虚拟显 示驱动发送的第一视频数据帧和第一视频数据帧处理指令;  The embodiment of the present invention further provides a video redirection system 900, including a server 800 and a client 1000. The server 800 is configured to intercept the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver;
当客户端 1000支持视频数据帧解码时,向所述客户端 1000发送 第二视频数据帧和所述第一视频数据帧处理指令,以使所述客户端 1000 根据所述第一视频数据帧处理指令对所述第二视频数据帧进行 处理, 所述第一视频数据帧处理指令包括视频数据帧解码指令、视频 数据帧后处理指令和视频数据帧呈现指令,所述第二视频数据帧为所 述第一视频数据帧或为所述第一视频数据帧经过视频编码格式转换 获得的视频数据帧; When the client 1000 supports video data frame decoding, the second video data frame and the first video data frame processing instruction are sent to the client 1000, so that the client 1000 processes according to the first video data frame. The instruction processes the second video data frame, where the first video data frame processing instruction includes a video data frame decoding instruction and a video a data frame post-processing instruction and a video data frame presentation instruction, where the second video data frame is the first video data frame or a video data frame obtained by converting the first video data frame by a video encoding format;
当所述客户端 1000 不支持视频数据帧解码时, 向所述客户端 1000 发送第二视频数据帧和第二视频数据帧处理指令, 以使所述客 户端 1000根据所述第二视频数据帧处理指令对所述第二视频数据帧 进行处理, 其中, 所述第二视频数据帧处理指令包括视频数据帧后处 理指令和视频数据帧呈现指令; 当所述第一视频数据帧为解码后的视 频数据帧时, 所述第二视频数据帧为所述第一视频数据帧, 所述第二 视频数据帧处理指令为所述第一视频数据帧处理指令; 当所述第一视 频数据帧为未解码的视频数据帧时 ,所述第二视频数据帧为解码后的 所述第一视频数据帧 ,所述第一视频数据帧处理指令包括视频数据帧 解码指令、 所述视频数据帧后处理指令和所述视频数据帧呈现指令。  When the client 1000 does not support video data frame decoding, sending a second video data frame and a second video data frame processing instruction to the client 1000, so that the client 1000 is configured according to the second video data frame. The processing instruction is configured to process the second video data frame, where the second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is decoded The second video data frame is the first video data frame, the second video data frame processing instruction is the first video data frame processing instruction, and the first video data frame is In the case of an undecoded video data frame, the second video data frame is the decoded first video data frame, and the first video data frame processing instruction includes a video data frame decoding instruction, and the video data frame post processing The instructions and the video data frame present instructions.
客户端 1000用于根据服务器 800发送的视频数据帧处理指令对 视频数据帧进行处理。  The client 1000 is configured to process the video data frame according to the video data frame processing instruction sent by the server 800.
本发明实施例通过截获视频播放器向虚拟显示驱动发送的视频 数据帧和视频数据帧处理指令,并且根据客户端是否支持视频数据帧 解码, 将视频数据帧、 视频数据帧解码指令、 视频数据帧后处理指令 和视频数据帧呈现指令发送到客户端进行处理,或者将解码后的视频 数据帧、视频数据帧后处理指令和视频数据帧呈现指令发送到客户端 进行处理, 从而可以在客户端实现视频数据帧后处理, 保证用户视频 体验。同时只要服务器侧视频播放器支持视频重定向功能即可实现本 发明, 不需要依赖某一种特定的视频播放器。  The embodiment of the present invention intercepts the video data frame and the video data frame processing instruction sent by the video player to the virtual display driver, and according to whether the client supports video data frame decoding, the video data frame, the video data frame decoding instruction, and the video data frame. The post-processing instruction and the video data frame presentation instruction are sent to the client for processing, or the decoded video data frame, the video data frame post-processing instruction, and the video data frame presentation instruction are sent to the client for processing, thereby being implemented on the client side. The video data frame is post-processed to ensure the user's video experience. At the same time, as long as the server side video player supports the video redirection function, the present invention can be implemented without relying on a specific video player.
关于本实施例提供的系统 900的细节描述可参见服务器 800的描 述以及如图 6a、 图 6b、 图 6c、 图 6d和图 7所示的各实施例的描述, 在此不再赘述。 本发明实施例中提供的服务器 800和客户端 1000具 体可以为普通的计算机、 移动终端、 工作站或专用服务器等, 本发明 不作具体限定, 具有存储器和至少一个中央处理器, 所述存储器与所 述至少一个中央处理器通过总线相连, 存储器存储计算机指令, 所述 至少一个中央处理器用于执行存储器中的计算机指令,同时也包含通 用的外部接口等实现服务器功能的一些必要组件。 其中, 存储器中存 储的计算机指令的算法可以参照图 7所示流程图的描述,具体实现可 以参见图 6a、 图 6b、 图 6c和图 6d所示的具体实施例。 For a detailed description of the system 900 provided by the present embodiment, reference may be made to the description of the server 800 and the description of the embodiments as shown in FIGS. 6a, 6b, 6c, 6d, and 7. I will not repeat them here. The server 800 and the client 1000 provided in the embodiment of the present invention may be a general computer, a mobile terminal, a workstation, or a dedicated server, etc., which is not specifically limited by the present invention, and has a memory and at least one central processor, and the memory and the At least one central processing unit is coupled by a bus, the memory stores computer instructions, and the at least one central processing unit is configured to execute computer instructions in the memory, and also includes a common external interface and the like to implement some necessary components of the server function. The algorithm for the computer instructions stored in the memory may refer to the description of the flowchart shown in FIG. 7. For specific implementation, reference may be made to the specific embodiments shown in FIG. 6a, FIG. 6b, FIG. 6c, and FIG.
本发明实施例提供的基于虚拟桌面的远程桌面架构,本领域技术 人员可以意识到,本发明并不限于本实施例的所描述的虚拟桌面架构, 对于基于其他类型的虚拟桌面的远程桌面本发明同样适用。 另外, 本 发明并不仅仅适用于基于虚拟桌面的远程桌面架构,同样也适用于其 他向远程客户端提供视频数据的应用场景。  The virtual desktop-based remote desktop architecture provided by the embodiment of the present invention, those skilled in the art may appreciate that the present invention is not limited to the described virtual desktop architecture of the present embodiment, and the remote desktop invention based on other types of virtual desktops The same applies. In addition, the present invention is not only applicable to a virtual desktop-based remote desktop architecture, but also to other application scenarios that provide video data to remote clients.
本领域普通技术人员可以意识到,结合本文中所公开的实施例描 述的各示例的单元及算法步骤, 能够以电子硬件、 或者计算机软件和 电子硬件的结合来实现。 这些功能究竟以硬件还是软件方式来执行, 取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每 个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不 应认为超出本发明的范围。  Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods to implement the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.
所属领域的技术人员可以清楚地了解到, 为描述的方便和简洁, 上述描述的系统、 装置和单元的具体工作过程, 可以参考前述方法实 施例中的对应过程, 在此不再赘述。  A person skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.
在本申请所提供的几个实施例中, 应该理解到, 所公开的系统、 装置和方法, 可以通过其它的方式实现。 例如, 以上所描述的装置实 施例仅仅是示意性的, 例如, 所述单元的划分, 仅仅为一种逻辑功能 划分, 实际实现时可以有另外的划分方式, 例如多个单元或组件可以 结合或者可以集成到另一个系统, 或一些特征可以忽略, 或不执行。 另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以 是通过一些接口, 装置或单元的间接耦合或通信连接, 可以是电性, 机械或其它的形式。 开的, 作为单元显示的部件可以是或者也可以不是物理单元, 即可以 位于一个地方, 或者也可以分布到多个网络单元上。 可以根据实际的 需要选择其中的部分或者全部单元来实现本实施例方案的目的。 In the several embodiments provided herein, it should be understood that the disclosed systems, devices, and methods may be implemented in other ways. For example, the device embodiments described above are merely illustrative, for example, the division of the unit is merely a logical function. Dividing, there may be another way of dividing in actual implementation, for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be electrical, mechanical or otherwise. The components displayed as the unit may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
另外,在本发明各个实施例中的各功能单元可以集成在一个处理 单元中, 也可以是各个单元单独物理存在, 也可以两个或两个以上单 元集成在一个单元中。  In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
所述功能如果以软件功能单元的形式实现并作为独立的产品销 售或使用时, 可以存储在一个计算机可读取存储介质中。基于这样的 理解,本发明的技术方案本质上或者说对现有技术做出贡献的部分或 者该技术方案的部分可以以软件产品的形式体现出来,该计算机软件 产品存储在一个存储介质中,包括若干指令用以使得一台计算机设备 (可以是个人计算机, 服务器, 或者网络设备等)执行本发明各个实 施例所述方法的全部或部分步骤。 而前述的存储介质包括: NAS ( Network Attached Storage ), U盘、移动硬盘、 只读存储器(ROM, Read-Only Memory )、 随机存取存 4诸器 (RAM , Random Access Memory )、 磁碟或者光盘等各种可以存储程序代码的介质。  The functions may be stored in a computer readable storage medium if implemented in the form of a software functional unit and sold or used as a standalone product. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product, which is stored in a storage medium, including The instructions are used to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: NAS (Network Attached Storage), U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or A variety of media such as optical discs that can store program code.
以上所述, 仅为本发明的具体实施方式, 但本发明的保护范围并 不局限于此,任何熟悉本技术领域的技术人员在本发明揭露的技术范 围内, 可轻易想到变化或替换, 都应涵盖在本发明的保护范围之内。 因此, 本发明的保护范围应所述以权利要求的保护范围为准。 The above is only a specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily think of changes or substitutions within the technical scope of the present invention. It should be covered by the scope of the present invention. Therefore, the scope of the invention should be determined by the scope of the claims.

Claims

权利要求书 claims
1、 一种视频重定向的方法, 其特征在于, 包括: 1. A video redirection method, characterized by including:
截获视频播放器向虚拟显示驱动发送的第一视频数据帧和第 一视频数据帧处理指令; Intercept the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver;
当客户端支持视频数据帧解码时, 向所述客户端发送第二视 频数据帧和所述第一视频数据帧处理指令,以使所述客户端根据 所述第一视频数据帧处理指令对所述第二视频数据帧进行处理, 所述第一视频数据帧处理指令包括视频数据帧解码指令、 视频数 据帧后处理指令和视频数据帧呈现指令, 所述第二视频数据帧为 所述第一视频数据帧或为所述第一视频数据帧经过视频编码格式 转换获得的视频数据帧; When the client supports video data frame decoding, the second video data frame and the first video data frame processing instruction are sent to the client, so that the client performs the decoding of the video data frame according to the first video data frame processing instruction. The second video data frame is processed. The first video data frame processing instructions include video data frame decoding instructions, video data frame post-processing instructions and video data frame rendering instructions. The second video data frame is the first video data frame. The video data frame may be a video data frame obtained by converting the video encoding format of the first video data frame;
当所述客户端不支持视频数据帧解码时, 向所述客户端发送 第二视频数据帧和第二视频数据帧处理指令, 以使所述客户端根 据所述第二视频数据帧处理指令对所述第二视频数据帧进行处理, 其中, 所述第二视频数据帧处理指令包括视频数据帧后处理指令 和视频数据帧呈现指令; 当所述第一视频数据帧为解码后的视频 数据帧时, 所述第二视频数据帧为所述第一视频数据帧, 所述第 二视频数据帧处理指令为所述第一视频数据帧处理指令; 当所述 第一视频数据帧为未解码的视频数据帧时, 所述第二视频数据帧 为解码后的所述第一视频数据帧, 所述第一视频数据帧处理指令 包括视频数据帧解码指令、 所述视频数据帧后处理指令和所述视 频数据帧呈现指令。 When the client does not support video data frame decoding, send a second video data frame and a second video data frame processing instruction to the client, so that the client can process the video data frame according to the second video data frame processing instruction. The second video data frame is processed, wherein the second video data frame processing instructions include video data frame post-processing instructions and video data frame presentation instructions; when the first video data frame is a decoded video data frame When, the second video data frame is the first video data frame, and the second video data frame processing instruction is the first video data frame processing instruction; when the first video data frame is undecoded When the video data frame is a video data frame, the second video data frame is the decoded first video data frame, and the first video data frame processing instructions include a video data frame decoding instruction, the video data frame post-processing instruction and the Described video data frame rendering instructions.
2、 如权利要求 1所述的方法, 其特征在于, 还包括: 当所述 客户端支持视频数据帧解码时, 并且所述客户端不支持所述视频 播放器向所述虚拟显示驱动发送的所述第一视频数据帧的视频编 码格式时, 将所述第一视频数据帧进行视频编码格式转换, 获得 所述第二视频数据帧; 其中, 所述第二视频数据帧的视频编码格 式为所述客户端支持的视频编码格式。 2. The method of claim 1, further comprising: when the client supports video data frame decoding, and the client does not support the video data sent by the video player to the virtual display driver. When the video encoding format of the first video data frame is determined, the first video data frame is converted into a video encoding format to obtain the second video data frame; wherein, the video encoding format of the second video data frame is The video encoding format supported by the client.
3、 如权利要求 2所述的方法, 其特征在于, 所述将所述第一 视频数据帧进行视频编码格式转换, 获得所述第二视频数据帧, 具体包括: 所述虚拟显示驱动将所述第一视频数据帧进行视频编 码格式转换, 获得所述第二视频数据帧。 3. The method of claim 2, wherein: converting the first video data frame into a video encoding format to obtain the second video data frame, Specifically, it includes: the virtual display driver converts the video encoding format of the first video data frame to obtain the second video data frame.
4、 如权利要求 1所述的方法, 其特征在于, 还包括: 当所述客户端不支持视频数据帧解码时, 并且当所述第一视 频数据帧为未解码的视频数据帧时, 向所述客户端发送第二视频 数据帧和第二视频数据帧处理指令之前, 根据所述第一视频数据 帧处理指令中的所述视频数据帧解码指令对所述第一视频数据帧 进行解码, 以获得所述第二视频数据帧。 4. The method of claim 1, further comprising: when the client does not support video data frame decoding, and when the first video data frame is an undecoded video data frame, Before the client sends the second video data frame and the second video data frame processing instruction, the first video data frame is decoded according to the video data frame decoding instruction in the first video data frame processing instruction, To obtain the second video data frame.
5、 如权利要求 1 或 4所述的方法, 其特征在于, 还包括: 当所述客户端不支持视频数据帧解码时, 向所述客户端发送所述 第二视频数据帧和所述第二视频数据帧处理指令之前, 压缩所述 第二视频数据帧; 5. The method of claim 1 or 4, further comprising: when the client does not support video data frame decoding, sending the second video data frame and the third video data frame to the client. Before the second video data frame processing instruction, compress the second video data frame;
则向所述客户端发送所述第二视频数据帧, 具体为向所述客 户端发送压缩后的第二视频数据帧。 Then the second video data frame is sent to the client, specifically the compressed second video data frame is sent to the client.
6、如权利要求 1至 5任一所述的方法,其特征在于,还包括: 所述截获视频播放器向虚拟显示驱动发送的第一视频数据帧和第 一视频数据帧处理指令之前, 获取所述客户端的视频数据帧处理 能力; 所述客户端的视频数据帧处理能力包括视频数据帧解码能 力, 以根据所述视频数据帧解码能力, 确定所述客户端是否支持 视频数据帧解码。 6. The method according to any one of claims 1 to 5, further comprising: before intercepting the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver, obtaining The video data frame processing capability of the client; The video data frame processing capability of the client includes a video data frame decoding capability, so as to determine whether the client supports video data frame decoding based on the video data frame decoding capability.
7、 如权利要求 2 , 3或 6所述的方法, 其特征在于, 还包括: 所述当客户端支持视频数据帧解码时, 向所述客户端发送第二视 频数据帧和所述第一视频数据帧处理指令之前, 获取所述客户端 支持的视频数据帧的视频编码格式信息, 以根据所述视频编码格 式信息, 确定所述客户端是否支持所述第一视频数据帧的视频编 码格式。 7. The method of claim 2, 3 or 6, further comprising: when the client supports video data frame decoding, sending a second video data frame and the first video data frame to the client. Before the video data frame processing instruction, obtain the video encoding format information of the video data frame supported by the client, so as to determine whether the client supports the video encoding format of the first video data frame based on the video encoding format information. .
8、 如权利要求 1至 7任一所述的方法, 其特征在于, 所述截 获视频播放器向虚拟显示驱动发送的第一视频数据帧和第一视频 数据帧处理指令, 具体为所述虚拟显示驱动截获所述第一视频数 据帧和所述第一视频处理指令。 8. The method according to any one of claims 1 to 7, characterized in that: the interception of the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver, specifically the virtual display driver The display driver intercepts the first video data frame and the first video processing instruction.
9、 一种服务器, 其特征在于, 包括: 9. A server, characterized by including:
截获单元, 用于截获视频播放器向虚拟显示驱动发送的第一 视频数据帧和第一视频数据帧处理指令; An interception unit, configured to intercept the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver;
发送单元, 用于当客户端支持视频数据帧解码时, 向所述客 户端发送第二视频数据帧和所述第一视频数据帧处理指令,以使 所述客户端根据所述第一视频数据帧处理指令对所述第二视频数 据帧进行处理, 所述第一视频数据帧处理指令包括视频数据帧解 码指令、 视频数据帧后处理指令和视频数据帧呈现指令, 所述第 二视频数据帧为所述第一视频数据帧或为所述第一视频数据帧经 过视频编码格式转换获得的视频数据帧; A sending unit, configured to send a second video data frame and a first video data frame processing instruction to the client when the client supports video data frame decoding, so that the client can process the second video data frame according to the first video data frame. The frame processing instructions process the second video data frame. The first video data frame processing instructions include video data frame decoding instructions, video data frame post-processing instructions and video data frame rendering instructions. The second video data frame It is the first video data frame or a video data frame obtained by converting the first video data frame through video encoding format conversion;
当所述客户端不支持视频数据帧解码时, 向所述客户端发送 第二视频数据帧和第二视频数据帧处理指令, 以使所述客户端根 据所述第二视频数据帧处理指令对所述第二视频数据帧进行处理, 其中, 所述第二视频数据帧处理指令包括视频数据帧后处理指令 和视频数据帧呈现指令; 当所述第一视频数据帧为解码后的视频 数据帧时, 所述第二视频数据帧为所述第一视频数据帧, 所述第 二视频数据帧处理指令为所述第一视频数据帧处理指令; 当所述 第一视频数据帧为未解码的视频数据帧时, 所述第二视频数据帧 为解码后的第一视频数据帧, 所述第一视频数据帧处理指令包括 视频数据帧解码指令、 所述视频数据帧后处理指令和所述视频数 据帧呈现指令。 When the client does not support video data frame decoding, send a second video data frame and a second video data frame processing instruction to the client, so that the client can process the video data frame according to the second video data frame processing instruction. The second video data frame is processed, wherein the second video data frame processing instructions include video data frame post-processing instructions and video data frame presentation instructions; when the first video data frame is a decoded video data frame When, the second video data frame is the first video data frame, and the second video data frame processing instruction is the first video data frame processing instruction; when the first video data frame is undecoded When the video data frame is a video data frame, the second video data frame is a decoded first video data frame, and the first video data frame processing instructions include a video data frame decoding instruction, a video data frame post-processing instruction and a video data frame processing instruction. Data frame rendering instructions.
10、 如权利要求 9 所述的服务器, 其特征在于, 还包括转换 单元, 所述转换单元用于当所述客户端支持视频数据帧解码时, 并且所述客户端不支持所述视频播放器向所述虚拟显示驱动发送 的所述第一视频数据帧的视频编码格式时, 将所述第一视频数据 帧进行视频编码格式转换, 获得所述第二视频数据帧; 其中, 所 述第二视频数据帧的视频编码格式为所述客户端支持的视频编码 格式。 10. The server according to claim 9, further comprising a conversion unit, the conversion unit is used when the client supports video data frame decoding, and the client does not support the video player. When sending the video encoding format of the first video data frame to the virtual display driver, the first video data frame is converted into a video encoding format to obtain the second video data frame; wherein, the second The video encoding format of the video data frame is a video encoding format supported by the client.
11、 如权利要求 10所述的服务器, 其特征在于, 所述转换单 元具体为所述虚拟显示驱动, 则所述将所述第一视频数据帧进行 视频编码格式转换, 获得所述第二视频数据帧, 具体包括: 所述 虚拟显示驱动将所述第一视频数据帧进行视频编码格式转换, 获 得所述第二视频数据帧。 11. The server according to claim 10, wherein the conversion unit is specifically the virtual display driver, and the first video data frame is Video coding format conversion to obtain the second video data frame specifically includes: the virtual display driver performs video coding format conversion on the first video data frame to obtain the second video data frame.
12、 如权利要求 9 所述的服务器, 其特征在于, 还包括解码 单元, 用于当所述客户端不支持视频数据帧解码时, 并且当所述 第一视频数据帧为未解码的视频数据帧时, 向所述客户端发送第 二视频数据帧和第二视频数据帧处理指令之前, 根据所述第一视 频数据帧处理指令中的所述视频数据帧解码指令对所述第一视频 数据帧进行解码, 以获得所述第二视频数据帧。 12. The server according to claim 9, further comprising a decoding unit for when the client does not support video data frame decoding, and when the first video data frame is undecoded video data frame, before sending the second video data frame and the second video data frame processing instruction to the client, the first video data is processed according to the video data frame decoding instruction in the first video data frame processing instruction. The frame is decoded to obtain the second video data frame.
13、 如权利要求 9或 12所述的服务器, 其特征在于, 还包括 压缩单元, 所述压缩单元用于当所述客户端不支持视频数据帧解 码时, 向所述客户端发送所述第二视频数据帧和所述第二视频数 据帧处理指令之前, 压缩所述第二视频数据帧; 则向所述客户端 发送第二视频数据帧, 具体为向所述客户端发送压缩后的第二视 频数据帧。 13. The server according to claim 9 or 12, further comprising a compression unit configured to send the first video data frame to the client when the client does not support video data frame decoding. Before the second video data frame and the second video data frame processing instruction, compress the second video data frame; then send the second video data frame to the client, specifically sending the compressed second video data frame to the client. Two video data frames.
14、 如权利要求 9至 13任一所述的服务器, 其特征在于, 还 包括获取单元, 所述获取单元用于所述截获单元截获所述视频播 放器向所述虚拟显示驱动发送的所述第一视频数据帧和所述第一 视频数据帧处理指令之前, 获取所述客户端的视频数据帧处理能 力; 所述客户端的视频数据帧处理能力包括视频数据帧解码能力, 以根据所述视频数据帧解码能力, 确定所述客户端是否支持视频 数据帧解码。 14. The server according to any one of claims 9 to 13, further comprising an acquisition unit, the acquisition unit being configured to intercept the information sent by the video player to the virtual display driver. Before the first video data frame and the first video data frame processing instruction, the video data frame processing capability of the client is obtained; the video data frame processing capability of the client includes a video data frame decoding capability, so as to obtain the video data frame processing capability according to the video data frame. Frame decoding capability determines whether the client supports video data frame decoding.
15、 如权利要求 10、 11 或 14所述的服务器, 其特征在于, 所述获取单元还用于根据所述客户端的视频数据帧处理能力, 确 定所述客户端支持视频数据帧解码时, 向所述客户端发送第二视 频数据帧和所述第一视频数据帧处理指令之前, 获取所述客户端 支持的视频数据帧的视频编码格式信息, 以根据所述视频编码格 式信息, 确定所述客户端是否支持所述第一视频数据帧的视频编 码格式。 15. The server according to claim 10, 11 or 14, wherein the acquisition unit is further configured to determine that the client supports video data frame decoding based on the video data frame processing capability of the client. Before the client sends the second video data frame and the first video data frame processing instruction, obtain the video encoding format information of the video data frame supported by the client, so as to determine the video encoding format information based on the video encoding format information. Whether the client supports the video encoding format of the first video data frame.
16、 如权利要求 9至 15任一所述的服务器, 其特征在于, 所 述截获单元具体为虚拟显示驱动, 所述虚拟显示驱动截获所述第 一视频数据帧和所述第一视频处理指令。 16. The server according to any one of claims 9 to 15, characterized in that: The interception unit is specifically a virtual display driver, and the virtual display driver intercepts the first video data frame and the first video processing instruction.
17、 一种服务器, 其特征在于, 包括存储器和中央处理器, 所述存储器和所述中央处理器通过系统总线相连, 所述存储器存 储计算机可执行指令, 当所述服务器运行时, 所述中央处理器执 行所述存储器存储的计算机可执行指令, 使所述服务器执行如下 步骤: 17. A server, characterized in that it includes a memory and a central processor, the memory and the central processor are connected through a system bus, the memory stores computer executable instructions, and when the server is running, the central processor The processor executes computer-executable instructions stored in the memory, causing the server to perform the following steps:
截获视频播放器向虚拟显示驱动发送的第一视频数据帧和第 一视频数据帧处理指令; Intercept the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver;
当客户端支持视频数据帧解码时, 向所述客户端发送第二视 频数据帧和所述第一视频数据帧处理指令,以使所述客户端根据 所述第一视频数据帧处理指令对所述第二视频数据帧进行处理, 所述第一视频数据帧处理指令包括视频数据帧解码指令、 视频数 据帧后处理指令和视频数据帧呈现指令, 所述第二视频数据帧为 所述第一视频数据帧或为所述第一视频数据帧经过视频编码格式 转换获得的视频数据帧; When the client supports video data frame decoding, the second video data frame and the first video data frame processing instruction are sent to the client, so that the client performs the decoding of the video data frame according to the first video data frame processing instruction. The second video data frame is processed. The first video data frame processing instructions include video data frame decoding instructions, video data frame post-processing instructions and video data frame rendering instructions. The second video data frame is the first video data frame. The video data frame may be a video data frame obtained by converting the video encoding format of the first video data frame;
当所述客户端不支持视频数据帧解码时, 向所述客户端发送 第二视频数据帧和第二视频数据帧处理指令, 以使所述客户端根 据所述第二视频数据帧处理指令对所述第二视频数据帧进行处理, 其中, 所述第二视频数据帧处理指令包括视频数据帧后处理指令 和视频数据帧呈现指令; 当所述第一视频数据帧为解码后的视频 数据帧时, 所述第二视频数据帧为所述第一视频数据帧, 所述第 二视频数据帧处理指令为所述第一视频数据帧处理指令; 当所述 第一视频数据帧为未解码的视频数据帧时, 所述第二视频数据帧 为解码后的第一视频数据帧, 所述第一视频数据帧处理指令包括 视频数据帧解码指令、 所述视频数据帧后处理指令和所述视频数 据帧呈现指令。 When the client does not support video data frame decoding, send a second video data frame and a second video data frame processing instruction to the client, so that the client can process the video data frame according to the second video data frame processing instruction. The second video data frame is processed, wherein the second video data frame processing instructions include video data frame post-processing instructions and video data frame presentation instructions; when the first video data frame is a decoded video data frame When, the second video data frame is the first video data frame, and the second video data frame processing instruction is the first video data frame processing instruction; when the first video data frame is undecoded When the video data frame is a video data frame, the second video data frame is a decoded first video data frame, and the first video data frame processing instructions include a video data frame decoding instruction, a video data frame post-processing instruction and a video data frame processing instruction. Data frame rendering instructions.
18、 如权利要求 17所述的服务器, 其特征在于, 还包括: 当 所述客户端支持视频数据帧解码时, 并且所述客户端不支持所述 视频播放器向所述虚拟显示驱动发送的所述第一视频数据帧的视 频编码格式时, 将所述第一视频数据帧进行视频编码格式转换, 获得所述第二视频数据帧; 其中, 所述第二视频数据帧的视频编 码格式为所述客户端支持的视频编码格式。 18. The server of claim 17, further comprising: when the client supports video data frame decoding, and the client does not support the video data sent by the video player to the virtual display driver. The view of the first video data frame When the video encoding format is selected, the first video data frame is converted into a video encoding format to obtain the second video data frame; wherein, the video encoding format of the second video data frame is a video encoding supported by the client. Format.
19、 如权利要求 18所述的服务器, 其特征在于, 所述将所述 第一视频数据帧进行视频编码格式转换, 获得所述第二视频数据 帧, 具体包括: 所述虚拟显示驱动将所述第一视频数据帧进行视 频编码格式转换, 获得所述第二视频数据帧。 19. The server according to claim 18, wherein said converting the first video data frame into a video encoding format to obtain the second video data frame specifically includes: the virtual display driver converts the first video data frame into a video encoding format. The first video data frame is subjected to video encoding format conversion to obtain the second video data frame.
20、 如权利要求 17所述的服务器, 其特征在于, 还包括当所 述客户端不支持视频数据帧解码时, 并且当所述第一视频数据帧 为未解码的视频数据帧时, 向所述客户端发送第二视频数据帧和 第二视频数据帧处理指令之前, 根据所述第一视频数据帧处理指 令中的所述视频数据帧解码指令对所述第一视频数据帧进行解码, 以获得所述第二视频数据帧。 20. The server of claim 17, further comprising: when the client does not support video data frame decoding, and when the first video data frame is an undecoded video data frame, Before the client sends the second video data frame and the second video data frame processing instruction, the first video data frame is decoded according to the video data frame decoding instruction in the first video data frame processing instruction, so as to Obtain the second video data frame.
21、 如权利要求 17或 20所述的服务器, 其特征在于, 还包 括: 当所述客户端不支持视频数据帧解码时, 向所述客户端发送 所述第二视频数据帧和所述第二视频数据帧处理指令之前, 压缩 所述第二视频数据帧; 则向所述客户端发送所述第二视频数据帧, 具体为向所述客户端发送压缩后的第二视频数据帧。 21. The server according to claim 17 or 20, further comprising: when the client does not support video data frame decoding, sending the second video data frame and the third video data frame to the client. Before the second video data frame processing instruction, compress the second video data frame; then send the second video data frame to the client, specifically sending the compressed second video data frame to the client.
22、 如权利要求 17至 21任一所述的服务器, 其特征在于, 还包括: 所述截获视频播放器向虚拟显示驱动发送的第一视频数 据帧和第一视频数据帧处理指令之前, 获取所述客户端的视频数 据帧处理能力; 所述客户端的视频数据帧处理能力包括视频数据 帧解码能力, 以根据所述视频数据帧解码能力, 确定所述客户端 是否支持视频数据帧解码。 22. The server according to any one of claims 17 to 21, further comprising: before intercepting the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver, obtaining The video data frame processing capability of the client; The video data frame processing capability of the client includes a video data frame decoding capability, so as to determine whether the client supports video data frame decoding based on the video data frame decoding capability.
23、 如权利要求 18、 19或 22所述的方法, 其特征在于, 还 包括: 根据所述客户端的视频数据帧处理能力, 确定所述客户端 支持视频数据帧解码时, 向所述客户端发送第二视频数据帧和所 述第一视频数据帧处理指令之前, 获取所述客户端支持的视频数 据帧的视频编码格式信息, 以根据所述视频编码格式信息, 确定 所述客户端是否支持所述第一视频数据帧的视频编码格式。 23. The method of claim 18, 19 or 22, further comprising: based on the video data frame processing capability of the client, when it is determined that the client supports video data frame decoding, reporting to the client Before sending the second video data frame and the first video data frame processing instruction, obtain the video encoding format information of the video data frame supported by the client, so as to determine whether the client supports the video encoding format information based on the video encoding format information. The video encoding format of the first video data frame.
24、 如权利要求 17至 23任一所述的服务器, 其特征在于, 所述截获视频播放器向虚拟显示驱动发送的第一视频数据帧和第 一视频数据帧处理指令, 具体为所述虚拟显示驱动截获所述第一 视频数据帧和所述第一视频处理指令。 24. The server according to any one of claims 17 to 23, characterized in that, the intercepted first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver, specifically the virtual display driver The display driver intercepts the first video data frame and the first video processing instruction.
25、 一种视频重定向系统, 包括客户端和如权利要求 9至 16 任一所述的服务器。 25. A video redirection system, including a client and a server as claimed in any one of claims 9 to 16.
26、 一种视频重定向系统, 包括客户端和如权利要求 17至 24 任一所述的服务器。 26. A video redirection system, including a client and a server as claimed in any one of claims 17 to 24.
27、 一种计算机可读存储介质, 其特征在于, 当计算机执行 所述计算机可读存储介质时, 所述计算机执行如下步骤: 截获视 频播放器向虚拟显示驱动发送的第一视频数据帧和第一视频数据 帧处理指令; 27. A computer-readable storage medium, characterized in that when the computer executes the computer-readable storage medium, the computer performs the following steps: intercept the first video data frame and the second video data frame sent by the video player to the virtual display driver. A video data frame processing instruction;
当客户端支持视频数据帧解码时, 向所述客户端发送第二视 频数据帧和所述第一视频数据帧处理指令,以使所述客户端根据 所述第一视频数据帧处理指令对所述第二视频数据帧进行处理, 所述第一视频数据帧处理指令包括视频数据帧解码指令、 视频数 据帧后处理指令和视频数据帧呈现指令, 所述第二视频数据帧为 所述第一视频数据帧或为所述第一视频数据帧经过视频编码格式 转换获得的视频数据帧; When the client supports video data frame decoding, the second video data frame and the first video data frame processing instruction are sent to the client, so that the client performs the decoding of the video data frame according to the first video data frame processing instruction. The second video data frame is processed. The first video data frame processing instructions include video data frame decoding instructions, video data frame post-processing instructions and video data frame rendering instructions. The second video data frame is the first video data frame. The video data frame may be a video data frame obtained by converting the first video data frame through video encoding format conversion;
当所述客户端不支持视频数据帧解码时, 向所述客户端发送 第二视频数据帧和第二视频数据帧处理指令, 以使所述客户端根 据所述第二视频数据帧处理指令对所述第二视频数据帧进行处理, 其中, 所述第二视频数据帧处理指令包括视频数据帧后处理指令 和视频数据帧呈现指令; 当所述第一视频数据帧为解码后的视频 数据帧时, 所述第二视频数据帧为所述第一视频数据帧, 所述第 二视频数据帧处理指令为所述第一视频数据帧处理指令; 当所述 第一视频数据帧为未解码的视频数据帧时, 所述第二视频数据帧 为解码后的第一视频数据帧, 所述第一视频数据帧处理指令包括 视频数据帧解码指令、 所述视频数据帧后处理指令和所述视频数 据帧呈现指令。 When the client does not support video data frame decoding, send a second video data frame and a second video data frame processing instruction to the client, so that the client can process the video data frame according to the second video data frame processing instruction. The second video data frame is processed, wherein the second video data frame processing instructions include video data frame post-processing instructions and video data frame presentation instructions; when the first video data frame is a decoded video data frame When, the second video data frame is the first video data frame, and the second video data frame processing instruction is the first video data frame processing instruction; when the first video data frame is undecoded When the video data frame is a video data frame, the second video data frame is a decoded first video data frame, and the first video data frame processing instructions include a video data frame decoding instruction, a video data frame post-processing instruction and a video data frame processing instruction. Data frame rendering instructions.
28、如权利要求 27所述的计算机可读存储介质,其特征在于, 还包括: 当所述客户端支持视频数据帧解码时, 并且所述客户端 不支持所述视频播放器向所述虚拟显示驱动发送的所述第一视频 数据帧的视频编码格式时, 将所述第一视频数据帧进行视频编码 格式转换, 获得所述第二视频数据帧; 其中, 所述第二视频数据 帧的视频编码格式为所述客户端支持的视频编码格式。 28. The computer-readable storage medium according to claim 27, further comprising: when the client supports video data frame decoding, and the client does not support the video player to send data to the virtual When the video encoding format of the first video data frame sent by the display driver is performed, the first video data frame is converted into a video encoding format to obtain the second video data frame; wherein, the second video data frame is The video encoding format is a video encoding format supported by the client.
29、如权利要求 28所述的计算机可读存储介质,其特征在于, 所述将所述第一视频数据帧进行视频编码格式转换, 获得所述第 二视频数据帧, 具体包括: 所述虚拟显示驱动将所述第一视频数 据帧进行视频编码格式转换, 获得所述第二视频数据帧。 29. The computer-readable storage medium of claim 28, wherein said converting the first video data frame into a video encoding format to obtain the second video data frame specifically includes: said virtual The display driver converts the video encoding format of the first video data frame to obtain the second video data frame.
30、如权利要求 27所述的计算机可读存储介质,其特征在于, 还包括当所述客户端不支持视频数据帧解码时, 并且当所述第一 视频数据帧为未解码的视频数据帧时, 向所述客户端发送第二视 频数据帧和第二视频数据帧处理指令之前, 根据所述第一视频数 据帧处理指令中的所述视频数据帧解码指令对所述第一视频数据 帧进行解码, 以获得所述第二视频数据帧。 30. The computer-readable storage medium of claim 27, further comprising: when the client does not support video data frame decoding, and when the first video data frame is an undecoded video data frame. When, before sending the second video data frame and the second video data frame processing instruction to the client, the first video data frame is processed according to the video data frame decoding instruction in the first video data frame processing instruction. Decoding is performed to obtain the second video data frame.
31、 如权利要求 27或 30所述的计算机可读存储介质, 其特 征在于, 还包括: 当所述客户端不支持视频数据帧解码时, 向所 述客户端发送所述第二视频数据帧和所述第二视频数据帧处理指 令之前, 压缩所述第二视频数据帧; 则向所述客户端发送第二视 频数据帧, 具体为向所述客户端发送压缩后的第二视频数据帧。 31. The computer-readable storage medium according to claim 27 or 30, further comprising: when the client does not support video data frame decoding, sending the second video data frame to the client and the second video data frame processing instruction, compress the second video data frame; then send the second video data frame to the client, specifically sending the compressed second video data frame to the client .
32、 如权利要求 27至 31所述的方法, 其特征在于, 还包括: 所述截获视频播放器向虚拟显示驱动发送的第一视频数据帧和第 一视频数据帧处理指令之前, 获取所述客户端的视频数据帧处理 能力; 所述客户端的视频数据帧处理能力包括视频数据帧解码能 力, 以根据所述视频数据帧解码能力, 确定所述客户端是否支持 视频数据帧解码。 32. The method of claims 27 to 31, further comprising: before intercepting the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver, obtaining the Video data frame processing capability of the client; The video data frame processing capability of the client includes video data frame decoding capability, so as to determine whether the client supports video data frame decoding based on the video data frame decoding capability.
33、 如权利要求 28、 29或 32所述的方法, 其特征在于, 还 包括: 根据所述客户端的视频数据帧处理能力, 确定所述客户端 支持视频数据帧解码时, 向所述客户端发送第二视频数据帧和所 述第一视频数据帧处理指令之前, 获取所述客户端支持的视频数 据帧的视频编码格式信息, 以根据所述视频编码格式信息, 确定 所述客户端是否支持所述第一视频数据帧的视频编码格式。 33. The method of claim 28, 29 or 32, further comprising: based on the video data frame processing capability of the client, when it is determined that the client supports video data frame decoding, reporting to the client Send the second video data frame and all Before processing the first video data frame instruction, obtain the video encoding format information of the video data frame supported by the client, so as to determine whether the client supports the first video data frame based on the video encoding format information. Video encoding format.
34、 如权利要求 27至 33任一所述的计算机可读存储介质, 其特征在于, 所述截获视频播放器向虚拟显示驱动发送的第一视 频数据帧和第一视频数据帧处理指令, 具体为所述虚拟显示驱动 截获所述第一视频数据帧和所述第一视频处理指令。 34. The computer-readable storage medium according to any one of claims 27 to 33, wherein the intercepted first video data frame and first video data frame processing instruction sent by the video player to the virtual display driver, specifically The first video data frame and the first video processing instruction are intercepted for the virtual display driver.
PCT/CN2013/071493 2013-02-07 2013-02-07 Video redirection method, device and system, and computer readable medium WO2014121477A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2013/071493 WO2014121477A1 (en) 2013-02-07 2013-02-07 Video redirection method, device and system, and computer readable medium
CN201380000046.7A CN104067627B (en) 2013-02-07 2013-02-07 Method, device, system and computer-readable medium that a kind of video is redirected

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2013/071493 WO2014121477A1 (en) 2013-02-07 2013-02-07 Video redirection method, device and system, and computer readable medium

Publications (1)

Publication Number Publication Date
WO2014121477A1 true WO2014121477A1 (en) 2014-08-14

Family

ID=51299186

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2013/071493 WO2014121477A1 (en) 2013-02-07 2013-02-07 Video redirection method, device and system, and computer readable medium

Country Status (2)

Country Link
CN (1) CN104067627B (en)
WO (1) WO2014121477A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105611394A (en) * 2015-12-21 2016-05-25 福建时迅信息科技有限公司 Method and system for redirecting video under VDI environment
US10979470B2 (en) 2015-12-28 2021-04-13 Industrial Technology Research Institute Server device, client device and dynamic image transmission method for virtual desktop infrastructure

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104202615A (en) * 2014-09-25 2014-12-10 湖北第二师范学院 Video redirection method based on cloud computing
CN106034241B (en) * 2015-03-19 2019-04-26 华为技术有限公司 A kind of method, client, server and system that multimedia redirects
CN105025359A (en) * 2015-07-07 2015-11-04 国鼎网络空间安全技术有限公司 Remote video display method and system based on SPICE protocol
CN106101794A (en) * 2016-06-22 2016-11-09 杭州华三通信技术有限公司 The processing method of a kind of virtual desktop video data, Apparatus and system
CN106888385A (en) * 2017-01-17 2017-06-23 武汉噢易云计算股份有限公司 Video broadcasting method and system of the client under virtualized environment

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101390397A (en) * 2006-02-24 2009-03-18 微软公司 Accelerated video encoding
CN102158553A (en) * 2011-03-31 2011-08-17 华为技术有限公司 Method and device for playing multi-media files for remote desktop
CN102547394A (en) * 2011-12-27 2012-07-04 福建升腾资讯有限公司 Universal audio/video redirection method for cloud access devices under VDI (Video Desktop Infrastructure) environment

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101582926B (en) * 2009-06-15 2012-05-16 中国电信股份有限公司 Method for realizing redirection of playing remote media and system
CN102541555A (en) * 2011-12-27 2012-07-04 福建升腾资讯有限公司 Method for remotely smoothly playing desktop video in API (application programming interface) interception mode
CN102695094B (en) * 2012-05-15 2014-12-10 福建升腾资讯有限公司 Method for carrying out software and hardware integrated audio/video redirection in VDI (Virtual Device Interface) environment
CN102761603B (en) * 2012-06-07 2014-10-22 福建升腾资讯有限公司 Webpage flash video redirection method in VDI environment

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101390397A (en) * 2006-02-24 2009-03-18 微软公司 Accelerated video encoding
CN102158553A (en) * 2011-03-31 2011-08-17 华为技术有限公司 Method and device for playing multi-media files for remote desktop
CN102547394A (en) * 2011-12-27 2012-07-04 福建升腾资讯有限公司 Universal audio/video redirection method for cloud access devices under VDI (Video Desktop Infrastructure) environment

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105611394A (en) * 2015-12-21 2016-05-25 福建时迅信息科技有限公司 Method and system for redirecting video under VDI environment
CN105611394B (en) * 2015-12-21 2018-11-06 福建时迅信息科技有限公司 Video reorientation method under VDI environment and system
US10979470B2 (en) 2015-12-28 2021-04-13 Industrial Technology Research Institute Server device, client device and dynamic image transmission method for virtual desktop infrastructure

Also Published As

Publication number Publication date
CN104067627B (en) 2017-08-04
CN104067627A (en) 2014-09-24

Similar Documents

Publication Publication Date Title
CN111399964B (en) Cloud desktop platform based on video streaming technology
WO2014089793A1 (en) Method, device, and system for video redirection and computer readable medium
US10721282B2 (en) Media acceleration for virtual computing services
WO2014121477A1 (en) Video redirection method, device and system, and computer readable medium
US10542301B2 (en) Multimedia redirection method, device, and system
EP3046331B1 (en) Media control method and system based on cloud desktop
WO2021143182A1 (en) Game processing method and apparatus, electronic device, and computer-readable storage medium
EP3706365B1 (en) Secondary-stream data transmission method and display method for use in conference, conference system, and peripheral device
WO2017124860A1 (en) Distributed wireless multi-screen virtual machine service system
KR20140035707A (en) Data redirection system and method for providing data redirection service
CN111240631A (en) Method and system for virtually transmitting Linux virtual machine screen picture based on AMD display card
WO2014139269A1 (en) Method and apparatus for playing and processing videos based on a virtual desktop
WO2020151660A1 (en) Stb cloudification method and system, thin stb, virtual stb and platform, and storage medium
WO2012151957A1 (en) Server, client, and method and system for playing video file remotely therewith
KR101931514B1 (en) Apparatus and method for simultaneous playback and backup of media in a web browser
WO2024037110A1 (en) Data processing method and apparatus, device, and medium
AlDuaij et al. Heterogeneous multi-mobile computing
CN102546803B (en) Remote desktop communication method based on capability set
CN114268626A (en) Window processing system, method and device
CN106658070B (en) Method and device for redirecting video
KR102199270B1 (en) System for cloud streaming service, method of cloud streaming service based on still image and apparatus for the same
WO2016066056A1 (en) Image remote projection method, server and client
KR20170105317A (en) Method for transmitting screen of virtual machine using graphic procedding unit based on hardware and apparatus using the same
WO2016107174A1 (en) Method and system for processing multimedia file data, player and client
KR102546633B1 (en) Method for transmitting screen of virtual machine using graphic procedding unit based on hardware and apparatus using the same

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13874658

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13874658

Country of ref document: EP

Kind code of ref document: A1