WO2014121477A1

WO2014121477A1 - Video redirection method, device and system, and computer readable medium

Info

Publication number: WO2014121477A1
Application number: PCT/CN2013/071493
Authority: WO
Inventors: 陈普; 许天锡
Original assignee: 华为技术有限公司
Priority date: 2013-02-07
Filing date: 2013-02-07
Publication date: 2014-08-14
Also published as: CN104067627B; CN104067627A

Abstract

By intercepting video data frames and a video data frame processing instruction transmitted from a video player to a virtual display driver, according to whether a client supports video data frame decoding, the embodiments of the present invention transmit to the client the video data frames, a video data frame decoding instruction, a video data frame post-processing instruction and a video data frame presentation instruction for processing, or transmit to the client the decoded video data frames, the video data frame post-processing instruction and the video data frame presentation instruction for processing, and thus can realize video data frame post-processing at the client, ensuring the video experience of users. At the same time, provided that a server-side video player supports the video redirection function, the present invention can be realized, without relying on a certain specific video player.

Description

Method, device, system and computer readable medium for video redirection

The present invention relates to the field of information technology, and in particular, to a video redirection method, apparatus, system, and computer readable medium.

Background technique

With the continuous development of computer technology, remote desktop has become a popular technology. Remote desktop technology transfers the content displayed on the remote server screen to the local client for display by logging in to the remote server. Remote desktop applications not only refer to remote screen displays, but also remote video delivery.

In the prior art, when playing a video file using a video player in a remote desktop, a multimedia player based on the DirectShow® framework introduced by Microsoft® can be used to load a video file through the video player on a remote server. The Filter in DirectShow® intercepts the undecoded video data, inserts the Filter in the client's DirectShow® framework, receives the video data sent by the remote server for decoding and display, and sends the video data display information to the client, but the video is redirected. The solution relies on Microsoft® Media Player® video player and does not post-process video data, affecting the user experience.

Summary of the invention

The embodiment of the invention discloses a method, a device, a system and a computer readable medium for video redirection to implement post processing of video data frames.

In a first aspect, the embodiment of the present invention discloses a video redirection method, including: intercepting a first video data frame and a first video data frame processing instruction sent by a video player to a virtual display driver;

Transmitting, by the client, a second video data frame and the first video data frame processing instruction to the client, so that the client is configured according to the first The video data frame processing instruction processes the second video data frame, where the first video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, the second video The data frame is the first video data frame or a video data frame obtained by converting the first video data frame by a video encoding format; when the client does not support video data frame decoding, sending the data to the client The second video data frame and the second video data frame processing instruction, so that the client processes the second video data frame according to the second video data frame processing instruction, where the second video data frame processing The instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is a decoded video data frame, the second video data frame is the first video data frame, The second video data frame processing instruction is the first video data frame processing instruction; when the first video data frame is an undecoded video data frame The second video data frame is a decoded first video data frame, and the first video data frame processing instruction includes a video data frame decoding instruction, the video data frame post-processing instruction, and the video data frame presentation. instruction.

With reference to the first aspect of the present invention, in a first possible implementation manner, the method further includes: when the client supports video data frame decoding, and the client does not support the video player to the virtual When the video encoding format of the first video data frame that is sent by the driver is displayed, the first video data frame is subjected to video encoding format conversion to obtain the second video data frame; wherein, the second video data frame is The video encoding format is a video encoding format supported by the client.

With reference to the first possible implementation manner of the first aspect of the present invention, in a second possible implementation manner, the first video data frame is subjected to video encoding format conversion to obtain the second video data frame, Specifically, the virtual display driver performs video encoding format conversion on the first video data frame to obtain the second video data frame.

In conjunction with the embodiment of the first aspect of the present invention, in a third possible implementation, The method includes: when the client does not support video data frame decoding, and when the first video data frame is an undecoded video data frame, sending the second video data frame and the second video data frame processing to the client Before the instruction, the first video data frame is decoded according to the video data frame decoding instruction in the first video data frame processing instruction to obtain the second video data frame.

With reference to the first aspect of the present invention, or the third possible implementation manner of the first aspect, in a fourth possible implementation, the method further includes: when the client does not support video data frame decoding, Before the client sends the second video data frame and the second video data frame processing instruction, compressing the second video data frame; sending a second video data frame to the client, specifically to the The client sends the compressed second video data frame.

With reference to the first aspect of the present invention or any one of the first to fourth possible embodiments of the first aspect, in a fifth possible implementation, the method further includes: sending, by the intercepting video player, a virtual display driver Obtaining a video data frame processing capability of the client before a video data frame and a first video data frame processing instruction; the video data frame processing capability of the client includes a video data frame decoding capability, according to the video data frame decoding capability And determining whether the client supports video data frame decoding.

With reference to the first, second or fifth possible implementation manner of the first aspect of the present invention, in a sixth possible implementation, the method further includes: determining, according to the video data frame processing capability of the client When the client supports the decoding of the video data frame, before sending the second video data frame and the first video data frame processing instruction to the client, acquiring video encoding format information of the video data frame supported by the client, according to The video encoding format information determines whether the client supports a video encoding format of the first video data frame. In combination with the first aspect of the invention, or any of the first to sixth possible embodiments, In a seventh possible implementation manner, the intercepting the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver, specifically, the virtual display driver intercepts the first video data frame And the first video processing instruction.

In a second aspect, an embodiment of the present invention discloses a server, including:

An intercepting unit, configured to intercept a first video data frame and a first video data frame processing instruction sent by the video player to the virtual display driver;

a sending unit, configured to send, when the client supports video data frame decoding, a second video data frame and the first video data frame processing instruction to the client, so that the client is configured according to the first video data The frame processing instruction processes the second video data frame, where the first video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, and the second video data frame a video data frame obtained by converting the first video data frame or the first video data frame by a video encoding format;

When the client does not support video data frame decoding, sending a second video data frame and a second video data frame processing instruction to the client, so that the client processes the instruction according to the second video data frame. The second video data frame processing is performed, where the second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is a decoded video data frame The second video data frame is the first video data frame, the second video data frame processing instruction is the first video data frame processing instruction; when the first video data frame is undecoded In the video data frame, the second video data frame is a decoded first video data frame, and the first video data frame processing instruction includes a video data frame decoding instruction, the video data frame post-processing instruction, and the video. The data frame presents instructions.

With reference to the second aspect of the present invention, in a first possible implementation, the method further includes a converting unit, where the converting unit is configured to support video data frame decoding when the client supports When the client does not support the video encoding format of the first video data frame sent by the video player to the virtual display driver, the first video data frame is converted into a video encoding format to obtain the first And a video encoding format of the second video data frame is a video encoding format supported by the client.

With reference to the first possible implementation manner of the second aspect of the present invention, in a second possible implementation manner, the converting unit is specifically the virtual display driving, and the video data of the first video data frame is performed. The encoding format conversion, obtaining the second video data frame, specifically includes: the virtual display driver performs video encoding format conversion on the first video data frame to obtain the second video data frame.

With reference to the embodiment of the second aspect of the present invention, in a third possible implementation, the method further includes: a decoding unit, when the client does not support video data frame decoding, and when the first video data frame is The undecoded video data frame, before the second video data frame and the second video data frame processing instruction are sent to the client, according to the video data frame decoding instruction in the first video data frame processing instruction The first video data frame is decoded to obtain the second video data frame.

With reference to the second aspect of the present invention or the third possible implementation manner of the second aspect, in a fourth possible implementation, the method further includes a compression unit, where the compression unit is configured to: when the client does not support video data frames Decoding, before transmitting the second video data frame and the second video data frame processing instruction to the client, compressing the second video data frame; sending a second video data frame to the client, Specifically, the compressed second video data frame is sent to the client.

With reference to the second aspect of the present invention, or any one of the first to fourth possible embodiments of the second aspect, in a fifth possible implementation, the method further includes an acquiring unit, where the acquiring unit is used by the acquiring unit Obtaining before acquiring the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver The video data frame processing capability of the client; the video data frame processing capability of the client includes a video data frame decoding capability, to determine whether the client supports video data frame decoding according to the video data frame decoding capability.

With reference to the first, second or fifth possible implementation manner of the second aspect of the present invention, in a sixth possible implementation, the acquiring unit is further configured to: according to the video data frame processing capability of the client And determining, before the client supports video data frame decoding, acquiring a video encoding format of the video data frame supported by the client, before sending the second video data frame and the first video data frame processing instruction to the client And determining information, according to the video encoding format information, whether the client supports a video encoding format of the first video data frame.

With reference to the second aspect of the present invention, or any of the first to sixth possible implementation manners of the second aspect, in the seventh possible implementation, the intercepting unit is specifically a virtual display driver, and the virtual display driver Intercepting the first video data frame and the first video processing instruction.

In a third aspect, an embodiment of the present invention discloses a server, including a memory and a central processing unit, where the memory and the central processing unit are connected by a system bus, where the memory stores computer executable instructions when the server is running. The central processor executes the computer-executable instructions stored in the memory, so that the server performs the following steps: intercepting a first video data frame and a first video data frame processing instruction sent by the video player to the virtual display driver;

When the client supports video data frame decoding, sending a second video data frame and the first video data frame processing instruction to the client, so that the client processes the instruction according to the first video data frame Processing, the second video data frame instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, where the second video data frame is the first Video data frame or a video data frame obtained by converting the first video data frame by a video encoding format; when the client does not support video data frame decoding, sending the second video data frame and the second video data frame to the client Directing, by the client, to process the second video data frame according to the second video data frame processing instruction, where the second video data frame processing instruction includes a video data frame post-processing instruction and video data a frame presentation instruction; when the first video data frame is a decoded video data frame, the second video data frame is the first video data frame, and the second video data frame processing instruction is the a video data frame processing instruction; when the first video data frame is an undecoded video data frame, the second video data frame is a decoded first video data frame, and the first video data frame processing instruction A video data frame decoding instruction, the video data frame post-processing instruction, and the video data frame rendering instruction are included.

With reference to the embodiment of the third aspect of the present invention, in a first possible implementation, the method further includes: when the client supports video data frame decoding, and the client does not support the video player to the And performing a video encoding format conversion on the first video data frame to obtain the second video data frame, where the second video data frame is obtained. The video encoding format is the video encoding format supported by the client.

With reference to the first possible implementation manner of the third aspect of the present invention, in a second possible implementation manner, the first video data frame is subjected to video encoding format conversion to obtain the second video data frame, Specifically, the virtual display driver performs video encoding format conversion on the first video data frame to obtain the second video data frame.

With reference to the embodiment of the third aspect of the present invention, in a third possible implementation, the method further includes: when the client does not support video data frame decoding, and when the first video data frame is undecoded a video data frame, before the second video data frame and the second video data frame processing instruction are sent to the client, processed according to the first video data frame The video data frame decoding instruction in the instruction decodes the first video data frame to obtain the second video data frame.

With reference to the third aspect of the present invention or the third possible implementation manner of the third aspect, in a fourth possible implementation, the method further includes: when the client does not support video data frame decoding, to the client Before transmitting the second video data frame and the second video data frame processing instruction, compressing the second video data frame; sending the second video data frame to the client, specifically to the client The terminal sends the compressed second video data frame.

With reference to the third aspect of the present invention, or any one of the first to fourth possible embodiments of the third aspect, in a fifth possible implementation, the method further includes: sending, by the intercepted video player, a virtual display driver Obtaining a video data frame processing capability of the client before the first video data frame and the first video data frame processing instruction; the video data frame processing capability of the client includes a video data frame decoding capability, to decode according to the video data frame Capability, determining whether the client supports video data frame decoding.

With reference to the first, second or fifth possible implementation manner of the third aspect of the present invention, in a sixth possible implementation, the method further includes: determining the client according to the video data frame processing capability of the client When the video data frame is decoded, the video encoding format information of the video data frame supported by the client is acquired before the second video data frame and the first video data frame processing instruction are sent to the client, according to the The video encoding format information determines whether the client supports the video encoding format of the first video data frame.

With reference to the third aspect of the present invention, or any of the first to sixth possible implementation manners of the third aspect, in the seventh possible implementation manner, the intercepting video player sends the first video to the virtual display driver The data frame and the first video data frame processing instruction, in particular, the virtual display driver intercepts the first video data frame and the first video processing instruction.

In a fourth aspect, an embodiment of the present invention discloses a video redirection system, including a client. And the server according to any one of the second aspects of the embodiments of the present invention.

In a fifth aspect, the embodiment of the present invention discloses a video redirection system, including a client and a server according to any one of the third aspects of the embodiments of the present invention.

In a sixth aspect, an embodiment of the present invention discloses a computer readable storage medium. When the computer executes the computer readable storage medium, the computer performs the following steps: intercepting a first video sent by the video player to the virtual display driver. a data frame and a first video data frame processing instruction;

When the client supports video data frame decoding, sending a second video data frame and the first video data frame processing instruction to the client, so that the client processes the instruction according to the first video data frame Processing, the second video data frame instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, where the second video data frame is the first a video data frame or a video data frame obtained by converting the first video data frame by a video encoding format; when the client does not support video data frame decoding, sending the second video data frame and the second to the client The video data frame processing instruction, so that the client processes the second video data frame according to the second video data frame processing instruction, where the second video data frame processing instruction includes video data frame post processing Command and video data frame presentation instructions; when the first video data frame is a decoded video data frame, the second video data The frame is the first video data frame, the second video data frame processing instruction is the first video data frame processing instruction; when the first video data frame is an undecoded video data frame, the The second video data frame is a decoded first video data frame, and the first video data frame processing instruction includes a video data frame decoding instruction, the video data frame post-processing instruction, and the video data frame presentation instruction.

With reference to the sixth aspect of the present invention, in a first possible implementation, the method further includes: when the client supports video data frame decoding, and the client does not support the video When the player is in the video encoding format of the first video data frame sent by the virtual display driver, the first video data frame is subjected to video encoding format conversion to obtain the second video data frame; The video encoding format of the second video data frame is a video encoding format supported by the client.

With reference to the first possible implementation manner of the sixth aspect of the present invention, in a second possible implementation manner, the first video data frame is subjected to video encoding format conversion to obtain the second video data frame, Specifically, the virtual display driver performs video encoding format conversion on the first video data frame to obtain the second video data frame.

With reference to the sixth aspect of the present invention, in a third possible implementation, the method further includes: when the client does not support video data frame decoding, and when the first video data frame is an undecoded video data frame, Decoding the first video data frame according to the video data frame decoding instruction in the first video data frame processing instruction before sending the second video data frame and the second video data frame processing instruction to the client Obtaining the second video data frame.

With reference to the sixth aspect of the present invention or the third possible implementation manner of the sixth aspect, in a fourth possible implementation, the method further includes: when the client does not support video data frame decoding, to the client Before transmitting the second video data frame and the second video data frame processing instruction, compressing the second video data frame; sending a second video data frame to the client, specifically sending to the client The compressed second video data frame.

With reference to the sixth aspect of the present invention, or any one of the first to fourth possible embodiments of the sixth aspect, in a fifth possible implementation, the method further includes: sending, by the intercepted video player, a virtual display driver Obtaining a video data frame processing capability of the client before the first video data frame and the first video data frame processing instruction; the video data frame processing capability of the client includes a video data frame decoding capability, to decode according to the video data frame Capability, determining whether the client supports video data frame decoding. With reference to the first, second or fifth possible implementation manner of the sixth aspect of the present invention, in a sixth possible implementation, the method further includes: determining, according to the video data frame processing capability of the client, the client When the video data frame is decoded, the video encoding format information of the video data frame supported by the client is acquired before the second video data frame and the first video data frame processing instruction are sent to the client, according to the The video encoding format information determines whether the client supports the video encoding format of the first video data frame.

With reference to the sixth aspect of the present invention, or any of the first to sixth possible implementation manners of the sixth aspect of the present invention, in the seventh possible implementation manner, the intercepting video player sends the first to the virtual display driver a video data frame and a first video data frame processing instruction, specifically, the virtual display driver intercepts the first video data frame and the first video processing instruction.

The embodiment of the present invention intercepts the video data frame and the video data frame processing instruction sent by the video player to the virtual display driver, and according to whether the client supports video data frame decoding, the video data frame, the video data frame decoding instruction, and the video data frame. The post-processing instruction and the video data frame presentation instruction are sent to the client for processing, or the decoded video data frame, the video data frame post-processing instruction, and the video data frame presentation instruction are sent to the client for processing, thereby being implemented on the client side. The video data frame is post-processed to ensure the user's video experience. At the same time, as long as the server side video player supports the video redirection function, the present invention can be implemented without relying on a specific video player.

DRAWINGS

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description are only some of the present invention. For the embodiments, those skilled in the art can obtain other drawings according to the drawings without any creative work. FIG. 1 is a schematic diagram of an application scenario system according to an embodiment of the present invention;

2 is a schematic structural diagram of a remote desktop server according to an embodiment of the present invention;

3 is a flowchart of post processing of a video data frame according to an embodiment of the present invention;

4a and FIG. 4b are diagrams showing a post-processing effect of a video data frame according to an embodiment of the present invention; FIG. 5 is a diagram showing a rendering effect of a video data frame according to an embodiment of the present invention;

6a, 6b, 6c, and 6d are schematic diagrams of a virtual desktop structure according to an embodiment of the present invention; FIG. 7 is a flowchart of video redirection according to an embodiment of the present invention;

8 is a schematic structural diagram of a server according to an embodiment of the present invention;

FIG. 9 is a schematic structural diagram of a system according to an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

In the remote desktop environment, the client initiates a video play request, and the server loads the video file through the video player according to the video play request of the client, and transmits the video stream of the video file to the client. This process is called video redirection. The server in the embodiment of the present invention may be a general personal computer or other terminal, or may be a commercial server, etc., and is not specifically limited herein. The embodiment of the present invention provides a remote desktop service based on the Windows® operating system platform to the client. The invention is described in detail by way of example. However, the solution of the present invention is not limited to the Windows® operating system. According to the solution of the present invention, it can be extended to other common system platforms, such as the Linux® operating system.

The embodiment of the invention discloses a method for video redirection, which is applied to a remote desktop scenario. Remote desktop scenarios typically consist of a local computer and a remote computer. Local computer pass Access the remote computer and transfer the remote computer desktop to the local computer. The local computer, that is, the client in the embodiment of the present invention; the remote computer, that is, the server in the embodiment of the present invention. The video redirection in the embodiment of the present invention refers to transmitting the video data stream of the video file on the server to the client. One of the remote desktop scenarios is a virtual desktop infrastructure scenario, but the present invention is not limited to the architecture shown in FIG. 1. The solution of the present invention can be applied to other scenarios for transmitting remote video data to a client. The virtual desktop infrastructure scenario is shown in FIG. 1. This embodiment is merely an exemplary description, and only the components related to the embodiment of the present invention are given, but FIG. 1 is not specifically limited to the application of the present invention. The left side of FIG. 1 is a client of a virtual desktop, and the device form of the client may be a general computer 101a, a tablet 101b, a smart phone 101c, and the like. They access the remote desktop service over the network 102 using the Remote Desktop Protocol 103. The servers 204a....204n provide a carrier for the remote desktop, and the user's virtual desktop (Virtual Desktop) runs on the server in the form of virtual machines 205a, 205b-205n. The virtual desktop management system 106 is configured to provide functions such as mapping of a client's client to a virtual machine. The client first connects to the virtual desktop management system 106, obtains the virtual machine address of the user, and then connects to the virtual machine. The virtual desktop management system 106 can be a server or a general personal computer. The present invention does not specifically limit this. The user accesses the virtual desktop (ie, virtual machine) assigned to the user through the client, and the virtual desktop transmits the content accessed by the user to the user's client for display.

Taking the server 204a as an example, the hardware structure and the software structure are the most important components in the virtual desktop infrastructure, and its typical structure is shown in FIG. 2. A typical structure in the server 204a includes three parts, a hardware platform 300, a virtualization platform (also referred to as a hypervisor) 301, and a virtual machine execution space 302. The hard disk 310, the central processing unit 311, the network interface card 312, and the memory 313 in the physical structure of the server may be included in the hardware platform 300. Hypervisor 301 and virtual machine execution space 302 provide virtual machine respectively Hardware platform and software platform. The Hypervisor includes virtual machine monitors 320a, 320b-320n. Hypervisor is the middle layer between physical hardware and virtual machines, and is responsible for coordinating the access of each virtual machine to the hardware platform. The virtual machine monitor includes a virtual hardware platform that implements the virtual machine. In FIG. 2, the virtual machine monitor 320a includes a virtual hardware platform 321a. The virtual hardware platform includes a virtual memory 322, a central processing unit 323, a hard disk 324, a graphics card 325, a network interface card 326, and the like. Running in a virtual machine execution space 302 multiple virtual machines _205a, 205b · 205η ο 205a of the virtual machine software system running on a virtual machine monitor 320a provides a virtual hardware platform 321a, other virtual machine software system is also running in the corresponding Virtual machine monitor on the virtual hardware platform. The guest operating system 330 runs in the virtual machine 205a. The guest operating system 330 includes a device driver layer 331, and the device driver layer 331 is installed with drivers such as a virtual display driver 334 and a network interface card driver 336. A video player 332 is included in the virtual machine 205a. Among them, the guest operating system 330 can be a Windows® operating system platform provided by Microsoft®.

As shown in FIG. 2, when the user logs in to the virtual machine 205a to view the video using the client 101a, such as watching Youku® network video, or the virtual machine 205a local video file, the video file is loaded by the video player 332. When the user logs in to the virtual machine 205a using the client 101a, the video data frame processing capability of the client 101a is provided to the virtual machine 205a, wherein the video data frame processing capability includes the video data frame decoding capability, and the client 101a is also provided to the virtual machine 205a. Video encoding format information of supported video data frames, such as video files supporting video encoding formats such as H.264, AVI, and FLV. In the embodiment of the present invention, a local video is taken as an example, and a video play request from the client 101a is received by the desktop protocol server 335, and the video player 332 loads the video file according to the video play request. The video player 332 demultiplexes the loaded video file to obtain a video data frame. The video player 332 needs to determine whether the virtual machine 205a supports the video hardware acceleration function during initialization, and the video player 332 sends the video hardware plus to the virtual display driver 334. Speed capability query request. Video hardware acceleration refers to processing a video data frame by using a computer software instead of a software algorithm of a video player, and specifically includes decoding a video data frame and post-processing a video data frame.

For video data frame post-processing, in this embodiment, Mircosoft® video hardware acceleration (DirectX® Video Acceleration, referred to as DXVA) is taken as an example, and the video data frame is post-processed and the video data frame is post-processed. Presentation is described. DXVA not only provides video data frame decoding processing, but also video data frame post-processing, such as image scaling, color space conversion and de-interlacing. As shown in Figure 3, DXVA is a post-processing flowchart for video data frames. DXVA can process the main data stream and the 15-way sub-data stream. After the main data stream is subjected to noise filtering, de-interlacing processing, detail filtering and color space conversion, it is mixed with the color space-converted sub-data stream and mixed. The subsequent video data stream is subjected to image adjustment, wherein the image adjustment includes brightness, contrast, saturation adjustment, and the like. The image-scaled video stream is image scaled to adjust the image size. The image-scaled data is subjected to color space conversion, so that the video data stream is subjected to video data frame presentation. Among them, color space conversion, including conversion from RGB color space to YUV color space, or conversion from YUV color space to RGB color space, where RGB represents red, green and blue, respectively, YUV represents brightness, chromaticity and concentration, respectively. The embodiments of the present invention are not described herein again.

Figure 3 depicts the process of video post processing. Not all functions in the actual application will be used. In actual tests, most players such as Microsoft® Media Player, Storm Video 8, QQ® Video Player, etc. only use image scaling by default to scale the decoded video data frame from the original size to the playback window size. In the embodiment of the present invention, the video data frame post-processing is specifically performed by taking image scaling as an example, and the decoded video data frame is subjected to image scaling, and the original image size is adjusted to be suitable for presentation on the display. The specific effect is shown in Figure 4a. Video data frame post-processing can also include transparency blending, as shown 4b is shown. The video data frame post-processing in the embodiment of the present invention may further include brightness, contrast, and saturation adjustment. This allows the client to present a better video experience to the user.

The decoded video data frame is subjected to video data frame post-processing, and the video data frame post-processed by the video data frame is sent to the graphics card according to the video data frame presentation instruction, and is presented by the display. The video data frame presentation instruction carries a presentation parameter, and the presentation parameter includes a video data frame to be presented and a presentation area of the video data frame. As shown in FIG. 5, an image formed after post-processing of a video data frame includes four regions to be presented, denoted as 1, 2, 3, and 4, respectively, and a desktop image, which is presented on a display, wherein the desktop image has The area is used to present the user with the "display settings, audio settings, and subtitle settings" options. When the video is presented, the "display settings, audio settings, and subtitle settings" option areas need to be presented, and the "display" The setting, audio setting, and subtitle setting "option area corresponds to the intermediate area enclosed by the four presentation areas 1, 2, 3, and 4. Therefore, the video data frames corresponding to the intermediate areas surrounded by the four presentation areas 1, 2, 3, and 4 do not need to be presented, and the video data frames constituting the intermediate areas surrounded by the four presentation areas 1, 2, 3, and 4 are not presented. The video data frame. This allows the video picture to be integrated with the desktop, providing users with a better video viewing experience. The principle of the video data frame post-processing performed by the client 101a in the embodiment of the present invention is the same as that of the present invention and will not be described again.

In the embodiment of the present invention, the hardware of the virtual machine 205a, as understood by those skilled in the art, is not the real physical hardware of the virtual machine 205a, but the virtual hardware provided by the virtual machine monitor 320a. The video player 332 queries the virtual display driver 334 for the video hardware acceleration capability of the virtual machine 205a, and can be queried by calling the GetCaps function. Alternatively, the hardware acceleration device can be created by calling the CreateDevice or CreateDecodeDevice function to determine whether the virtual machine 205a supports video hardware acceleration. Features. When the video player 332 initiates a video querying the virtual display driver 334 for the virtual machine 205a In the hardware acceleration capability, the virtual display driver 334 returns a query request response supporting the video hardware acceleration capability to the video player 332, and the video player 332 determines the video hardware acceleration capability supported by the virtual machine 205a according to the query request response, wherein the video hardware acceleration Capabilities include video data frame decoding capabilities and video data frame post-processing capabilities.

The video player 332 sends the video data frame and the video data frame processing instruction to the virtual display driver 334. As shown in FIG. 6a, an implementation manner is that the virtual display driver 334 intercepts the video data frame and the video data frame processed by the video player 332. instruction.

The process by which the virtual display driver 334 intercepts the video data frame and video data frame processing instructions is specifically described below. Specifically, the video player 332 in the virtual machine 205a, according to the video play request sent by the corresponding client 101a, after the video player 332 loads the video file, starts the video hardware acceleration, and sends the video data frame and the video data to the virtual display driver 334. Frame processing instructions. After the video file is loaded, the loaded video file, that is, the video data stream is demultiplexed, to obtain a video data frame therein. Specifically, in terms of video playback, a video hardware acceleration interface is defined in the DirectX® runtime library 333, and the video hardware acceleration is initiated by the video player 332 calling the DirectX® runtime library 333, informing the virtual display driver 334 to process the instruction according to the video data frame. The video data frame transmitted by the video player 332 is processed. When the video data frame and video data frame processing instructions are sent to the virtual display driver 334, the virtual display driver 334 intercepts the video data frame and video data frame processing instructions. As shown in Figure 6a, on the Windows® operating system platform, the virtual display driver is Windows® Display Driver Model (referred to as WDDM display driver model) or Windows® XP Display Driver Model (referred to as XPDM display driver model). Specifically, the WDDM or XPDM display driver models include the Display Driver and Miniport Driver modules. In the embodiment of the present invention, an implementation scheme is that the display driver module of the virtual display driver intercepts the video data frame and the video data frame processing instruction. Specifically, an implementation scheme is a WDDM or XPDM display driver model. The video hardware acceleration interface is defined in the Display Driver module. When the video data frame and the video data frame processing instruction pass through the interface, the video data frame and the video data frame processing instruction are intercepted.

The embodiment of the present invention intercepts another embodiment of the video data frame and the video data frame processing instruction. As shown in FIG. 6b, the video player 332 in the virtual machine 205a performs video playback according to the video playback request sent by the corresponding client 101a. After the video file is loaded by the device 332, the video hardware acceleration is turned on, and the video data frame and the video data frame processing instruction are transmitted to the virtual display driver 334. After the video player 332 loads the video file, the loaded video file, that is, the video data stream is demultiplexed, to obtain a video data frame therein. Specifically, in terms of video playback, a video hardware acceleration interface is defined in the DirectX® runtime library 333, and the video hardware acceleration is initiated by the video player 332 calling the DirectX® runtime library 333, informing the virtual display driver 334 to process the instruction according to the video data frame. The video data frame transmitted by the video player 332 is processed. The DirectX® runtime library 333 invokes the virtual display driver 334 to process the video data frame, ie, after the video data frame and video data frame processing instructions pass through the DirectX® runtime 333, before the virtual display driver 334 is reached, the dynamic link The library intercepts the video data frame and video data frame processing instructions. Specifically, one of the dynamic link library implementations is to use the Hook technology to intercept the call of the virtual display driver 334 by the DirectX® runtime library 333, and implement a dynamic link library file (dll file) for the virtual display driver 334 Hook, and It is set to be loaded first in the registry to intercept the call of the DirectX® runtime 333 to the virtual display driver 334 to enable interception of video data frames and video data frame processing instructions.

The embodiment of the present invention intercepts another embodiment of the video data frame and the video data frame processing instruction. As shown in FIG. 6c, the video player 332 in the virtual machine 205a performs video playback according to the video playback request sent by the corresponding client 101a. After the video file is loaded by the device 332, the video hardware acceleration is turned on, and the video data frame and the video are transmitted to the virtual display driver 334. Frequency data frame processing instructions. After loading the video file, the video player 332 demultiplexes the loaded video file, that is, the video data stream, to obtain a video data frame therein. Specifically, in terms of video playback, a video hardware acceleration interface is defined in the DirectX® runtime 333, and the DirectX® runtime 333 is called by the video player 332 to send video data frames and video data frame processing instructions to the virtual display driver 334. The video hardware acceleration is initiated, and the virtual display driver 334 is notified to process the video data frame sent by the video player 332. When the video data frame and video data frame processing instructions pass through the DirectX® runtime library 333, the DirectX® runtime intercepts the video data frame and video data frame processing instructions.

The embodiment of the present invention intercepts another embodiment of the video data frame and the video data frame processing instruction. As shown in FIG. 6d, the video player 332 in the virtual machine 205a performs video playback according to the video playback request sent by the corresponding client 101a. After the video file is loaded by the device 332, the video hardware acceleration is turned on, and the video data frame and the video data frame processing instruction are transmitted to the virtual display driver 334. After the video player 332 loads the video file, the loaded video file, that is, the video data stream is demultiplexed, to obtain a video data frame therein. Specifically, in terms of video playback, a video hardware acceleration interface is defined in the DirectX® runtime library 333, and the video player 332 calls the DirectX® runtime library 333 to initiate video hardware acceleration, when transmitting video data frames and video data frame processing instructions. That is, before the video data frame and video data frame processing instructions arrive at the DirectX® runtime 333, the dynamic link library intercepts the video data frame and video data frame processing instructions. To do this, you need to implement a dynamic link library file (dll file) for the DirectX® runtime 333 Hook, and set it to be loaded first in the registry to intercept the video player 332 to the DirectX® runtime 333. Called to achieve interception of video data frames and video data frame processing instructions.

In the above-described schemes of capturing video data frames and video data frame processing instructions sent by the video player 332 to the virtual display driver 334 in FIGS. 6a, 6b, 6c, and 6d, the video player 332 queries the virtual display driver 334. Know the virtual machine 205a support The video data frame is decoded and the video data frame is post-processed, and the video data frame and the video data frame are captured when the client 101a supports the video data frame decoding and the video encoding format of the video data frame sent by the video player 332 to the virtual display driver 334. The processing instruction sends the intercepted video data frame and the video data frame to the desktop protocol server 335. The desktop protocol server 335 transmits the intercepted video data frame and video data frame processing instructions to the client 101a. The video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction. As a result of the server 204a, the computing resources and network transmission bandwidth of the server are saved, and the computing resources of the client are fully utilized. This embodiment is applicable to the following scenarios: intercepting a first video data frame and a first video data frame processing instruction sent by the video player 332 to the virtual display driver 334; and when the client 101a supports video data frame decoding, the client is 101a transmitting a second video data frame and the first video data frame processing instruction, so that the client 101a processes the second video data frame according to the first video data frame processing instruction, where the first The video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, and the second video data frame is the first video data frame.

In the above-described schemes of capturing video data frames and video data frame processing instructions sent by the video player 332 to the virtual display driver 334 in FIGS. 6a, 6b, 6c, and 6d, the video player 332 queries the virtual display driver 334. It is learned that the virtual machine 205a supports video data frame decoding and video data frame post-processing, and the client 101a supports video data frame decoding, but does not support the video encoding format of the video data frame transmitted by the video player 332 to the virtual display driver 334. a video data frame and a video data frame processing instruction, where the video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, and the desktop protocol server 335 receives the intercepted video. The data frame, the video codec module of the desktop protocol server 335 decodes the video data frame according to the video data frame decoding instruction, and performs the decoded video data frame according to the video encoding format of the video data frame supported by the client 101a. Encoding, generating a new video data frame decoding instruction, the new video data frame decoding instruction including a video encoding format parameter of the re-encoded video data frame. Transmitting the re-encoded video data frame and the video data frame processing instruction to the client 101a, where the video data frame processing instruction includes, a video data frame post-processing instruction, a video data frame presentation instruction, and a video data frame decoding instruction, where the video The data frame decoding instruction includes video encoding format parameters of the re-encoded video data frame. Thereby enabling the client to decode the re-encoded video data frame according to its own video encoding format processing capability. Due to the virtuality in the server 204a, the computing resources and network transmission bandwidth of the server are saved, and the computing resources of the client are fully utilized. This embodiment is applicable to the following scenarios: intercepting a first video data frame and a first video data frame processing instruction sent by the video player 332 to the virtual display driver 334; and when the client 101a supports video data frame decoding, the client is 101a transmitting a second video data frame and the first video data frame processing instruction, so that the client 101a processes the second video data frame according to the first video data frame processing instruction, where the first The video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, where the second video data frame is a video data frame obtained by converting the first video data frame by a video encoding format. .

In another embodiment, the video player 332 is virtualized in a scheme in which the virtual display driver 334 as shown in FIG. 6a intercepts the video data frame and the video data frame processing instructions transmitted by the video player 332 to the virtual display driver 334. The display driver 334 queries the learned virtual machine 205a to support video data frame decoding and video data frame post-processing, and the client 101a supports video data frame decoding, but does not support the video player 332 to virtual display. When the video encoding format of the video data frame transmitted by 334 is driven, the video codec module in the virtual display driver 334 decodes the captured video data frame according to the intercepted video data frame decoding instruction, according to the support of the client 101a. The video encoding format of the video data frame encodes the decoded video data frame to generate a new video data frame decoding instruction, where the new video data frame decoding instruction includes a video encoding format parameter of the re-encoded video data frame, The video data frame processing instruction includes a video data frame post-processing instruction, a video data frame presentation instruction, and a video data frame decoding instruction, where the video data frame decoding instruction includes a video encoding format parameter of the re-encoded video data frame. Thereby enabling the client to decode the re-encoded video data frame according to its own video encoding format processing capability. The desktop protocol server 335 sends the re-encoded video data frame and the video data frame processing instruction to the client 101a, where the video data frame processing instruction includes, the video data frame post-processing instruction, the video data frame presentation instruction, and the video data frame decoding. The instruction, wherein the video data frame decoding instruction includes a video encoding format parameter of the re-encoded video data frame. Thereby enabling the client to decode the re-encoded video data frame according to its own video encoding format processing capability. At the same time, the virtual machine 205a on the server 204a does not process the video data frame according to the video data frame post-processing instruction, but is processed by the client, which fully utilizes the computing resources of the client, and saves the computing resources of the server. This embodiment is applicable to the following scenarios: intercepting a first video data frame and a first video data frame processing instruction sent by the video player 332 to the virtual display driver 334; and when the client 101a supports video data frame decoding, the client is 101a transmitting a second video data frame and the first video data frame processing instruction, so that the client 101a processes the second video data frame according to the first video data frame processing instruction, where the first The video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, where the second video data frame is a video obtained by converting the first video data frame by a video encoding format. Data Frame.

In the above-mentioned schemes of capturing video data frames and video data frame processing instructions sent by the video player 332 to the virtual display driver 334 in FIGS. 6a, 6b, 6c, and 6d, when the client 101a does not support video data frame decoding. And the video player 332 queries the virtual display driver 334 to learn that the virtual machine 205a supports the video data frame post-processing but does not support the video data frame decoding. After the video player 332 loads the video file, the video file is demultiplexed to obtain the video data. a frame, the video player 332 decodes the video data frame to obtain a decoded video data frame, and the video player 332 sends the decoded video data frame and the video data frame processing instruction to the virtual display driver 334; wherein the video data video The decoded video data frame and the video data frame processing instruction sent by the player 332, the desktop protocol server 335 sends the decoded video data frame and the video data frame processing instruction to the client 101a. To reduce the network transmission bandwidth, before the desktop protocol server 335 sends the decoded video data frame and the video data frame processing instruction, the decoded video data frame may be compressed, which may be implemented by a compression module in the desktop protocol server 335. Since the virtual machine 205a in the server 204a does not process the video data frame according to the video data frame post-processing instruction, the computing resources and network transmission bandwidth of the server are saved, and the computing resources of the client are fully utilized. This embodiment is applicable to: capturing the first video data frame and the first video data frame processing instruction sent by the video player 332 to the virtual display driver 334; when the client 101a does not support video data frame decoding, The client 101a sends a second video data frame and a second video data frame processing instruction, so that the client 101a processes the second video data frame according to the second video data frame processing instruction, where The second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is a decoded video data frame, the second video data frame is the first Video data frame, The second video data frame processing instruction is the first video data frame processing instruction.

In another embodiment, in the scheme in which the virtual display driver 334 as shown in FIG. 6a intercepts the video data frame and the video data frame processing instruction sent by the video player 332 to the virtual display driver 334, when the client 101a does not support The video data frame is decoded, and the video player 332 queries the virtual display driver 334 to learn that the virtual machine 205a supports the video data frame post-processing but does not support the video data frame decoding. After the video player 332 loads the video file, the video file is demultiplexed. Obtaining a video data frame, the video player 332 decodes the video data frame to obtain a decoded video data frame, and the video player 332 sends the decoded video data frame and the video data frame processing instruction to the virtual display driver 334; Present the instruction. The decoded video data frame and the video data frame processing instruction sent by the video player 332 are intercepted, and the desktop protocol server 335 transmits the decoded video data frame and the video data frame processing instruction to the client 101a. To reduce the network transmission bandwidth, the virtual display driver 334 intercepts the decoded video data frame and the video data frame processing instruction, and can compress the decoded video data frame, which can be implemented by the compression module in the virtual display driver 334. Since the virtual machine 205a in the server 204a does not process the video data frame according to the video data frame post-processing instruction, the computing resources and network transmission bandwidth of the server are saved, and the computing resources of the client are fully utilized. This embodiment is applicable to: capturing the first video data frame and the first video data frame processing instruction sent by the video player 332 to the virtual display driver 334; when the client 101a does not support video data frame decoding, The client 101a sends a second video data frame and a second video data frame processing instruction, so that the client 101a processes the second video data frame according to the second video data frame processing instruction, where The second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is a decoded video data frame, the second video data frame is the first Video data frame, The second video data frame processing instruction is the first video data frame processing instruction.

In the above-mentioned FIG. 6a, FIG. 6b, FIG. 6c and FIG. 6d, in the scheme of intercepting the video data frame and the video data frame processing instruction sent by the video player 332 to the virtual display driver 334, when the client 101a does not support the video data frame decoding, And the video player 332 queries the virtual display driver 334 to learn that the virtual machine 205a supports video data frame decoding and video data frame post-processing. After the video player 332 loads the video file, the video file is demultiplexed to obtain a video data frame and a video. The player 332 transmits a video data frame and a video data frame processing instruction to the virtual display driver 334; wherein the video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction. The video data frame and the video data frame processing instruction sent by the video player 332 are intercepted, and the desktop protocol server 335 decodes the video data frame according to the video data frame decoding instruction in the video data frame processing instruction, and the desktop protocol server 335 sends the decoded data frame. The video data frame and video data frame processing instructions are sent to the client 101a, wherein the video data frame processing instructions include a video data frame post-processing instruction and a video data frame presentation instruction. To reduce the network transmission bandwidth, the desktop protocol server 335 can compress the decoded video data frame, which can be implemented by a compression module in the desktop protocol server 335, or can be implemented by a compression module in the virtual display driver 334, and then The compressed decoded video data frame and video data frame processing instructions are transmitted by the desktop protocol server 335 to the client 101a. Since the virtual machine 205a in the server 204a does not process the video data frame according to the video data frame post-processing instruction, the computing resources and network transmission bandwidth of the server are saved, and the computing resources of the client are fully utilized. This embodiment is applicable to: capturing the first video data frame and the first video data frame processing instruction sent by the video player 332 to the virtual display driver 334; when the client 101a does not support video data frame decoding, The client 101a sends a second video data frame and a second video data frame processing instruction, so that the client 101a processes the second video data frame according to the second video data frame processing instruction. The second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction. When the first video data frame is an undecoded video data frame, the second video data frame is The decoded first video data frame, the first video data frame processing instruction includes a video data frame decoding instruction, the video data frame post-processing instruction, and the video data frame rendering instruction.

In another embodiment, in the scheme shown in FIG. 6a, when the client 101a does not support video data frame decoding, and the video player 332 queries the virtual display driver 334 to learn that the virtual machine 205a supports video data frame decoding and After the video data frame is post-processed, after the video player 332 loads the video file, the video file is demultiplexed to obtain a video data frame, and the video player 332 sends a video data frame and a video data frame processing instruction to the virtual display driver 334; The data frame processing instructions include a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction. The virtual display driver 334 intercepts the video data frame and the video data frame processing instruction sent by the video player 332, decodes the video data frame according to the video data frame decoding instruction in the video data frame processing instruction, and sends the decoded data by the desktop protocol server 335. The video data frame and video data frame processing instructions are to the client 101a, wherein the video data frame processing instructions include a video data frame post-processing instruction and a video data frame presentation instruction. To reduce the network transmission bandwidth, the virtual display driver 334 may decode the decoded video data frame, and may be implemented by the compression module in the virtual display driver 334, or may be implemented by the desktop protocol server 335. The compression module is implemented, and then the compressed decoded video data frame and the video data frame processing instruction are sent by the desktop protocol server 335 to the client 101a. Since the server 204a processes the data, the computing resources and network transmission bandwidth of the server are saved, and the computing resources of the client are fully utilized. This embodiment is applicable to the following scenario: intercepting the first video data frame and the first video data frame processing finger sent by the video player 332 to the virtual display driver 334 When the client 101a does not support video data frame decoding, the second video data frame and the second video data frame processing instruction are sent to the client 101a, so that the client 101a is based on the second video. The data frame processing instruction processes the second video data frame, where the second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is not The second video data frame is the decoded first video data frame, and the first video data frame processing instruction includes a video data frame decoding instruction and the video data frame post-processing instruction. And presenting instructions to the video data frame.

A specific exemplary flowchart of the embodiment of the present invention is shown in FIG. 7 , and the specific steps are as follows:

Step 701: Determine whether the client has a video data frame decoding capability, and whether the virtual machine supports video data frame decoding and video data frame post-processing.

When the client has video data frame decoding capability, and the virtual machine supports video data frame decoding and video data frame post-processing, step 702a is performed;

When the client does not have the video data frame decoding capability, the virtual machine does not support the video data frame decoding but supports the video data frame post-processing, and the step 702b is performed;

When the client does not have the video data frame decoding capability, the virtual machine supports the video data frame solution. After the code and video data frame is processed, step 702c is performed.

Step 702a: intercepting a video data frame and a video data frame processing instruction sent by the video player to the virtual display driver; the video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction;

Step 703a: Determine whether the client supports the video encoding format of the video data frame sent by the video player.

When the client supports the video encoding format of the video data frame sent by the video player, step 704a is performed; when the client does not support the video encoding format of the video data frame sent by the video player, step 704a' is performed.

Step 704a: Send the intercepted video data frame and the video data frame processing instruction to the client;

Step 704a': Convert the video data frame into a video data frame of a video encoding format supported by the client.

Step 702b: intercepting the decoded video data frame, the video data frame post-processing instruction, and the video data frame presentation instruction sent by the video player to the virtual display driver;

Step 703B: Optionally, after step 702b, compressing the intercepted decoded video data frame;

Step 704C: Send the decoded video data frame and the video data frame processing instruction to the client; the video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction.

Step 702c: intercepting a video data frame and a video data frame processing instruction sent by the video player; the video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction;

Step 703c: Decode the video data frame according to a video data frame decoding instruction. Optionally, after step 703c, steps 703B and 704C may also be continued. For the specific implementation of the flowchart shown in FIG. 6a to FIG. 6d, details are not described herein.

Corresponding to the above embodiments, the embodiment of the present invention provides a server 800, as shown in FIG. 8, including an intercepting unit 801 and a sending unit 802. The intercepting unit 801 is configured to intercept the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver;

The sending unit 802 is configured to: when the client supports decoding of the video data frame, send the second video data frame and the first video data frame processing instruction to the client, so that the client is configured according to the first video The data frame processing instruction processes the second video data frame, where the first video data frame processing instruction includes a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction, the second video data The frame is the first video data frame or a video data frame obtained by converting the first video data frame by a video encoding format;

The sending unit 802 is configured to: when the client does not support video data frame decoding, send a second video data frame and a second video data frame processing instruction to the client, so that the client is according to the second The video data frame processing instruction processes the second video data frame, where the second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is The second video data frame is the first video data frame, and the second video data frame processing instruction is the first video data frame processing instruction; when the first video is When the data frame is an undecoded video data frame, the second video data frame is a decoded first video data frame, and the first video data frame processing instruction includes a video data frame decoding instruction, and the video data frame is Processing instructions and the video data frame presentation instructions.

The embodiment of the present invention intercepts a video data frame and a video data frame processing instruction sent by the video player to the virtual display driver, and according to whether the client supports the video data frame. Decoding, transmitting a video data frame, a video data frame decoding instruction, a video data frame post-processing instruction, and a video data frame presentation instruction to a client for processing, or decoding the decoded video data frame, a video data frame post-processing instruction, and video data The frame rendering instruction is sent to the client for processing, so that the video data frame post-processing can be implemented on the client to ensure the user's video experience. At the same time, as long as the server side video player supports the video redirection function, the present invention can be implemented without relying on a specific video player.

Further, a conversion unit 803 is further included, where the conversion unit 803 is configured to: when the client supports video data frame decoding, and the client does not support the video player to send to the virtual display driver The video encoding format of the first video data frame, the first video data frame is subjected to a video encoding format conversion to obtain the second video data frame; wherein the video encoding format of the second video data frame is the The video encoding format supported by the client. In a possible implementation, the converting unit 803 is specifically the virtual display driver, and the video data format conversion is performed on the first video data frame to obtain the second video data frame, which specifically includes: The virtual display driver performs video encoding format conversion on the first video data frame to obtain the second video data frame.

The server 800 provided by the embodiment of the present invention includes a decoding unit 801 and a sending unit 802, and a decoding unit 804, configured to: when the client does not support video data frame decoding, and when the first video data frame is not Decoding the video data frame, before transmitting the second video data frame and the second video data frame processing instruction to the client, according to the video data frame decoding instruction in the first video data frame processing instruction The first video data frame is decoded to obtain the second video data frame.

The server 800 provided by the embodiment of the present invention includes, in addition to the intercepting unit 801 and the sending unit 802, or the server 800, in addition to the intercepting unit 801, the sending unit 802, and the decoding unit 803, a compression unit 805, where the compression unit 805 is used. When the customer When the video data frame decoding is not supported, the second video data frame is compressed before the second video data frame and the second video data frame processing instruction are sent to the client, and then sent to the client. The second video data frame is specifically configured to send the compressed second video data frame to the client, thereby saving network transmission bandwidth.

The server 800 of the foregoing embodiment of the present invention further includes an obtaining unit 806, where the acquiring unit 806 is configured to intercept the first video data frame sent by the video player to the virtual display driver. Obtaining a video data frame processing capability of the client before the first video data frame processing instruction; the video data frame processing capability of the client includes a video data frame decoding capability, to determine a location according to the video data frame decoding capability Whether the client supports video data frame decoding. Further, the obtaining unit 806 further determines, according to the video data frame processing capability of the client, that the client supports the video data frame decoding, and sends the second video data frame and the first video data to the client. Before the frame processing instruction, the video encoding format information of the video data frame supported by the client is obtained, to determine, according to the video encoding format information, whether the client supports the video encoding format of the first video data frame.

In the server 800 of the above embodiment, the intercepting unit is specifically a virtual display driver, and the virtual display driver intercepts the first video data frame and the first video processing instruction.

The embodiment of the present invention further provides a video redirection system 900, including a server 800 and a client 1000. The server 800 is configured to intercept the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver;

When the client 1000 supports video data frame decoding, the second video data frame and the first video data frame processing instruction are sent to the client 1000, so that the client 1000 processes according to the first video data frame. The instruction processes the second video data frame, where the first video data frame processing instruction includes a video data frame decoding instruction and a video a data frame post-processing instruction and a video data frame presentation instruction, where the second video data frame is the first video data frame or a video data frame obtained by converting the first video data frame by a video encoding format;

When the client 1000 does not support video data frame decoding, sending a second video data frame and a second video data frame processing instruction to the client 1000, so that the client 1000 is configured according to the second video data frame. The processing instruction is configured to process the second video data frame, where the second video data frame processing instruction includes a video data frame post-processing instruction and a video data frame presentation instruction; when the first video data frame is decoded The second video data frame is the first video data frame, the second video data frame processing instruction is the first video data frame processing instruction, and the first video data frame is In the case of an undecoded video data frame, the second video data frame is the decoded first video data frame, and the first video data frame processing instruction includes a video data frame decoding instruction, and the video data frame post processing The instructions and the video data frame present instructions.

The client 1000 is configured to process the video data frame according to the video data frame processing instruction sent by the server 800.

For a detailed description of the system 900 provided by the present embodiment, reference may be made to the description of the server 800 and the description of the embodiments as shown in FIGS. 6a, 6b, 6c, 6d, and 7. I will not repeat them here. The server 800 and the client 1000 provided in the embodiment of the present invention may be a general computer, a mobile terminal, a workstation, or a dedicated server, etc., which is not specifically limited by the present invention, and has a memory and at least one central processor, and the memory and the At least one central processing unit is coupled by a bus, the memory stores computer instructions, and the at least one central processing unit is configured to execute computer instructions in the memory, and also includes a common external interface and the like to implement some necessary components of the server function. The algorithm for the computer instructions stored in the memory may refer to the description of the flowchart shown in FIG. 7. For specific implementation, reference may be made to the specific embodiments shown in FIG. 6a, FIG. 6b, FIG. 6c, and FIG.

The virtual desktop-based remote desktop architecture provided by the embodiment of the present invention, those skilled in the art may appreciate that the present invention is not limited to the described virtual desktop architecture of the present embodiment, and the remote desktop invention based on other types of virtual desktops The same applies. In addition, the present invention is not only applicable to a virtual desktop-based remote desktop architecture, but also to other application scenarios that provide video data to remote clients.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods to implement the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.

A person skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

In the several embodiments provided herein, it should be understood that the disclosed systems, devices, and methods may be implemented in other ways. For example, the device embodiments described above are merely illustrative, for example, the division of the unit is merely a logical function. Dividing, there may be another way of dividing in actual implementation, for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be electrical, mechanical or otherwise. The components displayed as the unit may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

The functions may be stored in a computer readable storage medium if implemented in the form of a software functional unit and sold or used as a standalone product. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product, which is stored in a storage medium, including The instructions are used to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: NAS (Network Attached Storage), U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or A variety of media such as optical discs that can store program code.

The above is only a specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily think of changes or substitutions within the technical scope of the present invention. It should be covered by the scope of the present invention. Therefore, the scope of the invention should be determined by the scope of the claims.

Claims

claims

1. A video redirection method, characterized by including:

Intercept the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver;

When the client supports video data frame decoding, the second video data frame and the first video data frame processing instruction are sent to the client, so that the client performs the decoding of the video data frame according to the first video data frame processing instruction. The second video data frame is processed. The first video data frame processing instructions include video data frame decoding instructions, video data frame post-processing instructions and video data frame rendering instructions. The second video data frame is the first video data frame. The video data frame may be a video data frame obtained by converting the video encoding format of the first video data frame;

When the client does not support video data frame decoding, send a second video data frame and a second video data frame processing instruction to the client, so that the client can process the video data frame according to the second video data frame processing instruction. The second video data frame is processed, wherein the second video data frame processing instructions include video data frame post-processing instructions and video data frame presentation instructions; when the first video data frame is a decoded video data frame When, the second video data frame is the first video data frame, and the second video data frame processing instruction is the first video data frame processing instruction; when the first video data frame is undecoded When the video data frame is a video data frame, the second video data frame is the decoded first video data frame, and the first video data frame processing instructions include a video data frame decoding instruction, the video data frame post-processing instruction and the Described video data frame rendering instructions.

2. The method of claim 1, further comprising: when the client supports video data frame decoding, and the client does not support the video data sent by the video player to the virtual display driver. When the video encoding format of the first video data frame is determined, the first video data frame is converted into a video encoding format to obtain the second video data frame; wherein, the video encoding format of the second video data frame is The video encoding format supported by the client.

3. The method of claim 2, wherein: converting the first video data frame into a video encoding format to obtain the second video data frame, Specifically, it includes: the virtual display driver converts the video encoding format of the first video data frame to obtain the second video data frame.

4. The method of claim 1, further comprising: when the client does not support video data frame decoding, and when the first video data frame is an undecoded video data frame, Before the client sends the second video data frame and the second video data frame processing instruction, the first video data frame is decoded according to the video data frame decoding instruction in the first video data frame processing instruction, To obtain the second video data frame.

5. The method of claim 1 or 4, further comprising: when the client does not support video data frame decoding, sending the second video data frame and the third video data frame to the client. Before the second video data frame processing instruction, compress the second video data frame;

Then the second video data frame is sent to the client, specifically the compressed second video data frame is sent to the client.

6. The method according to any one of claims 1 to 5, further comprising: before intercepting the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver, obtaining The video data frame processing capability of the client; The video data frame processing capability of the client includes a video data frame decoding capability, so as to determine whether the client supports video data frame decoding based on the video data frame decoding capability.

7. The method of claim 2, 3 or 6, further comprising: when the client supports video data frame decoding, sending a second video data frame and the first video data frame to the client. Before the video data frame processing instruction, obtain the video encoding format information of the video data frame supported by the client, so as to determine whether the client supports the video encoding format of the first video data frame based on the video encoding format information. .

8. The method according to any one of claims 1 to 7, characterized in that: the interception of the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver, specifically the virtual display driver The display driver intercepts the first video data frame and the first video processing instruction.

9. A server, characterized by including:

An interception unit, configured to intercept the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver;

A sending unit, configured to send a second video data frame and a first video data frame processing instruction to the client when the client supports video data frame decoding, so that the client can process the second video data frame according to the first video data frame. The frame processing instructions process the second video data frame. The first video data frame processing instructions include video data frame decoding instructions, video data frame post-processing instructions and video data frame rendering instructions. The second video data frame It is the first video data frame or a video data frame obtained by converting the first video data frame through video encoding format conversion;

When the client does not support video data frame decoding, send a second video data frame and a second video data frame processing instruction to the client, so that the client can process the video data frame according to the second video data frame processing instruction. The second video data frame is processed, wherein the second video data frame processing instructions include video data frame post-processing instructions and video data frame presentation instructions; when the first video data frame is a decoded video data frame When, the second video data frame is the first video data frame, and the second video data frame processing instruction is the first video data frame processing instruction; when the first video data frame is undecoded When the video data frame is a video data frame, the second video data frame is a decoded first video data frame, and the first video data frame processing instructions include a video data frame decoding instruction, a video data frame post-processing instruction and a video data frame processing instruction. Data frame rendering instructions.

10. The server according to claim 9, further comprising a conversion unit, the conversion unit is used when the client supports video data frame decoding, and the client does not support the video player. When sending the video encoding format of the first video data frame to the virtual display driver, the first video data frame is converted into a video encoding format to obtain the second video data frame; wherein, the second The video encoding format of the video data frame is a video encoding format supported by the client.

11. The server according to claim 10, wherein the conversion unit is specifically the virtual display driver, and the first video data frame is Video coding format conversion to obtain the second video data frame specifically includes: the virtual display driver performs video coding format conversion on the first video data frame to obtain the second video data frame.

12. The server according to claim 9, further comprising a decoding unit for when the client does not support video data frame decoding, and when the first video data frame is undecoded video data frame, before sending the second video data frame and the second video data frame processing instruction to the client, the first video data is processed according to the video data frame decoding instruction in the first video data frame processing instruction. The frame is decoded to obtain the second video data frame.

13. The server according to claim 9 or 12, further comprising a compression unit configured to send the first video data frame to the client when the client does not support video data frame decoding. Before the second video data frame and the second video data frame processing instruction, compress the second video data frame; then send the second video data frame to the client, specifically sending the compressed second video data frame to the client. Two video data frames.

14. The server according to any one of claims 9 to 13, further comprising an acquisition unit, the acquisition unit being configured to intercept the information sent by the video player to the virtual display driver. Before the first video data frame and the first video data frame processing instruction, the video data frame processing capability of the client is obtained; the video data frame processing capability of the client includes a video data frame decoding capability, so as to obtain the video data frame processing capability according to the video data frame. Frame decoding capability determines whether the client supports video data frame decoding.

15. The server according to claim 10, 11 or 14, wherein the acquisition unit is further configured to determine that the client supports video data frame decoding based on the video data frame processing capability of the client. Before the client sends the second video data frame and the first video data frame processing instruction, obtain the video encoding format information of the video data frame supported by the client, so as to determine the video encoding format information based on the video encoding format information. Whether the client supports the video encoding format of the first video data frame.

16. The server according to any one of claims 9 to 15, characterized in that: The interception unit is specifically a virtual display driver, and the virtual display driver intercepts the first video data frame and the first video processing instruction.

17. A server, characterized in that it includes a memory and a central processor, the memory and the central processor are connected through a system bus, the memory stores computer executable instructions, and when the server is running, the central processor The processor executes computer-executable instructions stored in the memory, causing the server to perform the following steps:

18. The server of claim 17, further comprising: when the client supports video data frame decoding, and the client does not support the video data sent by the video player to the virtual display driver. The view of the first video data frame When the video encoding format is selected, the first video data frame is converted into a video encoding format to obtain the second video data frame; wherein, the video encoding format of the second video data frame is a video encoding supported by the client. Format.

19. The server according to claim 18, wherein said converting the first video data frame into a video encoding format to obtain the second video data frame specifically includes: the virtual display driver converts the first video data frame into a video encoding format. The first video data frame is subjected to video encoding format conversion to obtain the second video data frame.

20. The server of claim 17, further comprising: when the client does not support video data frame decoding, and when the first video data frame is an undecoded video data frame, Before the client sends the second video data frame and the second video data frame processing instruction, the first video data frame is decoded according to the video data frame decoding instruction in the first video data frame processing instruction, so as to Obtain the second video data frame.

21. The server according to claim 17 or 20, further comprising: when the client does not support video data frame decoding, sending the second video data frame and the third video data frame to the client. Before the second video data frame processing instruction, compress the second video data frame; then send the second video data frame to the client, specifically sending the compressed second video data frame to the client.

22. The server according to any one of claims 17 to 21, further comprising: before intercepting the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver, obtaining The video data frame processing capability of the client; The video data frame processing capability of the client includes a video data frame decoding capability, so as to determine whether the client supports video data frame decoding based on the video data frame decoding capability.

23. The method of claim 18, 19 or 22, further comprising: based on the video data frame processing capability of the client, when it is determined that the client supports video data frame decoding, reporting to the client Before sending the second video data frame and the first video data frame processing instruction, obtain the video encoding format information of the video data frame supported by the client, so as to determine whether the client supports the video encoding format information based on the video encoding format information. The video encoding format of the first video data frame.

24. The server according to any one of claims 17 to 23, characterized in that, the intercepted first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver, specifically the virtual display driver The display driver intercepts the first video data frame and the first video processing instruction.

25. A video redirection system, including a client and a server as claimed in any one of claims 9 to 16.

26. A video redirection system, including a client and a server as claimed in any one of claims 17 to 24.

27. A computer-readable storage medium, characterized in that when the computer executes the computer-readable storage medium, the computer performs the following steps: intercept the first video data frame and the second video data frame sent by the video player to the virtual display driver. A video data frame processing instruction;

When the client supports video data frame decoding, the second video data frame and the first video data frame processing instruction are sent to the client, so that the client performs the decoding of the video data frame according to the first video data frame processing instruction. The second video data frame is processed. The first video data frame processing instructions include video data frame decoding instructions, video data frame post-processing instructions and video data frame rendering instructions. The second video data frame is the first video data frame. The video data frame may be a video data frame obtained by converting the first video data frame through video encoding format conversion;

28. The computer-readable storage medium according to claim 27, further comprising: when the client supports video data frame decoding, and the client does not support the video player to send data to the virtual When the video encoding format of the first video data frame sent by the display driver is performed, the first video data frame is converted into a video encoding format to obtain the second video data frame; wherein, the second video data frame is The video encoding format is a video encoding format supported by the client.

29. The computer-readable storage medium of claim 28, wherein said converting the first video data frame into a video encoding format to obtain the second video data frame specifically includes: said virtual The display driver converts the video encoding format of the first video data frame to obtain the second video data frame.

30. The computer-readable storage medium of claim 27, further comprising: when the client does not support video data frame decoding, and when the first video data frame is an undecoded video data frame. When, before sending the second video data frame and the second video data frame processing instruction to the client, the first video data frame is processed according to the video data frame decoding instruction in the first video data frame processing instruction. Decoding is performed to obtain the second video data frame.

31. The computer-readable storage medium according to claim 27 or 30, further comprising: when the client does not support video data frame decoding, sending the second video data frame to the client and the second video data frame processing instruction, compress the second video data frame; then send the second video data frame to the client, specifically sending the compressed second video data frame to the client .

32. The method of claims 27 to 31, further comprising: before intercepting the first video data frame and the first video data frame processing instruction sent by the video player to the virtual display driver, obtaining the Video data frame processing capability of the client; The video data frame processing capability of the client includes video data frame decoding capability, so as to determine whether the client supports video data frame decoding based on the video data frame decoding capability.

33. The method of claim 28, 29 or 32, further comprising: based on the video data frame processing capability of the client, when it is determined that the client supports video data frame decoding, reporting to the client Send the second video data frame and all Before processing the first video data frame instruction, obtain the video encoding format information of the video data frame supported by the client, so as to determine whether the client supports the first video data frame based on the video encoding format information. Video encoding format.

34. The computer-readable storage medium according to any one of claims 27 to 33, wherein the intercepted first video data frame and first video data frame processing instruction sent by the video player to the virtual display driver, specifically The first video data frame and the first video processing instruction are intercepted for the virtual display driver.