WO2017167285A1

WO2017167285A1 - Remote-desktop video-encoding method and device

Info

Publication number: WO2017167285A1
Application number: PCT/CN2017/079043
Authority: WO
Inventors: 胡振; 崔振峰; 吴迪; 黄小敏; 张宁; 曹颖; 陆剑峰; 秦延涛
Original assignee: 中兴通讯股份有限公司
Priority date: 2016-03-31
Filing date: 2017-03-31
Publication date: 2017-10-05
Also published as: CN107295340A

Abstract

The embodiments of the present invention provide a remote-desktop video-encoding method and device; said method comprises: monitoring the window status on a remote desktop and determining an active window of a current frame; extracting from a pre-established reference-frame list a target reference frame matching the active window of said current frame; according to the extracted target reference frame, encoding the current frame. The remote-desktop video-encoding method and device provided by the embodiments of the present invention solves the problem of current encoders occupying a large amount of CPU resources.

Description

Remote desktop video coding method and device

Technical field

Embodiments of the present invention relate to the field of video coding technologies, and in particular, to a method and an apparatus for remote desktop video coding.

Background technique

With the development of mobile communication technology, people can access cross-platform applications and the entire customer desktop through devices connected to the network through the desktop cloud. In the desktop cloud, remote desktop is a core technology. Remote desktop refers to how to transfer the desktop in the cloud to the remote cloud terminal in real time with high image quality, small delay and low bandwidth. on.

The more commonly used remote desktop protocols are Microsoft's Remote Desktop Protocol (RDP), PCWare (PC-over-IP), and Citrix's independent computing architecture (ICA, Independent). Computing Architecture) and open source simple protocol for independent computing environment (SPICE, Simple Protocol for Independent Computing Environment), etc., RDP has certain restrictions, such as the client can only connect to see the desktop after the windows system is started, can not see Including the basic input and output system (BIOS, Basic Input Output System) self-test and other boot process, RDP virtual machine must also be configured with an IP address client to connect; PCoIP, CTRIX are commercial closed source protocols, the above three protocols The research system is difficult to integrate; the SPICE protocol occupies too much bandwidth, and the user experience is not good under low bandwidth conditions.

Based on the above defects, the H264 protocol is widely used at home and abroad as a video codec standard. H264 protocol is a video coding standard. It is a new generation digital video compression format after MPEG4 jointly proposed by the International Organization for Standardization (ISO) and the International Telecommunication Union (ITU). It has the advantages of high quality, strong fault tolerance and strong network adaptability. . However, when the H264 protocol performs real-time encoding on the desktop, the encoder occupies a high CPU, especially when the user performs a switching window operation during use, the CPU and bandwidth usage are relatively high.

Summary of the invention

The embodiment of the invention provides a method and a device for remote desktop video coding, which can save the CPU calculation amount of the encoder and save network bandwidth.

The embodiment of the invention provides a method for remote desktop video coding, the method comprising:

Monitoring the status of the window in the remote desktop to determine the active window of the current frame;

Extracting a target reference frame that matches an active window of the current frame from a pre-established reference frame list;

The current frame is encoded according to the extracted target reference frame.

An embodiment of the present invention provides an apparatus for remote desktop video coding, where the apparatus includes:

An active window determining unit configured to monitor a window state in the remote desktop to determine an active window of the current frame;

a target reference frame determining unit, configured to extract, from a pre-established reference frame list, a target reference frame that matches an active window of the current frame;

The coding unit is configured to encode the current frame according to the extracted target reference frame.

Compared with the related art, the method and device for remote desktop video coding provided by the embodiment of the present invention can be related to the reference frame in the preset reference frame list by using the window of the remote desktop, so that when the window switching occurs The reference frame matching the active window of the current frame can be selected, and the selected reference frame can be remotely transmitted, and the reference frame list can be an encoded reference frame, then when the window switching occurs, the frame to be encoded is The reference frames are similar, so reference frames that match the active window of the current frame can be used, which reduces the computation time of the encoder CPU while reducing the network bandwidth usage.

Other features and advantages of the invention will be set forth in the description which follows, The objectives and other advantages of the invention may be realized and obtained by means of the structure particularly pointed in the appended claims.

DRAWINGS

The drawings are used to provide a further understanding of the technical solutions of the present invention, and constitute a part of the specification, which together with the embodiments of the present invention are used to explain the technical solutions of the present invention, and do not constitute a limitation of the technical solutions of the present invention.

FIG. 1 is a flowchart of a method for remote desktop video encoding according to an embodiment of the present invention;

2 is a flowchart of a window monitoring step in a remote desktop video encoding method according to an embodiment of the present invention;

FIG. 3 is a flowchart of a working process of an encoder in a remote desktop video encoding method according to an embodiment of the present disclosure;

4 is a functional block diagram of a device for remote desktop video coding according to an embodiment of the present invention.

detailed description

The embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the embodiments in the present invention and the features in the embodiments may be arbitrarily combined with each other without conflict.

The steps illustrated in the flowchart of the figures may be executed in a computer system such as a set of computer executable instructions. Also, although logical sequences are shown in the flowcharts, in some cases the steps shown or described may be performed in a different order than the ones described herein.

FIG. 1 is a flowchart of a method for remote desktop video encoding according to an embodiment of the present invention. As shown in FIG. 1, the method includes the following steps:

Step 11: Monitor the status of the window in the remote desktop to determine the active window of the current frame.

In the embodiment of the present invention, in the remote desktop, when switching from one window to another, the state of the window also changes. For example, when the first window is used, the first window may be in an active state, and when switching from the first window to the second window, the first window is no longer active, and the second The window is active.

FIG. 2 is a schematic diagram of a window monitoring step in a method for remote desktop video encoding according to an embodiment of the present invention. In the embodiment of the present invention, a global hook may be injected into an operating system where the remote desktop is located, so that the state of the window on the desktop is monitored by the global hook.

In the operating system, hooks can use the Application Programming Interface (API) to intercept and process system messages. A hook is actually a block that processes a message and can be hooked into the system through a system call. Whenever a specific message is sent, the hook program can capture the message before reaching the destination window. That is to say, the hook function can first obtain control, and then the hook function can analyze the message or continue without processing. Passing the message also forces the end of the delivery of the message. For each type of hook, a hook chain is maintained by the system. The last installed hook is placed at the beginning of the chain, and the first installed hook is placed at the end of the chain, that is, the hook added later gains control.

In the embodiment of the present invention, to implement the injection of a hook in the system, the hook function can be installed by calling the API function SetWindowsHookEx in the Software Development Kit (SDK). The prototype of this function is HHOOK SetWindowsHookEx(int idHook, HOOKPROC lpfn, HINSTANCE hMod, DWORD dwThreadId), where the first parameter idHook in parentheses is the type of the hook function; the second parameter HOOKPROC lpfn is the address of the hook function; the third parameter HINSTANCE hMod is the module containing the hook function Handle; the fourth parameter DWORD dwThreadId specifies the thread to monitor. If a specified thread is specified, it is a thread-specific hook; if it is specified to be empty, it is a global hook. Therefore, in the embodiment of the present invention, the fourth parameter can be designated as empty, so that the global hook can be injected into the system. The function of the global hook can be included in the Dynamic Link Library (DLL), and the thread-specific hook can also be included in the executable file. After the hook function that obtains control completes the processing of the message, if it wants the message to continue to be delivered, it can call the API function CallNextHookEx in another SDK to deliver the message. The hook function can also discard the message by directly returning TRUE and block the delivery of the message.

In the embodiment of the present invention, the global hook may be injected into the operating system where the desktop is located, and the window event in the desktop may be intercepted by the global hook; then the intercepted window event may be analyzed, so that the The active window of the current frame in the desktop; after determining the active window of the current frame, the active window of the current frame can be reported to the encoder for subsequent processing.

Step 12: Extract a target reference frame that matches the active window of the current frame from a pre-established reference frame list.

In the embodiment of the present invention, when the window is switched, in order to prevent the encoder from calculating the compression of the window in real time, the calculation pressure of the CPU is too large, and when the window is switched, the reference frame corresponding to the switched window may be searched for and referenced. The frame is encoded, so when the window switching occurs, since the frame to be encoded is similar to the reference frame, the reference frame matching the active window of the current frame can be used, thereby reducing the calculation time of the encoder CPU and reducing the time. Network bandwidth usage. In a specific embodiment, the current frame may be H264, and the reference frame in the reference frame list may be an H264 reference frame.

In the embodiment of the present invention, after obtaining the active window of the current frame, the encoder may extract a target reference frame that matches the active window of the current frame from the pre-established reference frame list. The reference frame list may include reference frames of a plurality of windows. In the embodiment of the present invention, each reference frame in the reference frame list has a corresponding identifier, and the identifier may be a digital code or a letter. coding. The windows in the desktop may also correspond to respective identifiers.

In the embodiment of the present invention, after the encoder obtains the active window of the current frame, the identifier corresponding to the active window of the current frame may be queried. At this time, the encoder may determine the same target identifier as the identifier of the active window of the current frame in the pre-established reference frame list. For example, if the identifier of the active window of the current frame is A, then the determined target identifier may be A. In this way, the reference frame corresponding to the target identifier can be determined as a target reference frame that matches the active window of the current frame. That is, the reference frame identified as A in the reference frame list may be determined as the target reference frame that matches the active window of the current frame. It can be seen from the above that by using the matching relationship between the identifiers, the reference frame matching the active window of the current frame can be quickly obtained.

In an actual application scenario, the target identifier that is the same as the identifier of the active window of the current frame may not exist in the pre-established reference frame list. In this case, the reference frame that matches the active window of the current frame cannot be obtained from the identified match. In the embodiment of the present invention, the reference frame corresponding to the previous frame may be determined to be the same as the active window of the current frame. The target reference frame is assigned. That is, it is assumed that the identifier of the reference frame corresponding to the previous frame is B, and the identifier of the active window of the current frame is C. After querying, it is found that there is no reference frame identified as C in the reference frame list, then this The reference frame corresponding to the previous frame (that is, the reference frame identified as B) may be determined as a reference frame that matches the active window of the current frame. Since the current frame is similar to the content of the previous frame, the same reference frame can be used to ensure the accuracy of the content.

Step 13: Encode the current frame according to the extracted target reference frame.

In the embodiment of the present invention, after determining the target reference frame that matches the active window of the current frame, the current frame is encoded according to the extracted target reference frame.

In addition, the reference frame list is updated with the encoded result. Specifically, in the embodiment of the present invention, the target reference frame may be deleted from the reference frame list, the encoded current frame is added to the reference frame list, and the encoded current frame is allocated. Reference frame identification.

It can be seen that the remote desktop video encoding method provided by the embodiment of the present invention can select and current when the window switching occurs by associating the window of the desktop with the reference frame in the preset H264 reference frame list. The reference frame matched by the active window of the frame may be an encoded reference frame, and when the window switching occurs, since the frame to be encoded is similar to the reference frame, the active window with the current frame may be used. The matching reference frames can reduce the computation time of the encoder CPU while reducing the network bandwidth usage.

FIG. 3 is a flowchart of a working process of an encoder in a remote desktop video encoding method according to an embodiment of the present invention. In a specific application scenario, after acquiring the current frame to be encoded, the encoder may search for a target reference frame that matches the current frame in the reference frame list. After the target frame is obtained by querying, the target frame may be encoded to obtain a coding result, and the reference frame list may be updated by using the coding result.

The present invention also provides an apparatus for remote desktop video encoding. As shown in FIG. 4, the apparatus includes:

The active window determining unit 100 is configured to monitor the status of the window in the desktop, and determine The active window of the current frame;

The target reference frame determining unit 200 is configured to extract, from a pre-established reference frame list, a target reference frame that matches an active window of the current frame;

The encoding unit 300 is configured to encode the current frame according to the extracted target reference frame, and update the encoded result to the reference frame list.

In a preferred embodiment of the present invention, the active window determining unit 100 includes:

a global hook injection module is configured to inject a global hook in an operating system where the remote desktop is located, and intercept the window event in the remote desktop by using the global hook;

The analysis module is configured to analyze the intercepted window event to determine an active window of the current frame in the remote desktop.

In a preferred embodiment of the present invention, the target reference frame determining unit 200 includes:

a finding module, configured to determine, in a pre-established reference frame list, a target identifier that is the same as an identifier of an active window of the current frame;

a determining module, configured to determine a reference frame corresponding to the target identifier as a target reference frame that matches an active window of the current frame; when there is no activity in the pre-established reference frame list When the window identifies the same target identifier, the reference frame corresponding to the previous frame of the current frame is determined as the target reference frame that matches the active window of the current frame.

In a preferred embodiment of the present invention, the apparatus further includes: an updating unit 400, configured to delete the target reference frame from the H264 reference frame list, and add the encoded current frame to the reference frame list. And assigning a reference frame identifier to the encoded current frame.

It should be noted that the specific implementation manners of the foregoing various functional modules are consistent with the descriptions in steps 11 to 13, and are not described herein again.

It can be seen that the apparatus for remote desktop video coding provided by the embodiment of the present invention can select the current frame when the window switching occurs by associating the window of the desktop with the reference frame in the preset reference frame list. The active frame matches the reference frame, and the reference frame list can be the encoded reference frame, then when the window switching occurs, due to the frame to be encoded and the reference The test frames are similar, so reference frames that match the active window of the current frame can be used, which can reduce the computation time of the encoder CPU while reducing the network bandwidth usage.

It will be apparent to those skilled in the art from the above description of the embodiments that the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the related art, may be embodied in the form of a software product, which may be stored in a storage medium such as a ROM/RAM, a disk, An optical disk or the like includes a number of instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform the methods described in various embodiments of the present invention or portions of the embodiments.

While the embodiments of the present invention have been described above, the described embodiments are merely for the purpose of understanding the invention and are not intended to limit the invention. Any modification and variation in the form and details of the embodiments may be made by those skilled in the art without departing from the spirit and scope of the invention. The scope defined by the appended claims shall prevail.

Industrial applicability

Embodiments of the present invention relate to the field of video coding technologies, and provide a method and apparatus for remote desktop video coding, where the method includes monitoring a window state in a remote desktop to determine an active window of a current frame; A target reference frame matching the active window of the current frame is extracted from the frame list; the current frame is encoded according to the extracted target reference frame. The invention can solve the problem that the current encoder occupies a high CPU, thereby reducing the calculation time of the encoder CPU and reducing the occupancy rate of the network bandwidth.

Claims

A method for remote desktop video encoding, comprising:

Monitoring the status of the window in the remote desktop to determine the active window of the current frame;

Extracting a target reference frame that matches an active window of the current frame from a pre-established reference frame list;

The current frame is encoded according to the extracted target reference frame.
The method of claim 1 wherein said monitoring a window state on a remote desktop to determine an active window of the current frame comprises:

Injecting a global hook in an operating system where the remote desktop is located, and intercepting a window event in the remote desktop by using a global hook;

The intercepted window event is analyzed to determine an active window of the current frame within the remote desktop.
The method according to claim 1, wherein the reference frames in the reference frame list respectively correspond to an identifier;

Extracting, from the pre-established reference frame list, a target reference frame that matches an active window of the current frame, including:

Determining, in the reference frame list, a target identifier that is the same as an identifier of an active window of the current frame;

The reference frame corresponding to the target identifier is determined as a target reference frame that matches an active window of the current frame.
The method of claim 3, wherein the method further comprises:

When the target identifier of the active frame of the current frame does not exist in the pre-established reference frame list, the reference frame corresponding to the previous frame of the current frame is determined to be active with the current frame. The target reference frame that the window matches.
The method of claim 1 wherein the method further comprises:

The encoded current frame is updated into the reference frame list.
The method of claim 5, wherein the updating the encoded current frame to the reference frame list comprises:

Deleting the target reference frame from the reference frame list, adding the encoded current frame to the reference frame list, and assigning a reference frame identifier to the encoded current frame.
A device for remote desktop video coding, comprising:

An active window determining unit configured to monitor a window state in the remote desktop to determine an active window of the current frame;

a target reference frame determining unit, configured to extract, from a pre-established reference frame list, a target reference frame that matches an active window of the current frame;

The coding unit is configured to encode the current frame according to the extracted target reference frame.
The apparatus according to claim 7, wherein the active window determining unit comprises:

a global hook injection module is configured to inject a global hook in an operating system where the remote desktop is located, and intercept a window event in the remote desktop by using a global hook;

The analysis module is configured to analyze the intercepted window event to determine an active window of the current frame in the remote desktop.
The apparatus according to claim 7, wherein the reference frames in the reference frame list respectively correspond to an identifier;

The target reference frame determining unit includes:

a finding module, configured to search for a target identifier that is the same as an identifier of an active window of the current frame in a pre-established reference frame list;

a determining module, configured to determine a reference frame corresponding to the target identifier as a target reference frame that matches an active window of the current frame; when there is no activity in the pre-established reference frame list When the window identifies the same target identifier, the reference frame corresponding to the previous frame of the current frame is determined as the target reference frame that matches the active window of the current frame.
The apparatus of claim 7 wherein said apparatus further comprises:

And an updating unit, configured to delete the target reference frame from the reference frame list, add the encoded current frame to the reference frame list, and allocate a reference frame identifier to the encoded current frame.
A storage medium, characterized in that the storage medium comprises a stored program, wherein the program is executed to perform the method of any one of claims 1 to 6.