WO2010087828A1

WO2010087828A1 - Communicating data in flows between first and second computers connected over a network

Info

Publication number: WO2010087828A1
Application number: PCT/US2009/032375
Authority: WO
Inventors: James M. King
Original assignee: Hewlett Packard Development Co LP
Current assignee: Hewlett Packard Development Co LP
Priority date: 2009-01-29
Filing date: 2009-01-29
Publication date: 2010-08-05
Anticipated expiration: 2011-07-29
Also published as: US20150358249A1; US9306995B2; GB2479309B; GB2479309A; US9391918B2; CN102301361B; DE112009004319T5; GB201112679D0; CN102301361A; US20110276715A1; BRPI0919984A2

Abstract

A network arrangement includes a first computer connected over a network to a second computer. Data over plural flows is communicated over the network between the first computer and second computer, where the second computer has a resource remotely accessible by the first computer over the data network, and where the second computer has a device driver to receive user input at a user input device attached to the first computer. According to different priorities assigned to the corresponding plural flows, at least a first one of the plural flows of data is caused to be throttled such that at least a second one of the plural flows is provided a greater portion of a bandwidth of the network, where the second flow is used for communication of data related to remote access of the resource of the second computer by the first computer.

Description

Communicating Data In Flows Between First And Second Computers Connected Over A Network

Background

[0001] Many enterprises are transitioning to a network arrangement in which computing resources of central servers are provided to local computers at which users are located. The computing resources (e.g., software applications, processing resources, storage resources, etc.) that are centralized at one or more central servers can be selectively allocated to a session established by a user at a local computer.

[0002] Protocols are provided to enable a user at a local computer to access and share the desktop of a remote computer (e.g. , a central server) over a computer network. One such protocol is the Remote Desktop Protocol (RDP), as provided by Microsoft Corporation, to provide remote display and input capabilities over network connections. Another protocol that can be used is the Remote Graphics Software (RGS) protocol from the Hewlett Packard Co. RGS is designed to take full advantage of the computer and graphics resources of a remote computer to deliver interactive remote access at the local computer. The desktop video data of the remote computer is transmitted over the network to the local computer, which displays the desktop video data locally in a window at the local computer. RGS is designed to provide fast capture, compression, and transmission of a desktop video data over a network. RGS also allows audio data to be sent from the remote computer to the local computer for output on an audio device of the local computer. RGS also captures user keyboard and mouse inputs at the local computer, and sends the keyboard and mouse inputs to the remote computer for processing by the operating system of the remote computer, and by applications running on the remote computer. RGS also allows data of other peripheral devices (e.g., such as storage devices) to be communicated from the local computer to the remote computer.

[0003] Certain peripheral devices such as storage devices (e.g., hard disk drives, optical disk drives, etc.) attached to the local computer are considered "bulk devices" since these devices can potentially transfer a relatively large volume of data ('bulk data") in a single transfer operation. [0004] Transferring bulk data over the network from a local computer to a remote computer consumes a relatively large portion of the available bandwidth of the network between the local computer and remote computer. As a result, when a bulk data transfer operation is occurring, the amount of remaining network bandwidth available to sending video and/or audio data from the remote computer to the local computer is reduced, which may adversely affect the user experience in viewing video data or listening to audio data at the local computer.

Brief Description Of The Drawings

[0005] Some embodiments of the invention are described, by way of example, with respect to the following figures:

Fig. 1 is a block diagram of an exemplary arrangement that includes a local computer and remote computer, in which an embodiment of the invention can be incorporated;

Fig. 2 is a flow diagram of a process performed by a local computer of throttling a data flow over a network between the local computer and a remote computer, in accordance with an embodiment; and

Fig. 3 is a flow diagram of a process performed by the remote computer of throttling a data flow over a network between the local computer and the remote computer, in accordance with another embodiment.

Detailed Description

[0006] Fig. 1 illustrates an arrangement in which a local computer 100 (at which a user is located) is connected to a remote computer 102 over a data network 104. Although just one local computer 100 and one remote computer 102 is depicted in Fig. 1, it is noted that there can be multiple local computers 100 and/or multiple remote computers 102.

[0007] The local computer 100 uses the resources of the remote computer 102 in sessions established between the local computer 100 and the remote computer 102. For example, the local computer 100 can use the graphics resources of the remote computer 102, in which the remote computer 102 delivers desktop video data of the remote computer over the network 104 to the local computer 100 for display in a display device 106 of the local computer 100. Also, the local computer can use audio resources of the remote computer 102, in which the remote computer 102 delivers audio data of the remote computer over the network 104 to the local computer 100 for output by an audio output device 107 (e.g., headphone, speakers, etc.) of the local computer 100.

[0008] Also, one or more user interface devices 108 are attached to the local computer 100. Changes in the state of the user interface device 108 are communicated from the local computer 100 over the data network 104 to the remote computer 102. The user interface device 108 can be a human interface device (HID) according to the HID standard, as described in Universal Serial Bus (USB), Device Class Definition For Human Interface Devices (HID), Firmware Specification, Version 1.11, dated June 27, 2001. The HID device 108 can be a mouse, a keyboard, a roller ball-type input device, a tablet, and so forth.

[0009] In addition one or more bulk devices 109 are attached to the local computer 100. A "bulk device" is a device that can potentially transfer a relatively large volume of data ('bulk data") in a single transfer operation from the local computer 100 to the remote computer 102 over the network 104. Examples of bulk devices include storage devices such as a hard disk drive, an optical disk drive, or any other type of storage device. A bulk device can also include an internal memory 139 of the local computer 100. An example scenario in which bulk data may be transferred from the internal memory 139 of the local computer 100 to the remote computer 102 is in the context of a cut-and-paste or copy-and-paste operation, in which a user may have selected a relatively large amount of data at the local computer to save into another application or document — the cut or copied data is transferred from the internal memory 139 over the network 104 to the remote computer 102.

[0010] Transferring bulk data over the network 104 from the local computer 100 to the remote computer 102 consumes a relatively large portion of the available bandwidth of the network 104. As a result, when a bulk data transfer operation is occurring, the amount of remaining network bandwidth available to sending video and/or audio data from the remote computer 102 to the local computer 100 is reduced, which may adversely affect the user experience in viewing video data or listening to audio data at the local computer 100.

[0011] In accordance with some embodiments, a mechanism is provided to assign different priorities to different flows of data between the local computer 100 and remote computer 102. To improve user performance at the local computer 100 in obtaining data (e.g., viewing video data and/or listening to audio data) from the remote computer 102, the priority(ies) assigned to flows of video data and/or audio data from the remote computer 102 to the local computer 100 is (are) set higher than the priority assigned to the transfer of any bulk data from the bulk device 109 to the remote computer 102. A "flow" of data refers to a distinctly identifiable communication of data. The distinct flows of data can be flows associated with different applications (e.g. , one flow to communicate video data for displaying video; another flow to communicate audio data for outputting audio; and a further flow to communicate bulk data; etc.).

[0012] In the example of Fig. 1, three flows of data are depicted: flow A (audio data from the remote computer 102 to the local computer 100); flow B (video data from the remote computer 102 to the local computer 100); and flow C (bulk data from the local computer 100 to the remote computer 102). There can be other flows between the local computer 100 and remote computer 102. Some or all of the flows can proceed concurrently.

[0013] In accordance with some embodiments, the priority assigned to flow C is lower than either of the priorities assigned to flows A and B, such that the audio and video data will be effectively allocated a greater portion of the bandwidth of the network 104 than the bulk data. In this way, if there is audio or video data to transmit, transmission of the bulk data in flow C is delayed until the audio or video data is sent in flow A or B. Allocation of priorities to the flows can be performed by a scheduler 117 in the remote computer 102.

[0014] The data network 104 can communicate data according to the Internet Protocol (IP). The HID device 108 and bulk device 109 can be attached to the local computer 100 over Universal Serial Bus (USB) link(s) 110 (wired or wireless USB link) to the local computer 100. The HID device 108 is connected to an HID controller 112, and the bulk device 109 is connected to a bulk device controller 113. In the above-described implementation, any data relating to the HID device 108 or bulk device 109 is in the form of USB data that is communicated in IP packets transferred over the data network 104 to the remote computer 102. Although reference is made to "USB" and "IP" in the embodiments described, it is noted that techniques according to some embodiments can be applicable to data packets according to other types of protocols. [0015] The local computer 100 is referred to as a "receiving system," and the remote computer 102 is referred to as a "sending system." As such, the local computer 100 includes receiver software 114, and the remote computer 102 includes sender software 116. The sender software 116 is used for sending desktop video data and audio data of the remote computer 102 (sending system) over the data network 104 to the receiver software 114 in the local computer 100 (receiving system), where the desktop video data is displayed at the display device 106, and the audio data is output by the audio output device 107. Note that the desktop video data and audio data sent by the sender software 116 are actual rendering video data and rendering audio data that can be rendered by a respective display device and audio output device. The rendering video data and rendering audio data are different from data contained in source video files (e.g., MPEG files) or source audio files that have to be converted to a format that can be rendered by respective output devices.

[0016] The sender software 116 in the remote computer 102 receives video data from a video subsystem 136 in the remote computer 102. The video data of the video subsystem 136 is displayable by a display device attached to the remote computer 102. The sender software 116 then applies compression to the video data that is sent to the receiver software 114, which can then perform decompression of the video data before displaying the video data at the display device 106.

[0017] Similarly, the sender software 116 receives audio data from an audio subsystem 137 in the remote compute 102. The audio data of the audio subsystem 137 can be output by an audio output device attached to the remote computer 102. The sender software 116 also applies compression to the audio data that is sent to the receiver software 114, which can then perform decompression of the audio data before outputting the audio data at the audio output device 107.

[0018] Note that an actual display device or audio output device does not have to be connected to the video subsystem 136 or audio subsystem 137 of the remote computer 102 in some implementations; however, in other implementations, a display device or audio output device can be connected to the remote computer 102.

[0019] In some embodiments, the sender software 116 and receiver software 114 are according to the Remote Graphics Software (RGS) protocol from the Hewlett-Packard Co. RGS is designed to take full advantage of computer and graphics resources of a remote computer to deliver interactive remote access from a local computer. In a different embodiment, the sender software 116 and receiver software 114 can operate according to the Remote Desktop Protocol (RDP) from Microsoft Corporation, to provide remote display and input capabilities over network connections. In further embodiments, the sender software 116 and receiver software 114 can be according to other technologies.

[0020] Corresponding device drivers 118 in the local computer 100 interact with the HID controller 112 and bulk device 113 to perform operations with respect to the HID device 108 and bulk device 109, respectively. For example, a corresponding device driver 118 can receive indications of change of the HID device 108 (such as when a user has moved a mouse, actuated a keyboard, moved a roller ball-type input device, moved an input device relative to a tablet, and so forth). In addition, another device driver 118 can receive bulk data from the bulk device 109 for transmission over the network 104.

[0021] The receiver software 114 sends respective HID data and bulk data over the network 104 to the remote computer 102. In accordance with some embodiments, the receiver software 114 includes a throttler 115 to throttle the transmission of bulk data from the bulk device 109 over the network 104 if the throttler 115 detects that there are active flows of data (e.g., flow A and/or flow B) from the remote computer 102. "Throttling" a flow of data refers to slowing the rate of transmission of data in the flow, delaying the transmission of data in the flow, and/or stopping the transmission of data in the flow.

[0022] The flows of data between the local computer 100 and remote computer 102 occur through network interfaces 120 and 122. Each of the network interfaces 120 and 122 includes a physical network interface controller as well as a protocol stack, including an IP protocol stack. The network interface 120 sends USB data (bulk data and/or HID data) in respective IP packets over the data network 104 to the remote computer 102. The IP packets are received by a network interface 122 in the remote computer 102, which extracts the USB bulk data and/or HID data from the IP packets.

[0023] Bulk data received at the sender software 116 is forwarded to a virtual interposer 124, which can cause the received bulk data to be stored in the persistent storage 150 in the remote computer 102. The virtual interposer 124 intercepts calls from device drivers 126 in the remote computer 102 that are intended for remotely located peripheral devices, such as the bulk device 109 and the HID device 108 attached to the local computer 100. The virtual interposer 124 prevents calls to the remotely located peripheral devices from reaching lower level (kernel) device drivers of the operating system in the remote computer 102. Respective device drivers 126 can create audio data and video data that are provided to the audio subsystem 137 and video subsystem 136, respectively, to be rendered by respective output devices, such as respective output devices connected to the remote computer 100 and the remote computer 102. Also, another device driver 126 can receive updates regarding actuation (e.g. , movement) of a user interface device such as the remotely located HID device 108.

[0024] The remote computer 102 also includes software applications 134. A corresponding software application 134 may have requested bulk data to be retrieved from the bulk device 109 attached to the local computer 100. Another software application 134 may be waiting for updated information from the HID device 108.

[0025] In accordance with some embodiments, the sender software 116 includes the scheduler 117 that assigns priorities to the flows for communicating data between the local computer 100 and remote computer 102. The scheduler 117 can communicate the assigned priorities to the throttler 115, such that the throttler 115 can determine whether communication of bulk data is to be throttled when the throttler 115 detects that there are other flows of data between the local computer 100 and remote computer 102.

[0026] The local computer 100 includes one or more central processing units (CPUs) 138, which is connected to memory 139. The software modules of the local computer 100, such as the receiver software 114 and device driver 118, are executable on the CPU(s) 138.

[0027] The remote computer 102 similarly includes one or more CPUs 140 connected to memory 132 and persistent storage 150. The software modules of the remote computer 102, such as the software application 134, device driver 126, virtual interposer 124, and sender software 116 are executable on the CPU(s) 140.

[0028] Fig. 2 illustrates a procedure performed by a receiving system (local computer 100). The receiver software 114 in the local computer 100 receives (at 202) the assigned priorities for corresponding flows of data between the local computer 100 and the remote computer 102. For example, the flows of data can include flows A, B, and C depicted in Fig. 1. The assignment of priorities for corresponding flows of data can be performed by the scheduler 117 in the remote computer 102. After priorities have been assigned, the sender software 116 sends information relating to the assigned priorities to the receiver software 114.

[0029] Next, the receiver software 114 identifies (at 204) data associated with different flows that are to be communicated. If there exists data to communicate for a higher priority one or more flows (such as flows A and B in Fig. 1), then the throttler 115 in the local computer 100 throttles (at 206) a lower priority flow, such as flow C in Fig. 1 for sending bulk data from the local computer 100 to the remote computer 102.

[0030] Fig. 3 illustrates a procedure according to an alternative embodiment that is performed at the sending system (remote computer 102). The procedure of Fig. 3 can be performed in place of the procedure of Fig. 2.

[0031] The scheduler 117 in the remote computer 102 assigns (at 302) priorities to respective flows of data between the remote computer 102 and the local computer 100. The sender software 116 next identifies (at 304) data associated with different flows that are to be communicated. Based on the identified data, the sender software 116 sends (at 306) indications to the receiver software 114 in the local computer 100 to inform the receiver software 114 which flows of data to throttle, such as the local priority bulk data flow C in Fig. 1.

[0032] Instructions of software described above (including the receiver software 114, throttler 115, device drivers 118, sender software 116, scheduler 117, virtual interposer 124, and device drivers 126 of Fig. 1) are loaded for execution on a processor (such as one or more CPUs 139, 140 in Fig. 1). The processor includes microprocessors, microcontrollers, processor modules or subsystems (including one or more microprocessors or microcontrollers), or other control or computing devices. A "processor" can refer to a single component or to plural components (e.g., one CPU or multiple CPUs).

[0033] Data and instructions (of the software) are stored in respective storage devices, which are implemented as one or more computer-readable or computer-usable storage media. The storage media include different forms of memory including semiconductor memory devices such as dynamic or static random access memories (DRAMs or SRAMs), erasable and programmable read-only memories (EPROMs), electrically erasable and programmable read-only memories (EEPROMs) and flash memories; magnetic disks such as fixed, floppy and removable disks; other magnetic media including tape; and optical media such as compact disks (CDs) or digital video disks (DVDs). Note that the instructions of the software discussed above can be provided on one computer-readable or computer-usable storage medium, or alternatively, can be provided on multiple computer-readable or computer-usable storage media distributed in a large system having possibly plural nodes. Such computer- readable or computer-usable storage medium or media is (are) considered to be part of an article (or article of manufacture). An article or article of manufacture can refer to any manufactured single component or multiple components.

[0034] In the foregoing description, numerous details are set forth to provide an understanding of the present invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these details. While the invention has been disclosed with respect to a limited number of embodiments, those skilled in the art will appreciate numerous modifications and variations therefrom. It is intended that the appended claims cover such modifications and variations as fall within the true spirit and scope of the invention.

Claims

What is claimed is:

1. A method for use in a network arrangement including a first computer connected over a network to a second computer, comprising: communicating data in plural flows over the network between the first computer and second computer, wherein the second computer has a resource remotely accessible by the first computer over the network, and wherein the second computer has a device driver to receive user input at a user input device attached to the first computer; and according to different priorities assigned to the corresponding plural flows, causing at least a first one of the plural flows of data to be throttled such that at least a second one of the plural flows is provided a greater portion of a bandwidth of the network, wherein the second flow is used for communication of data related to remote access of the resource of the second computer by the first computer.

2. The method of claim 1 , wherein the resource of the second computer remotely accessed is an audio resource or a video resource, the method further comprising: communicating audio data or video data in the second flow from the second computer to the first computer, wherein the audio data or video data is presented by an output device at the first computer.

3. The method of claim 2, further comprising: communicating data of a peripheral device attached to the first computer in the first flow from the first computer to the second computer.

4. The method of claim 3, wherein communicating the data of the peripheral device comprises communicating the data of a data storage device.

5. The method of claim 3, wherein communicating the data of the peripheral device comprises communicating Universal Serial Bus (USB) data of the peripheral device.

6. The method of claim 2, further comprising: communicating data associated with a cut-and-paste or copy-and-paste operation performed at the first computer in the first flow from the first computer to the second computer.

7. The method of claim 2, further comprising communicating data associated with the user interface device from the first computer to the device driver of the second computer.

8. The method of claim 7, wherein communicating the audio data or video data comprises communicating the audio data or video data of an audio subsystem or video subsystem of the second computer, wherein the audio data or video data is created by a device driver of the second computer.

9. The method of claim 1, wherein the communicating and causing steps are performed by the second computer, the method further comprising: a scheduler in the second computer assigning different priorities to the plural flows of data.

10. The method of claim 1, wherein the communicating and causing steps are performed by either the first or second computer.

11. A first computer comprising: a processor; sender software executable on the processor to send, over a network, at least one of audio data and video data to a second computer for output by an output device attached to the second computer, and to receive data relating to a user interface device attached to the second computer; and a scheduler executable on the processor to assign different priorities to flows of data between the first computer and the second computer, wherein the flows of data include at least one flow for communicating the at least one of audio data and video data, and a second flow for communicating data of a peripheral device attached to the second computer over the network to the first computer, wherein the sender software is to provide an indication to the second computer to cause the second computer to throttle communication of the data in the second flow.

12. The first computer of claim 11 , wherein the peripheral device is a bulk device for communicating bulk data.

13. The first computer of claim 11 , further comprising: a device driver; and a virtual interposer to intercept a call made by the device driver to access the peripheral device attached to the second computer.

14. The first computer of claim 11 , wherein the data of the peripheral device comprises Universal Serial Bus (USB) data.

15. An article comprising at least one computer-readable storage medium containing instructions that when executed cause a first computer to: send, over a network, at least one of audio data and video data to a second computer for output by an output device attached to the second computer; receive, at a device driver in the first computer, data relating to a user interface device attached to the second computer; assign different priorities to flows of data between the first computer and the second computer, wherein the flows of data include at least one flow for communicating the at least one of audio data and video data, and a second flow for communicating data of a peripheral device attached to the second computer over the network to the first computer; and provide an indication to the second computer to cause the second computer to throttle communication of the data in the second flow.