CN108804068B

CN108804068B - Client of audio device redirection system and related audio device redirection method

Info

Publication number: CN108804068B
Application number: CN201710311244.5A
Authority: CN
Inventors: 邱自强; 江晋毅
Original assignee: Elite Silicon Tech Inc
Current assignee: Elite Silicon Tech Inc
Priority date: 2017-05-05
Filing date: 2017-05-05
Publication date: 2021-01-12
Anticipated expiration: 2037-05-05
Also published as: CN108804068A

Abstract

The invention discloses a client of an audio device redirection system, which comprises a storage device and a processor, wherein the storage device stores a program code; when the processor loads and executes the program code, the program code instructs the processor to virtualize an audio device of the client as a virtual universal serial bus audio device on the client; and redirecting the virtual USB audio device to a server of the audio device redirection system and performing a virtual desktop infrastructure service via a network interface according to a USB redirection protocol.

Description

Client of audio device redirection system and related audio device redirection method

Technical Field

The invention relates to an audio device redirection system and a related method.

Background

VDI (virtual desktop infrastructure) is a desktop virtual means that executes and manages a desktop operating system in a data center (server side), and desktop images are transmitted to a terminal device (client side) through a network, so that the operating system and application programs are executed on the local device, but a user actually uses the operating system and application programs installed on the server side on the terminal device, wherein the terminal device (client side) can be a traditional personal computer or a mobile device. Because the operating system and applications are not executing on the client, the actual computational load on the client is greatly reduced, and the lifetime of the outdated devices can be extended by setting them as VDI clients. In addition, because all data is stored in the server side rather than the client side, the VDI has better data security protection.

Universal Serial Bus (USB) redirection is a technique that allows a user to access a client device through a server after inserting an external device into a USB port of the device, and involves transferring the function of the USB device from the client to the server through a network, and organizations that support USB redirection in a virtual desktop use servers that are VDI software compatible with remote desktop protocols. Generally, the USB devices that need to be redirected include flash memory or other storage devices, smart card devices, printers, display devices, etc., however, for some non-USB devices such as audio devices, it will consume a lot of computing power if the virtual audio device is virtualized on the server side when redirected to the server side, especially when a large number of clients are linked with the server side to perform VDI functions.

Disclosure of Invention

It is therefore an objective of the claimed invention to provide a client in an audio device redirection system and a method for audio device redirection.

According to one embodiment of the present invention, a client in an audio device redirection system is disclosed, wherein the client comprises a storage device and a processor, the storage device is used for storing a program code, when the processor loads and executes the program code, the program code instructs the processor to perform the steps of virtualizing an audio device of the client as a virtual USB audio device on the client; and redirecting the virtual USB audio device to a server of the audio device redirection system and performing a VDI service via a network interface according to a USB redirection protocol.

According to an embodiment of the present invention, a method for redirecting an audio device applied to a server end is disclosed, comprising virtualizing an audio device of a client end on the client end as a virtual USB audio device; and redirecting the virtual USB audio device to a server of the audio device redirection system and performing a VDI service via a network interface according to a USB redirection protocol.

Drawings

Fig. 1 is a schematic diagram of an audio device redirection system according to an embodiment of the present invention.

FIG. 2 is a diagram of an original operating system and a VDI guest operating system in a server according to an embodiment of the present invention.

FIG. 3 is a diagram of an electronic device according to an embodiment of the invention.

Detailed Description

Certain terms are used throughout the description and following claims to refer to particular components. As one of ordinary skill in the art will appreciate, manufacturers may refer to a component by different names. In the present specification and the claims that follow, the terms used herein do not denote any order or importance, but rather the terms used herein denote any order or importance, such as a difference in functionality. In the following description and in the claims, the terms "include" and "comprise" are used in an open-ended fashion, and thus should be interpreted to mean "include, but not limited to. Furthermore, the term "coupled" is used herein to encompass any direct and indirect electrical connection, such that if a first device is coupled to a second device, that connection may be through a direct electrical connection, or through an indirect electrical connection via other devices and connections.

Figure 1 is a schematic diagram of an audio device redirection system 10 according to one embodiment of the present invention, the audio device redirection system 10 comprises a client 100 and a server 200, wherein the client 100 and the server are connected through a network interface 300 to execute a Virtual Desktop Infrastructure (VDI) service, as shown in fig. 1, the server includes an audio device 110, an audio device driver 120, a guest Operating System (OS) sound system 130, a guest audio application 140, a virtual USB audio device module 150, and a USB redirection module 160, wherein the audio device 110 includes a microphone and a speaker (not shown in figure 1) for transmitting/receiving audio, and the audio device 110 is a system-on-chip (SOC) device installed in the client 100; the audio device driver 120 is a software driver for driving the audio device 110; the guest OS sound system 130 is a sound system of the operating system of the client 100. It should be noted that the client 100 may be a computer, a set-top box (set-top box) or a smart phone, and the operating system of the client 100 may be any type of operating system, but the invention is not limited to the type of the operating system, in this embodiment, the operating system of the client 100 is an Android system, and therefore the client OS audio system 103 is an audio system under the Android system. The client audio application 140 is an application running in the operating system of the client 100, wherein the client audio application 140 can be any application used in the audio device 110, for example, the client audio application 140 is an instant messaging application or a multimedia player; the virtual USB audio device module 150 is used to virtualize the audio device 110 of the client 100 into a virtual USB audio device on the client 100; the USB redirect module 160 redirects the virtual USB audio device to the server 200 of the audio device redirection system 10 and performs the VDI service via the network interface 300 according to a USB redirect protocol.

It should be noted that the audio device driver 120, the guest OS sound system 130, the guest audio application 140, the virtual USB audio device module 150, and the USB redirection module 160 are implemented by software in this embodiment, but this is not a limitation of the present invention, and in other embodiments, the modules may also be implemented by hardware or firmware according to the consideration of the designer, and when implemented by software, the client 100 further includes a storage device (not shown in fig. 1), wherein a program code in the storage device is used to implement the modules.

As shown in FIG. 1, the server 200 comprises a server audio application 210, a server OS sound effect system 220, a USB audio driver 230, a USB host core driver 240 and a USB redirection module 250, wherein the operating system of the server 200 may be any type of operating system, in this embodiment, the operating system of the server 200 is a Linux system, and thus the server OS sound effect system 220 is a sound effect system of the Linux system. The server audio application 210 is an application running in the operating system of the server 200, wherein the server audio application 210 can be any application using the audio device 101, for example, the server audio application 210 is an instant messaging application or a multimedia player. The USB audio driver 230 is a software driver for driving the virtual USB audio device after the audio device 110 is virtualized into the virtual USB audio device by the virtual USB audio device module 150 and redirected to the server 200 by the USB redirection module 160 through the network interface 300, and for inputting/outputting the audio file according to the USB audio file format. In this embodiment, the USB audio driver 230 may be a standard USB device driver for driving the virtual USB audio apparatus. The USB host core driver 240 is used to perform device initialization, i.e., to handle the plug-in and configuration process of the virtual USB audio device after the audio device 110 is virtualized as the virtual USB audio device by the virtual USB audio device module 150 and redirected to the server 200 by the USB redirection module 160 through the network interface 300, and the USB host core driver 240 is further used to manage the USB Request Block (URB) sent by the USB audio driver 230. The USB redirect module 250 includes a virtual USB host controller driver 251 and a network stack 252, wherein the virtual USB host controller driver 251 is used to process the URB managed by the USB host core driver 240 and issue a USB I/O request as if it were an actual USB host controller, and the network stack 252 is similar/identical to the network stack of the prior art, and the detailed description thereof is omitted for brevity. The USB redirection module 250 of the server 200 and the USB redirection module 160 of the client 100 communicate with each other according to the USB redirection protocol. In this embodiment, the USB redirection Protocol is a USB Network transfer Protocol (USB Network Transmit Protocol) proposed by the limited company of the crystal surge technologies, it should be noted that the present invention is not limited to the type of the USB redirection Protocol.

As shown in fig. 1, the USB redirection module 160 comprises a virtual USB device controller driver 161 and a network stack 162, wherein the virtual USB device controller driver 161 is configured to respond to USB input/output requests sent by the virtual USB host controller driver 251 for the virtual USB audio device, and the network stack 162 is the same as the network stack 252 of the server 200, in order to complete data exchange between the device and the server 200. The virtual USB audio device module 150 comprises a USB device core driver 151, a USB audio function driver 152, and an audio interface bridge module 153, wherein the USB device core driver 151 is configured to respond to the USB host core driver 240 to complete initialization of the virtual USB audio device, the USB audio function driver 152 receives audio output (speaker output) data from the USB audio stage driver 230 and transmits audio input (microphone input) data to the server 200 according to a USB audio data format, and the audio interface bridge module 153 is configured to receive the audio output (speaker output) data from the USB audio function driver 152 and transmit the audio output (speaker output) data to the guest OS sound system 130, and further configured to retrieve the audio input (microphone input) data from the audio device 110 and transmit the audio input (microphone input) data to the USB audio function driver 152. The audio interface bridge module 153 transmits the audio data to and retrieves the audio data from the guest OS sound effects system 130 through an Application Programming Interface (API). Note that in this embodiment, the USB device core driver 151, the USB audio function driver 152, and the audio interface bridge module 153 are described as three different modules in the virtual USB audio device module 150, however, in other embodiments, the three modules may be combined into one or two different modules having the same functions as the three modules, or may be disassembled into more than three (e.g., four or five) different modules having the same functions as the three modules.

By virtualizing the audio device 110 as the virtual USB audio device on the client 100 and redirecting it to the server 200 to execute VDI services, the client 100 and the server 200 can simultaneously use the audio device 110 for audio output/input, the guest OS sound system 130 determines which audio data to dial based on the desktop currently being the client 100 or the server 200 displayed on the display device of the client 100, for example, assuming that the client 100 runs the guest audio application 140 under the android system and uses the VDI services controlled by the server 200, wherein the server 200 is executing the server audio application 210, and both the guest audio application 140 and the server audio application 210 require the audio device 110 for audio input/output, when the user selects to execute on the desktop of the android system, the guest OS sound system 130 determines to dial the audio data of the guest audio application 140 through the audio device 110 and redirects it to the server 200 to execute the VDI services Muting audio of the server audio application 210; when the user selects to run the VDI desktop, the guest OS sound effects system 130 determines to dial the audio data of the server audio application 210 through the audio device 110 and mutes the audio of the client audio application 140. For a USB audio device, when the user switches the operating system, for example, from the operating system of the server to the operating system of the client, the real USB audio device must go through the process of plugging/unplugging the USB device (which can be implemented by software), which will take several seconds to remove/load the USB device, when the operating system is switched, since the USB audio device is not loaded (or redirected) to the server 200, the server audio application 210 will be forced to be closed, and thus, when the operating system is switched back again, the server audio application 210 will no longer run, which may cause inconvenience to the user. However, the audio device 110 is a system-on-a-chip device, rather than a USB device, the client 100 does not have a USB host controller driver for a USB audio device installed, but has a virtualized USB audio device to accomplish the USB redirection, so that not only the USB plugging process is omitted, when the operating system is switched, the server audio application 210 will be muted instead of being turned off, therefore, after the os is switched back, the user can immediately use the server audio application 210, in other words, the user can switch between the client os and the server os at will without any inconvenience of audio input/output, and the guest OS sound effects system 130 can selectively dial audio data of one of the client audio application 140 and the server audio application 210 and mute the other of the client audio application 140 and the server audio application 210 depending on the system being used by the user. The load on the server 200 can be greatly reduced by virtualizing the audio device 110 as a virtual USB audio device on the client 100 and using the computing power of the client 100 to execute part of the application programs, especially when a large number of clients are connected to the server 200 at the same time to execute VDI services.

The modules in the server 200 shown in fig. 1 are not limited to be implemented in the original operating system of the server 200, but may also be implemented in a VDI guest operating system dedicated to perform VDI services in the server 200, wherein the VDI guest operating system is linked to the client 100 to enable the client to perform the VDI services. Fig. 2 is a schematic diagram of an original os 201 and a VDI guest os 202 in a server 200 according to an embodiment of the present invention, as shown in fig. 2, sub-diagram (a), all the modules (only USB redirection module 250 is shown for brevity) in the server 200 shown in fig. 1 are implemented in the original os 201 of the server 200, so that when multiple clients are connected to the server 200 to execute VDI service and each client redirects its USB device (or virtual USB device) to the server 200, the original os 201 of the server 200 may not be able to accurately recognize each USB device group and its corresponding client. Referring to the sub-diagram (B) of fig. 2, all the modules (only the USB redirection module 250 is depicted for brevity) in the server 200 shown in the embodiment of fig. 1 are implemented in the VDI guest operating system 201 of the server 200 corresponding to the server 100, where the VDI guest operating system 201 can be regarded as a virtual machine for performing VDI services for the client 100, and the USB device (or virtual USB device) of the client 100 is redirected to the VDI guest operating system 201 of the server 200 through the network interface 300 to simplify and avoid the above-mentioned problem of identifying the device and its home client.

Fig. 3 shows an electronic device 400 according to an embodiment of the invention, wherein the electronic device 400 can be implemented by the client 100 in fig. 1, the electronic device 400 includes a processor 401 and a storage device 402 containing a program code PROG, wherein the storage device 402 may further comprise the guest OS sound effects system 130 and the guest audio application 140 (not shown in FIG. 3) of FIG. 1, when implemented in software, the program code PROG of the storage device 402 may include the virtual USB audio device module 150 and the USB redirection module 160 of the client 100, and when the program code PROG is loaded and executed by the processor 401, the program code PROG instructs the processor 401 to execute the functions of all the modules shown in fig. 1, and the implementation process thereof can be easily understood by a person skilled in the art after reading the above paragraphs, and the detailed description is omitted here for brevity.

The above description is only a preferred embodiment of the present invention, and all equivalent changes and modifications made in accordance with the claims of the present invention should be covered by the present invention.

[ notation ] to show

10 Audio device redirection system

100 client

200 server side

110 audio device

120 audio device driver

130 guest OS sound effect system

140 client audio application

150 virtual USB Audio device Module

160. 250 USB redirecting module

151 USB device core driver

152 USB Audio function driver

153 audio interface bridge module

161 virtual USB device controller driver

162. 252 network stack

251 virtual USB host controller driver

240 USB host core driver

230 USB audio level driver

220 server OS sound effect system

210 server audio application

300 network interface

201 original operating system

202 VDI guest operating system

400 electronic device

401 processor

402 storage device

PROG program code

Claims

1. A client of an audio device redirection system, comprising:

a storage device for storing a program code; and

a processor, wherein when the program code is loaded into and executed by the processor, the program code instructs the processor to perform the steps of:

virtualizing an audio device of the client as a virtual Universal Serial Bus (USB) audio device on the client; and

redirecting the Virtual USB audio device to a server of the audio device redirection system, and executing a Virtual Desktop Infrastructure (VDI) service via a network interface according to a USB redirection protocol; wherein the storage device additionally stores a client Operating System (OS) sound effect system and a client audio application program, wherein after the virtual USB audio device is redirected to the server to execute the VDI service, when the client audio application program and a server audio application program of the server run simultaneously, the client OS sound effect system running on the processor selects to play or record audio data of one of the client audio application program and the server audio application program through the audio device, and mutes audio data of the other of the client audio application program and the server audio application program.

2. The client of claim 1, wherein the program code comprises:

a USB device core driver, which is used for responding to a USB host core driver of the server end to complete the initialization of the virtual USB audio device.

3. The client of claim 1, wherein the program code comprises:

the USB audio function driver is used for receiving audio output data from a USB audio level driver (class driver) of the server side according to a USB audio data format and transmitting audio input data to the server side.

4. The client of claim 3, wherein the program code further comprises:

and the audio interface bridge module is used for receiving the audio output data from the USB audio function driver, transmitting the audio output data to the guest OS sound effect system, capturing the audio input data from the guest OS sound effect system and transmitting the audio input data to the USB audio function driver.

5. The client of claim 4 wherein a USB host core driver on the server side is used to virtualize the virtual USB audio device and manage USB Request Blocks (URBs) sent by the USB audio level driver on the server side, and the USB audio level driver is used to input or output the audio data according to the USB audio data format.

6. The client of claim 1, wherein the program code comprises:

a client network stack for communicating with a server network stack of the server via the network interface according to the USB redirect protocol;

a virtual USB device controller driver for responding to a USB input request or a USB output request sent from a virtual USB host controller driver at the server side of the audio device redirection system.

7. The client of claim 6, wherein the server network stack and the virtual USB host controller driver are included in a server operating system of the server.

8. The client of claim 6, wherein the server network stack and the virtual USB host controller driver are included in a VDI guest operating system of the server.

9. An audio device redirection method applied to a client comprises the following steps:

redirecting the Virtual USB audio device to a server of the audio device redirection system, and executing a Virtual Desktop Infrastructure (VDI) service via a network interface according to a USB redirection protocol;

after redirecting the virtual USB audio device to the server side for performing the VDI service:

when a client audio application program of the client and a server audio application program of the server are simultaneously operated, an Operating System (OS) sound effect System is executed on the client to select to play or record audio data of one of the client audio application program and the server audio application program through the audio device, and mute audio data of the other of the client audio application program and the server audio application program.

10. The method of claim 9, further comprising:

executing a USB device core driver to respond to a USB host core driver of the server end to complete the initialization of the virtual USB audio device.

11. The method of claim 9, further comprising:

executing a USB audio function driver to receive an audio output data from a USB audio driver (class driver) of the server according to a USB audio data format and transmit an audio input data to the server.

12. The method of claim 11, further comprising:

and executing an audio interface bridge module to receive the audio output data from the USB audio function driver and transmit the audio output data to the guest OS sound effect system, and capturing the audio input data from the guest OS sound effect system and transmitting the audio input data to the USB audio function driver.

13. The method of claim 12 wherein a USB host core driver at the server side is used to virtualize the virtual USB audio device and manage USB Request Blocks (URBs) sent by the USB audio level driver at the server side, and the USB audio level driver is used to input or output the audio data according to the USB audio data format.

14. The method of claim 13, further comprising:

executing a client network stack to communicate with a server network stack of the server via the network interface according to the USB redirection protocol;

executing a virtual USB device controller driver in response to a USB input request or a USB output request sent from a virtual USB host controller driver on the server side of the audio device redirection system.

15. The method as claimed in claim 14, wherein the server network stack and the virtual USB host controller driver are included in a server operating system of the server.

16. The method as claimed in claim 14, wherein said server network stack and said virtual USB host controller driver are included in a VDI guest operating system on said server side.