WO2017153891A1

WO2017153891A1 - Method and system for a computer to interface with wirelessly connected peripheral devices

Info

Publication number: WO2017153891A1
Application number: PCT/IB2017/051295
Authority: WO
Inventors: Ramesh Mantha; Simon Tian
Original assignee: Neptune Computer Inc.
Priority date: 2016-03-07
Filing date: 2017-03-06
Publication date: 2017-09-14

Abstract

A method is provided for operating a wireless interface between a wearable computing hub and one or more peripheral devices. The computing hub provides all primary computer processing and connectivity functions while the peripheral is a thin client device that functions primarily as an input/output device. The method of the invention establishes the wireless interface while (i) supporting different types of peripheral devices, (ii) minimizing computational requirements for the interface at both the computing hub and at each peripheral, (iii) minimizing power consumption of the interface at both the computing hub and the peripheral, and (iv) adapting to any noise or interference in the wireless link between the computing hub and the peripheral.

Description

METHOD AND SYSTEM FOR A COMPUTER TO INTERFACE WITH WIRELESSLY CONNECTED

PERIPHERAL DEVICES

FIELD OF THE INVENTION

[oooi] The present invention relates generally to a system and method for operating a central computing hub that is wirelessly connected to one or more computing peripheral devices. More particularly, the present invention relates to a method for establishing a wireless communications protocol between a central computing hub and one or more wirelessly connected peripherals that enables the peripherals to be used as fully functioning devices even though computing and external communications functions occur primarily at the computing hub. In this specification, the terms "dumb terminal" and "dumb peripheral" mean a peripheral that has minimal computing power and associated memory and storage, with the operating system (OS) and application layer software running primarily on the central computing hub. In this context, "fully functioning device" means a device with the

characteristics of modern mobile computing devices from an end user perspective (e.g. a smartphone, tablet, laptop, etc.). This includes elements such as a high resolution display, audio input/output capabilities, one or more cameras, motion sensors, input methods such as touch on the display, a touchpad, keyboard, etc. In addition,

BACKGROUND OF THE INVENTION

[0002] The concept of a central computer hub connected to a dumb terminal, also known as a "thin client" dates to the very early days of computing, where the central computing hub was a mainframe computer and the dumb terminals consisted of displays and keyboards that were connected to the hub with a wired connection. More recently, new computing platforms have emerged that are mobile (initially laptops and more recently smartphones). These newer mobile devices have been used in conjunction with wirelessly connected peripherals which function primarily as display devices having no interaction with an end user. Examples of wireless protocols that support this type of capability include WiFi Display (aka Miracast) and Apple AirPlay. These protocols support very high data rates and are suitable for sending video and audio signals from the computing hub to a peripheral display device but support minimal or no interactivity from the peripheral, such as keyboard and audio input. Alternately, there are protocols that are designed specifically to provide input from a peripheral device to a wirelessly connected computing hub. The most well known of these is Bluetooth, which supports a wide variety of peripheral input and output (I/O) devices, such as speakers, microphones, keyboards, video game controllers, etc. However, Bluetooth is very limited in bandwidth and does not support high data rates associated with I/O such as a video signal intended for a display or the output of a high-resolution camera such as normally found in modern mobile devices. More general purpose, high bandwidth protocols, such as WiFi Direct, do exist to enable direct connections between devices but these are not optimized to minimize power consumption and support full interactivity between a computing hub and peripheral device.

SUMMARY OF THE INVENTION

[0003] It is an object of the present invention to provide a novel system and method for a central computing hub to wirelessly interface to one or more peripherals in a manner that supports the full scope of I/O typical in a modern mobile device.

[0004] Accordingly, it is desired to have a system and method which can implement the wireless interface between a central computing and connectivity hub and one or more peripheral devices in a manner where the peripheral devices function as standalone devices with their own computing and connectivity (i.e. like a typical smartphone, tablet, laptop, etc.).

[0005] Thus, the present invention can implement the required wireless interface while (i) supporting a wide range of peripheral devices, (ii) minimizing computational requirements for the interface at both the computing hub and at the peripherals, (iii) minimizing power consumption of the interface at both the computing hub and at the peripherals, and (iv) adapting to any noise or interference in the wireless link between the computing hub and the one or more peripherals.

[0006] Other features and advantages of the present invention are described more fully below.

BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Preferred embodiments of the present invention will now be described, by way of example only, with reference to the attached Figures, wherein:

Figure 1 is a block diagram of a wireless interface for wirelessly connecting a computing hub and a peripheral, according to an embodiment.

Figure 2 is a signal flow diagram showing different types of data being transferred over the wireless interface between the computing hub and wirelessly connected peripheral, according to an embodiment.

Figure 3 shows the frame structure of a wireless link protocol used to implement the wireless interface of Figures 1 and 2, according to an embodiment.

Figure 4 shows segments of a transmission frame for data output to the peripheral and data input from the peripheral.

Figure 5 is a block diagram showing a computing hub and three different peripheral displays to which the hub can be connected, as well as user interfaces for the three different displays, according to an embodiment.

Figure 6 is a flowchart showing a method for processing output video data in the computing hub for transmission to a peripheral display, according to an embodiment.

Figure 7 is a flow chart showing a method implemented in the hub to determine when to send a complete video frame to the peripheral display, according to an embodiment.

Figure 8 schematic representation of various modes by which a peripheral camera input signal can either be transmitted to the computing hub or displayed directly on its own display, according to an embodiment.

DETAILED DESCRIPTION OF THE INVENTION

[0008] One embodiment of a system with a computer wirelessly connected to a peripheral device is indicated at 100 in Figure 1 . System 100 includes at least two physical devices, a computing hub 104 and one or more peripheral devices, of which one peripheral device 108 is shown in Figure 1 . It is contemplated that other embodiments may include more than one peripheral device. The computing hub 104 is a computing and communications device that includes a central processing unit (CPU) 1 12, RAM 1 16, persistent storage 120, and two wireless interfaces: an interface 124 for external connectivity (i.e. to the internet) and an interface 128 for wireless I/O transfer between the hub 104 and the peripheral 108. In one embodiment, these two wireless interfaces may share common physical components such as antennas while in another embodiment they may each comprise their own physical components. It is contemplated that in other embodiments, the computing hub may additionally have many more components, such as a display, microphones, speakers, sensors, buttons, etc. In yet other embodiments, the computing hub can have a wired external communications interface, rather than a wireless one; or no external interface at all. The computing hub 104 runs an operating system (e.g. Android, iOS, Windows, Linux, etc.) and computer programs (or applications). The illustrated embodiment of peripheral device 108 includes a CPU 140, RAM 144, persistent storage 148, a wireless interface 152 for I/O transfer to the hub 152 104 as well as input and output devices. In the embodiment of Figure 1 , the peripheral device 108 has one output device: a display 156 and one input device: a touch panel 160. In other embodiments, the peripheral device 108 may have additional input devices and additional output devices, such as speakers, microphones, camera, keyboards, buttons, LEDs, etc. In yet other embodiments, the peripheral 108 may have no input devices or output devices. In some embodiments, the hub 104 may have the form factor of a wearable device such as a smartwatch while the peripheral 108 may have the form factor of a smartphone. In other embodiments, the peripheral 108 may have the form factor of a tablet, a laptop, a tablet/laptop hybrid device, an HDMI dongle for an external display, an AR headset, a VR headset, a hybrid AR/VR headset, a drone, a display embedded in a vehicle, a display embedded in a home appliance, a display embedded in an industrial machine, etc.

[0009] The wireless interface between the computing hub 104 and the peripheral device 108 comprises a protocol stack where the protocol layer preferably operates on top of an existing set of wireless protocols. Examples of such protocols include 802.1 1 ac (aka WiFi), 802.1 1 ad (aka WiGig). As discussed in greater detail below, according to an exemplary embodiment, the data transferred between the computing hub 104 and peripheral 108 can work with any underlying wireless communications protocol that supports a direct connection between the peripheral and hub. In other embodiments, the underlying wireless

communications protocol can have an indirect connection between the hub and the peripheral, with intervening communication nodes.

[ooio] Referring now to Figure 2, a signal flow diagram is provided that shows different types of data being transferred over the wireless interface between computing hub 204 and wirelessly connected peripheral 208. As discussed below, the different types of data include control data that lets the hub manage various aspects of the wireless interface; channel quality measurement data that can be used to measure the quality of the link (e.g. packet error rate) and the input data that travels from the peripheral to the hub 204 and the output data that travels from the hub to the peripheral 208. These data elements are shown to be multiplexed (MUX) together on the transmit side and de-multiplexed (DE-MUX) on the receive side. The hub 204 transmits the peripheral's combined output data to the peripheral 208 over the wireless interface. The peripheral transmits its combined input data to the hub over the same wireless interface.

[ooii] Returning to Figure 2, a system 200 is shown that comprises various types of data that are combined (i.e. multiplexed or MUX) and separated (i.e. de-multiplexed or DE-MUX) as part of the operation of a wireless interface between a computing hub 204 and a peripheral 208. This embodiment shows one peripheral 208 but other embodiments can include more than one peripheral. Figure 2 shows data types and associated processing steps involved in transferring data over the wireless interface between the hub 204 and the peripheral 208, according to an embodiment of the invention. Two directions of data flow are provided: the downlink (DL) 212 direction carries data from the hub 204 to the peripheral 208, while the uplink (UL) direction 216 carries data from the peripheral 208 to the hub 204.

[0012] The DL data comprises output data for the peripheral 220, DL control data 224 and data for measuring the DL channel quality 228. This DL data is multiplexed together using a multiplexing function 232. The combined data is then transferred into the hub wireless protocol stack 236, which then transmits it to the peripheral 208 where it is received by the peripheral side wireless protocol stack 240. The peripheral side of the wireless protocol stack separates the received DL control data 244, DL measurement data 248 and DL output data 252. The UL data consists of peripheral input data 256, UL control data 260 and data for measuring the UL channel quality 264. The UL data is multiplexed together using a

multiplexing function 268. The combined data is then transferred into the peripheral wireless protocol stack 240, which then transmits it to the hub 204 where it is received by the hub side wireless protocol stack 236. The hub side of the wireless protocol stack separates the received UL control data 272, UL measurement data 276 and UL input data 280. In one embodiment, this wireless protocol stack uses the 802.1 1 ac/WiFi standard for wireless communications between the hub 204 and the peripheral 208. In another embodiment, there may be no control data because the quality of the wireless interface between the hub and the peripheral is measured directly at either the receiver for the hub wireless protocol stack 236, the receiver for the peripheral wireless protocol stack 240, or both.

[0013] Figure 3 shows the frame structure of a wireless link protocol used to implement the wireless interface of Figures 1 and 2, according to an embodiment. Communications protocols typically group data into temporal transmission windows known as frames. As discussed below with reference to Figure 4, one frame can be divided in time between data that is transmitted from the hub 104/204 to the peripheral 108/208 (which is referred to herein as downlink, or DL, data) and data that is transmitted from the peripheral to the hub, (which is referred herein as uplink, or UL, data). A frame interval can either be a fixed value (e.g. 1 ms) or be variable. Within a frame, sub-sections of time are allocated for different purposes, such as for measurement data, control data and input and output data, as discussed below with reference to Figure 4 shows the sub-sections of time within a frame. Since the peripheral 204/208 may have one or more output devices and one or more input devices, the frame segments allow accommodate multiple output data elements (labeled Οι , O2 up to On) and multiple input data elements (labeled h , I2 up to In) . Each of these output data elements may have different durations in time (labelled tio, t2o. . .tno) . Each of these input data elements may have different durations in time (labelled ti i, ta. - .tni) .

[0014] Returning to Figure 3, one embodiment is shown of a frame structure 300 used to define how DL and UL data is transferred between the hub 104/204 and the peripheral 108/208. In the illustrated embodiment, the DL and UL data transfer intervals are shown to occur sequentially. In other embodiments, the DL and UL data transfer can occur

concurrently. The frame interval 304 shows the duration of a single frame. The value of the frame interval can be selected to minimize the latency of the wireless interface between the hub and peripheral. A typical frame interval value that can be used in one embodiment is 1 .667 ms. The DL part of the frame, which is referred to herein as the DL sub-frame 308, comprises the DL measurement data 312, the DL control data 316 and the DL peripheral output data 320. The illustrated embodiment shows the sequence of transmission as measurement data first, control data second and output data third, but in other embodiments the sequence of transmission can have a different order. The embodiment in Figure 3 shows a time gap 324 between the DL and UL part of the frame but the gap can be of zero duration in other embodiments. Similar to the DL part of the frame, the UL part of the frame, referred to herein as the UL sub-frame 310, comprises the UL measurement data 328, the UL control data 332 and the UL peripheral input data 336. In other embodiments, there can be more than one DL sub-frame and UL sub-frame within the frame interval 304 corresponding to more than one peripheral device concurrently connected to the computing hub 104/204. In other embodiments, multiple peripherals can be connected wherein data is received on the same DL sub-frame and where the output data is common to the multiple peripherals (e.g. they all show the same display output).

[0015] Figure 4 shows an exemplary embodiment of frame structure 400 showing additional details concerning the composition of the DL output data part of the frame 404 and the UL input data part of the frame 408, corresponding to the DL peripheral output data 320 and the UL peripheral input data 336, respectively, as shown in Figure 3. The peripheral 108/208 includes at least one output device and/or input device. However, in most embodiments, the peripheral includes more than one output device and more than one input device. A typical embodiment may include a display, multiple speakers, multiple

microphones, a touch panel, a headphone jack, buttons, LEDs, etc. Therefore, the data from the different outputs must be contained within the DL output data part of the frame 404. The DL output part of the frame 404, in turn, comprises parts that contain data for each output device. These intervals Oi 412, O2 416 up to On 420 have durations of ti₀ 424, .20 428 up to tno 432, as discussed above, for n distinct output devices in the peripheral, and they contain the output data for each of those peripherals respectively. The UL input part of the frame 408 in turn consists of parts that contain data from each input device. These intervals I1 436, I2 440 up to In 444 have durations of tii 448, t2i 452 up to tni 456, as discussed above, for n distinct input devices in the peripheral, and they contain the input data for each of those peripherals respectively.

[0016] The specific frame and subframe structure used for the wireless interface between the hub 104/108 and the peripheral 108/208 is determined by the hub and communicated to the peripheral and depends on the specific input and output attributes of the peripheral device. These attributes include the number of input and output devices and the amount of data that must be transferred for each input or output device within a frame interval. The duration of the DL output data UL and input data depends on the bandwidth requirements of the output and input devices in the peripheral device. For example, a peripheral 108/208 with a display having a 1920x1080 pixel resolution would require a higher bandwidth than a peripheral display having 1280x720 pixel resolution. When a peripheral 108/208 connects to a computing hub 104/204, the structure of the frame must be established in order to enable the input and output data to transfer. In one embodiment, establishing this structure includes specifying the duration of the frame interval 304; specifying the duration and temporal position within the frame of the DL sub-frame 308; specifying the duration and temporal position within the frame of the UL sub-frame 310; specifying the duration and temporal position within the frame of the DL measurement data 312; specifying the duration and temporal position within the frame of the DL control data 316; specifying the duration and temporal position within the frame of the DL output data 320; specifying the duration and temporal position within the frame of the gap 324; specifying the duration and temporal position within the frame of the UL measurement data 328; specifying the duration and temporal position within the frame of the UL control data 332; and specifying the duration and temporal position within the frame of the UL output data 336.

[0017] Another embodiment of the communications protocol between the computing hub and one or more peripherals does not use a pre-defined frame structure but instead uses the mechanisms of a packet data protocol scheme such as internet protocol. In this embodiment, the computing hub and each peripheral device are assigned IP addresses. Furthermore, each individual data field, either input data, output data, control data or measurement data are assigned IP port numbers. The combination of IP address and port number allows each packet of data to be uniquely associated with a particular data field and enables the multiplexing (MUX) and demultiplexing (DE-MUX) of the various data fields.

[0018] There are multiple mechanisms for the hub 104/204 to initiate the establishment of a wireless connection to a particular peripheral 108/208. In one embodiment, physical proximity as detected by a close range wireless technology such as NFC can be the criteria for establishing a connection. In this embodiment, the end user brings the hub and peripheral sufficiently close that the close range wireless technology in the hub detects the peripheral and the detection of this proximity can then be the criteria for initiating the process for establishing a connection. In another embodiment, the hub can be configured a priori to initiate the process of establishing a connection whose identifiable radio signal it detects on the wireless interface between the hub and peripherals. In one embodiment of the I/O radio interface, the wireless protocol stack comprises WiFi Direct running on top of 802.1 ^"l ac. Using this protocol stack, the peripheral 108/208 can transmit to the hub 104/204 a unique physical identifier (i.e. a MAC address), which can be used by the hub to determine whether it should automatically initiate the process for connection to the hub. In one embodiment, a

combination of approaches is also possible where a user initially uses NFC based proximity detection to establish a first connection between a hub and peripheral but subsequent connections are made automatically when the hub detects the presence of the peripheral over the wireless interface used for the I/O data connection between the hub and peripheral. In another embodiment, the peripheral could send the hub a message signed with a cryptographically generated digital signature that indicates it is an authorized device. This would allow the hub to choose to only connect if the digital signature comes from an authorized device.

[0019] There are multiple mechanisms for hub 104/204 to determine the frame structure for the wireless interface between it and a particular peripheral 108/208. In one embodiment, the hub 104 is initially "paired" with one peripheral. This pairing process involves

communication between the hub 104/204 and peripheral 108/208 that provides information to the hub that informs it of attributes of the peripheral device. In one embodiment of the pairing process, this information can be transferred between the hub 104/204 and the peripheral 108/208 using near field communications (NFC). In another embodiment of the pairing process the user can directly enter information about the peripheral device using the input interfaces of the hubl 04/204, such as entering text though an on-screen keyboard and touch panel. In another embodiment of the pairing process, the hub 104/204 may be able to identify information pertaining to the peripheral 108/208 based on information that the peripheral transmits as it attempts to establish a wireless connection to the hub using the I/O radio interfaces 152 and 128 in the peripheral and hub, respectively. In one embodiment of the I/O radio interface, the wireless protocol stack comprises WiFi Direct running on top of 802.1 1 ac. Using this protocol stack, the peripheral 108/208 can transmit a unique physical identifier (i.e. a MAC address) to the hub 104/204 , which can be used by the hub to determine the device attributes of the peripheral. [0020] In one embodiment, the information regarding the peripheral's attributes comprises the peripheral's input and output device details. These details can specify the number of distinct input and output devices in the peripheral and the information required to determine the data rates required to transfer the data associated with each of the input and output devices. This information can be the data rates directly or physical device attributes that enable the hub 104/204 to compute the required data rates. An example of such an attribute is the resolution of the peripheral's display. With this information, the hub can calculate the parameters of the frame structure and communicate them to the peripheraH 08/208. In another embodiment, the peripheral only provides information that uniquely identifies its device type. The hub 104/204 can then use a priori known information of the device attributes of all peripheral types to determine the parameters of the frame structure for the particular peripheral. In one embodiment, this a priori known information can be locally stored in the hub's storage 120. In another embodiment, this a priori known information can be stored in an external device that the hub 104/204 can connect to with an external networking interface. In one embodiment, this external networking interface is the external radio interface 124.

[0021] A computing hub 104/204 can wirelessly connect to more than one peripheral 108/208 concurrently. In one embodiment, a hub is already connected to one or more peripherals and an additional peripheral is also paired with the intent of establishing a connection. The hub 104/204 determines the frame structure required to wirelessly connect to the new peripheral and also determines any changes to the frame structure for the existing, connected peripherals. In this case, the hub first informs the existing, connected peripherals of the changes to their frame structures, begins communicating to the existing peripherals using this new frame structure and then establishes the new, additional connection to the new peripheral using the frame structure that the hub determines for the new peripheral.

[0022] A computing hub 104/204 may switch from being connected to one peripheral to a different peripheral. In some circumstances, the hub may be displaying content on the first peripheral display when this switch happens (e.g. content generated by the OS or an application). In such an event, the hub determines the display attributes of the new peripheral and provides this information to the hub OS or the application creating the display content, which then adjusts the properties of the content (e.g. resolution) to match the display attributes of the new peripheral. [0023] There are several ways for the peripheral 108/208 to inform the hub 104/204 of either a unique identifier or a message signed with a digital signature. In one embodiment, this information may be transferred between the hub and the peripheral using near field communications (NFC). In another embodiment, the hub can establish a wireless connection to the hub but with only a DL 316 and UL control channel 332. This control channel can be used to convey a unique identifier or a message signed with a digital signature from the peripheral to the hub that could then be used by the hub to determine if the peripheral is authorized device.

[0024] In Figure 5, a system 500 is shown comprising a computing hub 504 connected to peripheral devices 508, 512 and 516. Hub 504 contains a CPU that runs an operating system (OS) (e.g. Android, iOS, Windows, Linux, etc.). In one embodiment when the computing hub 504 is connected to a peripheral that has a display, the OS creates a user interface (Ul) that is shown on the display. In the embodiment of Figure 5, three different peripherals 508, 512 and 516 with different screen sizes may be connected to the hub 504 via a wireless interface. The illustrated peripherals include a smartphone-sized display 508, a laptop/tablet sized display 512 and large screen display 516. In one embodiment, the display size attribute information for the peripheral that is known to the hub 504 and is used to determine the frame structure of the wireless interface between the hub and the peripheral is also used to determine the attributes of the Ul that is shown on the peripheral device's display. In another embodiment, the user is provided with an option of selecting a preferred Ul by informing the hub using the input of either the hub or the peripheral device itself. When a hub is connected to multiple peripherals, each with one or more displays, the operating system on the hub can display either the same or a different Ul on each of the displays. Figure 5 shows examples of Uls for different screen sizes that are typically used for the respective classes of device for smartphone display 508, laptop/tablet sized display 512 and large screen display 516.

[0025] Figure 6 shows an embodiment of the sequence of steps 600 run in a program on the hub CPU to establish a connection between the hub 104/204/504 and a peripheral device 108/208/508/512/516. The first step 604 is a determination of the presence of the peripheral, followed by a step 608 that determines if the peripheral is authorized to connect to the hub. If the peripheral is detected and determined to be authorized, the hub then determines the frame structure in step 612 and informs the peripheral and establishes the wireless communications link to the peripheral. The hub then determines the Ul type in step 620 and begins transmitting video frames associated with the selected Ul to the peripheral using the DL output data sub-frame for display output.

[0026] Referring now to Figure 7, computing steps are shown for processing video data that has its source in the hub and is intended to be transmitted to an output display in a peripheral connected wirelessly to the hub. In one embodiment, the video output data represents a frame that comprises a numeric value for specifying the color and intensity of each pixel in a display. The display has a resolution (e.g. the display may comprise an array of pixels values with dimensions 1920x1080 where each pixel value represents color and intensity). The OS of the hub creates a new frame to be displayed by the peripheral at a fixed rate. In one embodiment, this rate is 60 frames per second (i.e. a new video frame is created every 1/60 seconds, or with a frame period of 1 .6667 ms). For each current video frame 704 and previous video frame 708, the frames are compared at step 712. If the frames are different, compression may be applied (step 716) to reduce the size of the video frame and then this video frame data is inserted into the DL sub-frame of the wireless interface between the hub and the peripheral (step 720). If the frames are identical, then step 724 results in no data being inserted into the DL sub-frame. In another embodiment, if the frames are identical, a small amount of data is inserted to inform the receiver at the peripheral that a full video frame is not required. In another embodiment, only the difference in pixel values from one frame to the next are inserted into the DL sub-frame.

[0027] Referring now to Figure 8, a system 800 is shown comprising a computing hub 802 that is wirelessly connected to a peripheral device 804 having at least one camera for input and a display for output. In one mode of operation, the camera input data 808 is transmitted from the peripheral wireless protocol stack 816 to the hub wireless protocol stack 820 and this camera input data 824 is received at the hub. The output data for the display 828 is transmitted from the hub wireless protocol stack 820 to the peripheral wireless protocol stack 816, from which the display output data 812 is sent to the display. In a second mode of operation, an application running on the hub 802 intends for the image from the peripheral camera to be displayed on the same peripheral's display. In this case, the hub 802 sends control data 832 through the hub wireless protocol stack 836 to the peripheral wireless protocol stack 840, which then outputs the received control data 844. This control data commands the peripheral to direct the camera input data 848 to an adding function 852 that visually superimposes the camera input data on top of the display output data 856 received from the hub. In one embodiment, the camera input data is not transmitted to the hub but instead stays on the peripheral. In another embodiment, the application implementing these steps is a camera application that intends the peripheral camera preview image to be displayed on the display. The camera application may have Ul elements that it will overlay on top of the camera preview image so there is still display output data 860 being transmitted from the hub to the peripheral that is visually superimposed onto the camera input 848 with the adding function 852 to create the display output data 864 that will be displayed on the peripheral display.

[0028] The radio transmitters at the hub and the peripheral that transmit the DL and UL data respectively in the embodiments discussed above, have adjustable transmit power levels. Reducing the power of these transmitters reduces system power consumption and increases battery life. The system comprising the wireless protocol stack in the hub and the wireless protocol stack at the peripheral reduces the transmit power to the minimum possible level that still maintains the required link quality, as measured with packet error rate (PER), signal to noise ratio (SNR) or another metric for channel quality. In one embodiment, the receiver at the peripheral measures the quality of the received DL signal and sends commands to the hub using the UL control channel that direct the hub to reduce its transmit power. Similarly, the receiver at the hub measures the quality of the received UL signal and sends commands to the peripheral using the DL control channel that directs the peripheral to reduce its transmit power. In another embodiment, the hub uses measurements of the UL signal received from the peripheral to adjust its DL transmit power to the minimal level required to maintain adequate link quality. Similarly, the peripheral may use measurements of the DL signal received from the hub to adjust its UL transmit power to the minimal level required to maintain adequate link quality. In one embodiment, these measurements can include the received signal strength indication (RSSI) of the desired signal, the RSSI of interfering signals, measurements of the signal to noise ratio (SNR), etc.

[0029] The wireless interface between the hub and the peripheral in the embodiments discussed above, may encounter interference from other wireless transmitters operating in the same radio frequency (RF) channels. Alternately, the received signal strength on the current RF channel may drop below a minimum threshold required to maintain acceptable link performance. In the event that the hub detects interference in the RF channel on which it is operating or detects the signal strength drop below the acceptable threshold, it can select an alternate RF channel with better performance, inform any connected peripherals that they should use the new channel and then switch to the new channel. In another embodiment, there may not be another channel available to use. In that case, the hub can switch the modulation and coding scheme (MCS) that it uses to transmit data to and receive data from the one or more peripherals to one that is more robust (i.e. that can operate at higher interference levels). When using a more robust MCS, the data rate drops. If this reduced data rate is insufficient to support the data rates of the I/O traffic between the hub and peripheral, the compression ratio of any compression schemes applied to the output or input data can be increased to reduce the data rate to a level that can be supported with the new MCS. In one embodiment, the display output data transmitted from the hub to the peripheral is

compressed. When a MCS that corresponds to a lower data rate is used, the compression ratio of this compression processing is increased, thereby reducing the associated data rate of the compressed display output data.

[0030] In summary, the embodiments of the invention set forth herein provide a method and system for wirelessly connecting a computing hub to one or more peripherals while (i) supporting different types of peripheral devices, (ii) minimizing computational requirements for the interface at both the computing hub and at the peripheral, (iii) minimizing power consumption of the interface at both the computing hub and at the peripheral, and (iv) adapting to any noise or interference in the wireless link between the computing hub and the one or more peripherals. Unlimited different types of peripheral devices are supported as a result of the hub dynamically changing the frame structure of the wireless protocol between the hub and the peripheral to reflect the I/O attributes of each specific peripheral device. Computational and power consumption requirements are minimized by avoiding the transmission of output display data over the wireless interface and camera input data over the wireless interface in scenarios where it is not required. Power consumption is also minimized by implementing mechanisms that can maintain the quality of the wireless interface while minimizing the transmit power required to operate the interface at the hub and the one or more peripherals. Finally, the negative impact of interference is mitigated by either changing radio channels or using a more robust transmission scheme over the wireless interface.

Claims

CLAIMS:

1 . A method for wirelessly connecting a computing hub to one or more peripheral devices having different device attributes using an adaptive protocol, said method comprising: transmitting data between the computing hub and one or more peripheral devices, said data comprising one or more data types selected according to said different device attributes; multiplexing said data for transmission from the computing hub to said one or more peripheral devices and from said one or more peripheral devices to the computing hub; demultiplexing said data for reception by the computing hub from said one or more peripheral devices and by said one or more peripheral devices from the computing hub; exchanging device attribute information between the computing hub and each of said one or more peripheral devices for identifying said respective different device attributes; exchanging formatting information between the computing hub and each of said one or more peripheral devices for establishing a format for multiplexing and demultiplexing said data; and wherein said formatting information is based on a priori knowledge of said device attribute information.

2. The method of claim 1 , wherein said protocol uses a time-based repeating frame structure whereby each data field is assigned a starting point and duration within a repeating temporal frame.

3. The method of claim 1 , wherein said protocol uses Internet Protocol (IP) whereby the computing hub and of each of said one or more computing peripheral devices is assigned an IP address and each data field is assigned a port number, such that each data type in a direction of transmission between the computing hub and said one or more computing peripheral devices comprises a unique network socket address.

4. The method of claim 1 , wherein the data transmitted between the computing hub and said one or more peripheral devices comprises input data from input devices of said one or more peripheral devices and output data to output devices of said one or more peripheral devices.

5. The method of claim 1 , wherein the data transmitted between the computing hub and said one or more peripheral devices comprises control data for transmitting commands from the computing hub to said one or more peripheral devices and transmitting measurement information from said one or more peripheral devices to the computing hub.

6. The method of claim 5, wherein the control data is transmitted using a transmission protocol that ensures guaranteed delivery.

7. The method of claim 6, wherein the transmission protocol is TCP/IP.

8. The method of claim 5, wherein the peripheral measurement information represents remaining battery life of each of said one or more peripheral devices.

9. The method of claim 1 , wherein the data transmitted between the computing hub and said one or more peripheral devices comprises measurement data for determining the quality of a radio link between the computing hub and said one or more peripheral devices.

10. A method for adapting a user interface for displaying data received from a computing hub on a peripheral device, said computing hub and peripheral device being wirelessly connected, the method comprising: determining at the computing hub a physical form factor and input/output attributes of the peripheral device, said physical form factor including at least one of display size and resolution and said input/output attributes including presence of at least one of a touchscreen, keyboard and mouse, and wherein the computing hub determines said physical form factor and input/output attributes by exchanging information with the peripheral device and storing the physical form factor and input/output attributes for subsequent wireless connection of the computing hub to the peripheral device; and generating at the computing hub a graphical user interface (GUI) optimized for said physical form factor and input/output attributes of the peripheral device.

1 1 . A method for reducing power consumption associated with a wireless link connecting a computing hub to a peripheral device, said computing hub and peripheral device each having a transmitter and receiver, the method comprising: detecting received signal quality at one of either the computing hub or peripheral device; comparing at one of either the computing hub or peripheral device the received signal quality with a required signal quality to maintain the wireless link; and if the received signal quality is higher than the required signal quality then one of either reducing power of the transmitter at the computing hub or sending a control message from the computing hub to the peripheral device commanding the peripheral device to reduce power of the transmitter at the peripheral device, until the received signal quality is equal to said required signal quality; or if the received signal quality is less than the required signal quality then one of either increasing power of the transmitter at the computing hub or sending a control message from the computing hub to the peripheral transmitter commanding the peripheral device to increase power of the transmitter at the peripheral device, until the received signal quality is equal to said signal required quality.

12. The method of claim 1 1 , wherein if the received signal quality does not equal the required signal quality at maximum transmit power the computing hub adjusts a modulation and coding scheme (MCS) of said wireless link for operation at a lower signal quality and transmits a control signal for causing the peripheral device to communicate according to the adjusted MCS.

13. A method for adjusting the data rate of a wireless link for transmitting video data generated by a computing hub for display at a peripheral device, the method comprising: comparing at the computing hub a current video frame with a previous video frame; if the current video frame is different from the previous video frame then transmitting the current video frame over the wireless link; and

if the current video frame is the same as the previous video frame then preventing transmission of the current video frame over the wireless link.

14. A method of claim 13, wherein the computing hub uses a video compression algorithm to compress video frames to be transmitted over the wireless link, said video compression algorithm being adjusted to ensure that the amount of data being transmitted is less than the available capacity of the wireless link.

15. A method for reducing the data rate of a wireless link for transmitting video data generated by a computing hub for display at a peripheral device, the method comprising: dividing each video frame at the computing hub into a fixed number of regions that are characterized by their locations within the video frame;

comparing at the computing hub pixel values for each of said regions of a current video frame with pixel values for the same regions of a previous video frame; and only transmitting the pixel values for regions that differ between the current video frame and previous video frame.

16. A method for reducing the data rate of a wireless link for video data generated by a computing hub for display at a peripheral device, the method comprising: comparing at the computing hub a current video frame with a previous video frame; determining at the computing hub any difference between individual pixels in the current video frame and the previous video frame; and

transmitting over the wireless link only the pixels that are different between the current video frame and the previous video frame.

17. A method for optimizing performance of a wireless link between a computing hub and one or more peripheral devices, the method comprising: selecting at the computing hub a radio frequency from among a plurality of radio frequency channels on respective radio frequency bands over said wireless link; calculating at a radio frequency receiver in the computing hub a signal quality for each of said radio frequency channels; selecting at the computing hub a radio frequency channel that provides optimal signal performance; and in the event the signal quality falls below a level at which the wireless link can be maintained selecting at the peripheral device a new radio frequency channel that is a priori known by both the computing hub and the peripheral device and attempting to establish the wireless link with the computing hub on the new radio frequency channel.

18. The method of claim 17, wherein the radio frequency receiver in the computing hub periodically scans different radio frequency channels.

19. The method from claim 17, wherein the radio frequency receiver in the computing hub begins scanning different radio frequency channels when the signal quality on a current radio frequency channel falls below a predetermined minimum quality threshold.

20. A method for reducing the amount of data transmitted between a computing hub and peripheral device over a wireless link, said peripheral device having a camera and display for displaying a video signal from the camera, the method comprising: transmitting commands over the wireless link from the computing hub to the peripheral device to request the peripheral device to transmit the video signal from the camera to display; redirecting the video signal at the peripheral device from the computing hub to the display; generating and transmitting over the wireless link one or more graphical elements from the computing hub to the peripheral display; and superimposing at the peripheral device the one or more graphical elements received from the computing hub onto the video signal from the camera and displaying a composite video signal on the display.

21 . The method of any one of claims 1 , 10, 1 1 , 13, 15 - 17 and 20, wherein the computing hub is a computing and communications device having a central processing unit (CPU), RAM, persistent storage, and at least two wireless interfaces for external connectivity and connectivity to said one or more peripheral devices over said wireless link.

22. The method of claim 21 , wherein the computing hub is a wearable device.