WO2017190233A1

WO2017190233A1 - Cross-device interaction verification

Info

Publication number: WO2017190233A1
Application number: PCT/CA2017/050528
Authority: WO
Inventors: Timothy Jing Yin SZETO
Original assignee: Nanoport Technology Inc.
Priority date: 2016-05-05
Filing date: 2017-05-01
Publication date: 2017-11-09

Abstract

A mobile device includes a plurality of connectors each for interconnecting the device with at least one of a plurality of other devices. Each of the plurality of connectors is located in a defined location on the mobile device. Memory in the device stores a data store that indicates connectivity information for each other device interconnected to the mobile device, including a relative spatial location of each of these other devices relative to the mobile device, based on the defined location of each of the plurality of connectors. The memory further stores processor executable instructions that identify an event for cross-device interaction with another device, and in response verify from the data store that the other device is in an expected relative spatial location to the mobile device.

Description

CROSS-DEVICE INTERACTION VERIFICATION

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority from U.S. Provisional Patent Application No. 62/332,215, filed May 5, 2016; the entire contents of which are hereby incorporated by reference herein.

FIELD

[0002] The following relates to mobile devices that may interact with each other and are capable of determining spatial relationships to interconnected devices, and methods.

BACKGROUND

[0003] Many known computing devices (e.g. mobile phones, tablets, laptop computers, etc.) allow cross-device interaction.

[0004] PCT Publication No. WO 2015/070321 , for example, discloses interacting mobile devices that allow users to provide input by way of way of touch screens, the contents of which are hereby incorporated by reference. Proximate devices may combine their screens to form a larger display area and user interface.

[0005] The combined screens may appear as a single user interface. Each of the screens may, however, be independently managed by the operating system/user interface of one device. User interactions may, however, span the two screens.

[0006] Yet other devices allow for more conventional cross-device interaction - often resulting from pairing the devices. For example devices may be paired to each, using Bluetooth or other protocols. Yet other devices provide access to physically interconnected peripherals or computing devices.

[0007] In conventional arrangements, however, unauthorized access by unknown or physically distant devices may be possible.

[0008] Accordingly, there exists a need for new methods and/or devices that cross-device interaction to be verified. SUMMARY

[0009] In accordance with one aspect, there is provided a mobile device comprising: a plurality of connectors each for interconnecting the device with at least one of a plurality of other devices, each of the plurality of connectors located in a defined location on the mobile device; a processor and a memory, the memory storing a data store that indicates connectivity

information for each of the other devices that are interconnected to the mobile device including a relative spatial location of each of these other devices relative to the mobile device based on the defined location of each of the plurality of connectors; and the memory further storing processor executable instructions that when executed cause the processor to identify an event for cross-device interaction with another device, and in response verify from the data store that the another device is in an expected relative spatial location to the mobile device.

[0010] In accordance with another aspect, there is provided a method of facilitating cross- device interaction at a mobile device that comprises a plurality of connectors each for interconnecting the device with at least one of a plurality of other devices, each of the plurality of connectors located at a defined location on the mobile device, the method comprising:

maintaining a data store that indicates connectivity information for each of the other devices that are interconnected to the mobile device including a relative spatial location of each of these other devices relative to the mobile device based on the defined location of each of the plurality of connectors; identifying an event for cross-device interaction with another device, and in response to the identifying the event for cross-device interaction, verifying from the data store that the another device is in an expected relative spatial location to the mobile device.

BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Embodiments will now be described by way of example only with reference to the appended drawings wherein:

[0012] FIG. 1 is a schematic block diagram of a two interconnected devices located in proximity to one another that verify cross-device interaction, according to an embodiment;

[0013] FIG. 2 is a block diagram of example hardware components of a first mobile computing device of FIG. 1 , according to an embodiment; [0014] FIG. 3 is a block diagram of example software components in the first mobile computing device of FIG. 1 , according to an embodiment;

[0015] FIG. 4 depicts a data store at the device of FIG. 2, according to an embodiment;

[0016] FIG. 5 is a flow chart illustrating verification of cross-device interaction, at a device initiating such interaction, in accordance with an embodiment; and

[0017] FIG. 6 is a flow chart illustrating verification of cross-device interaction, at a device responding to such interaction initiated at another device, in accordance with an embodiment.

DETAILED DESCRIPTION

[0018] FIG. 1 depicts two devices 100 and 102, each include a housing 104 defined by respective external surfaces 106. Devices 100, 102 can be any suitable electronic devices that interface with one another and provide complementary functions as described herein. At least one of devices 100, 102 may be a mobile computing device.

[0019] Example devices 100, 102 may take the form of cellular phones, cellular smart- phones, wireless organizers, pagers, personal digital assistants, computers, laptops, handheld wireless communication devices, wirelessly enabled notebook computers, portable gaming devices, tablet computers, or any other portable electronic device with processing and communication capabilities. Example mobile devices 100, 102 may also take the form of peripheral devices, such as displays, printers, touchscreens, projectors, digital watches, cameras, digital scanners and other types of auxiliary devices that may communicate with another computing device.

[0020] In an example, each of devices 100, 102 may be a smartphone, or one may be a smartphone and a peripheral device (e.g. a speaker, mouse, keyboard, display screen, camera, battery, etc.). In another example, one device 100 may be a mobile device including a touch surface and the other may be a communication peripheral (e.g. a router) for connecting to other devices. As will be apparent, other types of devices 100 and 102 can be envisaged that benefit from interconnection and interoperability.

[0021] Further, in the depicted embodiment, devices 100 and 102 may be of the same type - generally identical in structure and components. In other embodiments exemplified below, device 100 (or a similar device) may communicate with other different yet compatible devices. [0022] Each of devices 100, 102 may include an input interface such as a touch display 110 that cooperates with another complementary touch display 110 when the spatial locations of devices is established relative to one another (e.g. to provide one larger touch screen).

[0023] Touch display 110 may, for example, be a capacitive display screen that includes a touch sensing surface. These may be integrated as a single component. Alternatively, touch display 110 may include suitably arranged separate display and touch components. Touch display 110 may be adapted for sensing a single touch, or alternatively, multiple touches simultaneously. Touch display 110 may sense touch by, for example, fingers, a stylus, or the like. Touch display 110 may return the coordinates of any touch or touches for use by a processor of device 100/102. Likewise, touch display 110 may be used to display pixelated graphics - in the form of computer rendered graphics, video and the like.

[0024] In an example embodiment, a larger interconnected screen allows input to be received on either one of touch display 110 of devices 100, and 102.

[0025] Each of mobile devices 100 and 102 includes respective connectors 108 and 112 for allowing interconnection of devices 100 and 102. In the example illustrated in FIG. 1 , device 100 includes four connectors 108A, 108B, 108C, 108D (individually and collectively

connector(s) 108) and device 102 includes four connectors 112A, 112B, 112C, 112D

(individually and collectively connector(s) 112).

[0026] Connectors 108 and connectors 112 may for example be physical connectors to a serial communications port, such as a universal serial bus (USB) port, or the like. In a particular embodiment, connectors 108 and connectors 112 may be magnetic connectors, including magnetic elements, as detailed in PCT Publication No. WO 2015/070321.

[0027] Although connectors 108 and 112 have been shown at the center of the top/bottom and sides of devices 100 and 102, other locations of connectors 108 and 112 are possible. For example, the connectors on each of the devices 100 and 102 can be located at the corners of each edge of the devices, as for example illustrated in PCT Publication No. WO 2015/070321. Additionally, although four connectors have been shown, the number of connectors provided on each of devices 100 and 102 is somewhat arbitrary, and may vary from device to device, and may depend on the type of device 100, 102.

[0028] Devices 100 and 102 shown in FIG. 1 have been illustrated with particular exemplary connector and device form factor and geometry. Of course, alternate

configurations/layout/positioning for the connectors and alternate size/layout of the device are possible. Similarly, although two interconnected devices (e.g. devices 100 and 102) are shown in FIG. 1 , multiple (e.g. three or more interconnected devices) can be envisaged having alternate connector configurations/layout/positioning and alternate size/layout of device 100. Example devices having different geometries are for illustrated in US Patent Application No. 15/013,750, the contents of which are hereby incorporated by reference.

[0029] FIG. 2 is a simplified block diagram of device 100 (an example mobile computing device), according to an example embodiment. Mobile device 100 includes a processor 202, display 110, an I/O interface 208, connectors 108, a communication subsystem and network interface 210 which allows communication to external devices (e.g. interconnected devices such as device 102), a memory 212.

[0030] Processor 202 controls the overall operation of mobile device 100. Communication functions, including data and voice communications, are performed through a communication subsystem and network interface 210. Communication subsystem and network interface 210 enables device 100 to communicate with other devices (e.g. device 102). In some

embodiments, device 100 may communicate with device 102 via connectors 108 by way of a bus or point to point communications (as shown in FIG. 2). Additionally, device 100 may further communicate with device 102 via communication subsystem and network interface 210.

[0031] In other embodiments, connectors 108 provide a mechanical/physical connection and the data communication between devices 100 and 102 is established instead via the communication subsystem and network interface 210 (e.g. using wireless communications such as Wi-Fi, Bluetooth, Wireless USB, capacitive coupling communications). In such embodiments connectors 108 may not be connected to I/O interface 208 (not shown). In addition to establishing data communication between devices 100, 102 and communicating regarding whether device 100 is interconnected to device 102, wireless data communication can also be used to share connectivity information (e.g. for establishing data communications) prior to any mechanical connections being made.

[0032] In one example, connectors 108 of device 100 may utilize communication subsystem 210 to receive messages from and send messages to interconnected devices (e.g. request and receive additional spatial information from interconnected devices, such as from device 102). Accordingly, in one embodiment, device 100 can communicate with other interconnected devices using a USB or other direct connection, as may be established through connectors 108/112. In another embodiment, device 100 communicates with interconnected devices (e.g. device 102) using Bluetooth, NFC, or other types of wireless communications as envisaged by a person skilled in the art.

[0033] Memory 212 may include a suitable combination of any type of electronic memory that is located either internally or externally such as, for example, flash memory, random-access memory (RAM), read-only memory (ROM), compact disc read-only memory (CDROM), electro- optical memory, magneto-optical memory, erasable programmable read-only memory

(EPROM), and electrically-erasable programmable read-only memory (EEPROM), or the like.

[0034] I/O interface 208 enables device 100 to communicate via connectors 108, e.g., to exchange data and establish communication with other devices 102. I/O interface 208 may also enable device 100 to interconnect with various input and output peripheral devices. As such, device 100 may include one or more input devices, such as a keyboard, mouse, camera, touch screen (e.g. display 110), a microphone, and may also include one or more output devices such as a display screen (e.g. display 110) and a speaker.

[0035] Device 100 may be adapted to operate in concert with one or more interconnected devices (e.g. device 102). In particular, device 100 includes an operating system and software components, which are described in more detail below. Device 100 may store the operating system and software code in memory 212 and execute that software code at processor 202 to adapt it to operate in concert with one or more interconnected devices (e.g. device 102). The software code may be implemented in a high level procedural or object oriented programming or scripting language, or a combination thereof. The software code may also be implemented in assembly or machine language.

[0036] As exemplified in PCT Publication No. WO 2015/070321 , device 100 and

interconnected devices (e.g. device 102) may each store software code which when executed, provides a coordinator at each of devices 100, 102 which performs various functions, including detection and registration of devices connected to each of devices 100, 102. Additionally, coordinator of each device 100, 102 may coordinate task sharing between devices and task assignment from one device (e.g. device 100) to another (e.g. device 102). The coordinator may also coordinate data transfer between the devices 100, 102. Thus, a coordinator at a first device 100 can communicate with a coordinator at other devices (e.g. device 102) by way of a bus or a network or both (not shown). By way of these communications, the respective coordinators of devices 100, 102 may establish a peer to peer relationship or a master-slave relationship, depending on the nature of the desired communication as may be established between device 100 and/or interconnected devices 102. [0037] Those skilled in the art will appreciate that portions of operating system 300 and remaining software components, such as specific device applications, or parts thereof, may be temporarily loaded into a volatile store forming part of memory 212. Memory 212 or a portion thereof may be on processor 202. Other software components can also be included, as is well known to those skilled in the art.

[0038] As disclosed in US Patent Application No. 15/013,750, device 100 maintains connectivity information for each of its connectors 108 in a data store that may be used to determine the spatial relationship of devices (e.g. device 102) that are interconnected (e.g. mechanically and/or electrically and/or wirelessly) to device 100. The connectivity information for mobile device 100 can include information about whether a connection exists for each physical connector 108 on mobile device 100 with another device (e.g. device 102); the defined relative physical location of each of connectors 108 on device 100 (e.g. x, y parameters relative to the device; general location descriptors such as top, bottom, left, right). Based on knowledge of the location of connectors 108, the relative spatial location of device 102 may be deduced. For example, interconnection with connector 108A will allow deduction that device 102 is connected to left of device 100. Additionally, this connectivity information may optionally be augmented with more specific information about interconnected devices (e.g. size of any interconnected device, type of device, device identification information, location of connectors on an interconnected device, and devices interconnected with an interconnected device, etc.).

[0039] In the example of FIG. 1 , connectors 108B and 112C are physically (e.g.

mechanically) connected to one another in a side by side arrangement. In addition to the physical/mechanical connection, devices 100 and 102 are in data communication with one another.

[0040] Such data communication may occur through electrical conduction of signals between electrical contacts of the respective interconnected connectors (e.g. connectors 108B and 112C). This type of communication may be provided as a USB compatible bus is established through the interconnected device connectors (e.g. connectors 108B and 112C). Alternatively, data communication may be made through a suitable wireless interfaces at device 100, 102 - for example established as a result of the proximity of the device 100 to the device 102. Possible wireless interfaces include WiFi interfaces; Bluetooth interfaces; NFC interfaces; and the like. Extremely high frequency (EHF) communication is also contemplated. An example of such EHF communications is described in http://keyssa.com and U.S. Patent Publication No. 2015/0065069, both of which are hereby incorporated in their entirety. Other forms of wireless interfaces/communication will be appreciated to those of ordinary skill.

[0041] Once a mechanical/physical connection is established between respective connectors (e.g. connectors 108B, 112C), devices 100, 102 can sense the physical

interconnection (e.g. directly via the connectors and/or with external sensors). In embodiments in which connectors 108/112 provide a mechanical connection and data connectivity, a change in the electrical characteristics at the electrical contacts of the respective interconnected connectors (e.g. connectors 108B and 112C) such as but not limited to: a change in voltage, impedance, etc., can be used to indicate a physical coupling of the respective connectors (e.g. 108B and 112C).

[0042] In other embodiments, devices 100/102 may communicate using extremely short range wireless communication, and devices 100/102 can detect an EHF signal (e.g. received from an interconnected device 102 at the device 100) which can be used to indicate that the electronic connector elements (e.g. as contained within connectors 108B, 112C) are located within a few mm of one another.

[0043] In some embodiments, connectors 108 and 112 include magnets utilized to physically connect the devices 100 and 102 both mechanically and electrically (as discussed in PCT Publication No. WO 2015/070321). In other embodiments, at least some of connectors 108 may be adapted to physically mate with particular ones of respective connectors 112 such that when mated, connectors 108 and 112 allow interconnected devices 100 and 102 to connect both mechanically and/or electrically. In this embodiment, connectors 108 may optionally allow device 100 to transfer or receive power and/or data to or from interconnected devices such as device 102.

[0044] In some embodiments, sensors (e.g. Hall Effect sensors) on devices 100/102 can be used to detect a magnetic field of one or more magnets in a proximate connector 108/112. Such sensors may be integrated within each of connectors 108, 112 or provided as a separate external component. Other mechanical sensors may alternatively be used. For example, if a connector (e.g. connector 108B) includes a moveable magnetic element, a pressure sensor (not shown) can be used to detect attractive force of another connector (e.g. connector 112C) on that element and thereby detect a mechanical connection of the connectors 108B and 112C, as for example discloses in U.S. Provisional Patent Application No. 62/280329, the contents of which are hereby incorporated by reference. [0045] An indication of the physical/mechanical connectivity of devices 100 and 102 by way of one or more connectors 108, 112 can trigger a first device 100 to determine the relative spatial location of an interconnected device 102 relative to the first device 100, as for example detailed in US Patent Application No. 15/013,750. Likewise, device 102 may perform a similar method, and also determine its relative spatial location of interconnected device 100.

[0046] FIG. 3 illustrates an organizational block diagram of software components at device 100/102 as stored within the memory of FIG. 2 for allowing detection of spatial relationships of other interconnected mobile devices (e.g. device 102). As illustrated, software components include an operating system 300, connectivity module 302, a device identification module 304, a communication module 306, a spatial relationship synthesizer module 308, a data store 312, a cross-device communication module 316, and a cross-device request queue 314. Data store 312 includes information related to one or more of: connectivity, device and connector information for the device 100. The operating system and components may be loaded from persistent computer readable memory onto device 100/102.

[0047] Operating system 300 may allow basic communication and application operations related to the mobile device. Generally, operating system 300 is responsible for determining the functions and features available at device 100, such as keyboards, touch screen,

synchronization with applications, email, text messaging and other communication features as will be envisaged by a person skilled in the art. In an embodiment, operating system 300 may be Android™ operating system software, Linux operating system software, BSD derivative operating system software, or any other suitable operating system software.

[0048] Connectivity module 302 operates in conjunction with connectors 108, and coordinates detection of when a connection in made or lost at each of the connectors 108 on the device 100. Connectivity module 302 further maintains data store 312 which includes connectivity information that indicates whether a connection exists for each of the connectors 108 on the mobile device 100. Data store 312 may have any suitable format within memory 212. Further, in response to sensing that a new connection has been made or lost with a particular connector 108, connectivity module 302 updates the connectivity information in data store 312. Examples of such connectivity information are shown within data store 312 in FIG. 4.

[0049] Device identification module 304 causes processor 202 to store connector information including a pre-defined physical location of each of connectors 108 relative to the device (e.g. x- y parameters indicating location; general location parameters-TOP, BOTTOM, RIGHT, and LEFT) within the memory 212. The pre-defined physical location of each of the connectors may be defined upon fabrication and/or programming of the device 100 and/or connectors 108.

[0050] Device identification module 304 further maintains and/or updates device information including for example, the type of connectors 108; potential types of devices that can be coupled to each connector 108 (e.g. smartphone, peripheral devices, etc.) within memory 212. The relative physical location of each connector 108 is typically known with reference to a coordinate system attached to device 100 (e.g. in mm from a defined corner). Examples of connector information indicating relative location of connectors 108 is also shown in data store 312 of FIG. 4.

[0051] Additionally, in one embodiment, device identification module 304 further includes device information, such as but not limited to: size of device 100 (e.g. 100 mm x 200 mm), type of device (e.g. model), display 110 characteristics (e.g. pixel size, pixel color depth, pitch, etc.) and other device information that may be used to derive spatial information. In another exemplary embodiment, device identification module 304 further includes information about the location of touch sensors on device 100 (e.g. relative to the device's coordinate system). The device information may be stored in the memory 212. The location information of the touch sensors) which may be pre-defined (e.g. upon fabrication and/or programming of the device 100) and stored within memory 212.

[0052] Thus, based on connector information provided by device identification module 304 (e.g. connector locations on the device), device type, device size, and touch screen

information), connectivity module 302 can determine the relative spatial location of each of the other devices interconnected to mobile device 100. In the example configuration of FIG.1 , connectivity module 302 indicates, by way of the information in data store 312 shown in FIG. 4 that interconnected device 102 is located on the right side of device 100. Additional information (e.g. device type, device size, and touch screen information) can be used by connectivity module 302 to further refine the determined relative spatial location of each of the other devices interconnected to mobile device 100 and for use by software applications of the devices for processing input/output display operations (e.g. determining merging of the multiple display screens for screen stitching).

[0053] Based on connectivity information, device 100 can determine the relative spatial location of one or more interconnected devices (e.g. device 102 is connected on the right side of device 100). [0054] As well, once devices 100 and 102 are interconnected with one another, devices 100 and 102 can communicate with one another and exchange information as needed over a communication channel established between the devices - for example by way of a USB or other connection between device 100 and 102.

[0055] In one embodiment, once a connection is established between respective connectors 108, 112 of devices 100, 102 and the relative spatial location of the interconnected mobile device 102 is determined relative to the mobile device 100, the spatial awareness may be used to facilitate and/or verify cross-device interaction.

[0056] Examples of cross-device interaction can include, providing access to computing resources at one device to the other (e.g. memory; screen; speaker; or other computing and/or peripheral resources); detecting an input gesture spanning across both devices 100, 102;

stitching together displays 110 to create a larger display for rendering graphics (e.g. that allows device 100 to render graphics on the displays 110); etc.

[0057] Once two devices are proximate each other, cross-device interaction may be detected and verified. As required, cross-device interaction and communication between devices may be effected with aid of spatial information in data store 312. Cross-device interaction may be verified by cross-device communication module 316.

[0058] As illustrated in FIG. 5 cross-device interaction may be initiated by device 100, and responded to by device 102, as illustrated in FIG. 6.

[0059] As illustrated device 100 may initially detect an event that may qualify as a request for cross-device interaction in block S502. As will be appreciated, not all events detected at device 100 will be events indicating cross-device interaction. Suitable event types that may, however, be identified as events that may initiate cross-device interaction may be stored within memory 212.

[0060] Such event may take any number of forms. For example, the event may be a gesture or input at screen 110 of device 100. The event may have particular characteristics indicative of cross-device interaction. For example, the event may be an input at a particular input location, a gesture having a particularly starting location, path, or ending location, a gesture path, or press of a dedicated "pairing" button that may be on display 110 or elsewhere on device 100 (e.g. on a bezel surrounding display 110 or on the housing . Input locations may correspond to a particular icon, button, region, etc. of device 100. [0061] The desired cross-device interaction may include pairing initiating device 100 with device 102; granting access to a resource at device 100 to device 102 (e.g. memory; a peripheral (camera, speaker, etc.); etc.); requesting access to device 102; transferring a file from device 100 to device 102; or transferring power from device 102 to device 100; and/or sensing cross-device gestures. Numerous other cross-device interactions will be appreciated by those of ordinary skill.

[0062] As noted, the event may be a gesture detected in block S502. For example, a swipe gesture may be detected as a first detection of a touch caused by an implement such as a finger, stylus, or the like touching down on touch screen 110 of device 100. The gesture may continue, without lifting the implement, with the implement pulled across touch screen 110 in contact therewith, thereby tracing a path across touch screen 110 before being lifted off touch screen 110 at a second point, the lift off may also be part of the gesture - and detected.

Processor(s) 202 may receive indications of all of these events such as, for example, over a bus from display 110. In some embodiments, multiple indications may be received or generated corresponding to each event. For example, a gesture may result in a series of touch events, each of which may be detected, and the collection of touch events, where appropriate, may be interpreted as a gesture. Multiple touch events may result in the generation of a message indicative of the gesture.

[0063] An example cross-device gesture G across display 110 of device 100 and display 110 of device 102 is depicted in FIG. 1. At device 100, the cross-device gesture may be characterized by an origination point somewhere on display 110 of device 100 and a termination point on display 110 of device 100, near an edge interconnecting at device 102. At device 102, the cross-device gesture may be characterized by an origination point proximate an edge of display 110 of device 102 interconnected with device 100, and a termination point somewhere on device 102.

[0064] In blocks S504-S506, cross-device communication component 316 at device 100 may verify that the event is consistent with the spatial relationship of devices 100 and 102 as maintained in data store 312, to verify the cross-device interaction.

[0065] Specifically, in block S504 cross-device communication module 316 at device 100 may adapt processor 202 of device 100 to assess the expected spatial location of a receiving device (e.g. device 102) for the particular event, by retrieving expected spatial location data corresponding to the cross-device request. Prescribed expected spatial location data for each expected cross-device interaction may also be stored within memory 212. [0066] In block S506, cross-device communication module 316 at device 100 may confirm that another receiving device 102 is at a prescribed expected location for a given cross-device interaction, consistent with the event detected in block S502.

[0067] For example, to the extent that the receiving device 102 merely needs to be interconnected at one of edges of device 100, device 100 may determine that mere

interconnection of device 102 is required in block S504. Next, in block S506, device 100 using data store 312 may determine if device 102 is interconnected with one of connectors 108 of device 100.

[0068] If device 102 is expected to be in a particular position relative to device 100, this too may be verified in block S506. For example, if the imputed interaction with device 102 requires device 100 to be to the left of device 102, as determined in block S504, the relative spatial position of device 102 to the right of device 100 may be verified in block S506.

[0069] The spatial relationship of interconnected device 102 may be determined by examining data store 312 for each request, or in advance for multiple subsequent requests. Alternatively, or additionally, data store 312 may be updated with each request by performing the methods set out in US Patent Application No. 15/013,750.

[0070] Optionally, initiating device 100 may determine that a specific device or a specific type of device (based on an identifier) is located at a desired location. To that end, the spatial interrelationship of another device, including a device type may be maintained in memory 212 for each cross-device interaction and/or associated event - e.g., verification data for a request for a cross-device interaction may include an identifier of a desired responding device 102.

[0071] For example, a touch input or gesture spanning two devices 100, 102 i.e., beginning at first device 100 and terminating at second device 102, may be verified as follows. The touch input may represent, for example, a user request to initiate communication from device 100 (initiating device) to device 102 (responding device).

[0072] At device 100 the gesture starts and results in lift off at device 100 proximate an edge of device 100. Device 100 may treat this gesture as an event representative of a possible request for cross-device interaction in block S502. Device 100 may retrieve verification data for this gesture in block S504. This verification data may include data identifying that a device 102 is expected to be at that edge of device 100. The verification data may further required device 102 to have a touch sensitive interface. [0073] If device 100 determines (e.g. in block S506) that no device 102 exists at this edge, then processing of the user input/gesture as a cross-device input may terminate in block S506 and onward. Optionally, initiating device 100 may thereafter attempt to process the user input/gesture as a single-device input, local to device 100, rather than a cross-device input/gesture at block S512. Such processing may include interpreting the gesture as a single device gesture at device 100 and processing it accordingly, or presenting a user interface notifying a user that suspected cross-device interaction has been detected without a second device spatially located as required or expected.

[0074] If the cross-device interaction request is verified in block S506, a record of the request may also be stored at initiating device 100 for retrieval by device 102 in block S508.

[0075] To that end, device 100 may maintain a further data structure 314 reflecting each request for cross-device interaction. Data structure 314 may, for example, have the following form:

[0076] Other fields may be included in structure 314, and will be appreciated to those of ordinary skill. [0077] Each request for cross-device interaction may be stored in structure 314, each time block S500 are performed. Multiple requests may be queued for a given responding device 102; or multiple responding devices.

[0078] Alternatively, instead of queuing requests, initiating device 100 may transmit (push) requests to the responding device (e.g. device 100) immediately, without queuing. This may possibly be accomplished using an established communication channel between device 100 and device 102.

[0079] To the extent required at device 100, the verified request for cross-device interaction may be further processed in block S510. For example, if the request is a request to pair device 100 with device 102, pairing may be performed in block S512.

[0080] Steps performed under control of cross-device communication module 316 (or similar software) at responding device 102 are illustrated in FIG. 6. Device 102 may detect or impute cross-device interaction in block S602. For example, device 102 under software control may receive a message from device 100 indicative of a request for cross-device interaction, originated for example in block S510 at device 100.

[0081] Alternative, device 102 may also detect an event that may be used to deduce a request for cross-device interaction, initiated at another device (e.g. device 100). Such an event typically follows a corresponding event at device 100. For example, the event at device 102 may be a second portion of a cross-device gesture, detected at device 102. Such a cross- device gesture may be detected by detecting a gesture commencing (rather than ending) at an edge of display 110 of device 102. Again, suitable event types that may be identified as events that correspond to cross-device interaction initiated at another device may be stored within memory 212.

[0082] In block S604 cross-device verification data may be retrieved, in much the same way as cross-device verification data is retrieved in block S504. In particular, relative expected spatial information of an initiating device (e.g. device 100) may be retrieved. Again, cross- device verification data may be stored corresponding to each request for cross-device interaction that corresponds to a request initiated at another device to be recognized at device 102.

[0083] The spatial location of originating device 100 may verified in block S606, by comparing the expected spatial information retrieved in block S604 with spatial information maintained in data store 312 of device 102. As explained, this relative spatial information may be or have been determined as described US Patent application No. 15/013,750. This allows device 102 to confirm that an initiating device 100 is positioned at appropriate location relative to device 102 in block S606.

[0084] If the responding device 102 determines that no device exists at the appropriate location in block S606, then processing of the event as a cross-device event at device 102 terminates. Optionally, responding device 102 may attempt to process the event (e.g. user input) as a single-device input at device 102 block S608 and onward rather than a cross-device input.

[0085] If, however, the event is verified as a request for cross-device interaction, in block S606 device 102 may retrieve any queued requests for device 102 from device 100.

Responding device 102 may further contact initiating device 100 in block S610 to determine if there are any additional requests are queued for it, for example in data structure 314.

[0086] Alternatively, queued requests may be pushed to responding device 102. Requests may be queued first in, first out, or in using any suitable queuing scheme. Requests may be matched to responding device based on an Edge ID and/or Responding Device ID.

[0087] Each request, as retrieved from the initiating device 100 may then be processed at the responding device 102 in block S612 and onward.

[0088] As will be appreciated each device 100 and 102 may optionally act as both initiating device and responding device.

[0089] The above method is particularly useful for pairing devices - such as devices 100 and 102. In the USB context, pairing refers to establishing a host-slave relationship between the two devices. Either a USB slave or a USB host device may avoid USB handshaking until it receives a request as described above.

[0090] Further, user input may define whether each device takes on a host role or a slave role. For example, the direction of a user gesture spanning two devices may define which device serves as a USB host and which device serves as a USB slave.

[0091] In particular, the method may be used to pair two devices 100 and 102 only upon verifying co-location (by verifying that the devices are physically juxtaposed) and sensing an event indicative of user intent to pair (by acting in response to user input). [0092] In some embodiments, one or both of the initiating device 100 and responding device 102 may have no touch screen interface, and only present a single push button (e.g., a "sync" button or a "pair" button) to initiate or accept a request.

[0093] In some embodiments, receiving device 102 may not have any user interface. In this case, receiving device 102 may periodically check for interconnected devices and poll interconnected devices for queued requests.

[0094] For example, initiating device 100 may be a smartphone and the receiving device 102 may be an integral circuit controlled battery pack. A user may input a gesture extending from the smartphone (device 100) to the battery pack (device 102), the gesture representing a request to draw power from the battery pack. The battery pack may not have any user interface.

[0095] It will also be appreciated that the examples and corresponding diagrams used herein are for illustrative purposes only. Different configurations and terminology can be used without departing from the principles expressed herein. For instance, components and modules can be added, deleted, modified, or arranged with differing connections without departing from these principles.

[0096] The steps or operations in the flow charts and diagrams described herein are just for example. There may be many variations to these steps or operations without departing from the invention or inventions. For example, blocks may be performed in a differing order, or steps may be added, deleted, or modified.

[0097] Although the above has been described with reference to certain specific examples, various modifications thereof will be apparent to those skilled in the art.

Claims

WHAT IS CLAIMED IS:

1 . A mobile device comprising: a plurality of connectors each for interconnecting said device with at least one of a plurality of other devices, each of the plurality of connectors located in a defined location on the mobile device; a processor and a memory, said memory storing a data store that indicates connectivity information for each of the other devices that are interconnected to the mobile device including a relative spatial location of each of these other devices relative to the mobile device based on the defined location of each of the plurality of connectors; and said memory further storing processor executable instructions that when executed cause said processor to identify an event for cross-device interaction with another device, and in response verify from said data store that said another device is in an expected relative spatial location to said mobile device.

2. The mobile device of claim 1 , wherein said event comprises an event identifying a request for cross-device interaction initiated at said mobile device.

3. The mobile device of claim 1 , wherein each of said connectors comprises a

magnetic connector for magnetically interconnecting said device to one of said plurality of other devices.

4. The mobile device of claim 2, wherein said memory further stores expected

connectivity information for each request for cross-device interaction.

5. The mobile device of claim 2, wherein said memory further stores processor executable instructions that when executed cause said processor to, in response to verifying a request for cross-device interaction, process said request as a cross-device request at said mobile device.

6. The mobile device of claim 2, wherein said memory further stores processor executable instructions that when executed cause said processor to, in response to failing to verify a request for cross-device interaction, process said request as a request local to said mobile device.

7. The mobile device of claim 2, wherein said request comprises at least one of a cross-device gesture, a request to pair said mobile device with said another device, and a request to use resources at said another device.

8. The mobile device of claim 2, further wherein said processor executable

instructions when executed further cause said processor when executed to queue said request in said memory.

9. The mobile device of claim 2, wherein said processor executable instructions when executed further cause said processor when executed to provide said request to said another device.

10. The mobile device of claim 2, wherein said processor executable instructions when executed further cause said processor when executed to provide said request to said another device from a queue in said memory.

1 1 . The mobile device of claim 2, further comprising a touch surface, and wherein said request comprises a touch gesture on said touch surface terminating proximate an edge of said mobile device, and wherein said processor executable instructions when executed further cause said processor to verify that said another device is connected at said edge.

12. The mobile device of claim 2, wherein said memory further stores processor executable instructions to verify that a specific device or a specific type of device is at said expected relative spatial location to said mobile device.

13. The mobile device of claim 1 , wherein said event follows an event identifying a request for cross-device interaction initiated at said another mobile device.

14. The mobile device of claim 13, wherein said processor executable instructions when executed further cause said processor to retrieve said request from said another mobile device.

15. The mobile device of claim 13, wherein said processor executable instructions when executed further cause said processor when executed to verify that said another device is at a defined physical location.

16. The mobile device of claim 15, wherein said mobile device comprises a touch surface, and wherein said request comprises a touch gesture on said touch surface commencing proximate an edge of said mobile device, and wherein said processor executable instructions when executed further cause said processor when executed to verify that said another device is connected at said edge.

17. A method of facilitating cross-device interaction at a mobile device that

comprises a plurality of connectors each for interconnecting said device with at least one of a plurality of other devices, each of the plurality of connectors located at a defined location on the mobile device, said method comprising: maintaining a data store that indicates connectivity information for each of the other devices that are interconnected to the mobile device including a relative spatial location of each of these other devices relative to the mobile device based on the defined location of each of the plurality of connectors; identifying an event for cross-device interaction with another device, and in response to said identifying said event for cross-device interaction, verifying from said data store that said another device is in an expected relative spatial location to said mobile device.

18. The method of claim 17, wherein said event comprises an event identifying a request for cross-device interaction initiated at said mobile device.

19. The method of claim 18, wherein said request comprises at least one of a cross- device gesture, a request to pair said mobile device with said another device, and a request to use resources at said another device.

20. Computer readable memory storing processor executable instructions adapting a processor to perform the method of claim 17.