GB2499494A

GB2499494A - Distributing Information Display

Info

Publication number: GB2499494A
Application number: GB1222763.3A
Authority: GB
Inventors: Jonathan Drazin
Original assignee: Individual
Current assignee: Individual
Priority date: 2011-12-19
Filing date: 2012-12-18
Publication date: 2013-08-21
Also published as: GB201121796D0; GB201222763D0

Abstract

A network-connected master device (14) (e.g. smartphone, tablet) is able to discover slave display devices (13) (e.g. TV, STB) coupled to the network and is further able to upload items of primary information including a link to secondary information across the network. The device includes a screen representing the items to a user (63) and provides a user input which upon receiving a selection of one of the items causes the display of the selection on the discovered display(s) using the link. Slaves may acquire keys from servers across the Internet for multiple applications located on the master. During a session a master displays primary information to a user who selects an item of secondary information for display by the slave, whereupon the master sends a command and a link to the item to the slave for display and sharing with multiple viewers.

Description

1

A System for Distributing Display of Information

Background of the Invention

The emergence of Internet connected televisions (TVs) and set-top-boxes (STBs) in recent years has created new opportunities for users to interact and benefit from TV.

5 One field is electronic retailing of products to consumers via TV. However, retailing via TV is less popular with consumers compared to via alternative media such as personal computers (PCs) and personal networked devices (PNDs) such as smart phones and tablets. TVs make comparatively poor retailing devices because they are difficult to interact with and are not perceived to afford sufficient privacy for 10 transactions due to their shared screens and common use. Further, the accessibility and flexibility of PNDs make them increasingly preferred devices for TV remote control over conventional infra-red remote control units (RCUs). TV remote control applications for PNDs such as the Google TV remote application already allow users to control their TVs across a local area network (LAN).

15 Users describe a number of problems when they use PNDs as retailing devices. Smart phones are often too small to view a product clearly. The same can be true of tablets when it is considered that retail applications need often to display simultaneously a product description and illustrations for user convenience. Some purchases require a consensus between persons so that PND users may want to display products of interest 20 to other users prior to making a purchase, but shared viewing on PNDs is difficult due to their limited size.

A hybrid system of TV and PND is desired where some classes of product information (e.g. photographs and text descriptions) are available for shared viewing while other information is viewed privately (e.g. price, terms of payment and transaction security).

25 For the foregoing reasons, it would be desirable for a PND to become a "master"

device to control a second "slave" device such as a TV or other appliance that may display products or product information. It is further desired that the master should quickly select and cause the most proximate device to become a slave device without

2

requiring the user to have to make a decision unnecessarily over the preferred display device or express the device's selection to the master. A system is further desired where the master displays an item of product information (e.g. a product catalogue or partial description) that causes related items to be displayed on the slave. For example, 5 a user may search for an item of interest in the product catalogue displayed on the master which in turn causes certain related information to be displayed on the slave.

New technologies are emerging that ease interaction between TV clients and controlling devices. Systems based on protocols from the RVU Alliance, for example, transform the master controlling device into a thin client server requiring 10 interoperability with TVs and serving devices such as digital video recorders (DTRs)

and network attached storage (NAS) devices. However, thin client protocols can require significant computational power to generate the client display, encode it in real time and stream it. Thin client servers are as a consequence not easily hosted within PNDs and may introduce unacceptable battery drain. It is consequently desired that 15 the master should instead serve links to the slave and that the master should control the slave to play the links in time synchronisation with processes in the master.

It is further desired that a buyer should be able to share product information readily with friends and family, and where an administrator of a household or location should be able to transmit configuration information to other users of PNDs in a quick, 20 simple and secure way that does not require users to recall or key information manually. The Universal Plug and Play (UPnP) open architecture protocol has further provided a framework for home appliances to be linked on a local area network without manual network configuration where devices can be configured as slaves for the purpose of rendering content. However, several TVs may exist on a home network 25 and it is desired that masters should discriminate and authenticate themselves to the correct slave quickly with minimum involvement from the user. It is necessary therefore to overcome a difficulty of registering a master to a particular network used by a slave. Users commonly experience difficulty with wireless networks, identifying the desired network identity (SSID), security scheme (e.g. WEP, WPA) and security 30 keys or passphrases because they may not be readily to hand. Even when they are available, difficulties are experienced keying in the necessary access information.

3

Summary of the invention

According to one aspect of the invention, there is provided a handheld device comprising a screen, a user input device, means for coupling to a network, means for discovering one or a plurality of display devices that are coupled to the network, 5 means for uploading one or a plurality of items of primary information for display on the screen and comprising a link to secondary information, means to represent portions of at least one of the items on the screen, means for the user input device to receive from the user of the handheld device a selection of at least one of the display devices and means for receiving an instruction to display the selection from the user 10 input device.

The input device may be a touch sensitive display or an arrangement of physical keys. The instruction may be a selection of a touch sensitive cell or a key or a combination of keys that denote a display action.

The display devices may be discovered through communication with the handheld 15 devices across the network. The network may be a wireless communication network.

The network may be coupled to the Internet.

According to another aspect of the invention the master handheld device is adapted to indicate a presence of a display device conditionally upon its availability to the handheld device and comprises means to send the link across the network to the 20 display device responsive to the instruction. The presence may indicated by displaying a message on the screen.

According to a further aspect of the invention the link may comprise a command for processing by the display device.

According to a further aspect of the invention the link is sent to the display device 25 conditionally upon availability of the discovered display device.

According to a further aspect of the invention means are provided for determining whether a discovered display device is in proximity by detecting a broadcast signal from the display device. The signal may be a radio (i.e. electromagnetic) signal. Alternatively the signal may be acoustic (i.e. propagated over air). The handheld 30 device may be adapted to recover the identifiers of the display devices or a message

4

from the broadcasts. The handheld device may be adapted to decode the broadcasts using a near field communication protocol.

According to yet another aspect of the invention there is provided means for the handheld device to send the link to a discovered display device that is in proximity.

5 According to yet another aspect of the invention there is provided means for displaying a menu including identifiers of the discovered display devices conditionally upon determining that none of the discovered display devices is in proximity, means for receiving a selection of one of the displayed identifiers from the user input device and means for sending the link to the display device corresponding to the selected 10 identifier.

According to yet another aspect of the invention there are provided means for the handheld device to upload a key or a certificate across the Internet and means for authenticating the key to the discovered display devices. The handheld device may upload multiple keys or certificates, each key or certificate corresponding to an 15 application that is stored on the handheld device. According to the same aspect there are provided means for the display devices to upload a key or a certificate across the Internet. The keys or certificates may be uploaded to the respective handheld devices and display devices in real time during sessions with a user, or they may be uploaded periodically prior to sessions.

20 According to yet another aspect of the invention there are provided means for displaying a graphic user interface to the screen conditionally upon detecting a presence of input tags in the secondary information, means for receiving touch inputs from the graphic user interface and means for sending data corresponding to the touch inputs to the display device.

25 Brief Description of the Drawings

Various aspects of the invention will now be described by way of example only and with reference to the accompanying drawings, where:

• Figure 1 is a block diagram of the system of the invention.

• Figure 2 is an illustration of an exemplary master device.

5

• Figure 3 is a block diagram of a client application and a retail application within an exemplary system architecture on a master device.

• Figure 4 shows a block diagram of an exemplary master device.

• Figure 5 is a block diagram of a client application coupled to a browser within 5 an exemplary system architecture on a slave device.

• Figure 6 is a block diagram of an administrator registering a TV slave to a server according to step 9-2 as later described.

• Figure 7 is a block diagram of an administrator registering a STB slave to a server according to step 9-2 as later described.

10 • Figure 8 shows a user pairing a master with a TV slave in near field according to step 9-7 as later described.

• Figure 9 shows the overall steps in a process to engage in a master - slave session.

• Figure 10 is a data flow diagram of an administrator configuring a TV slave to

15 a server according to step 9-2 as later described.

• Figure 11 is an exemplary illustration of a scene on a user's master device according to step 9-4 as later described.

• Figure 12 is a data flow diagram for an administrator configuring a slave via a master according to step 9-4 as later described.

20 • Figure 13 is a data flow diagram for an administrator granting a certificate to a user according to step 9-6 as later described.

• Figure 14 is a block diagram of an administrator granting certificates to a user according to step 9-6 as later described.

• Figure 15 is an exemplary scene on an administrator's master's screen when

25 granting a certificate to a user according to step 9-6 as later described.

• Figure 16 is an exemplary scene on a user's master's screen when it is receiving a certificate.

• Figure 17 is an exemplary data table compiled and stored within a master device.

30 • Figure 18 is a flow chart showing the steps of discovering a slave and authenticating master according to step 9-7 as later described.

6

• Figure 19 is a data flow diagram for a user pairing a master with a slave in near field proximity according to step 9-7 as later described.

• Figure 20 shows an exemplary scene on a user's master's screen when selecting a slave for pairing in alternative step 18-1 A.

5 • Figure 21 shows the composition of metadata that describe a scene displayed by retail application on a master that references secondary information for display by a slave.

• Figure 22 is an exemplary scene on a user's master's screen displayed by a retail application when about to be paired .

10 • Figure 23 shows steps in the operation of a master - slave retail session.

• Figure 24 is a data flow diagram for operation of a master - slave session across a LAN according to the sub-steps of 9-9 as later described.

• Figure 25 is an exemplary scene on a slave's display according to step 9-8 as later described.

15 • Figure 26 is an exemplary scene of a slave control user interface with arrow navigation displayed to master screen during step 9-8.

• Figure 27 is an exemplary scene of a slave control user interface with touch and stroke navigation displayed to master screen during step 9-8.

System of the invention

20 Figure 1 shows a system of the invention that comprises one or a plurality of home or office local area networks (LANs) 10 connected to the Internet 11. Each LAN may comprise one or a plurality of routers or hubs 16 and employ wired (e.g. Ethernet, mains such as defined by the Homeplug Power Alliance) or wireless (e.g. as defined by IEEE 802.11, Bluetooth, Zigbee standards) methods.

25 Each location 10 may be connected via a gateway or router 18 to the Internet and may contain two classes of networked device: master devices 12 and slave devices 13. One or a plurality of master devices 14 may be assigned to administrators. Similarly, one or a plurality of master devices 15 may be assigned to users. An administrator's master device 14 and a user's master device 15 may further be collocated in a

30 common device. All devices may enter into bi-directional communication with entities in the Internet 11.

7

Preferably masters 12 are connected to LAN 10 via router 18 using wireless means, such as Wi-Fi (IEEE 802.11) or Bluetooth. Similarly, slave 13 may be connected to LAN 10 wirelessly. Masters 12 and slaves 13 may also be adapted for Internet communication via the wide area public mobile telecommunications wireless 5 networks (WANs). Master and slave devices each comprise one or a plurality of processors, and one or a plurality of data storage units to store and execute applications, store and retrieve the results of intermediate steps, in order to perform the steps of this invention. Preferably the application software are loaded by the user to the device from a third party in the Internet or, alternatively, an application or a 10 portion of it may be preloaded into the master device during manufacture. Master 12

and slave 13 may communicate with an administration server 17 via the Internet 11. Master 12 and slave 13 may also communicate with one or a plurality of retail servers 19 via the Internet 11.

Figure 2 shows master 12 which is preferably portable, handheld and battery powered. 15 Examples of masters include smart phones (e.g. Apple's iPhones and Android phones), tablet computers (e.g. Apple's iPads and Samsung's Galaxy Tab). Master's computer may receive user input from an arrangement of keys 20, communicate across WAN or LAN using a wireless transceiver 21 and output graphics to an attached screen 22 which may be touch sensitive. Master may have a bi-directional, 20 near field transceiver 23 comprising a data transmitting device 24 and a data receiving device 25.

Masters 12 may incorporate a standard software architecture 30 such as the Android System Architecture shown in Figure 3 and familiar to skilled persons. The control processes described are implemented by an application layer 31 comprising a master 25 client application 32 that interoperates with one or a plurality of retail applications that may originate from third parties 33 and cause display of product and/or service item information content on master screen 22 and other or same content on a slave's screen as described later. Examples of retail applications 33 include applications that display a catalogue of products on master screen and illustrate product content with 30 photographs and audio-video clips on a slave screen.

8

Retail application 33 and additional applications such as a browser 34 are layered over the Android Application Framework 35, libraries 36 and Linux kernel and device drivers 37. Master client 32 is abstracted from the hardware and link layers and receives data inputs 39 and outputs 38 through the layered architecture 35, 36 and 37.

5 Client application 33 forwards universal resource locations (URLs or "links") to master client 32 as inter-process communications 33A via Linux kernel 37. Master client 32 is booted immediately each time the master is powered up and reloads the certificates (described later) from non-volatile memory so as to be available to retail application 33.

10 Figure 4 is a block diagram of the functions within an exemplary master 12 which in the embodiment described is a tablet computer containing an Exynos 510 audio video decoder comprising computer processor (ARM Cortex A9) 40 and OpenGL 2.0ES display processor coupled to a 10.1 inch 1920x1200 pixels colour LCD panel 41 with touch sensitive screen 42, 1 GByte volatile DRAM (dynamic random access memory) 15 and 512 Mbyte non-volatile NAND flash memory 43, IEEE802.1 In wireless network adaptor 44, loudspeaker and microphone 45 arranged to operate as near field transceiver 46 coupled via a data bus 47. Non-volatile portion of memory 43 is programmed to contain firmware for the software sub-system 30 which is loaded into volatile portion of memory 43 and booted into volatile portion of memory 43 on 20 power up.

Figure 5 shows the embodiment to be described, where slaves 13 is an Android TV that incorporates the Android System Architecture 59 including WebKit browser and Chrome V8 JavaScript engine 50. Preferably, browser 50 is adapted to support the public CE-HTML and European Hybrid Broadcast Broadband TV (HbbTV) browser 25 specifications so as to render HTML pages to TV screens predictably.

The control processes described are implemented by a slave client application 51 interoperating with browser application 50 in application layer 52 over the Android Application Framework 53, libraries 54 and Linux kernel operating system and device drivers 55. Slave client 51 is abstracted from the hardware and link layers and receives 30 data inputs 56 and outputs data 57 through the layered architecture 53 thru 55. Slave client 51 forwards universal resource location (URL) references to browser 50 as

9

inter-process communications 58 via Linux kernel 55. Slave client 51 is booted immediately each time the slave is powered up and reloads administrators signature (to be described later) from non-volatile memory so as to be available to master client 32.

5 Slave 13 is a display device preferably adapted to play audio and video on a relatively large screen compared to the master screen 22 to be visible simultaneously to multiple viewers. Figure 6 shows a typical home where the slave is a TV or a TV projection system 13 with screen 60 and coupled 61 to a LAN 10.

There are two types of operators of the masters and slaves: administrators 63 and other 10 users. Administrators 63 may, in addition to other users, confer access certificates to other users according to processes described later.

Slave 13 may comprise a set-top-box (STB) 70 and TV coupled by an HDMI or SCART cable 71 as shown in Figure 7. Slave may be connected to LAN 10 and to a server 17 via the Internet.

15 Figure 8 shows master 12 and slave 13 adapted to communicate in the same proximity

(e.g. within a room) over a short range of a few metres in a near field bi-directional link 80 with each other through use of a slave data receiver 81 coupled to master's transmitter 24, and a slave transmitter 82 coupled to master's receiver 25. In the embodiments described, the near field transmission method is acoustic where 20 transmitting components 82 and 24 are loudspeakers and receiving components 81

and 25 are microphones where digital signal processing is used to embed a short data message within audio content and to recover it using methods known to skilled users (Cano et al, 2002). In alternative embodiments, the near field acoustic transmission is in the inaudible ultrasonic 20kHz to 50kHz frequency range. In other embodiments 25 the acoustic transmission is the audio content from the television, where the master device is communicatively coupled to the slave across the LAN to permit the master to identify the slave in proximity by recognising the audio content the slave is playing using audio content and fingerprint recognition techniques known to skilled persons. Acoustic transmission is preferred because such transmissions are generally limited to 30 within the room in which the user and master device are located.

10

■

Alternative arrangements for transmitter components 82 and 24 may comprise their respective host's built-in screen devices 22 and 60 to display data in formats such as barcodes or smart posters according to formats such as published by the Near Field Communication (NFC) Forum, and where 81 and 25 are cameras adapted 5 to receive and decode the same. In yet other embodiments, devices 81, 82, 25 and 24

may employ wireless data transmission methods such as Bluetooth, contactless card communication (e.g. ISO/IEC 14443 and FeliCa), infra-red (e.g. IrDA), RFID, wireless network (e.g. IEEE802.Ha/b/g/n) or ZigBee.

Process of the invention

10 Referring to Figure 9 the process of the invention divides into four phases:

1) Administrator Set-up comprises:

a. loading and initialisation of administrator's master client 32: step 9-1;

b. registration of home configuration with server and generation of a signature to be applied in the later phases: step 9-2;

15 2) Slave Set-up comprises:

a. initialisation of slaves: step 9-3;

b. configuration of slaves to hold the administrator's signature and access the network: step 9-4;

3) User/master Setup comprises:

20 a. user loading master client application to his/her master: step 9-5;

b. grant of administrator's master grants certificates to users' masters: step 9-6;

4) User Session comprises:

a. discovery of and authentication with a slave ("pairing"): step 9-7; 25 b. operation of retail application on user's master: step 9-8;

c. interaction between master and slave: step 9-9.

Each phase is described below.

Initialisation of administrator's master client 32: step 9-1

11

Master client 32 is loaded to administrator's master 14 (e.g. by downloading from Android Market in the case of Android tablets or smart phones or pre-installing the application in the factory).

Registration: step 9-2

5 At least one user is an administrator 63 of masters 12, slaves 13 and LAN 10. The administrator registers his/her details with a server 17 via his/her master 14 or another networked device (e.g. personal computer). Figure 10 shows the interactions between administrator 63, master 14 and remote server 17 in more detail. Administrator 63 keys his/her personal account name A,\dmin. password PAdmin, network name, 10 description and comments N\dmin, and access keys Lr, for each wireless network r

(steps 10-1, 10-2, 10-3 and 10-4). Server stores A^min , PAdmin, NAdmin and Lj in nonvolatile memory for later reference (step 10-5).

Administrator configures the slaves to recognise certificates later presented to them by authorised users' masters in step 9-7 described later by initialising each slave with a 15 signature, R, generated as a hash function H() of administrator's password PAdmin and where in this embodiment H() is the 128-bit MD2 algorithm published as RFC 1319.

Administrator's master 14 generates R and erases PAdmin (step 10-6) to prevent illicit recovery of PAdmin from master device if stolen. AAdmin , R and L- are stored in master 14 in non-volatile memory for later reference (step 10-7).

20 Figure 11 shows how master client 32 displays information 113 to administrator via screen 22 during configuration process 9-2. In the embodiment described, master client application 32 receives touch selection from administrator of labelled cells 110 via screen 22 causing selected cells 111 to be marked or highlighted differently (shown as shaded fill in Figure 11).

25 Master client 32 causes a soft keyboard 112 to be scrolled onto scene 113 when character values are to be input from administrator causing them to be displayed simultaneously in a labelled 114 non-touch sensitive cell 115 and keyboard 112 is withdrawn when a delimiting key such as return is touched.

Administrator may periodically repeat step 9-2 throughout the process of this 30 invention to maintain configuration details.

12

Initialise slaves: step 9-3

Administrator 63 prepares each slave 13 by loading first slave client 51 to each device (e.g. by downloading from Android Market in the case of Android tablets or smart phones or by pre-installing in the factory).

5 Configuration of slaves: step 9-4

Administrator configures each slave according to step 9-4 whose sub steps are shown in Figure 12 in further detail and described as follows. Administrator logs onto to server 17 via his/her slave 13 to enter his/her account name AAdmin and PAdmin (steps 12-1 and 12-2). Server sends to administrator the list of current networks NAdmin 10 (steps 12-3 and 12-4). Master client displays a column list of action options 116 which includes an option to pair to slave (shown as "Connect" in example of Figure 11), an option to grant certificates (shown as "Give access" in figure) according to step 9-6 to be described later, and an option (shown as "Manage TVs" in figure) to configure slaves according to this step 9-4 currently described. Master client 32 displays the list 15 of network names in column 117 and detects administrator's selection of a particular network (shown as "Home" in example of Figure 11) r, from the list. Client 32 receives a further selection from administrator to configure a new slave (shown as "Add new TV" in figure) whereupon administrator is prompted 114 and enters a particular slave's 13 name, S, via soft keyboard 112 or keys 20 (step 12-5). S is a 20 unique text hostname referenced by addressing devices to resolve a slave's internet protocol (IP) address, r and S are backed up from slave to server (step 12-6).

Server additionally returns the access information Lr for network r to master 14 (step 12-7). Slave is continuously in a receive state to detect near field commands from masters 12 (step 12-8) and is adapted to be configured from master 14 directly as 25 follows. Master 14 broadcasts in near field a command, ConfigureSlave, with argument values for hash key R, network access details Lr and slave identifier S in near field (step 12-9). Slave 13 receives and decodes the near field broadcast from master 14 to recover the command and arguments ConfigureSlave (R, Lr, S). Slave 13 may determine whether it has been previously configured with different argument 30 values L- Sand, if so, displays a warning to administrator. Slave attaches to LAN 10 using access details Lr via application framework 53 and verifies that it can ping

13

server 17 (step 12-10). Slave uploads from administration server 17 via LAN 10 and Internet 11a list of identities of retailers, RetailerList, for which user session processes 9-9 are later to be permitted (step 12-10A). In the embodiment described RetailerList is a text list of numeric identifiers, RetailerlD, of each retailer or retail 5 application 33. In other embodiments RetailerlD may further comprise authentication details, such as a key or certificate, which slave client application 51 uses to authenticate the retail application 33 during the later user session processes 9-9. Slave client saves RetailerList to non-volatile memory so that it can later be referenced during subsequent user sessions 9-9. In other embodiments, slave may periodically 10 download updated versions of RetailerList from administration server 17, e.g. every day or every week, so as to permit user sessions with new versions of retail application 33. Slave confirms to administrator via master that it is successfully configured (step 12-11). Thereafter, slave broadcasts continuously and repetitively every few seconds its hostname S in near field proximity throughout its remaining 15 operation (step 12-12). Administrator's master stores S and r to non-volatile memory for reference when administrator grants a user certificate as described later (step 12-13). Lr, Rand S are written to non-volatile memory in slave 13 for reference when paired with a user's master device as described later (step 12-14). Circumstances may arise where either a master or slave is not adapted for near field communication. In 20 such cases, steps 12-8, 12-9 and 12-11 may be replaced by a administrator manually keying R, Lr and S into slave as a menu setting using a conventional RCU.

Step 9-4 may be repeated so that a plurality of slaves are configured by exchanging administrator's signature R and slave identifiers S with administrator's master 14 and/or server 17.

25 User loads application to master: step 9-5

User ensures master client 32 is loaded to his/her master 15 as described previously for administrator's master 14 in step 9-1.

Administrator grants certificates to user: step 9-6

Administrator confers certificates in this step to allow users to control particular slaves 30 configured in step 9-4. A certificate is transferred from administrator to user as a

14

signature, Q. In the embodiment described, Q is a 16 byte word calculated as a hash function H() of three components: S, MUser and R.

Figure 13 shows the sub steps for granting a certificate to user's slave to be described. Administrator 63 cooperates with other user 83 request for a certificate (step 13-1) and 5 places transceiver 23 on his/her master 14 in a near field link 80 with transceiver 23 of other user's master 15 as shown in Figure 14. If required administrator wakes his/her master 14 from standby and starts client application 32 in master 14, step 13-2. Administrator logs into server 17 with his/her account name and password (steps 13-3 and 13-4) via his/her master and selects a cell whose action corresponds to granting of 10 a certificate as shown by the cell 150 labelled "Give access" in the example of

Figure 15. Administrator's master fetches the network list, N\dmjn, from the server,

step 13-5, and lists networks 117 for selection to scene 113 displayed on master's 14 screen 22, step 13-6. Administrator selects the cell 151 corresponding to LAN r, to which access is going to be given to user (labelled "Home" in the example of 15 Figure 15) causing a list 152 of one or more slave devices, S, attached to LAN r during step 9-4 to be displayed for selection of the desired device 153 (labelled "Bedroom TV" in the example of Figure 15) for which access is be granted (step 13-7). Master uploads the network access key L- and LAN hostname S from server if present (steps 13-8 and 13-9 respectively) and displays instructions for administrator 20 154 and a cell 155 to indicate the status of communication of certificate, Q, across link

80.

If required other user 83 wakes his/her master 15 from standby and starts master client 32 in his/her master 15 (step 13-10). Figure 16 shows exemplary scene 161 generated by master client 32 on other user's 83 master 13. User 83 selects an action 160 from a 25 list of actions 116 that corresponds to a request for access (step 13-11) and follows instructions 162. User's master 15 displays a cell 163 denoting status of link 80 and transmits a data token corresponding to a request for access RequestAccess, its unique numeric identifier MUser, user's text name TNuser, (such as "Freddy's tablet") several times per second in a data stream to master 14 in near field (step 13-12). 30 Administrator's master displays instructions in cell 156 for administrator to authorise transfer of certificate (labelled "Press to authorise Freddy" in example of Figure 15)

15

(step 13-13). Administrator confirms TNuser by touching cell 156 (step 13-14) causing master client in administrator's master 63 to calculate signature Q = H(S MUser - R) (step 13-15). Q and Lr, if present, are transmitted to user's master device 15 (step 13-16) and stored in non-volatile memory, 13-19. User's master updates cell 163 to 5 indicate that a certificate has been received (e.g. by displaying "Received") and returns a token, CertificateReceived, to administrator's master to confirm receipt of certificate (step 13-17). Administrator's master updates cell 155 to confirm that the certificate has been successfully sent and received by user (e.g. by displaying "Sent") and clears cell 156 (step 13-18).

10 Circumstances may arise where either administrator's or other user's master is not adapted for near field communication. In such cases, steps 13-12 to 13-16 may be replaced by master 15 client 32 transferring Mrser, TNuser to administrator master 14 client 32 via public gateway 18, Internet 11 and server 17, for administrator master client 32 to calculate Q according to step 13-15 and return Q, Lr and S to other user's 15 master 15 via the same route.

Step 9-6 may be repeated so that a user's master 15 may accumulate a plurality of certificates and network access details, L, in non-volatile memory for particular slaves, S, under one or more administrators' control. Referring to Figure 17, user's master client 32 adds a new record for the acquired certificate to a table 170 stored in 20 masters' non-volatile memory 43 each time step 9-6 is followed. Each record contains an index to the acquired certificate, the hostname S for the slave with which the certificate is associated and the index r to the LAN in which the slave is located. In the example of Figure 17, the first record refers to a certificate associated with a slave whose hostname is "HOME123 " on a LAN whose access details are Li.

25 Discovery and authentication: step 9-7

A master and slave are paired for a session when a master has discovered a slave and authenticated itself. A master may contain certificates for multiple slaves and it is required therefore that master should, when requested by retail application 33, select and authenticate itself with one of such slaves with a minimum of interaction from the 30 user. To prevent a master inadvertently controlling a slave that its user cannot view

(e.g. because the slave is in another room) it is further required that a master should

16

inform and present user with an option to use a slave for viewing a portion of the content displayed by retail application 33 conditionally upon a slave being in near proximity to the master. In the preferred embodiment, near field communication is employed to further filter out slaves on the same LAN but out of proximity. A slave 5 may be unavailable or occupied (e.g. because it is in a session with another user) and so it is further required that master should engage a slave conditionally on its availability. Master must further determine a slave from a possible plurality of slaves 13 for which master contains certificates.

The flow chart of Figure 18 shows the overall steps of discovery and authentication 10 and is described as follows. User starts retail application 33 (step 18-1) and application 33 detects execution of master client application 51 (e.g. by filtering for the process name in the output of the Linux ps command) (step 18-2). If client 32 is not found, retail application displays a message to inform that slave functionality is not available but may be installed for future sessions (Message A) band continues 15 without engaging a slave in steps 9-8 and 9-9 for the present session (step 18-6). If master client 51 is present, application 33 commences discovery of a slave according to step 18-3.

The communication diagram of Figure 19 show the process of step 18-13 in further detail specifically for a slave in near field and is described as follows. Client 51 sends 20 an inter process command to client 32 to discover slaves in near field (step 18-3A). As described for step 12-12, slaves 13 broadcast their hostnames S repetitively every few seconds to near field proximity. Master 12 listens for hostname S (step 18-3B) and resolves slave's network by looking up table 170 for the network identity r corresponding to S, configures wireless LAN transceiver 21 and attaches to LAN r 25 (step 18-3C). Master client sends a request to slave via the LAN to reserve a master-

slave session (step 18-3D). Slave client 51 receives the request and returns confirmation to master client if it is available (e.g. because a session is not currently reserved with another master) (step 18-3E).

Authentication step 18-4 follows whereby master verifies itself to slave as follows: 30 Master looks up within certificate table 170 for certificates, Qs, with slave's hostname S and transmits these with a command to request authentication with its identity MUser

17

(step 18-4A). Slave client 51 receives Qs and attempts to replicate the certificate by calculating Q' = H(S, Mrser. R) and comparing to Qs. Qs is accepted as genuine if Q' and Qs are identical and user session steps 9-8 and 9-9 are permitted (step 18-4B). Slave sends a service confirmation message to retail application via master (steps 18-5 4C and 18-4D) if a genuine certificate is found, causing retail application to display a message to inform user that master and slave are paired e.g. "Tap product to view on Bedroom TV" (Message B, Step 18-5).

A slave may not be detected automatically during near field discovery steps 18-3A to 18-3E but may nevertheless be accessible to the user (e.g. because of interference or 10 because either master or slave is not fitted with a near field transceiver). If a slave was not found in near field (i.e. not in proximity of a few metres), master client looks up slave host names in table 170 against the LAN to which it is currently attached and polls each slave to determine whether it is contactable and reserves them according to the steps 18-3D and 18-3E previously described (step 18-3F). Steps 18-6 is followed 15 as previously described if no slaves were found (step 18-7). Authentication (step 18-4)

follows if a single slave is found.

Figure 20 shows how, if more than one slave is discovered, retail application 33 displays a pop-up menu 200 comprising a list of the identifiers 201 of discovered slaves, inviting user to select a slave 202 for interactive session 9-9 to be described. A 20 cell 201 is displayed for each slave found where a cell corresponding to the slave selected by user during a previous execution of this step 18-3G may be highlighted differently 202 (step 18-3G). User selects a slave by tapping the cell corresponding to the desired slave 202 (step 18-3H) and authentication step 18-4 follows as previously described.

25 In an alternative embodiment, authentication step 18-4 is performed between master and each slave discovered on the network immediately prior to step 18-7 to eliminate slaves discovered on the network for which retail application 33 is not authenticated. In such case pop-up menu 200 appears only if more than one authenticated slave can be paired. Message B is otherwise displayed immediately.

30 Interaction with retail application: step 9-8

Figure 21 shows the composition of data 210 that describe scenes 220 to be rendered

18

and displayed by retail application 33 on master 15. Figure 22 shows an exemplary displayed scene 220 from a retail application 33 on master 15.

Data 210 are typically in extended mark-up language (XML) or hypertext mark-up language (HTML) data formats, and contain subsets 211 of data that each describe a 5 product item (rendered as a band 222 in scene 220). Each product item 211 may contain one or a plurality of data sets 212. Each set 212 describes a cell 226 in band 222. Set 212 may contain a link or URL reference 213 to metadata 219 to be rendered by slave browser 50 on screen 60 (to be described later) such as to data 214 that describes scene 250 to be displayed by browser 50 on display 60 (to be described 10 later). Furthermore data 214 may contain sets 215 that contain downstream links or references 217 to other scenes that user may select with browser 50.

Area 221 contains a column list of one or a plurality of sub-areas or horizontal bands 222 that each represent an item that satisfies a search word or phrase 223 keyed by the user into touch screen 22 and displayed in a second area 224. Each band 222 may 15 display a name or descriptive note 225 and one or a plurality of cells 226 that may each contain one or a combination of text 227A, photographs or illustrations 227B, or videos 227C. Each cell may be frozen, animated, flashing or playing in any combination. User may scroll band 222 horizontally to reveal other cells 226 by swiping band 222 by touch to left or right.

20 Cell 228A displays a label 228B containing Message A or Message B as described.

An icon 228C is displayed adjacent to or within each product band 222 conditionally upon step 9-9 being operable when a cell 226 is selected to indicate that the master is paired with a slave (i.e. the slave has been discovered, authenticated and selected) for interactive session 9-9 steps to be described. Message B and icon 228C indicate to the 25 user that one or more parts of item 222 may be displayed to the slave for session 9-9

and saves the user inadvertently selecting a cell 226 for which session 9-9 is unavailable. Retail application 33 preferably displays icons 228C conditionally upon discovering the slave 13, conditionally upon the presence of a URL 213 within the page displayed on the master 210 that references a page for display on the slave 219 30 and conditionally upon receiving a confirmation from slave client application 51 that application 33 is valid. Alternatively icons 228C may be displayed adjacent to or

19

overlaid upon or adjacent to each of individual cells 226 to denote that each is operable according to process 9-9, or the cells 226 may be rendered or highlighted differently compared to other cells to denote that they can be displayed to slave (e.g. displayed in a particular colour or border).

5 One of the sub-areas 222A may be highlighted differently compared to the other bands 222 to denote that it is in focus and is selected by the user.

Cell 229A displays transaction or purchase information, such as price or terms of business for the band highlighted 222A. Cell 229B contains descriptive notes for the band highlighted 222A. User may cause the focus (shown as reverse field in 10 Figure 22) to move to a different band 222 by touching different band with a finger,

causing the content of 229A and 229B to adjust accordingly.

Message B is displayed if a slave is paired, and user may tap one of the cells 226 to cause the paired slave to display the content featured in selected cell 226. User may scroll area 221 to reveal other bands 222 by swiping area 221 by touch up or down.

15 Interaction with slave: step 9-9

Figure 23 shows the sub-steps of the master-slave control session. Figure 24 shows the data flow between the entities. User gives an instruction to the master to send URL 213 by selecting cell 226 (e.g. by tapping the cell) (step 23-1) and retail application 33 forwards its identifier RetailerlD, the URL 213 that corresponds to the 20 selected cell 226 to the master client application 32 (step 23-2). In addition to the link (e.g. "www.acme-retail.com/products/dresses/pink-dress/front") URL 213 preferably comprises also a symbol corresponding to a command for slave client application 51 to interpret (e.g. "display") as an instruction to resolve and render the link on display 60. In the preferred embodiment described, URL 213 references content that 25 is hosted by and served from retail server 17 or elsewhere in the Internet. However, in alternative embodiments, URL 213 may reference content that is stored on master 15 and hosted by a server application (e.g. Mongoose or Apache) in layer 31. Alternatively URL 213 may reference content served from the Internet via a proxy server application located in layer 31 master 15. Master client application forwards 30 RetailerlD and URL to slave client application 51 (step 23-2A). Slave client application looks up RetailerlD to verify that it is present in the approved list of

20

retailers RetailerList uploaded during earlier step 12-10A (step 23-2B). Other embodiments of the invention may take additional steps to verify that retail application 33 issuing the URL is bona fide by authenticating RetailerlD as a valid key or certificate. If RetailerlD is valid, then slave client 51 returns a confirmation 5 message of validity to retail application 33 via master client application 32.

Slave client application 51 saves its operating state (e.g. currently viewed broadcast channel, position within a user interface such as a programme guide) and forwards the URL to slave browser 50 if retail application 33 is valid (steps 23-2C).

Slave browser 50 resolves and gets the resources referenced by URL 213 (step 23-10 3A), renders the page to scene to display 60 according to their type (step 23-3B), e.g. an HTML web page or scene or a still image or a video clip, shown in exemplary scene 250 of Figure 25. Master client application 32 highlights icon 228C associated with selected cell 226 as flashing or rotating during resolving and render steps 23-3A, B and C to notify user 83 that the cell has been selected and is being rendered or 15 played by the slave until a confirmation status is received from slave browser 50 that rendering or playing is complete. Scene 250 is rendered from page metadata 214 (referring to Figure 21) which may contain one or more URLs 217 that link to other pages or scenes 218 that may be rendered by slave. Page metadata 214 may contain one or more input tags 216 that correspond to visual fields where user can input 20 characters (e.g. to key in a name or a password). Retail application 33 determines whether URL 213 points directly or indirectly to input 216 tags either by reading a pre-processed metadata tag 212A for URL 213 within cell data 212 or by following link 213 and all downstream URLs (e.g. 217 in Figure 21) recursively to determine whether user input is needed to master 15 (step 23-4). If input is required retail 25 application 33 updates cell 228A with a message 228B inviting user to swap master scene for remote control of slave (e.g. "Tap here to input to Lounge TV"), step 23-5. User selects cell 228A by tapping it (step 23-6) to cause retail application 33 to request master client 32 (step 23-7) to display a remote control user interface scene 260 (step 23-8) to screen 22 shown in exemplary Figure 26. In alternative 30 embodiments retail application may swap scene automatically from 220 to 260

without prompting user. Master client forwards data corresponding to user inputs to

21

arrow cells 261 and OK cell 265 to slave browser (step 23-9). User may further input text data via soft keyboard 266.

Figure 27 shows an alternative arrangement of master control scene 260 where touch sensitive key cells 261, 262, 263, 264 and 265 are replaced by touch panel 270 and 5 arrow movement is interpreted by master client application framework 35 from finger swipes and taps across panel 270. Any combination of touch, swiping, tapping and input via physical keys is possible.

Slave browser renders link referenced by cell 226 to screen 60 as shown in exemplary scene 250 of Figure 25, marks links 251 (if any) and highlights one link 252 to the 10 user as in focus. User manipulates arrow keys 261, 262, 263 and 264 to move focus 252 and presses OK 265 to select link 252 where master client forwards user input data to browser (step 23-10) and browser renders accordingly (step 23-11). User presses cell 267 to terminate input (step 23-12) causing retail application 33 to swap display on screen 22 from scene 260 to scene 220 (step 23-13).

15 Masters and slaves may be composed of different architectures, application frameworks and operating systems and be interoperable according to the processes described. Other software architectures for 30 and 59 are possible. For example, various components of architectures 30 and 59 do not have to be separate processes but can be part of a single process performed by operating system, 37 or 55, or a 20 single application.

As described, multiple retail applications 33 may be installed to a master and control slaves via communication interface 33A with a master client application. Retail application 33 may be a browser adapted to parse meta elements in tags that characterise cells 206 according to the processes described.

25 In the embodiment described master presents certificates to specified slave to enable the functionality of steps 9-8 and 9-9. Other embodiments may convey certificates to enable different functionalities such as may be performed by other applications (e.g. access to slave's native features such as power on/off, change channel or access to the Internet 11 via browser 50).

22

As described for the preferred embodiment, role of server 17 is to back up information held on administrator's master 14 and restore portions back to administrator or other users' masters and slaves according to the process described. Alternative embodiments may incorporate some or all of the functionality of server 17 into 5 administrator's master 14. Alternative embodiments may employ different cryptographic algorithms or methods to change or improve the level of security conferred to the system of the invention.

The invention and all of the functional operations described herein can be implemented in digital electronic circuitry, or in computer hardware, firmware, 10 software, or in combinations of them. The invention can be implemented as a computer program product, i.e., a computer program tangibly embodied in an information carrier, e.g., in a machine-readable storage device or in a propagated signal, for execution by, or to control the operation of, data processing apparatus, e.g., a programmable processor, a computer, or multiple computers. A computer program 15 can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program can be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and 20 interconnected by a communication network.

Method steps of the invention can be performed by one or more programmable processors executing a computer program to perform functions of the invention by operating on input data and generating output. Method steps can also be performed by, and apparatus of the invention can be implemented as, special purpose logic circuitry, 25 e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit) or other customized circuitry.

Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive 30 instructions and data from a read-only memory or a random access memory or both.

The essential elements of a computer are a processor for executing instructions and

23

one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. Information carriers suitable for embodying computer 5 program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in special purpose logic circuitry.

10 To provide for interaction with a user, the invention can be implemented on a device having a screen, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and an input device, e.g., a keyboard, a mouse, a trackball, and the like by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for 15 example, feedback provided to the user can be any form of sensory feedback, e.g.,

visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input.

The invention can be implemented in, e.g., a computing system, a handheld device, a telephone, a consumer appliance, or any other processor-based device. A computing 20 system implementation can include a back-end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front-end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the invention, or any combination of such back-end, middleware, or front-end 25 components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a LAN and WAN, e.g., the Internet.

A skilled person will appreciate that variations of the order of the steps, processes and disclosed arrangements are possible.

30 Accordingly the above description of the specific embodiment is made by way of example only and not for the purpose of limitation. It will be clear to the skilled

24

person that minor modifications may be made without significant changes to the operation described.

5

References

1. A Review of Algorithms for Audio Fingerprinting (P. Cano et al. In International Workshop on Multimedia Signal Processing, US Virgin Islands, December 2002)

25

Claims

1. A handheld device / user interface comprising a screen, a user input device, means for discovering one or a plurality of display devices coupled to the network, means for uploading one or a plurality of items of primary information including a link to secondary information across the network, means for representing at least one of the items on the screen, means for receiving a selection of at least one of the items from the user input device and means for receiving an instruction from the user input device to display the selection on one of the discovered display devices.

2. A handheld device according to claim 1 comprising means for indicating that the represented item may be displayed.

3. A handheld device according to any of claim 1 to claim 2 comprising means for sending the link across the network to the discovered display device responsive to the instruction.

4. A handheld device according to any of claim 1 to claim 3 comprising means for sending the link conditionally upon availability of the discovered display device.

5. A handheld device according to any of claim 1 to claim 4 comprising means for determining whether a discovered display device is in proximity.

6. A handheld device according to claim 5 comprising means for determining the identifier of the discovered display device in proximity.

7. A handheld device according to any of claim 5 to claim 6 comprising means for sending the link to the discovered display device in proximity.

8. A handheld device according to any of claim 5 to claim 7 wherein the link comprises a command.

9. A handheld device according to claim 5 comprising means for displaying a menu including identifiers of the discovered display devices conditionally upon determining that none of the discovered display devices is in proximity, means for receiving a selection of one of the displayed identifiers from the user input device and means for sending the link to the discovered display device corresponding to the selected identifier.

10. A handheld device according to claim 5 wherein the determining means comprise means for receiving a broadcast.

26

11 .A handheld device according to claim 10 wherein the broadcast is electromagnetic.

12. A handheld device according to claim 10 wherein the broadcast is acoustic.

13. A handheld device according to any of claim 11 to claim 12 wherein the receiving means comprise decoding a near field communication.

14. A handheld device according to any of claim 10 to claim 13 wherein the determining means comprise detecting a message or recovering the identifier embedded within the broadcast.

15. A handheld device according to any of the preceding claims wherein the handheld device comprises means for uploading a key across the network and means for authenticating the key to the discovered display device.

16. A handheld device according to any of the preceding claims comprising means for displaying a graphic user interface to the screen conditionally upon detecting a presence of input tags in the secondary information, means for receiving touch inputs from the graphic user interface and means for sending data corresponding to the touch inputs to the discovered display device.

17. A handheld device according to any of the preceding claims wherein the screen and the user input are an integral touch sensitive unit.

18. A handheld device according to any of the preceding claims wherein the item is a description of a retail product or service.

19. A handheld device according to any of the preceding claims wherein the handheld device is a telephone or a tablet computer or a laptop computer.

20. A display device comprising a display unit, means for communicating across a network, means for being discovered by a handheld device, means for receiving a link, means for uploading information referenced by the link across the network and means for displaying the information to the display unit.

21 .A display device according to claim 20 comprising means for indicating proximity.

22. A display device according to claim 21 wherein the indicating means comprise receiving a broadcast in proximity to the discovered display device.

23. A display device according to claim 22 wherein the broadcast is electromagnetic.

24. A display device according to claim 22 wherein the broadcast is acoustic (i.e. propagated over air).

25. A display device according to any of claims 22 to 24 comprising means for encoding a message.

27

26. A display device according to any of claims 22 to 24 comprising means for encoding a key.

27. A display device according to any of claims 20 to 26 comprising means for uploading a key from a display device and means for authenticating the key against a stored second key.

28. A display device according to claim 27 comprising means for uploading the second key across the network and means for storing the second key.

29. A display device according to any of claims 20 to 28 comprising means for receiving user inputs across the network from the handheld device and means for updating the displayed information responsively to the user inputs.

30. A display device according to any of claims 20 to 29 wherein the display device is a television.

31 .A display device according to any of claims 20 to 29 wherein the display device is a set-top-box and the display unit is a television.

32. A display device according to any of claims 20 to 29 wherein the display device is a computer.

33. A method for distributing displays across multiple devices comprising discovering one or a plurality of display devices coupled to a network, uploading one or a plurality of items of primary information including a link to secondary information across the network, representing one of the items on a screen, receiving a selection of a subset of the items from a user input device and receiving an instruction from the user input device that the selection is to be displayed.

34. A method according to claim 33 comprising indicating that the represented item may be displayed.

35. A method according to any of claim 33 to claim 34 comprising sending the link across the network to one of the discovered display devices responsive to the instruction.

36. A method according to any of claim 33 to claim 35 wherein sending the link is conditional upon availability of the discovered display device.

37. A method according to any of claim 33 to claim 36 comprising determining whether the discovered display device is in proximity.

38. A method according to claim 37 comprising determining the identifier of the discovered display device in proximity.

28

39. A method according to any of claim 37 to claim 38 comprising sending the link to the discovered display device in proximity.

40. A method according to claim 37 comprising displaying a menu comprising identifiers of the discovered display devices conditionally upon none of the discovered display devices being determined to be in proximity, receiving a selection of one of the displayed identifiers from the user input device and sending the link to the discovered display device corresponding to the selected identifier.

41 .A method according to claim 37 comprising receiving a broadcast.

42. A method according to claim 41 wherein the broadcast is electromagnetic (i.e. radio).

43. A method according to claim 41 wherein the broadcast is acoustic (i.e. propagated over air).

44. A method according to any of claims 41 to 43 comprising decoding a near field communication.

45. A method according to any of claims 41 to 44 comprising detecting a message embedded within the broadcast.

46. A method according to any of claims 33 to 45 comprising uploading a key across the network and authenticating the key to the display device.

47. A method according to any of claims 33 to 46 comprising displaying a graphic user interface to the screen conditionally upon detecting a presence of input tags in the secondary information, receiving touch inputs from the graphic user interface and sending data corresponding to the touch inputs to the display device.

48. A computer program, preferably on a data carrier or computer readable medium, the computer program having code or instructions for discovering one or a plurality of display devices, uploading across a network one or a plurality of items of first information which each comprise a link to secondary information, representing one or a plurality of the items on a screen, indicating that at least one of the represented items may be displayed, receiving a selection of a subset of the items from a user input device, receiving an instruction to display the selection from the user input device and sending the link across the network to one of the discovered display devices.

29

49. A handheld device / user interface comprising a processor, a data storage device, a network adapter, a screen, a user input device, the processor being operatively connected to the data storage device to execute a sequence of steps, the processor being operatively connected to the network adapter to discover one or a plurality of display devices coupled to the network, the processor being operatively connected to the network adapter to upload one or a plurality of items of primary information including a link to secondary information across the network, the processor being operatively connected to the screen for representing at least one of the items on the screen, and the processor being operatively connected to the user input device for receiving a selection of at least one of the items from the user input device and receiving an instruction to display the selection from the user input device on one of the discovered devices and the processor being operatively connected to the network adapter for sending the link across the network to one of the discovered display devices.