GB2552016A

GB2552016A - A System and method for communicating with a plurality of mobile devices from a single transmitter

Info

Publication number: GB2552016A
Application number: GB1611859.8A
Authority: GB
Inventors: Morrison Alex; Cummins Andy; Rubinstein Ben
Original assignee: Cognitive Applications Ltd
Current assignee: Cognitive Applications Ltd
Priority date: 2016-07-07
Filing date: 2016-07-07
Publication date: 2018-01-10
Also published as: GB201611859D0

Abstract

A plurality of mobile devices 11, possibly belonging to audience members at a performance, which monitor the identification data of a wireless transmitter 12 using a software program or an application 10, whereby detecting a modification in the data, they perform a predetermined function, which may be: displaying a colour or image or sequence; output audio tone or clip; set a torch on, off or flashing; activate a vibrating mechanism. The mobile device 11 may require further input (such as light, sound or physical input such as shaking or waving), before the mobile device 11 performs the predetermined function. The identification data may contain a compressed or encrypted control commands in an SSID of a WiFi (RTM) access point 12 or advertising data of a Bluetooth (RTM) Low-energy (BT LE) peripheral or beacon 12. The control commands may have verification and recognition features for the mobile devices 11 to validate them before performing the function. The transmission parameters of the WiFi (RTM) or Bluetooth (RTM) arrangements 12 may be manipulated to shorten the interval between beacon frames to decrease latency of reception.

Description

(54) Title of the Invention: A System and method for communicating with a plurality of mobile devices from a single transmitter

Abstract Title: Mobile devices monitor changing identification data in transmitter broadcast beacon (57) A plurality of mobile devices 11, possibly belonging to audience members at a performance, which monitor the identification data of a wireless transmitter 12 using a software program or an application 10, whereby detecting a modification in the data, they perform a predetermined function, which may be: displaying a colour or image or sequence; output audio tone or clip; set a torch on, off or flashing; activate a vibrating mechanism. The mobile device 11 may require further input (such as light, sound or physical input such as shaking or waving), before the mobile device 11 performs the predetermined function. The identification data may contain a compressed or encrypted control commands in an SSID of a WiFi (RTM) access point 12 or advertising data of a Bluetooth (RTM) Low-energy (BT LE) peripheral or beacon 12. The control commands may have verification and recognition features for the mobile devices 11 to validate them before performing the function. The transmission parameters of the WiFi (RTM) or Bluetooth (RTM) arrangements 12 may be manipulated to shorten the interval between beacon frames to decrease latency of reception.

FIG 1

1/7

FIG 1

217

ΑΛ Λ display blue

FIG 2

3/7

Instruction:

Command: Display Image named Fire, for 10 seconds

Encoded:

Command code: DI

Numeric parameter: 10 String parameter: Fire

Byte sequence (7 bytes):

O'	T	10	'F'	T	'r'	'e'
^444444444444-	«444444444444444444444444444-	«44444444444444444444444444444444444444444-	^44444444444-^

Encrypted with a checksum(14 bytes):

BO 24 8E 84 7A DO 92 48 F8 67 EO 77 91

Transformed to bit stream:

0000010010110000001001001000111010000100011110101101000010010010010

010001111100001100111111000000111011110010001

Split into chunks of six bits, zero padded as necessary (19 chunks):

000001 001011 000000 100100 100011 101000 010001 111010 110100 001001 001001 001000 111110 000110 011111 100000 011101 111001 000100

Shifted to represent printable characters, used as WiFi SSID or BTLE peripheral name ; 1 B0aZeHwq998/6VWTv4 ^444444444444444^*444444444444444^^

FIG 3

4/7 access point device access point device

ω	ω	<<<<<:ί&
o	o	:<<<<<£>
05	05	<<<<<:©
3	□	i<<<<3

o— command issued command detected—o

FIG 4a time

<<<<<<<<<<<<φ<<<<<<<<<<<<<;		V>
O	<<<<<<<<<<IGS<<<<<<<<<<<<<	a
05		o>
□	<<<<<<<<<<<<3<<<<<<<<<<<<<	3

o— command issued o— command detected time

FIG 4b

5/7 peripheral central peripheral central

			03
Q.	Q.	:±:Ωίϊ:	Q.
<	<	<	<
Φ	Φ		Φ
Z5-	Z5-		Z5-
ω	ω		ω
Φ	Φ	φ	Φ

o— command issued o— command detected

		03	03		03
Q.		Q.		Q.		Q.
<	<	<		<		<
Φ	ΚΦύϊύϊ	Φ		Φ		Φ
Z5-	Z3-	Z3-		Z3-		Z3-
ω	Am	ω		ω		ω
Φ		Φ		Φ		Φ

FIG 5a

adverti	adverti	adverti	adverti	adverti
ω	ω	ω	ω	ω
Φ	Φ	Φ	Φ	Φ

o— command issued o— command detected

-¹-► time time

FIG 5b

6/7

FIG 6 in

Technician issues command

JL (qQo)

WSSSSSSSSS»^^

Command transmitted
1	r
Audience devices running ! app detect command \| transmission 1
	r
App	waits

Performer sings line fl

z /Qp A£

A

Audience shake their devices

Γ ^{α Λ}
II nV		Devices respond as determined by
1		the command (emit audio, flash
		torch, vibrate, display image, etc)

FIG 7

Intellectual

Property

Office

Application No. GB1611859.8

RTM

Date :28 December 2016

The following terms are registered trade marks and should be read as such wherever they occur in this document:

Apple: page 4 iOS: page 4

Android: page 4 iBeacon: page 8

WiFi: page 2-6; figures 3 and 6; claims 6-7, 15-16

Bluetooth: page 2-3, 6; figures 3 and 6; claims 8, 17

BTFE: page 2-5, figures 3 and 6; claims 9, 18

Intellectual Property Office is an operating name of the Patent Office www.gov.uk/ipo

A system and method for communicating with a plurality of mobile devices from a single transmitter

Technical Field

This invention relates to a system and method for communicating with a plurality of mobile devices from a single transmitter, and particularly for communicating with the mobile devices of a live audience to enhance live events, such as musical performances.

Background

Live events such as musical performances typically employ sound systems, light systems, large video projections or video walls, and may use other effects such as pyrotechnics, confetti cannons etc. to increase the excitement and entertainment value of the event. These techniques are well established and effective but they require substantial hardware with corresponding cost to purchase or hire. Additionally, as most performers take their shows on tour often playing only a single performance at each venue, these techniques require considerable and expensive crews to transport, install, and operate the equipment.

Additionally, these techniques are all 'broadcast' from the stage to the audience. The audience may be excited or entertained by the effects, but they remain passive consumers.

There have been attempts made and various methods proposed that would create a very specific effect: a large image created across the audience using either special hardware or the audience's mobile devices. Both of these require the specific location of a device to be defined in advance of the effect being used. This is either through user input or pre-show preparation and designation of people to specific locations. See for example Canadian patent no. CA2749208C COMMUNICATION TO AN AUDIENCE AT AN EVENT; COMMUNICATION A UN AUDITOIRE LORS DUNE ACTIVITE - and US Patent no. US7697925B Synchronized light shows on cellular handsets of users at a gathering.

There have been attempts to use the audience as part of the light-show of an event, by distributing special hardware [ref: Xylobands: http://www.xylobands.com/1. This is expensive, cumbersome and of limited application relative to the invention that is the subject of this application.

Other methods have been proposed to allow the mobile devices of audience members to contribute to a performance. These have described several methods of coordinating the audience effect with the performance, including:

A) timed scheduling, which is unreliable and inaccurate. See patent US7697925B referred to above

B) audio signals, requiring special set-up and constraining the musical performance. See WhamCityLights: http://www.whamcitylights.com/; and

C) connected radio frequency control, using either WiFi or Bluetooth where the audience's devices establish a connection with the transmitter. These are adequate for small audiences, but require very large numbers of access points to support larger audiences/larger venues - which then requires considerable equipment, a substantial overhead for installation, etc. See for example US patent US2014009366A SYSTEMS AND METHODS FOR COORDINATING PORTABLE DISPLAY DEVICES.

Summary of the Invention

The present invention provides a system and method for communicating with a plurality of mobile devices from a single transmitter, the transmitter broadcasting identifying data publicly for all mobile devices to receive, in which the mobile devices are each running a software application which monitors the identifying data and, on detection of a change in the identifying data, the software application causes each mobile device to perform a predetermined function.

The predetermined function may be one or more of the following:

i) display a colour, image, or image sequence;

ii) output an audio tone, sequence, or clip;

iii) switch the 'torch' light on or off, or start it flashing; or iv) actuate the device's vibrate mechanism.

In a preferred embodiment, a further input is required before the mobile device performs the predetermined function. The further input may be selected from the following: a light input; a sound input; a physical input (such as shaking or waving the mobile device).

The identifying data is preferably an encrypted or compressed combination of one or more commands, with verification and recognition features to allow the mobile device to recognize and validate the commands, before performing the predetermined function.

In a preferred embodiment, the identifying data is transmitted by manipulating the SSID of a WiFi access point, without requiring the mobile devices to form a connection with the access point.

The normal transmission parameters of the WiFi access point may be modified by reducing the interval between the start of each beacon frame transmission to decrease the latency of reception. A preferred interval is in the range of 20-40ms.

In an alternative preferred embodiment, the identifying data is transmitted by manipulating the advertising data of a Bluetooth Low-Energy peripheral, without requiring the mobile devices to form a connection with the peripheral. The standard transmission method of the BTLE peripheral may be modified by broadcasting the advertising data with shorter intervals between the start of each advertising data transmission to decrease the latency of reception. A preferred interval is in the range of 30-70ms.

At least in preferred embodiments, the present invention uses a software application installed on the audience's mobile devices, receiving commands by radio frequency transmissions sent from a control console, which are encoded in a way that can be received using the existing standard hardware of consumer devices, without requiring an established connection between transmitter and receiver and preferably without contravening manufacturer restrictions on how the radio hardware can be accessed.

The present invention circumvents the problems discussed above with the prior art, as it allows very large numbers of devices to be controlled simultaneously from a single, portable transmitter.

By using the devices that the audience bring with them to the performance, with simple small control systems, the invention avoids the transportation, installation, and de-installation costs associated with prior art systems and methods. The invention does not require any advance set up with regards to client device locations.

Brief Description of the Drawings

FIG 1 is a general flowchart of how the system is used;

FIG 2 is a diagram illustrating an exemplary scenario of how the system might be used at a musical performance to enhance the audience experience;

FIG 3 describes a method of encoding commands into identifying data;

FIG 4 describes a modification to standard WiFi transmissions which reduces latency and improves the responsiveness of the system;

FIG 5 describes a modification to standard BTLE transmissions which reduces latency and improves the responsiveness of the system;

FIG 6 illustrates the high-level process of the app; and

FIG 7 illustrates an alternative embodiment in which the device function is affected by audience member input.

Detailed Description of a Preferred Embodiment

FIG 1 shows a flowchart of how the system of the invention is used in general, and FIG 2 shows the invention being employed in a musical performance.

A series of 'apps' are programmed, for each mobile device operating system which is to be targeted (for example, Apple iOS, Android, etc.). The application can access the radio devices built into the hardware of the device. For each operating system it may be necessary to use a different approach in order to comply with limitations of the hardware or limitations imposed by the manufacturers.

With reference to FIGS 1 and 2, this software application 10 is installed on the mobile devices of the audience members 11. A hardware device 12 ,the transmitter, is installed in the venue, which may be controlled by the performer or their technician, for example from the lighting control desk 13.

At a particular moment in the show, the technician at the lighting desk 13 may simultaneously make the stage lights show blue, and also issue a command to the transmitter 12. This may also occur as part of a pre-programmed sequence of light and video effects.

The transmitter 12 encodes the command according the method shown in FIG 3, and transmits it by modifying the SSID of a WiFi access point, or the peripheral name of a BTLE peripheral, or both.

One of the constraints of using the SSID or Peripheral Name to transmit commands is that by design they are human readable text strings. Although the relevant protocol specifications do not explicitly restrict the character set for these elements of the data packet, using binary data in these contexts would at least risk causing unexpected and unwanted side effects in some audience devices. Thus it is necessary to restrict the values to what is known in the art as 'printable characters'.

On the other hand, as plain text strings designed to be seen by users, and easily viewed using any modern smartphone, there is a risk of interference and non-intended effects. It is necessary to have some method of encoding the messages used in the system so that the receiving application can unambiguously verify the integrity and legitimacy of messages sent through this channel.

To avoid these issues, the present invention includes a scheme for encoding messages into a 'plain text' string of 'printable characters', which is illustrated in FIG 3.

To improve responsiveness and reduce latency, the transmitter may modify the standard WiFi SSID advertising timings in the manner illustrated in FIG 4b contrasted with FIG 4a; or the standard BTLE GAP advertising timings, in the manner illustrated in FIG 5b contrasted with FIG 5a.

Data in the 802.11 (WiFi) standard is sent in frames. The SSID of an access point is normally 'advertised¹ as part of a beacon frame (properly a management frame of subtype Beacon). Although the 802.11 standard generally doesn't dictate the speeds or intervals at which various frames should be sent, beacon frames are generally sent at very low speeds to be accessible to the widest range of devices, and because these are sent before an association has been negotiated which can define what speed is supported by both the client and access point. Of the many kinds of frame, a conventional access point sends a beacon frame only periodically because in normal use the client devices are only rarely looking for a new WiFi network to associate to, and because most of the bandwidth is reserved for data frames and actual network traffic. Typical access points use a beacon interval of 100ms, which provides reasonable performance in normal usage.

In this invention, the frequency with which the application 10 can scan for available WiFi access points is limited by the operating system (for power-conservation reasons). Hence not only is there a delay after a new command is to be transmitted before the next beacon frame is sent; but even after this, the beacon frame may be sent several times which don't happen to coincide with the device performing a scan, before they do coincide and it's possible for the application 10 to detect the transmission. FIG 4a illustrates a standard WiFi access point sending beacon frames at long intervals; and a device scanning periodically - and possibly scanning at a time when no beacon frame is being transmitted. In this illustration the grey boxes in the transmission stream from the access point indicate beacon frames; the grey box in the bottom indicates the device scan which detects the beacon frame.

To reduce latency therefore, in the preferred embodiment the transmitter, a modified access point, reduces the beacon interval when sending a transmission to a minimum, for example to a value within the range of about 20ms to about 40ms, so that an almost continuous stream of beacon frames is sent. This is illustrated in FIG 4b: the effect is the commands are detected much more swiftly.

There is a similar although slightly different issue when using Bluetooth LE as the transmission medium. In this case the 'peripheral' has no other data to send, but normally transmits only for brief intervals.

FIG 5a shows a standard Bluetooth LE GAP advertising profile: 'peripheral' advertises for short periods, with long and slightly randomised pauses between, to conserve power and allow other devices to advertise as well; 'central' device scans for short periods with substantial pauses between, to conserve power. A command is detected only when the 'advertise' and 'scan' periods overlap (see the grey overlapping boxes and the command detected indicator). Consequently there may be a considerable delay before this overlap occurs and the command is detected.

In a preferred system for use in the invention as shown in FIG 5b, a modified profile is used by the peripheral (there is no possibility to modify the 'central' device, which in this context is a mobile phone or other audience device); broadcasting the GAP data with very short pauses between, for example with an interval within the range of about 30ms to about 70ms. The effect is the commands are detected much more swiftly.

When the suitably encoded message is transmitted as described above, the app 10 running on audience devices 11 detects the transmission and decodes it to recover the transmitted message, in the process illustrated in FIG 6.

Having decoded the display blue message, the app 10 running on audience devices interprets the command, and displays a blue screen on the audience device 11.

An additional feature of the solution is that it can combine control or timing signals sent through the radio channel with signals sent through other channels, such as light or sound, to increase the diversity of effects; for example, using sound or light to trigger an effect can reduce latency; using light can allow the command to be sent with greater spatial specificity.

An additional feature of the solution is that it can take advantage of all elements of the mobile device, using not only the screen of the device to provide colour light effects, but also for those devices which have a camera flash or 'torch' facility, using this to provide extremely bright white light effects; and also using the audio features of the device to provide audio effects. Any of these can be used by themselves or in combination.

An additional feature of the solution is that it can perform effects which are specific to the event. For example, audio and graphic assets can be loaded into the application to complement a particular song in a performance.

An additional feature of the solution is that it can sub-divide the audience in different ways - by location within the venue, by individual choice (favourite song, favourite band member etc.), or statistically into groups. This can be used to create spatial colouring in the venue, to create more subtle effects, to create call-and-response or quiz effects, among other possibilities.

An additional feature of the solution is that it can accommodate individual input; statically by individual choice as above, but also dynamically, for example using the device's position and motion sensors. A show designer can specify a colour range; an individual audience member can cycle their device within this range by waving it. A show designer can specify a sound clip; an individual audience member can choose when to perform this clip by shaking their device (this is illustrated by FIG 7). By using techniques such as this audience engagement can be increased.

There may be further applications of this system in other contexts. For example (as an effective alternative to iBeacons) the same signalling system used with multiple controllers located in appropriate places, could be used to trigger effects on visitor's mobile devices as they circulate through a retail space, an outdoor space such as an ancient monument, an exhibition or museum display.

Claims

1. A system for communicating with a plurality of mobile devices from a single transmitter, the transmitter broadcasting identifying data publicly for all mobile devices to receive, in which the mobile devices are each running a software application which monitors the identifying data and, on detection of a change in the identifying data, the software application causes each mobile device to perform a predetermined function.

2. The system as in claim 1 in which the predetermined function is one or more of the following:

i) display a colour, image, or image sequence;

ii) output an audio tone, sequence, or clip;

3. The system as in claim 1 or 2 in which a further input is required before the mobile device performs the predetermined function.

4. The system as in claim 3, in which the further input is selected from the following: a light input; a sound input; a physical input (such as shaking or waving the mobile device).

5. The system as in any preceding claim in which the identifying data is an encrypted or compressed combination of one or more commands, with verification and recognition features to allow the mobile device to recognize and validate the commands, before performing the predetermined function.

6. The system as in any preceding claim in which the identifying data is transmitted by manipulating the SSID of a WiFi access point, without requiring the mobile devices to form a connection with the access point.

7. The system as in claim 6 in which the normal transmission parameters of the WiFi access point are modified by reducing the interval between beacon frame transmissions to decrease the latency of reception.

8. The system as in any of claims 1 to 5 in which the identifying data is transmitted by manipulating the advertising data of a Bluetooth Low-Energy peripheral, without requiring the mobile devices to form a connection with the peripheral.

9. The system as in claim 8 in which the standard transmission method of the BTLE peripheral is modified by broadcasting the advertising data with shorter intervals in between to decrease the latency of reception.

10. A method for communicating with a plurality of mobile devices from a single transmitter, the transmitter broadcasting identifying data publicly for all mobile devices to receive, in which the mobile devices are each running a software application which monitors the identifying data and, on detection of a change in the identifying data, the software application causes each mobile device to perform a predetermined function.

11. The method as in claim 10 in which the predetermined function is one or more of the following:

i) display a colour, image, or image sequence;

ii) output an audio tone, sequence, or clip;

12. The method as in claim 10 or 11 in which a further input is required before the mobile device performs the predetermined function.

13. The method as in claim 12, in which the further input is selected from the following: a light input; a sound input; a physical input (such as shaking or waving the mobile device).

14. The method as in any of claims 10 to 13 in which the identifying data is an encrypted or compressed combination of one or more commands, with verification and recognition features to allow the mobile device to recognize and validate the commands, before performing the predetermined function.

15. The method as in any preceding claim in which the identifying data is transmitted by manipulating the SSID of a WiFi access point, without requiring the mobile devices to form a connection with the access point.

5

16. The system as in claim 15 in which the normal transmission parameters of the WiFi access point are modified by reducing the interval between beacon frame transmissions to decrease the latency of reception.

17. The system as in any of claims 10 to 14 in which the identifying data is transmitted by

10 manipulating the advertising data of a Bluetooth Low-Energy peripheral, without requiring the mobile devices to form a connection with the peripheral.

18. The system as in claim 17 in which the standard transmission method of the BTLE peripheral is modified by broadcasting the advertising data with shorter intervals in between to

15 decrease the latency of reception.

Intellectual

Property

Office

Application No: GB1611859.8 Examiner: Mr Peter Herbert