GB2577238A

GB2577238A - Entertainment system and method of delivery of augmented content

Info

Publication number: GB2577238A
Application number: GB1812057.6A
Authority: GB
Inventors: Charles Regleri Jason; Clive Dearling Bruce
Original assignee: Individual
Current assignee: Individual
Priority date: 2018-07-24
Filing date: 2018-07-24
Publication date: 2020-03-25
Also published as: GB201812057D0

Abstract

A method of delivering augmented content to a device comprises: at a primary recipient device 42, receiving a data packet containing a primary payload (data) 34 and secondary payload 36, the primary payload for delivery by the primary recipient device through an associated sensory output (e.g. display); in response to identifying the presence of secondary payload 36, extracting this from the data packet and activating a transmitter in the primary recipient device to send the secondary payload (information) to (secondary) ancillary device(s) 51-54 registered with the primary device; at the ancillary device(s), receiving the secondary payload and then at least one of: storing the secondary payload; playing the secondary payload through an output device of the ancillary device; changing a mode of operation of the ancillary device; and controlling operation of the ancillary device e.g. to deliver advertisements. Also claimed is an entertainment system comprising a television or computer 42 wherein primary and (distinct) secondary payloads are similarly processed, with wireless coupling of the ancillary (secondary) device.

Description

Intellectual Property Office Application No. GII1812057.6 RTM Date:14 January 2020 The following terms are registered trade marks and should be read as such wherever they occur in this document: Diageo Enado Bluetooth Xyloband Intellectual Property Office is an operating name of the Patent Office www.gov.uk /ipo

ENTERTAINMENT SYSTEM

AND METHOD OF DELIVERY OF AUGMENTED CONTENT

Field of the Invention

This invention relates, in general, to an entertainment system and is particularly, but not exclusively, applicable to a system and associated method in which augmented content, including control information and/or complementary data, is delivered in a multi-media environment.

Summary of the Prior Art

In a browser-based environment, it is known to provide ancillary content, to the browser's user, in the form of product advertisements that, in some instances, are targeted at the user following profiling of the user in a registration-like process. This ancillary information is therefore pushed with access to a particular web-page, and the precise content potentially targeted according to partitioning of the user's profile into a particularly characteristic or demographic. Information is therefore provided directly to the device used to undertake web access and browsing activity.

Summary of the Invention

According to first aspect of the present invention there is provided a method of delivering augmented content to a device, the method comprising: at a primary recipient device, receiving a data packet containing a primary payload and secondary payload, the primary payload for delivery by the primary recipient device through a sensory output associated with the primary recipient device; in response to identifying the presence of secondary payload, extracting the secondary payload from the data packet and activating a transmitter in the primary recipient device to send the secondary payload to an ancillary device registered that is pre-registered with the primary recipient device; at the ancillary device, receiving the secondary payload and then at least one of: storing the secondary payload; playing the secondary payload through an output device of the ancillary device; changing a mode of operation of the ancillary device; and controlling operation of the ancillary device in response to the secondary payload.

According to a second aspect of the invention there is provided an entertainment system comprising: a television or computer having: an input over which is received, from a remote content provider, data containing a primary payload directly executable by the television or computer and secondary payload having content distinct to the primary s payload; a display; a transmitter; and a first controller executing control logic to control display of content, extracted from the primary payload, on the display and selectively to control sending of secondary payload using the transmitter; and an ancillary device wireless coupled to the television or computer, the ancillary device including: a receiver arranged to receive the secondary information sent from the transmitter; a second controller arranged to control operation of the ancillary device, wherein the second controller is further arranged to interpret said content of the secondary payload; and a sensory output from which the secondary payload can be presented, wherein presentation of the secondary payload at the ancillary device is dependent upon interpretation, by one of the first controller and the second controller, of the content of the secondary payload.

In another aspect of the invention there is provided a computer program product in accordance with claim 17.

Advantageously, the preferred embodiments operate automatically to cascade secondary information -whether control data and/or content -into a locally registered ancillary recipient device that is connected to a relay device, such as a television or computer, arranged to deliver primary data that is sent to it. Connection of the ancillary recipient device to the relay device is typically a short-range wireless connection. The primary data is delivered from the relay device through a user interface, such as a screen. Data packets that contain the primary data are augmented with the secondary information that is to be cascaded to any locally registered ancillary device. These received data packets include a header that is interpreted by control logic at the relay device such that a local transmitter at the relay device is activated to push the secondary information to the locally registered recipient device. The secondary data can be used either to control the ancillary device to receive and/or deliver related multi-media content pertinent to the data, programme and/or advertisement being received and played at the relay device. The result is an augmented sense of connection with -or immersion in -the data, programme and/or advert being received and viewed, and a system in which a remote content provider can gain access to and deliver secondary data to an ancillary device which is not directly known to the content provider. By embedding the programme or data with such supplementary control information or supplementary data at the source, the option for receipt of this supplementary control information or supplementary data is controlled locally by the user, s e.g. by registering the ancillary recipient device with the television or computer such as to receive this supplementary control information or supplementary data.

In fact, the registration process may, optionally, he associated with a pre-set user profile, such as age, gender and interests, thereby allowing a local controller in the television or computer to decide whether the supplementary control information or supplementary data is pushed onwards and locally to the registered ancillary recipient device or otherwise withheld. This arrangement allows for flexibility and permits the content provider to globally encode all downlink communicated content, whether provided upon user interaction or broadcast.

Brief Description of the Drawings

Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which: FIG. 1 is an entertainment system embodying the present invention.

Detailed Description of a Preferred Embodiment

FIG. 1 shows an entertainment system 10 according to various embodiments of the present invention.

From a server or broadcaster side, a content provider 12 assembles content in the form of multi-media content and/or applications (collectively "media content"). This media content may include television programmes, audio files, advertisements (whether on-line or television-based), interactive or player-downloaded games and general web-page content and information. This media context may be streamed live, or otherwise delayed and stored, such as within a database 14, for delayed broadcast or user-based request access, e.g. in the context of catch-up TV or website information. The database is accessible by a controller 16, such as a server, that either directly administers operation of the context provider or otherwise provides the context provider with an ability to regulate, control, release and/or code any media content to which it has access.

Media content is encoded, via the controller 16, with metadata and additional payload.

The metadata provides instructions that can be interpreted locally by a recipient device to which the media content, containing the metadata, is transmitted, addressed or otherwise broadcast. The metadata might simply be an identify for a type of device that can make use of supplementary control data. The additional payload can he supplementary control information and/or supplementary data that is designed to be pushed onwards, in a cascaded fashion, to one or more ancillary devices that are locally registered with the recipient device. In this way, the header can be kept to a minimum length, and payload included only when there is an identified ancillary device detailed in the metadata. For example, the metadata may include an indication that the additional payload requires the ancillary device to include an output controllable tri-coloured LED device, an audio speaker and/or memory into which a program or application can be downloaded.

The content provider 12 -and thus the media content -is connected via a communications network 18, such as a wide area network that may include wireless and wireline aspects, to user devices at locations, such as houses or offices 20-24. The user devices may be uniquely addressable, or just responsive to a broadcast signal simultaneously receivable by multiple other user devices.

For the sake of explanation only, the following description equates the broadcast media content as a TV recording of a concert sponsored by a business group, such as Diageo®.

Delivery of the media content is generally not relevant to the present invention. Assuming, for the sake of explanation, that the media content is provided in a packet-based system (although other delivery formats are clearly possible and contemplated, as will be readily appreciated), each packet 28 (or group of packets) contains associated metadata 30. The metadata, and in fact the entire packet, may encoded/encrypted and there will usually be some form of header 32 containing, for example, at least one of an address [of a device], data relating to the payload and/or error correcting bits. The packet may therefore contain two distinct payloads: Payload A 34 relating to source-provided media content, e.g. the TV recording of the conceit; and Payload B 36 relating to supplementary control information or supplementary data, such as a URL providing a storable link to a complementary website of the business group and/or an instruction to engage with and control through a script a particularly identified form of registered ancillary device. The script may, in fact, he within the payload rather than the metadata per se.

The location of the script, in many respect, simply depends upon the nature of and number of bits of information that is being communicated in order for there to be effective local control and/or effective delivery of supplementary content to an ancillary device, so this is a design option.

Returning to FIG. 1 and referring to local configuration of hardware and software components in, for example, a first house 20. The configurations between houses may actually vary based upon the recipient device and the ancillary device(s).

Again, assuming a packet transmission of data, the data packet 28 is received at the house at a network interface 40; this may be a wireline router or a radio interface. The network interface conventionally supports interface and operation between two pieces of equipment or protocol layers used within the entirety of the system 10. The network interface 40 therefore can pass information uplink and downlink including packets of information to a targeted principal recipient device, such as television 42. Communication between the network interface 40 and the recipient device may wireless or via a wire, such as an Ethernet cable.

The principal recipient device 42 will typically have some form of user interface and a screen to display data recovered from the payload of received packets. The recipient device might have auxiliary audio speakers 44 or the like to enhance the basic functionality. The recipient device will all include some programmable memory 46 and at least one processor 48 to oversee operation thereof. For example, the user interface provides the user with the ability to select functions on the recipient device, e.g. changing channels or updating or downloading software to the principal recipient device 42. Such limited control may be realized by a screen-based graphic-user interface "GUI" accessed and controlled, typically, by a wireless remote controller 50 or a cell phone containing a suitable app, such as the EnadoTM interface from Wyrestorm® Limited The remote controller 50 therefore provides a known way to access to control level functions of the principal recipient device 42.

Although the recipient device may include a pre-installed app that functions to cascade the supplementary control information and/or supplementary data to a registered ancillary device, this software may he provided as a downloadahle app obtained from the content supplier 12 (or a third party). The software could, of course, be provided by different processes or on a discrete memory stick or CD ROM.

Returning to FIG. 1, the environment, e.g. the house, in which the principal recipient device is located also includes one or more registrable ancillary recipient devices 51-54. Typically, these ancillary recipient devices 51-54 are wireless connectable to the principal recipient device 42. Connectivity typically makes use of a short-range communications protocol, such as Bluetooth I'm or the like. Registration may take the form of a simply 'push-to-link' function on the ancillary recipient device, or via a user interface that involves confirmation of a dedicated password to establish a long-term association between the principal recipient device 42 and the ancillary recipient devices 51-54.

As to the nature of these ancillary recipient devices 51-54, these can be realized by one or more of: i) a cell phone or smartphone 51 having a memory, a display 56 and, typically, a light 58 (which may be white or a multi-coloured LED); ii) audio speakers and preferably wireless speakers (herein denoted "aux" to represent a variety of sensory-perceivable functions that can be generated and output); iii) an animated plush toy 52 having motor-controlled limbs 60, eyes 62 and/or a mouth and/or an audio speaker arranged to output audio that is either pre-stored in local memory or streamed for reception by and local broadcast from the plush toy; iv) a specific light box, e.g. a Xyloband LED wristband 54, or light controller connected between the electrical supply and the bulb; and/or a motorized device that has controllable motors.

Turning now to the functional configuration that may be adopted in each of the ancillary devices 51-54. As shown in FIG. 1, the ancillary device may contain a GUI 70 which may, in fact, double-up as the auxiliary [sensory] output 82. The ancillary device will include a microcontroller 74 for operation control thereof, which microcontroller (or processing module) is operatively coupled, as will be understood, via a bus 76, to a transceiver 78 and memory 80 arranged to store program code. They may be an additional auxiliary output which, in the context of an animated plush toy 52 may be a motor controller, microphone or audio circuit.

In terms of system operation, the principal recipient device 42 is programmed with logic (executed by its processor 48) that interprets the header and/or metadata communicated downlink, across the communications network 18, from a server (not shown) of the content provider 12. The download may be "pushed" content in that it is pushed independently by the server on a one-to-one (direct) or one-to-many (broadcast) basis, or otherwise may be requested "pulled" content that is delivered following an uplink request (from the client/user side and emanating from the principal recipient device 42). As such, the primary content is supplemented with augmenting secondary content at or with the instruction of the content provider.

Upon receipt of a data packet to the principal recipient device 42, its locally-installed control logic is arranged to activate a local transmitter 43 such that, when appropriate metadata is present, the transmitter 43 is selectively activated to communicate the augmenting secondary content onwards to locally registered ancillary devices, thereby automatically distributing this secondary content for immediate use, including immediate local control of the ancillary device or storage of such pushed/communicated data in memory 80 of the ancillary device 51-54. In this fashion, the secondary content (which may be data related to the internet of things) is targeted at the ancillary device, which secondary content may only he indirectly related to the primary content displayed/broadcast of, for example, a screen of the principal recipient device 42.

As such, the system and methodology of the preferred embodiments enable the content provider 12 (at the server side of the network) to communicate additional secondary data or secondary content or control data directly to a registered ancillary device 51-54 via an intermediate [relaying] principal recipient device 42 notwithstanding that the content provider 12 remains unaware of the existence of the ancillary device. The principal recipient device 42 therefore acts as a gatekeeper guarding release/access to the secondary content, with such secondary content only released locally to locally registered ancillary devices.

The payload includes primary media content as Payload A 34 and augmenting secondary media content as Payload B 36. The presence of the augmenting secondary content is identified by the setting of hits in the header and/or by the metadata. The precise communication protocol used to communicate media content, and therefore also the nature of the advisory header, is a design choice. It is suffice to say that the delivery of primary media content as Payload A 34 and augmenting secondary media content as Payload B 36 simply needs to be identifiable and the presence resolvable at the principal recipient device.

By linking the primary and secondary media content into a single packet, timing is maintained between the complementary media contents thereof. Of course, an alternative approach is to send the primary and secondary media content as separate transmissions and then to buffer and align these two media contents at the principal recipient device 42. However, this increases the processing burden at the principal recipient device 42, requires setting of an appropriate delay flag in the header as well as requiring the establishment of a link between potentially two disparate transmissions, although this alternative may allow for secondary content to he sourced from a separate server-side entity rather than for the content provider to pre-compile the all the media contents/information in advance. In this latter alternative, logic at the server side may make use of demographic information and registered user identification to locate, source and then cause the sending of secondary media content in near real-time to an identified principal recipient device 42.

The primary media content, delivered as Payload A 34, is basic context, e.g. the television programme or webpage. The augmenting secondary media content, delivered as Payload B 36, is data and/or control that is inserted by (or with the knowledge of) the content provider and relates to links or control that is to be cascaded automatically downwards from the principal recipient device 42 to registered ancillary devices 51-54 in a push operation. The augmenting secondary media content can then either be presented immediately at each registered (and, preferably, uniquely addressed) ancillary device, stored in local memory at the ancillary device for later recall and/or used to execute a local function to generate a sensory effect at the ancillary device. By way of non-limiting examples, the augmenting secondary media content could take the form of: a downloadable static or moving image from an affiliated sponsor of the primary content, e.g. a TV show, the downloadable image immediately displayable on the ancillary device and/or locally storable for later recall; a redeemably promotional coupon for an advert being presented as the primary content, thus allowing a smartphone to automatically receive and store the redeemably promotional coupon for subsequent use by the user at the point when the primary content is being screened/viewed; a voice or music file with associated queue points that allows for a registered ancillary device, such as a plush toy, to provide an interactive contemporaneous output that is distinct from that presented by or on the principal recipient device 42, i.e. TV. In this way, the augmenting secondary media content may be part of the primary content albeit actioned from a spatial distinct point relative to the TV; a scripted sequence of actuation codes that is complementary to contemporaneously presented primary content. For example, the primary content may he a concert and the secondary content is a control sequence that actuates changing patterns of coloured LEDs on a Xyloband® wristband (or actuates a light on a smartphone) in a synchronized fashion in time with the music in the primary content; and interactive content that links at least one local ancillary device into media that is playing through the TV, such that the local media device and TV (and the local environment in which the ancillary device is active) together become part of the "set" albeit that the set is personal to the vicinity/room in which the TV is situated. In this way, the ancillary device may he controlled, by the received and selective and onwardly communicated payload, to (for example) turn on a local motor and/or project speech or sound at a point in time that correlates to a related on-screen event. For example, movement of an actor's hand in a video projected from the television screen would see a motor energised on the ancillary device to reflect sensory perception of the hand's movement and its touching of an object. In another example, a crack of lightning in the video could lead to a vibration of the ancillary device to generate a local sensation of a shudder.

Unique addressing is preferably since some augmenting secondary media content may not be applicable to certain types of device, e.g. a webpage could not be displayed on a plush toy, so the metadata is used to indicate the ancillary devices for which augmenting secondary media content is relevant. This addressing function can therefore cut down unnecessary local wireless transmissions.

Once received at the principal recipient device, the header and/or metadata are interpreted by control logic executable by the microprocessor 48. The control logic can he downloaded and stored in the memory 46 of the principal recipient device.

Similarly, with the ancillary devices, these can be pre-programmed or otherwise programmed from a download with appropriate control logic and permits control instructions, received from the principal recipient device, to be interpreted and actioned in a timely and coordinated fashion.

If there is no locally registered ancillary device, then the augmenting secondary media content can be ignored by the principal recipient device and the principal recipient device tasked simply to deliver, in a conventional sense, the primary media content.

As used in this application, the terms "module", "system", "terminal", "server", "user/subscriber equipment/device" and the like are intended to refer to a computer-related entity, either hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component of the system (and as the context requires) can he, but is not limited to being, a process running on a processor (or interchangeably a "controller"), an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a computing device and the computing device can he a component. One or more components can reside within a process and/or thread of execution and a component can be localized on one processing board of a computing device, and/or distributed between two or more computing boards in many devices. In addition, these components can execute from various computer readable media having various data structures stored thereon. The components can communicate by way of local and/or remote processes such as in accordance with a signal having one or more data packets (e.g., data from one component -/1-interacting with another component in a local system, distributed system, and/or across a network such as the Internet with other systems by way of the signal).

It is understood that the specific order or hierarchy of steps in the processes disclosed s herein is an example of exemplary approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the processes may he rearranged while remaining within the scope of the present disclosure. The accompanying method claims present elements of the various steps in sample order, and are not meant to he limited to the specific order or hierarchy presented unless a specific order is expressly described or is logically required.

Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, methods and algorithms described in connection with the examples disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, methods and algorithms have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application while remaining, either literally or equivalently, within the scope of the accompanying claims.

Unless specific arrangements are mutually exclusive with one another, the various embodiments described herein can be combined to enhance system functionality and/or to produce complementary functions in the effective automatic pushed delivery of augmenting secondary content to ancillary devices. Such combinations will he readily appreciated by the skilled addressee given the totality of the foregoing description. Likewise, aspects of the preferred embodiments may be implemented in standalone arrangements where more limited and thus specific component functionality is provided within each of the interconnected -and therefore interacting -system components albeit that, in sum, they together support, realize and produce the described real-world effect(s). Indeed, it will be understood that unless features in the particular preferred embodiments arc expressly identified as incompatible with one another or the surrounding context implies that they are mutually exclusive and not readily combinable in a complementary and/or supportive sense, the totality of this disclosure contemplates and envisions that specific features of those complementary embodiments can he selectively combined to s provide one or more comprehensive, but slightly different, technical solutions.

It with, therefore, be appreciated that the above description has been given by way of example only and that modification in detail may he made within the scope of the present invention. For example, packet data delivery is but one data delivery option.

Claims

Claims 1. A method of delivering augmented content to a device, the method comprising: at a primary recipient device, receiving a data packet containing a primary payload and secondary payload, the primary payload for delivery by the primary recipient device through a sensory output associated with the primary recipient device; in response to identifying the presence of secondary payload, extracting the secondary payload from the data packet and activating a transmitter in the primary recipient device to send the secondary payload to an ancillary device registered that is preregistered with the primary recipient device; at the ancillary device, receiving the secondary payload and then at least one of: storing the secondary payload; playing the secondary payload through an output device of the ancillary device; changing a mode of operation of the ancillary device; and controlling operation of the ancillary device in response to the secondary payload.
2. The method of claim 1, wherein the primary recipient device is one of a television and a computer.
3. The method of claim 1 or 2, wherein the data packet includes a header that addresses the primary recipient device.
4. The method of claim 3, wherein the header includes a data field identifying the presence of the secondary payload.
5. The method of any preceding claim, wherein playing of content in the primary payload on the primary recipient device is coordinated in time with the playing, on the ancillary device, of content in the secondary payload.
6. The method of any preceding claim, wherein any change in operation of the ancillary device is coordinated in time with the playing of content in the primary payload on the primary recipient device.
7. The method of any of claims 1 to 4, wherein the secondary payload is actioned at the ancillary device independently of sensory presentation of the primary payload by the primary recipient device.
8. The method of any preceding claim, wherein the secondary payload is at least one of: a downloadable static or moving image from an affiliated sponsor of the primary payload; a redeemably promotional coupon for an advert being presented as the primary 10 payload; a voice or music file with associated queue points that allows for the registered ancillary device to provide an interactive contemporaneous output that is distinct from the primary payload presented by or on the primary recipient device; a scripted sequence of actuation codes that is complementary to contemporaneously presented primary payload; and interactive content that links the ancillary device into media being played as the primary payload on the primary recipient device.
9. An entertainment system comprising: a television or computer (42) having: an input over which is received, from a remote content provider, data containing a primary payload directly executable by the television or computer and secondary payload having content distinct to the primary payload; a display; a transmitter; and a first controller executing control logic to control display of content, extracted from the primary payload, on the display and selectively to control sending of secondary payload using the transmitter; and an ancillary device (51-54) wireless coupled to the television or computer, the ancillary device including: a receiver arranged to receive the secondary information sent from the transmitter; a second controller arranged to control operation of the ancillary device, wherein the second controller is further arranged to interpret said content of the secondary payload; and a sensory output from which the secondary payload can he presented, wherein presentation of the secondary payload at the ancillary device is dependent upon interpretation, by one of the first controller and the second controller, of the content of the secondary payload.
10. The entertainment system of claim 9, wherein the ancillary device includes a transmitter for communicating a registration request to a second receiver of the television or computer, wherein the registration request authorizes the receipt of secondary payload by the ancillary device.
11. The entertainment system of claim 9 or 10, wherein the television or computer (42) receives the data in a packet.
12. The entertainment system of claim 9 or 10, wherein the packet includes a header having a data field identifying the presence of the secondary payload.
13. The entertainment system of claim 9, 1() to 11, wherein the first and second controllers coordinate, in time, playing of the primary payload of the television or computer with playing of the secondary payload on the ancillary device.
14. The entertainment system of claim 9, 10 to 11, wherein any change in operation of the ancillary device is coordinated in time with the playing of the primary payload on the television or computer.
15. The entertainment system of any of claims 9 to 12, wherein the secondary payload is actioned at the ancillary device independently of sensory presentation of the primary payload by the television or computer.
16. The entertainment system of any of claims 9 to 15, wherein the secondary payload is at least one of: -/6-a downloadable static or moving image from an affiliated sponsor of the primary payload; a redeemably promotional coupon for an advert being presented as the primary payload; a voice or music file with associated queue points that allows for the registered ancillary device to provide an interactive contemporaneous output that is distinct from the primary payload presented by or on the primary recipient device; a scripted sequence of actuation codes that is complementary to contemporaneously presented primary payload; and interactive content that links the ancillary device into media being played as the primary payload on the primary recipient device.
17. A computer program product that, when executed by a first processor of the primary recipient device of any of claims 1 to 8 and a second processor of the ancillary device of any of claims 1 to 8, causes the primary recipient device and the ancillary equipment to execute procedure to effect the processes of claims 1 to 8.