WO2012085659A1 - Improvements to interactive multimedia systems - Google Patents

Improvements to interactive multimedia systems Download PDF

Info

Publication number
WO2012085659A1
WO2012085659A1 PCT/IB2011/003168 IB2011003168W WO2012085659A1 WO 2012085659 A1 WO2012085659 A1 WO 2012085659A1 IB 2011003168 W IB2011003168 W IB 2011003168W WO 2012085659 A1 WO2012085659 A1 WO 2012085659A1
Authority
WO
WIPO (PCT)
Prior art keywords
multimedia
decoder
data
video stream
supplementary data
Prior art date
Application number
PCT/IB2011/003168
Other languages
French (fr)
Inventor
Keith Huang
Wesley Ng
Ching Chit REX
Chan Tsz KIT
Felix TO
Chan Pui CHI
Original Assignee
Starbucks (Hk) Limited
Eip
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Starbucks (Hk) Limited, Eip filed Critical Starbucks (Hk) Limited
Publication of WO2012085659A1 publication Critical patent/WO2012085659A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • H04N21/25808—Management of client data
    • H04N21/25816—Management of client data involving client authentication
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
    • H04N21/2355—Processing of additional data, e.g. scrambling of additional data or processing content descriptors involving reformatting operations of additional data, e.g. HTML pages
    • H04N21/2358—Processing of additional data, e.g. scrambling of additional data or processing content descriptors involving reformatting operations of additional data, e.g. HTML pages for generating different versions, e.g. for different recipient devices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
    • H04N21/42226—Reprogrammable remote control devices
    • H04N21/42227—Reprogrammable remote control devices the keys being reprogrammable, e.g. soft keys
    • H04N21/42228—Reprogrammable remote control devices the keys being reprogrammable, e.g. soft keys the reprogrammable keys being displayed on a display screen in order to reduce the number of keys on the remote control device itself
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H04N21/4316—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
    • H04N21/4355—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream involving reformatting operations of additional data, e.g. HTML pages on a television screen
    • H04N21/4358—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream involving reformatting operations of additional data, e.g. HTML pages on a television screen for generating different versions, e.g. for different peripheral devices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4622—Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/475—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data
    • H04N21/4758—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data for providing answers, e.g. voting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65—Transmission of management data between client and server
    • H04N21/654—Transmission by server directed to the client
    • H04N21/6547—Transmission by server directed to the client comprising parameters, e.g. for client setup
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/16—Analogue secrecy systems; Analogue subscription systems
    • H04N7/173—Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal

Definitions

  • the present invention relates to a system, apparatus, method and computer software for improved multimedia systems, and in particular for providing an interactive multimedia experience to a group of users.
  • Broadcast television systems are well known.
  • a broadcast video stream is provided from a single source to a large number of receivers.
  • the broadcast video stream may be transmitted using terrestrial or satellite based radio frequency transmissions, or using physical cabling laid between the transmitter and the receivers.
  • IPTV Internet Protocol Television
  • This programme data may be unicast; that is the packets mal ing up a given programme are each sent from a source to a single receiver.
  • unicasting is inefficient in terms of network resources, in particular for television programmes that are broadcast to many receivers, since separate packets have to be sent to the separate receivers, even if the separate packets have the same payload and only differ in address.
  • Multicasting allows for a packet to be addressed to multiple receivers, and can dramatically reduce network usage since the packet need only be sent once from the transmitter (or source), and, is propagated through the network by bespoke intermediate network nodes in order to reach the receivers.
  • One popular type of television programme is a game show or quiz show.
  • a contestant in a television studio attempts to answer questions in an effort to win prizes.
  • a remote viewer's (i.e. one viewing the show on a screen and not in the television studio) participation in such shows is limited, in that the viewer can, at best, merely try and answer the question before the studio contestant and has no method of keeping a score, or to claim a prize, should they answer questions correctly.
  • a multimedia system comprising: a server system arranged to provide multimedia content, the multimedia content comprising a video stream and supplementary data; and a plurality of multimedia decoders, each multimedia decoder comprising: an interface for receiving the multimedia content and for outputting display data to a display device; a processor arranged to overlay onto a received said video stream a graphical element created on the basis of said received supplementary data, whereby to generate said display data, wherein the server system is arranged to: receive input relating to the multimedia content from a said multimedia decoder; broadcast the video stream to at least some of the multimedia decoders; and transmit, separately from the video stream, said supplementary data to said at least some of the multimedia decoders, and wherein the supplementary data transmitted to a given said multimedia decoder is selected in dependence on the input received from the given said multimedia
  • the input received from a said multimedia decoder may comprise an identity of the said multimedia decoder.
  • the system By transmitting the supplementary data separately from the video stream the system is able to ensure that the display data sent to the display device (and therefore viewed by a user) is personalized in some way beyond the broadcast video stream.
  • the display data for each multimedia decoder can be made unique to that decoder. More advantageously, the unique display data can be provided while still talcing advantage of the savings (in terms of network resources) afforded by broadcasting (or multicasting) the bulk of the data (i.e. the video stream).
  • the multimedia system may comprise: input means associated with each multimedia decoder, each input means arranged to receive user input, and, responsive to user input of a predetermined type, to cause a message indicative of the said user input to be sent to the server system, wherein the server system may be arranged, for each said message, to create acknowledgement data in dependence on the message, wherein the supplementary data transmitted to the multimedia decoder associated with the respective input means comprises at least a part of the created acknowledgement data, and wherein the graphical element created by the processor of a given multimedia decoder includes a visual indication of said acknowledgement data.
  • the system by sending at least part of the acknowledgement data to the multimedia decoder (as part of the supplementary data), is able to provide the user with visual confirmation that, not only has their input been received by the input means, but that the input has been received by the server system (where it can be properly recorded in a memory or database).
  • the system is able to utilize IP precedence support in the network. That is, the data making up the acknowledgement can be given a priority higher than that of the video stream and/or other network traffic.
  • Each interface of the multimedia decoders may comprise said associated input means, and wherein the multimedia decoder is arranged to send the message indicative of the user input via the interface.
  • the input means is a part of the decoder, being for example a remote control or other input device.
  • the input means is separate from the decoder, for example being an application on a computer or smartphone, and has an independent network connection to the server system.
  • the server system may be arranged to access a storage system arranged to hold records corresponding to a respective multimedia decoder.
  • the server system may be arranged to create a said record upon receipt of an initialization message from a said multimedia decoder.
  • the server system may be arranged to modify, in response to a message received from a multimedia decoder, a record corresponding to the respective multimedia decoder.
  • the server system may be arranged to generate the supplementary data transmitted to the multimedia decoder based on the corresponding record.
  • the server system is able to create a record in memory to log the user inputs.
  • the supplementary data, and thus the display data displayed to the user is created, not only to contain a part of the acknowledgement data from the user input, but also in dependence on the stored record, ensuring that the personalization of the display data can reflect to past input as well as current input.
  • the display data may request user input, for example by asking a question.
  • the question may be presented in the broadcast video stream; that is the question is presented to all users simultaneously. This enables users to compete against each other when attempting to provide an answer.
  • the question may be indicated by the supplementary data.
  • the server system may be arranged to start a timer upon transmitting a portion of the multimedia content.
  • the server system may be arranged such that when a message indicative of user input is received from a multimedia decoder, the modification of the corresponding record is dependent on the value of the timer when the message data was received.
  • the multimedia system When the multimedia system is used, for example, to provide an interactive game show, it is important that the timing of the user input is properly recorded. Therefore the system records when the e.g. a game show question is asked in the multimedia content so that the answers can be correctly logged.
  • the server system and the plurality of decoders may be connected via a network arranged to carry traffic of a first type and traffic of a second type, the traffic of the first type having a priority higher than traffic of the second type, and the supplementary data and optionally the message indicative of the user input may comprise traffic of the first type.
  • the system By sending the supplementary data (and the messages indicative of user input) separately and providing them with a higher priority than other traffic on the network (that is the traffic of the second type), the system is able to ensure that critical data (such as a user's answer in a game show) gets to its destination.
  • the video stream may comprise an audio stream
  • the processor may be arranged to combine the audio stream with an audio element created on the basis of said received supplementary data, whereby to generate audio data to be output by the interface .
  • a multimedia server system a decoder, a computer program or a suite of programs comprising a set of instructions to be executed on said server system and decoders as described herein.
  • Figure 1 is a schematic diagram illustrating a multimedia system in which embodimen ts of the invention may be used
  • Figure 2 is a schematic diagram illustrating a multimedia decoder in which embodiments of the invention may be used
  • Figure 3A illustrates examples of a broadcast video stream, graphical element which may be superimposed on the broadcast video stream, and the resultant display data in accordance with embodiments of the invention
  • Figure 3B illustrates examples of a broadcast video stream and graphical elements which may be superimposed on the broadcast video stream in accordance with embodiments of the invention.
  • FIGS. 4A, 4B and 4C illustrate methods according to embodiments of the invention. Detailed Description of the Invention
  • Embodiments of the invention are concerned with the transmission of multimedia content to multiple multimedia decoders (i.e. multimedia receivers). Such embodiments may conveniently be implemented as a multimedia system, in this embodiment an IPTV system, which will be described with reference to Figure 1. Many of the components of the system 100 are well known in the art and these will, consequently not be described in detail.
  • a television studio schematically represented by camera 102, records a live quiz show.
  • the studio 102 is connected to video broadcast server 104 via connection 106.
  • Video broadcast server 104 is connected to network 108 through connection 110.
  • Network 108 may be a dedicated .Internet Protocol (IP) network, a public IP network such as the Internet, or a combination of the two.
  • IP Internet Protocol
  • the network 108 may include fixed (i.e. wired) infrastructure and/or wireless infrastructure (such as WiFi and/or mobile telecommunications such as 3G or LTE).
  • the video broadcast server 104 is designed to broadcast (that is, in this embodiment, multicast) the live show recorded in the studio along connection 1 10 to any decoder 122 A, 122B connected to network 108 wishing to receive the video stream containing the live show.
  • the video broadcast server is also connected to a data server 112 via connection 1 14. This connection may be made via the network 108 or be a dedicated connection 114 as shown.
  • the data server 112 is connected to a storage system 116 via connection 118.
  • the data server 112 is additionally connected to network 108 through connection 120.
  • the data server 1 12 is designed to send and receive data to and from the decoders connected to network 108 (and in particular those receiving the video stream broadcast from the video broadcast server 104). Part of this data is supplementary data, to be described in detail below, which is unicast from the data server 1 12 to the decoders.
  • the storage system 1 16 connected to the data server 112 holds records relating to the decoders 122A, 122B connected, via the network 108, to the data server 112 and the video broadcast server 104.
  • a number of other servers may be connected to the video broadcast server 104 and the data server 1 12. These servers may include, for example, accounting and authorization servers to ensure that services provided by the system are paid for. The operation, of such servers is known in the art arid will not be described in detail here. Overall the interconnected servers form a multimedia server system.
  • a number of decoder systems are connected to network 108. For simplicity only two decoder systems, referenced 'A' and 'B', are shown, however typically many more will be connected to the network 108. The two decoder systems have analogous features, differentiated by the suffix A or B.
  • Each decoder system comprises a decoder 122 connected to the network 108 via connection 124 through which the decoder 1.22 sends and receives data.
  • the decoder 122 has a connection. 126 to a display device 128.
  • This display device may be a television screen, computer monitor, projector or any similar device capable of displaying an image.
  • the decoder 122 has a user input system. Any known means for enabling user input may be used.
  • the user input is provided by remote control 130 which has buttons which may be pressed by the user, and a transmitter, which uses, for example, infrared, Bluetooth, or other form of wireless signalling to signal the buttons being pressed to the decoder 122.
  • the studio 102 provides a live video stream to the video broadcast server 104 which broadcasts (i.e. multicasts) the video stream over the network 108 to the decoders 122 A and 122B.
  • the data server 112 selects or otherwise defines supplementary data which is unicast to the decoders 122A and 122B. Each decoder will have supplementary data selected for it. This selection is performed on the basis of data stored in the storage system 1 16 and/or on the basis of data received from the relevant decoder.
  • the supplementary data provides a way of personalizing the image produced by the relevant decoder. Consequently, the supplementary data may include an indication of whether the user of the decoder has logged into the system, and if so may provide personalized statistics and the like. Equally, if the user has just provided a user input, the supplementary data may comprise an acknowledgement of that user input.
  • the unicast supplementary data and/or the user input may be sent with a higher priority over the broadcast video stream, and/or over other traffic on the network 108.
  • the decoders 122A and 122B thus each receive the multicast video stream from the video broadcast server 104, and the unicast supplementary data from the data server 112.
  • the decoders 122 A and 122B use the video stream and supplementary data to produce display data which is sent to the displays 128 A and 128B respectively, to be viewed, by a user.
  • a user uses the remote controls 13 OA and 130B to provide user input to the decoders 122A and 122B respectively.
  • This user input is transmitted by the decoders 122 A and 122B to the data server 112 in one or more packets or messages.
  • the data server 112 receives these messages, and may change the supplementary data transmitted to the respective decoders 122 A and 122B based on the content of the messages whereby to provide a user with an acknowledgement of the user input being received by the data server 1 12.
  • the data server 1 12 may create or update a record associated with the decoders 122 A and 122B based on the messages received .
  • decoder system of an embodiment of the invention for example decoder system A or B, will now be described in more detail with reference to Figure 2.
  • network 108 is connected to decoder 122 via connection 124.
  • Display 128 is connected to the decoder 122 via connection 126.
  • Remote control 130 provides user input.
  • the decoder 122 is provided with a network interface 132 through which the decoder 122 communicates with network 108 via connection 124.
  • This network interface may, for example, be an Ethernet, WiFi, or mobile telecommunications interface, and the connection 124 may therefore be wired or wireless.
  • network 108, network interface 132 and/or connection 124 may include a number of networking elements such as a domestic IP router, firewall, NAT server, DCHP server and the like which are typically used in a domestic internet connection. These are well known and will not be described in any more detail.
  • the network interface is connected to a processor 134.
  • the processor is shown with four elements, a general processor/controller 136, which performs and controls the majority of the functions of the processor, and three dedicated elements 138, 140 and 142 whose purpose and interconnections will be described below.
  • the dedicated elements include a video stream decoder 138 and a graphical element generator 140.
  • the video stream decoder 138 and the graphical element generator 140 are in turn connected to a third dedicated element, overlay engine 142.
  • the overlay engine 142 provides output to a display interface 144 which is connected via connection 126 to display 128.
  • a remote control interface 146 is provided in the decoder 122 to receive signals from remote control 130. This remote control interface has a connection to processor 134.
  • the remote control is provided with four differently coloured buttons (red, green, yellow and blue), which will be used to provide user input as described below).
  • multimedia content that is a multicast video stream and unicast supplementary data
  • the video stream is passed to the video stream decoder 138 which decodes the video stream to provide image data which is sent to the overlay engine 142.
  • the supplementary data is provided to the graphical element generator 140 where a graphical element is generated based on the supplementary data. This graphical element is also provided to the overlay engine 142.
  • the graphical element is overlaid onto the hiiage data from the multicast video stream.
  • the resultant image data is then output by the processor to the display interface 144 and from there is provided to the display 128 for viewing by the user.
  • User input that is generated by the user pressing button(s) on the remote control 130 is signalled by the remote control 130 to the remote control interface 146.
  • the user input is then passed to the processor 134 where (typically in general processor/controller 136) it is encapsulated into the appropriate data format (i.e. data packets) to be output to the network interface 132 and from there be sent to the data server 1 12 shown in Figure 1.
  • the system is providing a game show programme, which has a live video stream whereby a studio game show host asks multiple choice questions.
  • the decoder systems A, B and C present four multiple choice answers (on the display) for a user to select.
  • the buttons on the remote control the users of the decoder systems can select one of the four answer options.
  • Figure 3A shows an example of how the broadcast video stream and a graphical element may be combined to provide display data to be output to the display device.
  • Example image 302 represents the broadcast video stream.
  • image 302 is an image of the quiz host 304. Above the host, the current question 306 is shown ("What is the capital of Peru?").
  • a timer 308 indicates the time left for an answer to be given (in this example the timer's value is 12 indicating that 12 seconds are left to answer).
  • box 310 which contains the four multiple choice answers to the question presented in a 2x2 grid format (the answers being, "Lima”; “Lama”; “Laos” and “Liechtenstein”).
  • the four boxes are each associated with, a different colour (red, green, yellow and blue) corresponding to the four buttons on the remote control 130, described, above with reference to Figure 2. To provide this effect the four answer boxes may be highlighted with a respective colour (this is not shown in Figure 3 A)
  • Example image 312 shows an example of a graphical element which is to be overlaid onto the video stream.
  • Some personal details of the user are presented in box 3 16, in this example these are the user's username (“Bob") and the user's average score in previous games ("6").
  • the user has selected an answer (corresponding to the top left answer, "Lima”), and therefore a highlighting box 314 is provided to acknowledge the selection.
  • the decoder is designed to overlay the graphical element 312 onto the broadcast video stream 302.
  • the result is shown by image 318, where it can be seen that the highlighting box 314 is overlaid over the answer box 310 from the video stream.
  • the result is that the answer "Lima” appears highlighted.
  • the user's personal details (box 31.6) are visible over the live feed.
  • the user is provided witli a personalized visual experience, while still viewing a live, broadcast, video stream.
  • the user is able to see their personal details, and an acknowledgement of the auswer (user input) provided.
  • other users who receive the same broadcast video stream, will see a different overlay graphical element and therefore be provided with a different overall image.
  • Users who have not subscribed to the game show (in this case simply viewers) will see only the video stream, with no overlay, and therefore will be able to view the game show, but will not be able to compete.
  • Figure 3B shows schematic examples representing the broadcast video stream and the graphical elements which may be generated for the decoders A, B and C.
  • Each decoder will overlay the graphical element identified with it onto the broadcast video stream in the manner described in reference to Figure 3 A.
  • tlie result is that each user views a different overall video stream.
  • the left hand column 320 shows example images from the broadcast video stream at different time points.
  • the remaining three columns 322, 324 and 326 (titled “Graphical Element for Decoder X", where X is replaced by A, B and C respectively), show the graphical elements which are generated in the respective decoders from the supplementary data received by the decoders at the same time points.
  • Each row represents a different time point in the game: the first row, referenced 330 is when the question has just been asked by tlie quiz host, and the timer has its initial value of 30.
  • the second row 332 is when the timer has a value of 12.
  • the third row 334 is when the timer has reached 3.
  • tlie last row 336 shows tlie situation when tlie time has reached zero, and the answer is revealed, along with whether the answers given were correct.
  • the quiz host asks the question.
  • the question is written across the top of the screen, the answers are provided in a box near the bottom of the screen and a timer is provided in the comer. It will be apparent that these elements are universal, that is they are to be shown to all users, therefore these elements are presented in the broadcast video stream.
  • the graphical element produced from the supplementary data for all three decoders shows only the personal data for each of the users/decoders (since no user has provided an answer at this time).
  • the live feed shows the host, the question, and the timer value (which is now 12), User A (of decoder A) has selected an answer (in this case the top left answer in the 2x2 grid of options corresponding to "Lima". Consequently, the graphical element is modified to acknowledge the selection; in this example this is done by providing a higlilighting box in the position of the selected answer. Neither User B nor User C (of decoders B and C respectively) have provided input to select an answer, and the corresponding graphical elements present only the personal information for the respective users.
  • the video stream indicates the correct answer by moving the correct answer away from the rest.
  • the game show host may announce the correct answer verbally.
  • the graphical elements for each of the users maintain the highlighting of the provided answer (if applicable) but in addition present the user with an indication of whether their answer was correct.
  • each user sees a different series of images on their display, providing a personalized experience and, importantly, providing them with an acknowledgement when they attempt an answer.
  • FIG 4A S the method starts at step 402A with the question being transmitted in the broadcast video stream.
  • the timer has a value of 30.
  • the video broadcast server 104 transmits a signal to the data server 112 indicating that the question has been asked.
  • step 404A the data server 1 12 receives the start signal, and transmits supplementary data to decoder A.
  • the data server 1 12 retrieves data from the record corresponding to decoder A from the storage system 116 and constructs supplementary data showing the usemame and average score (as described in relation, to Figure 3A).
  • the data server then arranges supplementary data, referenced as "SDOA", to include an indication of this data and sends the supplementary data to the decoder A.
  • SDOA supplementary data
  • step 406A the decoder A receives the supplementary data "SDOA”, and generates the corresponding overlay graphical element. This graphic is overlaid onto the live stream and the result is displayed to the user.
  • the above three steps correspond to the first row 330 in Figure 3B.
  • the next six steps (408A to 418A) correspond to the second row 332 where the timer has a value of 12.
  • the timer value is represented by the box 408 A.
  • step 41 OA the user decides that "Lima” is the answer.
  • each of the four answers are associated with a different colour.
  • the remote control is provided with four coloured buttons corresponding to the four colours of the four answer options. Therefore, to select "Lima", the user presses the coloured button on the remote control corresponding to the colour associated with "Lima”. This user input is transmitted by the remote control to the decoder, and in response, the decoder transmits a message to the data server 112 indicative of the button being pressed.
  • step 41.2 A die data server 112 receives the message indicating the user input.
  • step 414 A the data server 1 12 creates an acknowledgement to be sent back to the decoder A.
  • the data server 112 is able to determine from the user input message (indicative of the colour of button being pressed) "which option was selected. In this case it was the top right option in the 2x2 grid. Accordingly the data server 112 generates an acknowledgement of this selection.
  • This acknowledgement is sent as supplementary data (referenced as "SDl") and indicates an acknowledgement of the selection of the top left answer.
  • me data server 112 updates the memory 116 to record the result.
  • step 416A the supplementary data "SDl" is received by the decoder A.
  • the corresponding overlay graphical element is generated and overlaid onto the broadcast video stream and the result displayed to the user. As is shown in Figure 3B, the resultant graphical element highlights the answer "Lima”.
  • step 420 A the broadcast server transmits an end signal to the data server 1 12 indicating that the period for answering the question has ended.
  • this end signal is received by the data server 112.
  • the data server 112 determines that the decoder A had provided the correct answer ("Lima") within the allotted time. Therefore, the server 112 selects supplementary data (reference as "SD2") representing that the user is correct and sends this data to the decoder.
  • SD2 supplementary data
  • step 424A the decoder receives the supplementary data "SD2", and generates the corresponding overlay graphical element.
  • the graphic shows that the answer provided was correct.
  • the graphic is overlaid onto the live stream and the result displayed to the user.
  • steps 402B to 406B are the similar to steps 402A to 406A (although the supplementary data transmitted, referenced "SDOB" is selected for the decoder B - that is the username and average score presented are those of the user of decoder B).
  • the steps 408B to 416B occur when the timer has a value of 3 (as described above with reference to Figure 3B).
  • the steps are analogous to those in Figure 4A, and involve, hi step 41 OA, the user selecting the answer "Lama" (using a coloured button), and the user input being transmitted to the data server 11.2.
  • step 41.2B and 414B the server 112 reacts to the answer by generating an acknowledgement through selecting the appropriate supplementary data to represent the selection of answer "Lama”.
  • This supplementary data (referenced "SD3") is sent to the decoder B, where it causes the display of the graphical element shown in column 324, row 334 of Figure 3B.
  • the database 1 16 is updated to store the answer provided by the user.
  • the streaming server 104 again signals the data server ⁇ 2 in step 420B.
  • the data server 112 determines that while decoder B provided an answer (i.e. User B answered the question), the answer was wrong, and therefore the supplementary data should be provided to reflect that the incorrect answer was given.
  • the appropriate supplementary data (referenced as "SD4") is selected and transmitted to the decoder B.
  • step 424B the decoder receives the supplementary data and presents the user with the appropriate graphic overlay as shown in Figvire 3B.
  • steps 402C to 406C correspond to steps 402A to 406A (albeit with the personalized supplementary data "SD0C")
  • tliere are no steps in which the user provides an answer.
  • the broadcast video server 104 sends a signal indicating that a prompt time point has been reached. This prompt causes the data server 112 to determine if any decoders have not submitted an answer, and to send supplementary data representing an appropriate prompt if one is required.
  • step 428C the data server 1 12 receives the prompt from the video server 104, determines that no answer has been provided by the decoder C, and consequently selects and transmits supplementary data (referenced "SD5") which provides the user with an on-screen prompt.
  • step 430C the supplementary data "SD5" is received by the decoder C and consequently the graphical element includes a prompt ("Hurry Up!) which is displayed to the user as shown, in Figure 3B.
  • step 420C the timer once again reaches zero, and the broadcast server
  • the data server 1 12 determines that no answer was received before the timer expired, and tlierefore that the supplementary data should indicate to the user than no answer was given (referenced as "SD6")- This supplementary data is sent to the decoder C.
  • step 424C the decoder receives the supplementary data "SD6", and displays the corresponding overlay graphic.
  • the User C in an act of desperation, presses an answer button after the timer has expired.
  • a corresponding message may still be sent by the decoder to the data server 1.12; however the data server 1 12 will determine that the answer has arrived too late, and therefore the answer will not be accepted.
  • the data server 112 may be programmed to select and send supplementary data which signals to the user that the input was received (i.e. acknowledges the user input), but also indicates that the user input was received after the timer had expired.
  • the supplementary data has been provided by the data server 112 as 'push' data (i.e. it is sent to the decoder without being requested).
  • a 'pull' system could be used, in which the decoder periodically requests an update of the supplementary data and downloads this update accordingly.
  • a combination of 'push' and 'pull' may be used.
  • the acknowledgement of user input may be 'pushed', while the user's personal information (e.g. username and average score) are 'pulled'.
  • the choice of 'push' or 'pull' may be made based on whether the data is time critical.
  • the above described embodiments use the supplementary data to provide some personal details (username and average score) and an indication of the answer selected.
  • the supplementary data may be used to provide more, or different, information.
  • the graphical element described in relation to Figures 3A and 3B is deliberately simplistic. Substantially more complex graphical elements are envisaged.
  • the graphical element may be provided with moving elements, enabling the selected answer to 'pulse' for example.
  • a user may customize the nature of the graphical element which is overlaid onto the video stream, for example specifying the size and font of text.
  • the overlay of the graphical element may involve a modification of the underlying image, without masking the underlying image. For example, the overlay may darken, lighten or invert specific parts of the video stream image, without masking the image underneath. This may be used to highlight the selected answer, while keeping the text of the answer (which is presented in the video stream) visible.
  • the graphical element is determined from the supplementary data.
  • the supplementary data may contain all the elements which are to be displayed (i.e. to contain image data representing the overlay graphical element).
  • the decoder may receive and store (on e.g. initialization) a series of game related graphics and computer readable code.
  • the supplementary data may provide one or more variables which are used by this code to recall stored graphics whereby to generate the graphical element in. software.
  • the broadcast server 104 may broadcast non-video data in. addition to the video stream, and that this additional non-video data is interpreted by the decoder and included into the overlay graphical element.
  • the broadcast server may provide a video stream showing the game show host, alongside non- video data representing the question, answer options and the timer. The decoders may then interpret this data to generate a graphical overlay element which contains the question, answer, and the timer.
  • the supplementary data may be used to modify this non-video broadcast data to acknowledge an answer.
  • the supplementary data may simply provide an indication that the top right option has been selected, and the decoder may accordingly generate the overlay graphical element to reflect this selection,
  • the broadcast programme providing the game show may have an introduction period during winch each user can login,
  • a user When a user first selects the game show programme (i.e. changes to the relevant channel), the user may be presented with a request to press a particular button to login. This request may be provided in the video stream, and is therefore presented to all viewers/users during the introduction period.
  • the system may create a record for the user (in storage system 1 16), and send supplementary data to the user's decoder.
  • the supplementary data may only be provided once the user has indicated that they wish to compete in the game show. The supplementary data may then be used to confirm login and request and confirm any further details (if required).
  • the supplementary data may be used to request that the user logs in.
  • the system may be arranged to provide a form of default supplementary data, which is sent to all decoders until user input is received.
  • the supplementary data will be selected by the data server 112 to be specific to that decoder. Consequently the user is again provided with a personalized experience, while the broadcast video stream can be played in the background enabling the game show hosts to, for example, advertise the prizes available to any user viewing the channel.
  • the data server 1 12 may create a corresponding record in the storage system 116.
  • an existing record corresponding to that decoder may be updated or otherwise modified so as to be able to store the user's results. This record may then be used to log the answers of the user.
  • the decoder may identify itself in messages sent to the data server (by using, for example, a serial number). Alternatively the decoder may be identified by the server system based on, for example, an IP address.
  • the record may be associated with an identity of the decoder.
  • the records may be associated with a user, and therefore a given user can use different decoders.
  • Embodiments of the invention may be used for many other television 5 programmes to provide a personalise experience (i.e. displaying statistics during a sports fixture) or to allow the user to express an opinion (for example in a talent contest or online debate).
  • the supplementary data may be used to provide a form of interactive and/or personalized advertising (for example, during commercial breaks during the game show).
  • audio data may be treated in. an analogous way.
  • the broadcast video stream will be provided with an audio track recorded in the studio.
  • the supplementary data may be used to provide audio feedback specific to the user/decoder.
  • an acknowledgement sound may be played when an answer is provided by the user, and celebratory or commiseratory sounds may be played, when the answer is revealed (depending on whether the answer was right or wrong).
  • the video broadcast server 104 20 and the data server 112 are shown as being separate entities and as having separate connections to the network. This may not necessarily be the case, and any appropriate arrangement of one or more servers which allows for the video broadcast data to be multicast, the supplementary data to be unicast and for the messages from the decoders to be received is envisaged.
  • a timing signal is described as being sent from, the video broadcast server 104. Such a timing signal may not be required (depending on the game format). Equally a coordinating server or the like may be used to provide timing signals.
  • IP multicasting to send the broadcast video 30 stream to the decoders.
  • any form of broadcasting can be used to provide the video stream.
  • a satellite or terrestrial broadcast system could be used for the broadcast stream, alongside a domestic internet connection for the supplementary data and messages,
  • the decoder system is shown as having separate decoder, display and remote control.
  • the decoder may take the form of a set-top-box connected to a television.
  • a game console, media player or DVD player may be used.
  • embodiments of the invention may be used on a personal computer, wherein the display is a computer monitor, and the remote control is replaced by a keyboard or mouse.
  • a tablet PC, PDA or smartphone could perform the function of a decoder system, with the buttons or a touch screen providing the user input.
  • the user input means is in communication with the decoder, which is thus able to receive the user input and transmit messages indicative of the user input to the data server 112.
  • the user input is provided in a separate system to the decoder.
  • This separate system may have an entirely separate connection to the data server 1 12, for example being a smartphone or other mobile device connected using a cellular network.
  • the user device is associated with the decoder in some manner (such as by a unique username), and the answers provided using the user input are transmitted directly to the data sever 112. Nevertheless, despite this separate connection, the broadcast video stream and the supplementary data are still provided to the decoder in the usual manner to provide the user with a personalized visual (and audio) experience.
  • references to a user are references to the associated decoder.
  • the user selects an answer by pressing a coloured button on the remote control.
  • the answers may be provided with numbers (corresponding to number keys) or in a North, South, East, West configuration corresponding to four arrow keys (Up, Down, Left and Right).
  • the processor 134 was described as being a single entity comprising a number of discrete elements. It will be apparent that this arrangement was purely exemplary, and that any processor, chipset, or system that is able to decode the video stream, generate the graphical element, overlay the graphical element onto the video stream, and communicate with the network, display and remote control interfaces will be able to replace tire processor 134.

Abstract

A multimedia system has a server system which provides multimedia content comprising a video stream and supplementary data to a plurality of multimedia decoders. Each multimedia decoder has a processor which overlays onto a received video stream a graphical element created on the basis of received supplementary data. The server system is arranged such that the supplementary data to be sent to a multimedia decode is separately selected for each multimedia decode based on an input from the multimedia decoder.

Description

Improvements to Interactive Multimedia Systems
Field of the Invention
The present invention relates to a system, apparatus, method and computer software for improved multimedia systems, and in particular for providing an interactive multimedia experience to a group of users.
Background of the Invention
Broadcast television systems are well known. In such systems, a broadcast video stream is provided from a single source to a large number of receivers. The broadcast video stream may be transmitted using terrestrial or satellite based radio frequency transmissions, or using physical cabling laid between the transmitter and the receivers.
Recently, with the advent of high speed packet switched networks, a further form of television system has emerged. Internet Protocol Television (IPTV) uses a packet switched data network to transmit television programme data. This programme data may be unicast; that is the packets mal ing up a given programme are each sent from a source to a single receiver. However, unicasting is inefficient in terms of network resources, in particular for television programmes that are broadcast to many receivers, since separate packets have to be sent to the separate receivers, even if the separate packets have the same payload and only differ in address.
Consequently, television programmes are typically broadcast using a teclmique known as multicasting. Multicasting allows for a packet to be addressed to multiple receivers, and can dramatically reduce network usage since the packet need only be sent once from the transmitter (or source), and, is propagated through the network by bespoke intermediate network nodes in order to reach the receivers.
One popular type of television programme is a game show or quiz show. In such a programme, a contestant in a television studio attempts to answer questions in an effort to win prizes. A remote viewer's (i.e. one viewing the show on a screen and not in the television studio) participation in such shows is limited, in that the viewer can, at best, merely try and answer the question before the studio contestant and has no method of keeping a score, or to claim a prize, should they answer questions correctly.
Consequently, there is a need for a system which enables a viewer to interact with a live, broadcast, game show. By allowing the viewer to interact, a viewer may both compete and win prizes and thereby become a contestant. Moreover, by allowing multiple viewers to compete across a broadcast system, novel formats of game show are enabled, where large numbers of viewers (as opposed to small numbers of studio contestants) can compete for a prize.
There is a need for such a system to be able to offer a viewer (now also a contestant) a personalized experience. That is, there is a need for a system which is able to offer the viewer/contestant (hencefoitli a user) an aclaiowledgement of their answer and a record of, for example, their score or other personalized data.
Finally, there is a need in such a system which enables a user to receive clear confirmation that an answer made during a live quiz has been received by the system (i.e. the quiz hosts). This last requirement is important since, if a substantial prize is being offered in the quiz, an error in receiving a user's answer may result in, at best, a bad user experience, and, at worst, complaints or a legal action.
Summary of the Invention
In accordance with at least one embodiment of the invention, methods, devices, systems and software are provided for supporting or implementing functionality to enable the provision of imiltimedia content, as specified in the independent claims.
This is achieved by a combination of features recited in each independent claim. Accordingly, dependent claims prescribe further detailed implementations of the present invention. More specifically, according to a first aspect of the invention there is provided a multimedia system comprising: a server system arranged to provide multimedia content, the multimedia content comprising a video stream and supplementary data; and a plurality of multimedia decoders, each multimedia decoder comprising: an interface for receiving the multimedia content and for outputting display data to a display device; a processor arranged to overlay onto a received said video stream a graphical element created on the basis of said received supplementary data, whereby to generate said display data, wherein the server system is arranged to: receive input relating to the multimedia content from a said multimedia decoder; broadcast the video stream to at least some of the multimedia decoders; and transmit, separately from the video stream, said supplementary data to said at least some of the multimedia decoders, and wherein the supplementary data transmitted to a given said multimedia decoder is selected in dependence on the input received from the given said multimedia decoder.
The input received from a said multimedia decoder may comprise an identity of the said multimedia decoder.
By transmitting the supplementary data separately from the video stream the system is able to ensure that the display data sent to the display device (and therefore viewed by a user) is personalized in some way beyond the broadcast video stream. In particular, by selectmg the supplementary data on the basis of input received from the respective multimedia decoder, the display data for each multimedia decoder can be made unique to that decoder. More advantageously, the unique display data can be provided while still talcing advantage of the savings (in terms of network resources) afforded by broadcasting (or multicasting) the bulk of the data (i.e. the video stream).
The multimedia system may comprise: input means associated with each multimedia decoder, each input means arranged to receive user input, and, responsive to user input of a predetermined type, to cause a message indicative of the said user input to be sent to the server system, wherein the server system may be arranged, for each said message, to create acknowledgement data in dependence on the message, wherein the supplementary data transmitted to the multimedia decoder associated with the respective input means comprises at least a part of the created acknowledgement data, and wherein the graphical element created by the processor of a given multimedia decoder includes a visual indication of said acknowledgement data.
It is important that a user receives confirmation of a user input, especially in situations such as a game show, where the user response is both important (in terms of their continued participation in the game) and time dependent. Consequently the system, by sending at least part of the acknowledgement data to the multimedia decoder (as part of the supplementary data), is able to provide the user with visual confirmation that, not only has their input been received by the input means, but that the input has been received by the server system (where it can be properly recorded in a memory or database). Importantly by separating the acknowledgement from the video stream, the system is able to utilize IP precedence support in the network. That is, the data making up the acknowledgement can be given a priority higher than that of the video stream and/or other network traffic.
Each interface of the multimedia decoders may comprise said associated input means, and wherein the multimedia decoder is arranged to send the message indicative of the user input via the interface.
Preferably the input means is a part of the decoder, being for example a remote control or other input device. However it is possible that the input means is separate from the decoder, for example being an application on a computer or smartphone, and has an independent network connection to the server system.
The server system may be arranged to access a storage system arranged to hold records corresponding to a respective multimedia decoder. The server system may be arranged to create a said record upon receipt of an initialization message from a said multimedia decoder. The server system may be arranged to modify, in response to a message received from a multimedia decoder, a record corresponding to the respective multimedia decoder. For each multimedia decoder, the server system may be arranged to generate the supplementary data transmitted to the multimedia decoder based on the corresponding record.
Advantageously the server system is able to create a record in memory to log the user inputs. The supplementary data, and thus the display data displayed to the user, is created, not only to contain a part of the acknowledgement data from the user input, but also in dependence on the stored record, ensuring that the personalization of the display data can reflect to past input as well as current input.
To provide an interactive user experience, the display data may request user input, for example by asking a question. The question may be presented in the broadcast video stream; that is the question is presented to all users simultaneously. This enables users to compete against each other when attempting to provide an answer. Alternatively, if the question is specific to that user, then the question may be indicated by the supplementary data.
The server system may be arranged to start a timer upon transmitting a portion of the multimedia content.
The server system may be arranged such that when a message indicative of user input is received from a multimedia decoder, the modification of the corresponding record is dependent on the value of the timer when the message data was received.
When the multimedia system is used, for example, to provide an interactive game show, it is important that the timing of the user input is properly recorded. Therefore the system records when the e.g. a game show question is asked in the multimedia content so that the answers can be correctly logged.
The server system and the plurality of decoders may be connected via a network arranged to carry traffic of a first type and traffic of a second type, the traffic of the first type having a priority higher than traffic of the second type, and the supplementary data and optionally the message indicative of the user input may comprise traffic of the first type.
By sending the supplementary data (and the messages indicative of user input) separately and providing them with a higher priority than other traffic on the network (that is the traffic of the second type), the system is able to ensure that critical data (such as a user's answer in a game show) gets to its destination.
The video stream may comprise an audio stream, and the processor may be arranged to combine the audio stream with an audio element created on the basis of said received supplementary data, whereby to generate audio data to be output by the interface .
The use of a broadcast video stream and supplementary data is not limited to visual data. Consequently audio data can be transmitted in a similar manner with similar advantages.
According to further aspects of the invention there is provided a multimedia server system, a decoder, a computer program or a suite of programs comprising a set of instructions to be executed on said server system and decoders as described herein.
Further features and advantages of the invention will become apparent from the following description of preferred embodiments of the invention, given by way of example only, which is made with reference to the accompanying drawings.
Brief Description of the Drawings
A system for providing multimedia content will now be described as an embodiment of the present invention, by way of example only, with reference to the accompanying figures in which:
Figure 1 is a schematic diagram illustrating a multimedia system in which embodimen ts of the invention may be used;
Figure 2 is a schematic diagram illustrating a multimedia decoder in which embodiments of the invention may be used; Figure 3A illustrates examples of a broadcast video stream, graphical element which may be superimposed on the broadcast video stream, and the resultant display data in accordance with embodiments of the invention;
Figure 3B illustrates examples of a broadcast video stream and graphical elements which may be superimposed on the broadcast video stream in accordance with embodiments of the invention; and
Figures 4A, 4B and 4C illustrate methods according to embodiments of the invention. Detailed Description of the Invention
Embodiments of the invention are concerned with the transmission of multimedia content to multiple multimedia decoders (i.e. multimedia receivers). Such embodiments may conveniently be implemented as a multimedia system, in this embodiment an IPTV system, which will be described with reference to Figure 1. Many of the components of the system 100 are well known in the art and these will, consequently not be described in detail.
Within the system 100, a television studio, schematically represented by camera 102, records a live quiz show. The studio 102 is connected to video broadcast server 104 via connection 106. Video broadcast server 104 is connected to network 108 through connection 110. Network 108 may be a dedicated .Internet Protocol (IP) network, a public IP network such as the Internet, or a combination of the two. The network 108 may include fixed (i.e. wired) infrastructure and/or wireless infrastructure (such as WiFi and/or mobile telecommunications such as 3G or LTE). The video broadcast server 104 is designed to broadcast (that is, in this embodiment, multicast) the live show recorded in the studio along connection 1 10 to any decoder 122 A, 122B connected to network 108 wishing to receive the video stream containing the live show.
The video broadcast server is also connected to a data server 112 via connection 1 14. This connection may be made via the network 108 or be a dedicated connection 114 as shown. The data server 112 is connected to a storage system 116 via connection 118. The data server 112 is additionally connected to network 108 through connection 120. The data server 1 12 is designed to send and receive data to and from the decoders connected to network 108 (and in particular those receiving the video stream broadcast from the video broadcast server 104). Part of this data is supplementary data, to be described in detail below, which is unicast from the data server 1 12 to the decoders.
The storage system 1 16 connected to the data server 112 holds records relating to the decoders 122A, 122B connected, via the network 108, to the data server 112 and the video broadcast server 104.
In addition, although not shown, a number of other servers may be connected to the video broadcast server 104 and the data server 1 12. These servers may include, for example, accounting and authorization servers to ensure that services provided by the system are paid for. The operation, of such servers is known in the art arid will not be described in detail here. Overall the interconnected servers form a multimedia server system.
A number of decoder systems are connected to network 108. For simplicity only two decoder systems, referenced 'A' and 'B', are shown, however typically many more will be connected to the network 108. The two decoder systems have analogous features, differentiated by the suffix A or B.
Each decoder system comprises a decoder 122 connected to the network 108 via connection 124 through which the decoder 1.22 sends and receives data. The decoder 122 has a connection. 126 to a display device 128. This display device may be a television screen, computer monitor, projector or any similar device capable of displaying an image.
In addition the decoder 122 has a user input system. Any known means for enabling user input may be used. In this embodiment, the user input is provided by remote control 130 which has buttons which may be pressed by the user, and a transmitter, which uses, for example, infrared, Bluetooth, or other form of wireless signalling to signal the buttons being pressed to the decoder 122.
In operation, the studio 102 provides a live video stream to the video broadcast server 104 which broadcasts (i.e. multicasts) the video stream over the network 108 to the decoders 122 A and 122B.
In addition, the data server 112 selects or otherwise defines supplementary data which is unicast to the decoders 122A and 122B. Each decoder will have supplementary data selected for it. This selection is performed on the basis of data stored in the storage system 1 16 and/or on the basis of data received from the relevant decoder.
As will be described in more detail below, the supplementary data provides a way of personalizing the image produced by the relevant decoder. Consequently, the supplementary data may include an indication of whether the user of the decoder has logged into the system, and if so may provide personalized statistics and the like. Equally, if the user has just provided a user input, the supplementary data may comprise an acknowledgement of that user input. The unicast supplementary data and/or the user input may be sent with a higher priority over the broadcast video stream, and/or over other traffic on the network 108.
The decoders 122A and 122B thus each receive the multicast video stream from the video broadcast server 104, and the unicast supplementary data from the data server 112. The decoders 122 A and 122B use the video stream and supplementary data to produce display data which is sent to the displays 128 A and 128B respectively, to be viewed, by a user.
A user uses the remote controls 13 OA and 130B to provide user input to the decoders 122A and 122B respectively. This user input is transmitted by the decoders 122 A and 122B to the data server 112 in one or more packets or messages.
The data server 112 receives these messages, and may change the supplementary data transmitted to the respective decoders 122 A and 122B based on the content of the messages whereby to provide a user with an acknowledgement of the user input being received by the data server 1 12. In addition the data server 1 12 may create or update a record associated with the decoders 122 A and 122B based on the messages received .
A decoder system of an embodiment of the invention, for example decoder system A or B, will now be described in more detail with reference to Figure 2.
Features appearing in Figure 1 will be given the same reference numerals for consistency. Therefore, network 108, as shown, is connected to decoder 122 via connection 124. Display 128 is connected to the decoder 122 via connection 126. Remote control 130 provides user input.
In addition, the decoder 122 is provided with a network interface 132 through which the decoder 122 communicates with network 108 via connection 124. This network interface may, for example, be an Ethernet, WiFi, or mobile telecommunications interface, and the connection 124 may therefore be wired or wireless. While not shown, network 108, network interface 132 and/or connection 124 may include a number of networking elements such as a domestic IP router, firewall, NAT server, DCHP server and the like which are typically used in a domestic internet connection. These are well known and will not be described in any more detail.
Within the decoder 122, the network interface is connected to a processor 134. For the purposes of this example, the processor is shown with four elements, a general processor/controller 136, which performs and controls the majority of the functions of the processor, and three dedicated elements 138, 140 and 142 whose purpose and interconnections will be described below.
The dedicated elements include a video stream decoder 138 and a graphical element generator 140. The video stream decoder 138 and the graphical element generator 140 are in turn connected to a third dedicated element, overlay engine 142. The overlay engine 142 provides output to a display interface 144 which is connected via connection 126 to display 128. A remote control interface 146 is provided in the decoder 122 to receive signals from remote control 130. This remote control interface has a connection to processor 134. In this exemplary embodiment, the remote control is provided with four differently coloured buttons (red, green, yellow and blue), which will be used to provide user input as described below).
In operation, multimedia content, that is a multicast video stream and unicast supplementary data, is received by network interface 132. The video stream is passed to the video stream decoder 138 which decodes the video stream to provide image data which is sent to the overlay engine 142. In addition, at least some of the supplementary data is provided to the graphical element generator 140 where a graphical element is generated based on the supplementary data. This graphical element is also provided to the overlay engine 142.
Within the overlay engine 142, the graphical element is overlaid onto the hiiage data from the multicast video stream. The resultant image data is then output by the processor to the display interface 144 and from there is provided to the display 128 for viewing by the user.
User input that is generated by the user pressing button(s) on the remote control 130 is signalled by the remote control 130 to the remote control interface 146. The user input is then passed to the processor 134 where (typically in general processor/controller 136) it is encapsulated into the appropriate data format (i.e. data packets) to be output to the network interface 132 and from there be sent to the data server 1 12 shown in Figure 1.
An example of how the system described above provides multimedia content in embodiments of the invention will now be described with reference to Figures 3A, 3B, 4A, 4B and 4C. In this example, the system is providing a game show programme, which has a live video stream whereby a studio game show host asks multiple choice questions. There are three decoder systems, referenced A, B and C, all connected to the network 108 and all receiving the game show programme. When a question is asked, the decoder systems A, B and C present four multiple choice answers (on the display) for a user to select. Using the buttons on the remote control, the users of the decoder systems can select one of the four answer options. There is a timer of 30 seconds during which the user needs to answer. At the end of the 30 seconds, the user is informed whether the answer given was correct (or that they needed to have provided an answer).
Figure 3A shows an example of how the broadcast video stream and a graphical element may be combined to provide display data to be output to the display device.
Example image 302 represents the broadcast video stream. Within image 302 is an image of the quiz host 304. Above the host, the current question 306 is shown ("What is the capital of Peru?"). In the bottom right of the image a timer 308 indicates the time left for an answer to be given (in this example the timer's value is 12 indicating that 12 seconds are left to answer). At the bottom of the image is provided a box 310 which contains the four multiple choice answers to the question presented in a 2x2 grid format (the answers being, "Lima"; "Lama"; "Laos" and "Liechtenstein"). The four boxes are each associated with, a different colour (red, green, yellow and blue) corresponding to the four buttons on the remote control 130, described, above with reference to Figure 2. To provide this effect the four answer boxes may be highlighted with a respective colour (this is not shown in Figure 3 A)
Example image 312 shows an example of a graphical element which is to be overlaid onto the video stream. Some personal details of the user are presented in box 3 16, in this example these are the user's username ("Bob") and the user's average score in previous games ("6"). In addition, the user has selected an answer (corresponding to the top left answer, "Lima"), and therefore a highlighting box 314 is provided to acknowledge the selection.
As described above with reference to Figure 2, the decoder is designed to overlay the graphical element 312 onto the broadcast video stream 302. The result is shown by image 318, where it can be seen that the highlighting box 314 is overlaid over the answer box 310 from the video stream. The result is that the answer "Lima" appears highlighted. In addition, the user's personal details (box 31.6) are visible over the live feed.
As can be seen from Figure 3 A, the user is provided witli a personalized visual experience, while still viewing a live, broadcast, video stream. In particular the user is able to see their personal details, and an acknowledgement of the auswer (user input) provided. Equally, other users, who receive the same broadcast video stream, will see a different overlay graphical element and therefore be provided with a different overall image. Users who have not subscribed to the game show (in this case simply viewers) will see only the video stream, with no overlay, and therefore will be able to view the game show, but will not be able to compete.
Figure 3B shows schematic examples representing the broadcast video stream and the graphical elements which may be generated for the decoders A, B and C. Each decoder will overlay the graphical element identified with it onto the broadcast video stream in the manner described in reference to Figure 3 A. As will be evident, tlie result is that each user views a different overall video stream.
The left hand column 320 (heading "Broadcast Video Stream") shows example images from the broadcast video stream at different time points. The remaining three columns 322, 324 and 326 (titled "Graphical Element for Decoder X", where X is replaced by A, B and C respectively), show the graphical elements which are generated in the respective decoders from the supplementary data received by the decoders at the same time points.
Each row represents a different time point in the game: the first row, referenced 330 is when the question has just been asked by tlie quiz host, and the timer has its initial value of 30. The second row 332 is when the timer has a value of 12. The third row 334 is when the timer has reached 3. Finally, tlie last row 336 shows tlie situation when tlie time has reached zero, and the answer is revealed, along with whether the answers given were correct. In general, the live feed shows the quiz host, question, possible answers and the timer, while the graphical elements show personal information, give feedback as to whether an answer has been selected and, at the end (timer = 0) whether that answer is correct.
At timer = 30, the quiz host asks the question. As previously described with reference to Figure 3 A, the question is written across the top of the screen, the answers are provided in a box near the bottom of the screen and a timer is provided in the comer. It will be apparent that these elements are universal, that is they are to be shown to all users, therefore these elements are presented in the broadcast video stream.
At this time the graphical element produced from the supplementary data for all three decoders shows only the personal data for each of the users/decoders (since no user has provided an answer at this time).
At timer = 12, the live feed shows the host, the question, and the timer value (which is now 12), User A (of decoder A) has selected an answer (in this case the top left answer in the 2x2 grid of options corresponding to "Lima". Consequently, the graphical element is modified to acknowledge the selection; in this example this is done by providing a higlilighting box in the position of the selected answer. Neither User B nor User C (of decoders B and C respectively) have provided input to select an answer, and the corresponding graphical elements present only the personal information for the respective users.
At timer = 3 User B makes a guess, and selects the top right answer in the 2x2 grid of options corresponding to "Lama"). Accordingly, the graphical element changes to highlight and acknowledge this selection. User C has still not made any attempt to answer, and since the time is almost up, the User C is provided with a prompt, in this case "Hurry Up!" which is presented across the middle of the screen. This prompt is not provided to Users A and B, since they have both already answered. At timer = 0 (i.e. when the time has run out), the video stream and the graphical elements of each of the decoders changes to provide the users with an indication of the correct answer, and whether they were right (or in the case of decoder C, that no answer was provided).
The video stream indicates the correct answer by moving the correct answer away from the rest. In addition, the game show host may announce the correct answer verbally. The graphical elements for each of the users maintain the highlighting of the provided answer (if applicable) but in addition present the user with an indication of whether their answer was correct.
For decoder A, the answer was correct and the graphical element reflects this by presenting a confirmation (text "Correct!"). The answer provided is still highlighted, and consequently the highlighting box is moved to maintain its position with relation to the answer box (so that the correct highlighting effect is produced).
For decoder B, the answer was incorrect, and the text "Incorrect!" is displayed. The selected answer remains highlighted.
Finally, user C had not provided an answer by the time the timer had expired. Therefore the text "No Answer!" is displayed. No highlighting box is shown, since no answer was selected.
As can. be seen from Figure 3B, each user sees a different series of images on their display, providing a personalized experience and, importantly, providing them with an acknowledgement when they attempt an answer.
The signalling processes between the servers and the decoders (A, B and C) which correspond to the timelines in Figure 3B will now be described with reference to Figures 4A, 4B and 4C. In each Figure there are three columns, each representing the steps performed by a particular entity. The leftmost column represents the steps undertaken by the video broadcast server 104 from Figure 1. The middle column represents the steps undertaken by the data server 112 from Figure 1. Finally the rightmost column represents the steps performed by the respective decoders. In these Figures, steps which are the same, or similar (with only the variation dependent on the decoder) are given the same reference numerals, differentiated by the respective suffixes A, B and C.
In Figure 4AS the method starts at step 402A with the question being transmitted in the broadcast video stream. The timer has a value of 30. The video broadcast server 104 transmits a signal to the data server 112 indicating that the question has been asked.
In step 404A, the data server 1 12 receives the start signal, and transmits supplementary data to decoder A. The data server 1 12 retrieves data from the record corresponding to decoder A from the storage system 116 and constructs supplementary data showing the usemame and average score (as described in relation, to Figure 3A). The data server then arranges supplementary data, referenced as "SDOA", to include an indication of this data and sends the supplementary data to the decoder A.
In step 406A, the decoder A receives the supplementary data "SDOA", and generates the corresponding overlay graphical element. This graphic is overlaid onto the live stream and the result is displayed to the user.
The above three steps correspond to the first row 330 in Figure 3B. The next six steps (408A to 418A) correspond to the second row 332 where the timer has a value of 12. The timer value is represented by the box 408 A.
In step 41 OA, the user decides that "Lima" is the answer. As described above, each of the four answers are associated with a different colour. Equally, the remote control is provided with four coloured buttons corresponding to the four colours of the four answer options. Therefore, to select "Lima", the user presses the coloured button on the remote control corresponding to the colour associated with "Lima". This user input is transmitted by the remote control to the decoder, and in response, the decoder transmits a message to the data server 112 indicative of the button being pressed.
In step 41.2 A die data server 112 receives the message indicating the user input. In step 414 A the data server 1 12 creates an acknowledgement to be sent back to the decoder A. The data server 112 is able to determine from the user input message (indicative of the colour of button being pressed) "which option was selected. In this case it was the top right option in the 2x2 grid. Accordingly the data server 112 generates an acknowledgement of this selection. This acknowledgement is sent as supplementary data (referenced as "SDl") and indicates an acknowledgement of the selection of the top left answer. In addition, in step 18A, me data server 112 updates the memory 116 to record the result.
In step 416A the supplementary data "SDl" is received by the decoder A. The corresponding overlay graphical element is generated and overlaid onto the broadcast video stream and the result displayed to the user. As is shown in Figure 3B, the resultant graphical element highlights the answer "Lima".
There is no change when the timer reaches 3, therefore the next three steps occur when the timer reaches zero, ending the answer window for that question. These three steps correspond to the last row 336 of Figure 3B,
In step 420 A the broadcast server transmits an end signal to the data server 1 12 indicating that the period for answering the question has ended.
In steps 422 A, this end signal is received by the data server 112. The data server 112 then determines that the decoder A had provided the correct answer ("Lima") within the allotted time. Therefore, the server 112 selects supplementary data (reference as "SD2") representing that the user is correct and sends this data to the decoder.
In step 424A the decoder receives the supplementary data "SD2", and generates the corresponding overlay graphical element. The graphic, as shown in Figure 3B, shows that the answer provided was correct. The graphic is overlaid onto the live stream and the result displayed to the user.
The process in Figure 4B is similar in many ways to that in Figure 4A. 'Therefore a number of steps will be only briefly mentioned, the operation of the entities in these steps being the sarae or at least analogous to those described with reference to Figure 4A. More specifically, steps 402B to 406B are the similar to steps 402A to 406A (although the supplementary data transmitted, referenced "SDOB" is selected for the decoder B - that is the username and average score presented are those of the user of decoder B). However, unlike in. Figure 4 A, the steps 408B to 416B occur when the timer has a value of 3 (as described above with reference to Figure 3B). Nevertheless, the steps are analogous to those in Figure 4A, and involve, hi step 41 OA, the user selecting the answer "Lama" (using a coloured button), and the user input being transmitted to the data server 11.2.
In steps 41.2B and 414B the server 112 reacts to the answer by generating an acknowledgement through selecting the appropriate supplementary data to represent the selection of answer "Lama". This supplementary data (referenced "SD3") is sent to the decoder B, where it causes the display of the graphical element shown in column 324, row 334 of Figure 3B.
As in Figure 4A, the database 1 16 is updated to store the answer provided by the user.
When the timer reaches zero, the streaming server 104 again signals the data server Π2 in step 420B. However in this case, the data server 112 determines that while decoder B provided an answer (i.e. User B answered the question), the answer was wrong, and therefore the supplementary data should be provided to reflect that the incorrect answer was given. The appropriate supplementary data (referenced as "SD4") is selected and transmitted to the decoder B.
In step 424B the decoder receives the supplementary data and presents the user with the appropriate graphic overlay as shown in Figvire 3B.
As stated above, the User C failed, to answer. Accordingly, while steps 402C to 406C correspond to steps 402A to 406A (albeit with the personalized supplementary data "SD0C"), tliere are no steps in which the user provides an answer. Nevertheless, when the timer reaches 3, the system is designed to provide any user who has not provided an answer with a prompt (shown in Figure 3B as "Hurry Up!"). Consequently, in step 426C, the broadcast video server 104 sends a signal indicating that a prompt time point has been reached. This prompt causes the data server 112 to determine if any decoders have not submitted an answer, and to send supplementary data representing an appropriate prompt if one is required.
Therefore, in step 428C the data server 1 12 receives the prompt from the video server 104, determines that no answer has been provided by the decoder C, and consequently selects and transmits supplementary data (referenced "SD5") which provides the user with an on-screen prompt.
In step 430C the supplementary data "SD5" is received by the decoder C and consequently the graphical element includes a prompt ("Hurry Up!") which is displayed to the user as shown, in Figure 3B.
While these steps were not shown in Figures 4A and 4B, in both cases it is expected that the data server 112 would determine that an answer had been provided from the relevant decoder, and therefore make the decision not to change the supplementary data sent to the decoder (i.e. not to indicate "Hurry Up!").
At step 420C the timer once again reaches zero, and the broadcast server
104 transmits the end signal. Upon receipt of this signal, at step 422C, the data server 1 12 determines that no answer was received before the timer expired, and tlierefore that the supplementary data should indicate to the user than no answer was given (referenced as "SD6")- This supplementary data is sent to the decoder C.
In step 424C the decoder receives the supplementary data "SD6", and displays the corresponding overlay graphic.
It may be the case that the User C, in an act of desperation, presses an answer button after the timer has expired. In such cases, a corresponding message may still be sent by the decoder to the data server 1.12; however the data server 1 12 will determine that the answer has arrived too late, and therefore the answer will not be accepted. In such cases, the data server 112 may be programmed to select and send supplementary data which signals to the user that the input was received (i.e. acknowledges the user input), but also indicates that the user input was received after the timer had expired.
Adaptations and Modifications
The above embodiments are to be understood as illustrative examples of the invention. Further embodiments of the invention are envisaged.
In the above embodiments, the supplementary data has been provided by the data server 112 as 'push' data (i.e. it is sent to the decoder without being requested). This does not have to be the case, and a 'pull' system could be used, in which the decoder periodically requests an update of the supplementary data and downloads this update accordingly. Alternatively, a combination of 'push' and 'pull' may be used. For example, the acknowledgement of user input may be 'pushed', while the user's personal information (e.g. username and average score) are 'pulled'. The choice of 'push' or 'pull' may be made based on whether the data is time critical.
The above described embodiments use the supplementary data to provide some personal details (username and average score) and an indication of the answer selected. However, the invention is not limited to this. The supplementary data may be used to provide more, or different, information.
Equally, the graphical element described in relation to Figures 3A and 3B is deliberately simplistic. Substantially more complex graphical elements are envisaged. For example, the graphical element may be provided with moving elements, enabling the selected answer to 'pulse' for example. Equally a user may customize the nature of the graphical element which is overlaid onto the video stream, for example specifying the size and font of text. The overlay of the graphical element may involve a modification of the underlying image, without masking the underlying image. For example, the overlay may darken, lighten or invert specific parts of the video stream image, without masking the image underneath. This may be used to highlight the selected answer, while keeping the text of the answer (which is presented in the video stream) visible.
In the system described above, the graphical element is determined from the supplementary data. However this does not require the supplementary data to contain all the elements which are to be displayed (i.e. to contain image data representing the overlay graphical element). For example, the decoder may receive and store (on e.g. initialization) a series of game related graphics and computer readable code. Accordingly, the supplementary data may provide one or more variables which are used by this code to recall stored graphics whereby to generate the graphical element in. software.
The above embodiments have been described in terms of a broadcast video stream being overlaid by a graphical element derived from unicast supplementary data. However, it is possible for the broadcast server 104 to broadcast non-video data in. addition to the video stream, and that this additional non-video data is interpreted by the decoder and included into the overlay graphical element. For example, the broadcast server may provide a video stream showing the game show host, alongside non- video data representing the question, answer options and the timer. The decoders may then interpret this data to generate a graphical overlay element which contains the question, answer, and the timer.
In such an embodiment, the supplementary data may be used to modify this non-video broadcast data to acknowledge an answer. For example, the supplementary data may simply provide an indication that the top right option has been selected, and the decoder may accordingly generate the overlay graphical element to reflect this selection,
The methods described with reference to Figures 4A, 4B and 4C have been described in the context of a user providing an answer in a game show. However this is not the only use for embodiments of the invention. One use, still in the context of a game show or similar, is to enable a user to login or initialize for the game show. To login, the user may simply be required to press one of the coloured buttons (so as to indicate their desire to compete rather than simply view). Alternatively a more complex procedure may be required whereby a user provides a useraame and password.
As such, the broadcast programme providing the game show may have an introduction period during winch each user can login, When a user first selects the game show programme (i.e. changes to the relevant channel), the user may be presented with a request to press a particular button to login. This request may be provided in the video stream, and is therefore presented to all viewers/users during the introduction period. Upon, the user pressing the appropriate button the system may create a record for the user (in storage system 1 16), and send supplementary data to the user's decoder. As such, the supplementary data may only be provided once the user has indicated that they wish to compete in the game show. The supplementary data may then be used to confirm login and request and confirm any further details (if required).
Alternatively, the supplementary data may be used to request that the user logs in. As such, the system may be arranged to provide a form of default supplementary data, which is sent to all decoders until user input is received. Once die user has logged in, the supplementary data will be selected by the data server 112 to be specific to that decoder. Consequently the user is again provided with a personalized experience, while the broadcast video stream can be played in the background enabling the game show hosts to, for example, advertise the prizes available to any user viewing the channel.
When a user logs in to the game, the data server 1 12 may create a corresponding record in the storage system 116. Alternatively, an existing record corresponding to that decoder may be updated or otherwise modified so as to be able to store the user's results. This record may then be used to log the answers of the user. The decoder may identify itself in messages sent to the data server (by using, for example, a serial number). Alternatively the decoder may be identified by the server system based on, for example, an IP address. The W
23
record may be associated with an identity of the decoder. Alternatively, the records may be associated with a user, and therefore a given user can use different decoders.
Embodiments of the invention may be used for many other television 5 programmes to provide a personalise experience (i.e. displaying statistics during a sports fixture) or to allow the user to express an opinion (for example in a talent contest or online debate). Alternatively, the supplementary data may be used to provide a form of interactive and/or personalized advertising (for example, during commercial breaks during the game show).
10 The above system has been described solely in terms of visual multimedia content. However it is envisaged that audio data may be treated in. an analogous way. For example, the broadcast video stream will be provided with an audio track recorded in the studio. In addition the supplementary data may be used to provide audio feedback specific to the user/decoder. For
15 example, an acknowledgement sound may be played when an answer is provided by the user, and celebratory or commiseratory sounds may be played, when the answer is revealed (depending on whether the answer was right or wrong).
In the embodiment shown in Figure 1, the video broadcast server 104 20 and the data server 112 are shown as being separate entities and as having separate connections to the network. This may not necessarily be the case, and any appropriate arrangement of one or more servers which allows for the video broadcast data to be multicast, the supplementary data to be unicast and for the messages from the decoders to be received is envisaged.
25 Likewise, a timing signal is described as being sent from, the video broadcast server 104. Such a timing signal may not be required (depending on the game format). Equally a coordinating server or the like may be used to provide timing signals.
The above embodiments use IP multicasting to send the broadcast video 30 stream to the decoders. However this is not necessarily the case, and any form of broadcasting can be used to provide the video stream. For example a satellite or terrestrial broadcast system could be used for the broadcast stream, alongside a domestic internet connection for the supplementary data and messages,
The decoder system is shown as having separate decoder, display and remote control. For example the decoder may take the form of a set-top-box connected to a television. However this is purely exemplary and any system which is able to decode and display video data and receive user input is envisaged. For example, a games console, media player or DVD player may be used. Alternatively, embodiments of the invention may be used on a personal computer, wherein the display is a computer monitor, and the remote control is replaced by a keyboard or mouse. Equally, a tablet PC, PDA or smartphone could perform the function of a decoder system, with the buttons or a touch screen providing the user input.
In the above embodiments the user input means (remote control) is in communication with the decoder, which is thus able to receive the user input and transmit messages indicative of the user input to the data server 112. However, it is also possible that the user input is provided in a separate system to the decoder. This separate system may have an entirely separate connection to the data server 1 12, for example being a smartphone or other mobile device connected using a cellular network. In such embodiments, the user device is associated with the decoder in some manner (such as by a unique username), and the answers provided using the user input are transmitted directly to the data sever 112. Nevertheless, despite this separate connection, the broadcast video stream and the supplementary data are still provided to the decoder in the usual manner to provide the user with a personalized visual (and audio) experience.
In the above description, only one user has been described in relation to one decoder. However it is possible that multiple users may compete using one decoder system; cither by being provided with separate input devices, or through team work. Equally, it will be understood that references to a user are references to the associated decoder. In the embodiments described above, the user selects an answer by pressing a coloured button on the remote control. However, alternatives are envisaged. For example the answers may be provided with numbers (corresponding to number keys) or in a North, South, East, West configuration corresponding to four arrow keys (Up, Down, Left and Right).
The method described above in relation to the game show was described with only one answer being given. A more complex system may be provided in which, if a user makes a first selection, the acknowledgement reflects that answer and also displays "Are you sure?" or a similar prompt. The user may then press another button to reject the answer, or the same button again to confirm the selection. The flow of messages and supplementary data in such situations is analogous to that previously described, and serves to show the flexibility of embodiments of the invention.
The processor 134 was described as being a single entity comprising a number of discrete elements. It will be apparent that this arrangement was purely exemplary, and that any processor, chipset, or system that is able to decode the video stream, generate the graphical element, overlay the graphical element onto the video stream, and communicate with the network, display and remote control interfaces will be able to replace tire processor 134.
It is to be understood that any feature described in relation to any one embodiment may be used alone, or in combination with other features described, and may also be used in combination with one or more features of any other of the embodiments, or any combination of any other of the embodiments. Furthermore, equivalents and modifications not described above may also be employed without departing from the scope of the invention, which is defined in the accompanying claims.

Claims

Claims
1. A multimedia system comprising:
a server system arranged to provide multimedia content, the multimedia content comprising a video stream and supplementary data; and
a plurality of multimedia decoders, each multimedia decoder comprising: an interface for receiving the multimedia content and for outputting display data to a display device;
a processor arranged to overlay onto a received said video stream a graphical element created on the basis of said received supplementary data, whereby to generate said display data,
wherein the server system is arranged to:
receive input relating to the multimedia content from a said multimedia decoder;
broadcast the video stream to at least some of the multimedia decoders; and
transmit, separately from the video stream, said supplementary data to said at least some of the multimedia decoders, and
wherein the supplementary data transmitted to a given said multimedia decoder is selected in dependence on the input received from the given said multimedia decoder.
2. The multimedia system of claim 1, wherein the input received from a said multimedia decoder comprises an identity of the said multimedia decoder.
3. The multimedia system of claim 1 or claim 2, comprising:
input means associated with each multimedia decoder, each input means arranged to receive user input, and, responsive to user input of a predetermined type, to cause a message indicative of the said user input to be sent to the server system,
wherein the server system is arranged, for each said message, to create acknowledgement data in dependence on the message, wherein the supplementary data transmitted to the multimedia decoder associated with the respective input means comprises at least a part of the created acknowledgement data, and
wherein the graphical element created by the processor of a given multimedia decoder includes a visual indication of said acknowledgement data.
4. The multimedia system of claim 3, wherein each interface of the multimedia decoders comprises said associated input means, and wherein the multimedia decoder is arranged to send the message indicative of the user input via the interface.
5, The multimedia system of claim 3 or claim 4, wherein the server system is arranged to access a storage system arranged to hold records corresponding to a respective multimedia decoder.
6. The multimedia system of claim 5, wherein the server system is arranged to create a said record upon receipt of an initialization message from a said multimedia decoder.
7. The multimedia system of claim 5 or claim 6, wherei the server system is arranged to modify, in response to a message received from a multimedia decoder, a record . corresponding to the respective multimedia decoder.
8. The multimedia system of any of claims 5 to 7, wherein, for each multimedia decoder, the server system is arranged to generate the supplementary data transmitted to the multimedia decoder based on the corresponding record.
9. The multimedia system of any of claims 3 to 8, wherein the server system is arranged to start a timer upon transmitting a portion of the multimedia content.
10. The multimedia system of claim 9, wherein the server system is arranged such that when a message indicative of user input is received from a multimedia decoder, the modification of the corresponding record is dependent on the value of the timer when the message data was received.
11. The multimedia system of any of claims 3 to 10, wherein the server system and the plurality of decoders are connected via a network arranged to carry traffic of a first type and traffic of a second type, the traffic of the first type having a priority higher than traffic of the second type, and wherein, the supplementary data and the message indicative of the user input comprise traffic of the first type.
12. The multimedia system of any of claims 1 to 10, wherein the server system and the plurality of decoders are connected via a network arranged to carry traffic of a first type and traffic of a second type, the traffic of the first type having a priority higher than traffic of the second type, and wherein the supplementary data comprises traffic of the first type.
13. The multimedia system of any preceding claim, wherein the video stream comprises an audio stream, and the processor is arranged to combine the audio stream, with an audio element created on the basis of said received supplementary data, whereby to generate audio data to be output by the interface.
14. A method of outputting multimedia content for display on a display device, the multimedia content comprising a video stream and supplementary data, the method comprising:
receiving the multimedia content from a server system;
overlaying onto a received said video stream a graphical element created on the basis of the supplementary data, whereby to generate display data; and outputting the display data to a display device whereby to display the multimedia content.
15. A method of providing multimedia content to a plurality of multimedia decoders, the method comprising:
receiving input relating to the multimedia content from each said multimedia decoder;
broadcasting a video stream to the multimedia decoders;
selecting, in dependence on the input received from a given said multimedia decoder, supplementary data to be transmitted to the given said multimedia decoder; and
transmitting, separately from the video stream, said supplementary data to the given said multimedia decoder, wherein the multimedia content comprises said video stream and said supplementary data, and the supplementary data is indicative of a graphical element to be overlaid onto the video stream.
16. A computer program product having computer readable instructions stored thereon, the computer readable instructions being executable by a computerized device to cause the computerized devices to perform, a method of displaying xmiltimedia content, the multimedia content comprising a video stream and supplementary data, the method comprising; receiving the multimedia content from a server system.
overlaying onto a received said video stream a graphical element created on the basis of received supplementary data, whereby to generate display data; and
outputting the display data to a display device whereby to be display the multimedia content.
17. A computer program product having computer readable instructions stored thereo , the computer readable instructions being executable by a computerized device to cause the computerized devices to perform a method of providing multimedia content to a plurality of multimedia decoders, the method comprising:
receiving input relating to the multimedia content from each said multimedia decoder;
broadcasting a video stream to the multimedia decoders;
selecting, in dependence on the input received from a given said multimedia decoder, supplementary data to be transmitted to the given said multimedia decoder; and
transmitting, separately from the video stream, said supplementary data to the given said multimedia decoder, wherein the multimedia content comprises said video stream and said supplementary data, and the supplementary data is indicative of a graphical element to be overlaid onto the video stream.
PCT/IB2011/003168 2010-12-24 2011-12-23 Improvements to interactive multimedia systems WO2012085659A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GB1022003.6A GB2486744A (en) 2010-12-24 2010-12-24 Multimedia system with separately transmitted supplementary data overlaid onto broadcast video stream based on user input
GB1022003.6 2010-12-24

Publications (1)

Publication Number Publication Date
WO2012085659A1 true WO2012085659A1 (en) 2012-06-28

Family

ID=43599027

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2011/003168 WO2012085659A1 (en) 2010-12-24 2011-12-23 Improvements to interactive multimedia systems

Country Status (2)

Country Link
GB (1) GB2486744A (en)
WO (1) WO2012085659A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9837124B2 (en) 2015-06-30 2017-12-05 Microsoft Technology Licensing, Llc Layered interactive video platform for interactive video experiences
CN108415780A (en) * 2018-01-25 2018-08-17 阿里巴巴集团控股有限公司 A kind of information displaying method and device
CN110850983A (en) * 2019-11-13 2020-02-28 腾讯科技（深圳）有限公司 Virtual object control method and device in video live broadcast and storage medium

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230206296A1 (en) * 2021-12-29 2023-06-29 Gripcompany Co., Ltd. Live-commerce system for determining winners through games and method thereof

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1997019556A1 (en) * 1995-11-22 1997-05-29 Philips Electronics N.V. Interactive television
CN1413028A (en) * 2001-10-15 2003-04-23 华为技术有限公司 Interactive video equipment and its caption superposition method
CN101076089A (en) * 2006-06-23 2007-11-21 腾讯科技（深圳）有限公司 Method for displaying captions
US20080002021A1 (en) * 2006-06-30 2008-01-03 Guo Katherine H Method and apparatus for overlay-based enhanced TV service to 3G wireless handsets
CN101500125A (en) * 2008-02-03 2009-08-05 突触计算机系统（上海）有限公司 Method and apparatus for providing user interaction during displaying video on customer terminal

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100295386B1 (en) * 1992-12-09 2001-09-22 마크 홀린저 Set-top terminals for cable television delivery systems
US7099951B2 (en) * 2001-05-24 2006-08-29 Vixs, Inc. Method and apparatus for multimedia system
US20090180025A1 (en) * 2002-05-28 2009-07-16 Sony Corporation Method and apparatus for overlaying graphics on video
JP2005277802A (en) * 2004-03-25 2005-10-06 Yamaha Corp Control method of remote control apparatus, control method of apparatus to be controlled, remote control apparatus, apparatus to be controlled, and program
US20100239222A1 (en) * 2009-03-20 2010-09-23 International Business Machines Corporation Digital video recorder broadcast overlays
US20100319043A1 (en) * 2009-06-11 2010-12-16 Microsoft Corporation Interactive television architecture

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1997019556A1 (en) * 1995-11-22 1997-05-29 Philips Electronics N.V. Interactive television
CN1413028A (en) * 2001-10-15 2003-04-23 华为技术有限公司 Interactive video equipment and its caption superposition method
CN101076089A (en) * 2006-06-23 2007-11-21 腾讯科技（深圳）有限公司 Method for displaying captions
US20080002021A1 (en) * 2006-06-30 2008-01-03 Guo Katherine H Method and apparatus for overlay-based enhanced TV service to 3G wireless handsets
CN101500125A (en) * 2008-02-03 2009-08-05 突触计算机系统（上海）有限公司 Method and apparatus for providing user interaction during displaying video on customer terminal

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9837124B2 (en) 2015-06-30 2017-12-05 Microsoft Technology Licensing, Llc Layered interactive video platform for interactive video experiences
CN108415780A (en) * 2018-01-25 2018-08-17 阿里巴巴集团控股有限公司 A kind of information displaying method and device
CN110850983A (en) * 2019-11-13 2020-02-28 腾讯科技（深圳）有限公司 Virtual object control method and device in video live broadcast and storage medium

Also Published As

Publication number Publication date
GB2486744A (en) 2012-06-27
GB201022003D0 (en) 2011-02-02

Similar Documents

Publication Publication Date Title
US20070300273A1 (en) Interactive television application and content enhancement
US6545722B1 (en) Methods and systems for providing television related services via a networked personal computer
CN110149526B (en) Live broadcast interactive system, control method and device thereof and storage medium
US8910208B2 (en) Interactive video system
EP0745948B1 (en) Method and apparatus for playing and providing a game of skill or chance
JP2000217094A (en) Method and device for transmitting interactive application
CN101237558B (en) Image display system
KR20030040097A (en) A transmission system for transmitting video streams relating to an event to spectators physically present at said event
US8715083B2 (en) Methods and systems for processing gaming data
US11949937B2 (en) Method and device to create interactivity between a main device and at least one secondary device
JPH10154062A (en) Display system showing information from plural sources
CN101472108A (en) Method and apparatus for playing video
JP2008053916A (en) Content distribution system, information terminal equipment, content distribution method, and image switching program
WO2012085659A1 (en) Improvements to interactive multimedia systems
CN103414856A (en) Method and device for carrying out channel preview
CN105228009A (en) The method and apparatus of the interactive interface of display result of the match guess
US20080057952A1 (en) Service roaming system and method
WO2007010416A2 (en) A method and a system for enabling real-time participation of a mass viewer audience in an interactive television program
JPH0946651A (en) Interactive teletext broadcasting system
JP4703047B2 (en) Method for providing communication between viewers in television broadcasting and television communication terminal used therefor
JP2004015087A (en) Viewer participating type two-way communication service system
JP2000115746A (en) Broadcast data transmitter and receiver
US7016992B2 (en) Electronic mail system
JP2000115095A (en) Broadcasting data transmitter and receiver
JP2000050230A (en) Bidirectional telecasting system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11850602

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11850602

Country of ref document: EP

Kind code of ref document: A1