Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
  • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
  • H04N21/43615—Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
  • H04N21/23424—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/41—Structure of client; Structure of client peripherals
  • H04N21/4104—Peripherals receiving signals from specially adapted client devices
  • H04N21/4126—The peripheral being portable, e.g. PDAs or mobile phones
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/41—Structure of client; Structure of client peripherals
  • H04N21/4104—Peripherals receiving signals from specially adapted client devices
  • H04N21/4126—The peripheral being portable, e.g. PDAs or mobile phones
  • H04N21/41265—The peripheral being portable, e.g. PDAs or mobile phones having a remote control device for bidirectional communication between the remote control device and client device
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
  • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
  • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
  • H04N21/43079—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of additional data with content streams on multiple devices
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
  • H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
  • H04N21/4345—Extraction or processing of SI, e.g. extracting service information from an MPEG stream
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
  • H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
  • H04N21/4348—Demultiplexing of additional data and video streams
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
  • H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
  • H04N21/4348—Demultiplexing of additional data and video streams
  • H04N21/4349—Demultiplexing of additional data and video streams by extracting from data carousels, e.g. extraction of software modules from a DVB carousel
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
  • H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
  • H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
  • H04N21/4622—Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
  • H04N21/81—Monomedia components thereof
  • H04N21/8166—Monomedia components thereof involving executable data, e.g. software
  • H04N21/8173—End-user applications, e.g. Web browser, game
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
  • H04N21/81—Monomedia components thereof
  • H04N21/8166—Monomedia components thereof involving executable data, e.g. software
  • H04N21/8186—Monomedia components thereof involving executable data, e.g. software specially adapted to be executed by a peripheral of the client device, e.g. by a reprogrammable remote control
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
  • H04N21/85—Assembly of content; Generation of multimedia applications
  • H04N21/854—Content authoring
  • H04N21/8545—Content authoring for generating interactive applications

Definitions

• the field of the invention is that of digital television (DTV, for "Digital Television” in English).
• a multimedia transport stream is distributed by cable, satellite, terrestrial hertzian or ADSL ("Asymmetric Digital Subscriber Line" in English).
• This multimedia transport stream contains one or more programs (services) each of which consists of images, sounds and other information (data) complementary. It is broadcast to the users (end customers) who have a specific decoder (also called “Set-top box”) or a connected television, or more generally a rendering terminal with functions of demultiplexing and decoding such a stream .
• the present invention applies in particular, but not exclusively, in the case of a multimedia transport stream compliant with the standards MPEG-2 TS ("Moving Picture Experts Group - 2 Transport Stream” in English) and DVB ("Digital Video Broadcasting"). In English), described in ISO / IEC 13818-1.
• the invention applies to interactive television, defined for example in the standard HBBTV (for "Hybrid Broadcast Broadband TV” in English) which provides, for connected TVs and set-top boxes (“set-top box”) , to add information to display on the TV screen while the user is watching a television program (television channel).
• HBBTV Hybrid Broadcast Broadband TV
• set-top box set-top box
• the number of programs involving viewers is growing. For example, during a TV reality show, the viewer can vote for a candidate, or during a game show, the viewer can answer a quiz. In all cases, it is for the viewer to provide an answer to a question that is presented to him during the broadcast.
• the question is put to the viewer by means of at least one image that is displayed by the television screen.
• the question can be formulated orally by the presenter of the program and is reproduced by the speakers of the television.
• the additional information to be displayed on the television screen is available directly in the multimedia stream (broadcasted information, or "broadcast” in English), or delivered via a connection (return channel) between the user terminal (connected TV or set-top box) and a server of the broadcaster or of a third party operating in relation to the broadcaster (information transmitted via point-to-point access, or "broadband”).
• the viewer has the opportunity to display the recipe on its HBBTV terminal, if the info is transmitted in the stream by the broadcaster.
• An HBBTV terminal i.e., compatible with the HBBTV standard
• a software architecture generally comprising system software over which interactive television middleware ("iTV middleware") is run.
• iTV middleware interactive television middleware
• an interactive application management module including a browser
• a table management module i.e., a browser
• an object carousel management module i.e., a browser
• the broadcaster When deploying a given interactive application, associated with a given program (television channel), the broadcaster transmits (within a multimedia stream) the given program comprising:
• Constituents in particular a software code, for example in HTML, in the form of a web page
• a software code for example in HTML, in the form of a web page
• DSM CC standard Digital Storage Media
• AIT Application Information Table
• the HBBTV terminal On receipt of this program, the HBBTV terminal displays the audio and video streams (thanks to the tables describing them).
• the table management module processes the AIT table and detects that a software code of the given interactive application has been transmitted in an object carousel; the carousel management module receives and reassembles the constituents of the given interactive application; and the application management module manages the given interactive application (reassembled by the carousel management module).
• the latter can for example take one of the following states: downloaded ("loaded"), paused ", active (" active ”) or destroyed (" destroyed ").
• the given (broadcast) program does not include the constituents of the given interactive application, but the AIT table specifies a URL where the given interactive application can be obtained.
• the table management module processes the AIT table and obtains this URL address; the browser of the application management module sends a request to this URL, in order to receive the given interactive application; and the application management module manages the given interactive application.
• a disadvantage of this operation is a visual discomfort for the user. Indeed, each time the user accesses, on a television channel (program), a particular interactive application associated with the program and offering access to a web portal, this leads to the display, in superposition of the TV program. current, a window containing a message or a logo indicating to the user that access to a portal is possible by pressing a key ("OK" key for example) of the remote control of the terminal HBBTV.
• a key "OK" key for example
• Second screen application ("second screen application” according to the English terminology) and provide complementary content to television programs according to their scope (for all channels, for a channel or for a program), on a separate device (for example a smartphone or a tablet) from that used to view the TV program (for example a television).
• additional content can be editorial or interactive (quizzes, polls, predictions, ...), synchronized or not with the TV program.
• a synchronization is required between the TV stream and the content displayed on the second screen application of the second device and can be implemented according to several different technologies, among which:
• Watermarking this technique consists of adding additional information to audiovisual content allowing the insertion of "markers" in the content. This technique is often used to insert a copyright. It is also used to insert audio markers in the TV stream, imperceptible by the user, that can be captured by the second screen application to trigger an action. Digital tattooing is implemented at the level of the broadcaster;
• the fingerprint (FingerPrinting" in English): this technique consists in generating a file of signatures from an audiovisual content. This file then allows an audio synchronization module listening to a stream, for example on the second device running the second screen application, to know the precise position in the content. For example, an audio signature of an episode of a series makes it possible to know to which position in the episode one is. This technique is independent of the diffuser and is not very reliable because it depends on the ambient noise and the quality of reception in the room where the viewer watches the TV;
• this technique consists of triggering events on the second screen application only on an hourly basis. This technique is not very reliable and does not manage the offsets that can occur with different means of receiving TV streams (IPTV, TNT, satellite, ).
• the main disadvantage of these second-screen applications lies in the lack of reliability or accuracy of synchronization techniques necessary for their proper operation.
• a method of synchronization between a reproduction of a multimedia stream by a rendering device and an execution of an operation by a software application by at least one execution device is proposed. restitution and execution being present within a local communication network.
• the method comprises the following steps, in the rendering device:
• the multimedia stream comprising additional data synchronized with a multimedia element contained in the multimedia stream
• the invention proposes a method of synchronizing the execution of an operation of processing additional data with the reproduction of a multimedia element contained in the multimedia stream, by providing a retransmission of additional data contained in the multimedia stream and received by a device for rendering the multimedia stream, to a separate device in charge of executing this processing operation relating to these additional data, via a software application for example previously downloaded and installed on said separate device execution.
• the general principle of the invention relies in particular on the presence, in the multimedia stream, of an indicator making it possible to detect that the additional data are intended to be processed by a device other than that which has received the multimedia stream. Once this indicator has been detected, the device for rendering the multimedia stream processes it to extract the additional data before transmitting it, for example via a communication module, to the destination execution device.
• these additional data are processed by a device separate from the stream rendering device, allowing better ergonomics for the user, who can continue to view the multimedia stream on the rendering device (for example a television program on a television ), while benefiting from additional information produced by the execution, by this distinct device (for example a smartphone or a tablet,) of the operation of processing additional data (this operation corresponding for example to a connection and display of data).
• this distinct device for example a smartphone or a tablet, of the operation of processing additional data (this operation corresponding for example to a connection and display of data).
• a given web page or an interactive application execution related to the content of the multimedia element may be processed by a device separate from the stream rendering device, allowing better ergonomics for the user, who can continue to view the multimedia stream on the rendering device (for example a television program on a television ), while benefiting from additional information produced by the execution, by this distinct device (for example a smartphone or a tablet,) of the operation of processing additional data (this operation corresponding for example to a connection and display of
• the invention allows a fine and precise synchronization between the reproduction of a multimedia element of the multimedia stream and the operation of processing the additional data.
• this processing operation includes a return of additional information by the execution device
• the invention allows a synchronization between the restitution, by the execution device, of this additional information and the restitution, by the device. of restitution, of the multimedia element.
• the additional data inserted into the stream being synchronized with the multimedia element and received by a single rendering device, the execution device in charge of their processing receives them and thus also processes them synchronously.
• the multimedia stream is an HBBTV type stream and the additional data corresponds to a second screen interactive application.
• the invention makes it possible to execute an interactive application of the second screen type, on a device separate from the device for rendering the HBBTV stream, the interactive application being inserted in the HBBTV stream and synchronized with the stream.
• HBBTV according to HBBTV synchronization means.
• the invention makes it possible to offer a user an interactive application service, of the second screen type for example, that is efficient in terms of synchronization accuracy, the interactive application being transported or made accessible by means of additional data contained in the HBBTV stream received by the rendering device then transmitted and operated synchronously by the software application of the execution device.
• the operations of restitution of the multimedia element by the device for rendering and processing additional data by the execution device are synchronous, since the additional data and the multimedia element are received synchronously in the multimedia stream, extracted from this stream and processed as soon as they are received by the rendering device on the one hand and by the execution device on the other hand.
• the transmission of the additional data triggers the processing operation of these additional data
• the reproduction of the multimedia element by the rendering device and the processing of these additional data by the execution device can be carried out in parallel. by these two devices.
• the latency between, on the one hand, a display, on the execution device, of data resulting from the processing of the additional data and, on the other hand, a display / a reproduction, on the rendering device, of the multimedia content depends, on the one hand, on the decoding time and / or processing, by the rendering device, of the multimedia content, and on the other hand, the transmission time of the additional data to the execution device to which adds the processing time by the execution device, these additional data.
• this latency remains well below one second, of the order of a few tens or hundreds of milliseconds. Additional data contained in the stream multimedia therefore form a "trigger" or command that triggers a processing device side. This treatment is immediate, triggered upon receipt of these additional data by this device.
• the destination information is carried by a descriptor present in an AIT signaling table, for Application Information Table in English, of the HBBTV stream.
• the invention provides, according to this embodiment, to add a descriptor in the AIT table of the HBBTV stream enabling indicate whether the application identified by this table AIT is intended for a device separate from the device for rendering the stream HBBTV.
• the rendering device upon detection of this descriptor, the rendering device knows that it must retransmit the additional data (corresponding to an interactive application) received in the stream, and identified by the table AIT, to another device, which will execute this interactive application.
• the method further comprises the following steps:
• data packets associated with the additional data previously inserted in the multimedia stream, and thus also intended for the execution device, can be detected in the multimedia stream, by the rendering device, so that to transmit them to the destination execution device.
• these data packets can be inserted into the media stream to update the previously received additional data, while benefiting from the precise synchronization of the multimedia stream.
• the data packet associated with the additional data corresponds to an event of the "Event Stream" HBBTV type.
• events of the "Event Stream” type can also be transmitted by the rendering device to the execution device, for example to notify synchronized events with the multimedia stream, when the additional data is in progress. processing on the execution device.
• Event Streams make it possible to notify the interactive application of the events during the execution of the interactive application.
• Event Streams can be used to notify the launch of a particular quiz once the Quiz application is launched.
• the reception and / or processing of the data packet associated with the additional data may be supported by the software application having received and processed the additional data, for example in view of a transmission of this data packet to the application. interactive for processing, or be directly supported by the interactive application.
• the latency between, on the one hand, a display, on the execution device, of data resulting from the processing of the data packet and, on the other hand, a display / a reproduction, on the rendering device, of the received multimedia content with this data packet depends, on the one hand, the decoding time and / or processing, by the rendering device, the multimedia content, and on the other hand, the transmission time of the data packet to the execution device to which is added the processing time by the execution device, of this data packet. In practice, this latency remains well below one second, of the order a few tens or even hundreds of milliseconds.
• the data packet contained in the multimedia stream thus forms a "trigger" or command that triggers a processing on the execution device side. This processing is immediate, triggered upon receipt of this data packet by this execution device.
• the method comprises the following steps:
• the extraction steps correspond to a demultiplexing of the additional data and / or the data packet associated with the additional data.
• the processing of the multimedia stream received by the rendering device consists of demultiplexing the additional data inserted into the stream, so that it can be transmitted, without interpreting or executing them, to an execution device separate, in charge of him to interpret and execute them.
• the demultiplexing consists in extracting this URL, so that it can be transmitted to the execution device.
• the demultiplexing consists in extracting the Event Stream, so as to transmit it to the execution device.
• the method further comprises a step of detecting, in the multimedia stream, at least one end of execution indicator, by the software application of the execution device, of the processing operation of the additional data previously transmitted to the execution device.
• the invention also provides that execution of the additional data by the execution device can be stopped, via an indicator detected in the multimedia stream and transmitted by the rendering device to the execution device. .
• the method further comprises a preliminary step of announcing the presence of the rendering device within the local communication network.
• the rendering device is indicated as being present in the communication local network to which it is connected, so that the execution device or devices also present in the same network.
• communication center may be aware of its presence, by example via a discovery process such as SSDP (for "Simple Service Discovery Protocol” in English) used for the discovery of UPnP entities (for "Universal Plug and Play” in English).
• SSDP Simple Service Discovery Protocol
• UPnP entities for "Universal Plug and Play” in English
• the method further comprises the following steps, in the execution device:
• This embodiment of the invention relates more particularly to the execution device, and the steps implemented in the latter.
• the execution device first acquires knowledge of the rendering device (s) connected to the same local network as itself, so as to be able, if necessary, to implement a data communication mechanism with this or these device (s). restitution.
• the execution device can discover it.
• the execution device receives additional data, previously received and retransmitted, according to the various embodiments of the invention, by the rendering device.
• the execution device receives the additional data from the rendering device via a communication module, which then transmits them to the (x) modules for interpreting and executing the additional data of the execution device , which execute, by means of a software application, a processing operation of these additional data.
• the transmission steps, in the rendering device, and reception, in the execution device are implemented according to a mechanism belonging to the group comprising:
• the invention provides for the use of several known mechanisms for transmitting the additional data, as well as associated data packets (such as events) of the device for rendering the multimedia stream to the device. execution of additional data.
• mechanisms at the initiative of the server or at the initiative of the client can be implemented ("polling", “http server push”, transmission "UDP”, ....
• the invention also relates to a method for generating a multimedia stream, comprising the following steps:
• another aspect of the invention relates to the generation of a multimedia stream, and in a first step, the insertion, in this multimedia stream, of additional data.
• the method of generating the multimedia stream inserts, in the multimedia stream, an indicator enabling the device for restitution of the stream to detect that the additional data is intended for an execution device distinct from the device for restitution of the stream.
• the invention also relates to a computer program, comprising program code instructions for implementing the synchronization and generation methods as described above, when the program is run on a computer.
• the invention also relates to a computer readable storage medium, storing a computer program comprising a set of instructions executable by a computer or a processor for implementing the synchronization and generation methods as described above.
• Another aspect of the invention relates to a device for rendering a multimedia stream present within a local communication network, comprising the following means:
• Such a device is for example a television, a decoder or a computer comprising an application for receiving a multimedia stream and implement the steps of the method.
• a rendering device corresponds to any device capable of interpreting and rendering a multimedia stream, whatever its mode of broadcasting / transmission.
• Another aspect of the invention relates to an additional data execution device present within a local communication network, comprising the following means:
• an execution module by a software application of the execution device, of a processing operation of the additional data received.
• a device is for example a smartphone or a tablet.
• Another aspect of the invention relates to a device for generating a multimedia stream comprising the following means:
• Figure 1 shows the main steps of the additional data transmission method according to a particular embodiment of the invention
• Figure 2 shows a block diagram of a system according to a particular embodiment of the invention
• FIG. 3 presents a sequence diagram generically showing an example of communication kinematics between the two devices of the system of FIG. 2;
• FIGS. 4a to 4c illustrate examples of a simplified structure of a rendering device, an execution device and a generation device, according to one embodiment of the invention.
• the multimedia stream 100 complies with the MPEG-2 TS and HBBTV standards, and that it comprises additional data 110 corresponding to an interactive application, for example a "second screen" application intended to be executed on an execution device that is physically distinct from the rendering arrangement.
• This interactive application may be an HBBTV application or not (an HBBTV application being an application that is encoded in HTML (HTML-CE) and / or Javascript and is transported in an HBBTV stream), a web application or not.
• the invention is applicable however to different types of application, and does not imply a particular mode of coding. It is sufficient that the software application of the execution device is capable of triggering execution, possibly by calling on the operating system of the execution device.
• the additional data 110 is synchronized, that is to say inserted in the multimedia stream synchronously, with a multimedia element 120 of the multimedia stream 100 (for example the current part of the multimedia stream, as a given program).
• a multimedia element 120 of the multimedia stream 100 for example the current part of the multimedia stream, as a given program.
• a device for reproducing a multimedia stream 100 for example an HBBTV television set, and an execution device 21, by a software application, for an additional data processing operation 110 (contained in FIG. multimedia stream 100 and synchronized with a multimedia element 120 contained in the multimedia stream 100), for example a smartphone.
• the multimedia stream HBBTV corresponds to a television game program
• the additional data correspond to an interactive application of Quiz, intended to be executed on a second screen, separate from the screen on which the game TV program is rendered.
• This application consists, for example, in enabling the viewer to answer Quiz questions via his smartphone, while continuing to ergonomically view the game television program on his connected television.
• the first steps of the synchronization method are implemented in the playback device 20, which receives, during a first reception step 10, the multimedia stream 100.
• a detection step 11 is then implemented, in order to detect, in the multimedia stream 100, a destination information, denoted l-Dest, indicating that the additional data 110 are intended for at least one separate execution device of the reproduction device 20, in this case the execution device 21.
• This detection step 11 is based, in this example, on the fact that an interactive application is identifiable in an MPEG2 TS multimedia stream by a DVB table.
• This is the AIT table (for "Application Information Table").
• This table contains indeed different information for executing the interactive application, and in particular, according to this embodiment of the invention, this table carries a private descriptor, noted here destination information l-Dest.
• This destination information l-Dest indicates the destination / destination terminal concerned by the interactive application. In the example considered, this is the execution device 21.
• This detection step 11 is implemented, according to this embodiment, by an additional HBBTV module 201 of the rendering device 20, denoted HBBTV-SUP.
• the playback device 20 comprises a "standard" HBBTV module 200, making it possible to implement all the "standard” HBBTV functions, and an additional HBBTV module 201 for implementing the invention, according to its different modes of operation. particular achievements.
• the next step of extraction 12 consists in demultiplexing the additional data 110, as well as the multimedia element 120, of the multimedia stream 100, so as to be able to transmit, during a transmission step 13, these additional data 110 to the device. execution 21.
• the multimedia element 120 is in turn restored, on the playback device 20, during a step 14 of restitution.
• this extraction step 12 is implemented by the "standard" HBBTV module 200 of the reproduction device 20.
• this extraction step 12 triggers an execution, by a software application 211 of the execution device 21, of a processing operation of these additional data 110 transmitted, so that the restitution, on the device of restitution 20, of the multimedia element 120 and the execution, on the execution device 21, of the processing operation of the additional data 110 are synchronized.
• the software application 211 of the execution device 21 is for example an application downloaded to the execution device 21, in order to process additional data received from a rendering device.
• This software application is for example a "native” application (this application can be downloaded from "Google Play” or “AppleStore”, depending on the operating system).
• This software application 211 is therefore implemented in the execution device 21 for the processing of the additional data and must be distinguished afterwards from the interactive application, for example of the HBBTV type and / or of the second screen type. , corresponding to the additional data 110 transmitted.
• the execution, by this software application 211, of this additional data processing operation 110 consists in fact, in this example, in triggering the execution of the interactive application.
• the HBBTV stream contains interactive application access information (for example, the AIT table contains a URL where the interactive application can be downloaded), or part or all of the software module or modules constituting the interactive application is encapsulated in the HBBTV stream in an interpretable or executable form by the device. 21 (for example in a DVB carousel, for example in the form of meta data interpretable by the software application 211 of the execution device 21).
• the operation of processing the additional data thus consists in obtaining the URL, sending an HTTP request to this URL address, in order to receive in return data constituting the interactive application in question, d.
• Execute the interactive application and display content resulting from this execution (Quiz questions in the example).
• the loaded data can be either Web pages, received and then displayed by the software application 211, or another (non-Web) application that can be interpreted and / or executed by the execution device 21 (by the software application 211 or by the operating system of the terminal 21).
• the operation of processing the additional data consists in interpreting the metadata constituting the interactive application that can be interpreted and / or executed by the execution device 21 (by the software application 211 or by the system operating the terminal 21), to execute the interactive application by the execution device 21 and to display a content resulting from this execution (quiz questions in the example in question).
• the transmission step 13 can be implemented by a communication module 203 of the rendering device 20, allowing a server-type communication to the execution device 21, and for example to a communication module 213 of the execution device 21.
• the communication module 213 of the execution device 21 receives the additional data 110 from the communication module 203 of the rendering device 20, and forwards them for example to the modules of the software application 211 (for example conventional modules "GUI” and "BUSINESS") in charge of interpreting them, as described above.
• the communication module 213 is integrated with the software application 211 itself.
• the communication module 213 of the execution device 21 sends periodic requests to the communication module 203 of the rendering device 20, to know if new data are available and if so, get them back.
• the http server of the rendering device 20 does not close the connection after sending its last response to the execution device 21.
• the connection remains open and the server stacks the events until the client l queries again to retrieve a new event.
• the server can issue notification requests via the UDP protocol.
• the sending of UDP packets can be done in unicast (exclusively for the client who has previously provided his IP address and the port concerned for the reception of UDP packets) or multicast (in the manner of UPnP events).
• the multicast approach makes it possible, for example, to inform several local network execution devices of the arrival of a new event.
• FIG. 3 illustrates an example of a "Long Polling" type communication, noted LONG P, between a rendering device 20, for example an HBBTV TV, and an execution device 21, for example a smartphone.
• the client that is to say the smartphone
• the server that is to say the TV HBBTV.
• a new event for example a Stream Event, noted SE
• a table AIT no response is made to the smartphone, and when a new event is received by the TV HBBTV, the TV HBBTV responds to the smartphone, specifically by transmitting data to the communication module 213 of the smartphone.
• This communication module 213 receives this data and transmits it to the smartphone module in charge of executing it, that is to say to the software application 211.
• a step prior to any data exchange consists in allowing the communication between the different devices of the local communication network involved in the implementation of the invention, according to its different particular embodiments.
• This prior step may consist of a "discovery" or announcement of presence of the rendering device 20 within the local communication network.
• the rendering device 20 is indicated as being present in the local communication network to which it is connected, so that the execution device 21 also present in the same local communication network.
• the execution device 21 can be aware of his presence, by example via a known discovery process such as UPnP's SSDP.
• the advertisements are sent by the rendering device 20 periodically.
• this discovery step can also consist in sending a search request for a device (s) for rendering, by the execution device 21, the rendering device (s) present in the local network of communication directly responding to the search query.
• rendering devices may be present in the local communication network.
• the execution device will be aware of the presence of these different rendering devices and will inform the user by presenting a list of playback devices, in which he can select the playback device to use, with which the enforcement system will have to communicate.
• the rendering and executing devices 21 are thus related. It is then possible for the rendering device 20 to transmit the additional data extracted from the multimedia stream to the execution device 21, when a suitable destination information is detected in this multimedia stream, as described above.
• the reproduction of the multimedia element of the multimedia stream, in the rendering device, and the execution of the operation processing additional data synchronized with this multimedia element, in the execution device are synchronized.
• the synchronization of the processing of the additional data and the restitution of the multimedia stream is made possible by the use of the table AIT comprising the additional data synchronized with the multimedia stream carrying this table AIT.
• this AIT table being received and processed by a single rendering device 20, the execution device 21 in charge of their processing receives them and thus also processes them synchronously.
• the synchronization of the additional data in the HBBTV stream is made possible by the fact that an HBBTV stream is encapsulated in an object carousel coded according to the DSM CC standard.
• This tool associated with MPEG2 TS manages in particular the transmission of synchronization events as well as additional data in the transport stream M PEG2.
• Stream Event consist of small data packets that can be transmitted synchronously in the program signal. They therefore allow the transmission of additional data at specific times, for example questions or answers to an interactive quiz.
• these packets are immediately processed after their demultiplexing within the rendering device 20 and immediately transmitted to the destination device 21. It is therefore possible to synchronize fine and precise additional data (such as an interactive application) with the multimedia stream of a TV channel.
• the interactive application is executed both on the device for rendering the HBBTV stream and also on the separate execution device.
• the table AIT comprises for example a descriptor indicating that the interactive application is intended to be executed on the execution device and a descriptor indicating that the application can also be executed on the rendering device.
• the interactive application is visually inconvenient for the user, he may prefer not to use his smartphone and interact with the application directly on his connected TV.
• the execution of the interactive application on the connected TV may be conditioned by a user action, such as a validation on a specific button on the remote control.
• the generation method inserts, into a multimedia stream, additional data intended to be processed synchronously with the reproduction of the part of the corresponding stream.
• additional data For example, in the case of an HBBTV stream, these additional data correspond to an interactive application, identifiable by an AIT table carried by the stream which contains the loading and execution information of this interactive application.
• the method of generating the multimedia stream inserts, in the multimedia stream, an indicator enabling the device for restitution of the stream to detect that the additional data is intended for an execution device distinct from the device for restitution of the stream.
• this flag corresponds to a descriptor of the AIT table, indicating on which device the interactive application is intended to be executed.
• the generation method transmits the multimedia stream to at least one rendering device, designed for, following the detection of the destination information, transmitting the additional data to a destination execution device, according to the different embodiments. particular of the invention.
• FIGS. 4a to 4c respectively, the simplified structure of a rendering device, the simplified structure of an execution device and the simplified structure of a generating device, according to an embodiment are presented. particular of the invention.
• such a rendering device comprises a memory 2000 comprising a buffer memory, a processing unit 2001, equipped for example with a ⁇ micropr microprocessor and driven by the computer program 2002, implementing the method of synchronization according to the invention.
• the code instructions of the computer program 2002 are for example loaded into a RAM before being executed by the processor of the processing unit 2001.
• the processing unit 2001 receives as input a stream multimedia.
• the microprocessor of the processing unit 2001 implements the steps of the synchronization method described above, according to the instructions of the computer program 2002, for synchronizing the reproduction of an element of the multimedia stream with the execution of the processing, on a separate device, additional data synchronized with the stream.
• the rendering device further comprises: a multimedia stream receiving module, the multimedia stream comprising additional data synchronized with a multimedia element contained in the multimedia stream; multimedia stream, a module for detecting, in the multimedia stream, at least one destination information indicating that the additional data is intended for at least one separate execution device of the rendering device, a module for extracting the multimedia element and additional data of the multimedia stream, a module for transmitting additional data to the execution device, the transmission triggering an execution, by a software application of the execution device, of a processing operation of the additional data transmitted, and a module for rendering the multimedia element.
• a multimedia stream receiving module the multimedia stream comprising additional data synchronized with a multimedia element contained in the multimedia stream
• multimedia stream a module for detecting, in the multimedia stream, at least one destination information indicating that the additional data is intended for at least one separate execution device of the rendering device, a module for extracting the multimedia element and additional data of the multimedia stream, a module for transmitting additional data to the execution device, the transmission triggering an execution, by a software application of
• These modules are controlled by the microprocessor of the processing unit 2001.
• such an execution device comprises a memory 2100 comprising a buffer memory, a processing unit 2101, equipped for example with a ⁇ micropr microprocessor, and driven by the computer program 2102, setting implement the synchronization method according to the invention.
• the code instructions of the computer program 2102 are for example loaded into a RAM before being executed by the processor of the processing unit 2101.
• the processing unit 2101 receives the data input additional.
• the microprocessor of the processing unit 2101 implements the steps of the synchronization method described above, according to the instructions of the computer program 2102.
• the device for executing the additional data further comprises: a discovery module of at least one rendering device announced within the local network, a module for receiving additional data from the rendering device and an execution module, by a software application of the execution device, an operation of processing of additional data received.
• such a generation device comprises a memory 40 comprising a buffer memory, a processing unit 41, equipped for example with a microprocessor ⁇ and driven by the computer program 42, implementing the method of generation according to the invention.
• the code instructions of the computer program 42 are for example loaded into a RAM before being executed by the processor of the processing unit 41.
• the processing unit 41 receives as input a stream multimedia.
• the microprocessor of the processing unit 41 implements the steps of the method of generation described above, according to the instructions of the computer program 42, for generating a multimedia stream comprising synchronized additional data.
• the device for generating a multimedia stream comprises: means for inserting, into the multimedia stream, additional data and at least one destination information, indicating that the additional data is intended for at least one device distinct execution of a device for rendering the multimedia stream, the devices being present within a local communication network, and means for transmitting the multimedia stream to at least one device for rendering the multimedia stream, designed for, following the detection of the destination information, transmit the additional data to a destination execution device.
• These modules are driven by the microprocessor of the processing unit 41.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Databases & Information Systems (AREA)
• Computer Security & Cryptography (AREA)
• General Engineering & Computer Science (AREA)
• Software Systems (AREA)
• Business, Economics & Management (AREA)
• Marketing (AREA)
• Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Priority Applications (1)

Applications Claiming Priority (2)

Related Child Applications (1)

Publications (1)

Family

ID=49578498

Family Applications (2)

Family Applications Before (1)

Country Status (6)

Families Citing this family (2)

Family Cites Families (15)

• 2013
  • 2013-09-26 FR FR1359311A patent/FR3011155A1/fr not_active Withdrawn
• 2014
  • 2014-09-26 KR KR1020167009978A patent/KR102205932B1/ko active IP Right Grant
  • 2014-09-26 EP EP23166037.4A patent/EP4224868A3/de active Pending
  • 2014-09-26 WO PCT/FR2014/052427 patent/WO2015044607A1/fr active Application Filing
  • 2014-09-26 EP EP14796206.2A patent/EP3050311A1/de not_active Ceased
  • 2014-09-26 JP JP2016516843A patent/JP2016536826A/ja active Pending
  • 2014-09-26 US US15/025,124 patent/US10244282B2/en active Active
• 2019
  • 2019-09-20 JP JP2019171040A patent/JP7253477B2/ja active Active

Non-Patent Citations (2)

Also Published As

Similar Documents

Legal Events