EP2761884B1 - Method of saving content to a file on a server and corresponding device - Google Patents
Method of saving content to a file on a server and corresponding device Download PDFInfo
- Publication number
- EP2761884B1 EP2761884B1 EP12762294.2A EP12762294A EP2761884B1 EP 2761884 B1 EP2761884 B1 EP 2761884B1 EP 12762294 A EP12762294 A EP 12762294A EP 2761884 B1 EP2761884 B1 EP 2761884B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- content
- time
- playback
- area network
- local area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims description 26
- 230000002123 temporal effect Effects 0.000 claims description 29
- 238000012545 processing Methods 0.000 claims description 17
- 230000002457 bidirectional effect Effects 0.000 claims description 4
- 230000006870 function Effects 0.000 description 14
- 230000001360 synchronised effect Effects 0.000 description 13
- 238000012546 transfer Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000007726 management method Methods 0.000 description 2
- 230000000295 complement effect Effects 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 238000013497 data interchange Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
- H04N21/43072—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of multiple content streams on the same device
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/2803—Home automation networks
- H04L12/2805—Home Audio Video Interoperability [HAVI] networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
- H04N21/2625—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists for delaying content or additional data distribution, e.g. because of an extended sport event
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
- H04N21/4334—Recording operations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
- H04N21/43615—Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/2803—Home automation networks
- H04L2012/2847—Home automation networks characterised by the type of home appliance used
- H04L2012/2849—Audio/video appliances
Definitions
- This invention relates to the domain of content distribution in general and more specifically to the availability of streamed content on a server in a local area network.
- Interoperability allows equipment to communicate with other equipment and to make functionality available to a user while ensuring simplicity of implementation. For example, it is convenient to be able to connect a printer to a local area network and to be able to print a document from a computer that is connected to the same network without having to first configure a set of parameters used for printing. To do this, there are interoperability standards that, implemented, allow equipment to automatically be detected after being connected to a network and to communicate its characteristics to other equipment, such as its make, model, main function, capacity, or even services that can be implemented through it.
- UPnP universal plug and play
- UPnP is defined by a set of documents available at http: // upnp.org / ) .
- UPnP is a standard whose purpose is to allow equipment to easily connect to a local area network and to simplify its implementation on the network.
- UPnP-compatible equipment also supports file sharing, communication, and data interchange for utilities and entertainment applications.
- UPnP enables the interoperability of equipment with greater simplicity for the user by defining and publishing UPnP control protocols based on the usual communication standards for networks.
- UPnP AV is the part of the standard dealing with audio and video aspects. The current version of UPnP AV is UPnP AV :4.
- UPnP can synchronize their respective playback on one or more playback devices.
- the same content can be a television program, and the components can be the audio and video or the videos corresponding to two different views of the same scene. Synchronization is possible if the components each contain temporal information related to their playback and the pieces of equipment share a common clock. Sharing a common clock applies to equipment connected to the same local area network.
- the common clock for a local area network is commonly called a "wall clock".
- Methods for saving content allow that content to be played back from one or more files stored on a local server, the playback being specified, for example, according to a UPnP standard.
- these methods have the disadvantage of not allowing access to streamed content, such as audiovisual content transmitted by DTT (Digital Terrestrial Television), by satellite, or even multicasted on a broadband network.
- DTT Digital Terrestrial Television
- DVD Digital Video Broadcasting
- ETSI Technical specification TS 102 905
- UPnP Content Directory services proposes a mapping of some UPnP functionalities in the DVB system.
- this technical specification does not address synchronizing a play back of a broadcast content with some other content.
- Patent application WO201102870A1 describes a method for synchronizing a playback of a locally stored audio file by two different audio playback devices.
- the patent application discloses a system based on UPnP and on a common wall clock in the home network.
- the common wall clock, shared by both audio devices is used to correct the time information so as to synchronize both audio devices as they play back a data stream.
- WO201102870A1 does not address the problem of synchronizing a play back of a broadcast content with some other content.
- This invention resolves at least one of the drawbacks of the prior art. Specifically, it aims to allow a data stream to be saved to a server, the saving being adapted to content playback, comprising playback of a content synchronized with another content playback, for example, according to the UPnP AV :4 standards.
- the invention relates to a method for saving content to a server that is connected to a local area network, the server being adapted to the synchronized playback of audiovisual content by one or more playback devices connected to the local area network and by the use of an interoperability protocol between the server and the playback device(s).
- the method comprises the steps of:
- the method further includes a playback step whose start time depends on temporal playback information.
- the playback step cannot begin before a time specified by the temporal playback information, the content being unavailable before the specified time.
- the playback step begins at a time specified by the temporal playback information.
- the playback step includes a step for transmitting the file to at least one destination at a time specified by the temporal playback information.
- the playback step includes a step for transmitting the file to at least one destination upon receipt of a request to transmit at least a portion of the content.
- the step for calculating temporal playback information depends on the amount of space available for saving content.
- the step for saving at least a portion of the content to a file uses a buffer file according to a circular buffer method.
- the server file contains all of the content after it is received.
- the broadcast channel is a bidirectional broadband channel.
- the server is a local media server that is compatible with the Universal Plug and Play AV interoperability standard.
- the temporal playback information is written to a content description service information field indicating a scheduled start time parameter (ScheduledStartTime).
- the temporal playback information can be accessed by equipment connected to the server in a content description service.
- the invention also relates to a device for saving content to a file.
- the device includes an input interface to receive transmitted content that includes timestamps, the timestamps indicating the time elapsed since the start of the content; a module for calculating at least a temporal playback information of a possible playback start time from at least one timestamp and a wall clock, the wall clock being a local area network time reference; and a module for saving that can save transmitted content to a file and save temporal playback information.
- the device includes a playback module that can play the file, based on the temporal playback information.
- the playback of the file can then be synchronized with the playback of another content (or another view of the same content).
- the device includes a module for saving the at least one piece of temporal playback information to a content description service information field indicating a scheduled start time parameter.
- the device includes an interface for transmitting content at a moment based on the temporal playback information.
- modules shown are functional units that may or may not correspond to physically distinguishable units.
- these modules or some of them can be grouped together in a single component, or constitute functions of the same software.
- some modules may be composed of separate physical entities.
- the embodiment falls within the scope of a home local area network connected to a wide area network by means of a network gateway, but the invention is not limited to this particular environment and can be applied within other environments.
- FIG. 1 shows a home local area network 2 connected to a wide area network 3, by means of a network gateway 1.
- the wide area network 3 is a bidirectional broadband network, commonly known as the Internet.
- the home local area network is an Ethernet (ISO/IEC 8802-3) network according to the embodiment of the invention that is well known to those skilled in the art and includes wired connections.
- the local area network contains wireless connections, such as those described in the wireless transmission standards IEEE802.11.
- Various equipment is connected to the home local area network 2. This equipment includes the network gateway 1, a personal computer 5, and two playback devices 9 and 4.
- the playback devices 9 and 4 are televisions, each with an Ethernet network connection interface for receiving a digital stream carrying audiovisual content and playing it back in the form of playing a video or sound.
- the network gateway 1 includes a modem and a router for connecting the home local area network 2 to the Internet and for accessing the Internet and related services such as telephony, the television, and video on-demand for the subscriber, a user of the home local area network 2.
- a content streamer 7 is also connected to the wide area network 3 and streams audiovisual content that has been encoded by the encoder 8 on the wide area network 3. Audiovisual content is streamed according to a mode of multicast transmission. Also, a receiver device that can receive a multicast data stream can subscribe to a multicast stream transmitted by the content streamer 7. Such a device can be, for example, the personal computer 5 or the network gateway 1.
- a content server 6 is also connected to the wide area network 3.
- the content server provides access, through the wide area network 3, to audiovisual content files encoded by the encoder 8.
- the personal computer 5 connected to the home local area network 2 can download a file and save it to a storage unit on the local network 2.
- a device connected to the home local area network 2 it is therefore possible, for a device connected to the home local area network 2, to receive a multicast data stream transporting audiovisual content streamed by the content streamer 7 and to download audiovisual content in the form of a file from the content server 6 for the purposes of saving it to a storage unit that is connected to the home local area network 2.
- the audiovisual content is thus available in the form of files and can include common temporal information since the content was encoded by the encoder 8, regardless of whether it is stored on the content server 6 or streamed by the content streamer 7.
- FIG. 2 shows the overall architecture of the network gateway 1 according to one embodiment of the invention.
- the network gateway 1 includes a modem/router 12 for connecting the local area network 2 and the wide area network 3.
- the gateway 1 further includes a local server 13 that is connected to the local area network 2.
- the local server 13 is compatible with the UPnP AV :4 interoperability standard and allows files to be stored and accessed from equipment that is connected to the home local area network 2. It is therefore possible to share content files on the local server 13.
- the system architecture that is shown allows sites that are accessible over the Internet, represented by the wide area network 3, to be viewed from a personal computer 5 that is connected to the home local area network 2. It is then possible to access the content server 6 connected to the wide area network 3 from the personal computer 5. Therefore, a user of the personal computer can download files that are available on the content server 6 to the personal computer 5 and save the downloaded files to the local server 13 for gateway 1.
- the local server 13 includes a saving unit 133 shown in Figure 3 that is accessible from the home local area network 3.
- Figure 3 illustrates the local server 13 for the network gateway 1.
- the local server 13 includes an interface for connecting to a local area network 131 for the connection to the home local area network 2.
- the local server also includes a calculator 132 that is connected to the interface 131 and a saving unit 133 that is connected to the calculator.
- the interface 131 is a bidirectional interface that can send and receive data to and from the saving unit 133 through the calculator 132. All of the management operations, control operations, and configurations to the network interface 131, the calculator 132, and the saving unit 133 are performed by the control and processing unit 134 by means of the control bus 135.
- the control and processing unit 134 includes a microcontroller and related memory (not shown) that execute routines used for implementing functions on the local server 13.
- the saving unit 133 includes a hard drive and a hard disk controller that can be configured by the control unit 134.
- the control unit 134 implements the services and functions specified by the UPnP AV:4 interoperability standard.
- the local server 13 is compatible with the UPnP AV :4 specifications and is seen as a Media Server, as defined by the specifications and all of the protocols described in UPnP AV :4.
- a client application executed by the personal computer 5 can detect multicast streams streamed by the content streamer 7 and prompt the user to have the local server 13 store a multicast stream in the form of a file containing audiovisual content.
- the stored file is available for a playback device that is connected to the local area network 2.
- the UPnP AV :4 standard defines a Media Server, a Media Renderer, and a Control Point. According to UPnP, the Control Point communicates with the Media Servers using established protocols that allow them to identify files on the Media Server and access a Content Descriptor Service (CDS).
- CDS Content Descriptor Service
- a piece of equipment that implements the UPnP Control Point function connected to a local area network can therefore request for a piece of equipment implementing the UPnP Media Renderer function and connected to the same local area network to play audiovisual content from a file that is accessible on the network and stored on equipment implementing the UPnP Media Server function.
- the synchronous playback of both streams of audiovisual content under UPnP is carried out by a SyncPlay function.
- the model for UPnP interaction between compatible equipment is such that the Control Point interacts with the Media Server(s) and the appropriate Media Renderer for playing back content.
- the Media Server and Media Renderer interact with one another directly via a protocol, depending on the type of local area network connecting them.
- the Control Point uses UPnP to initialize and configure the Media Server and the Media Renderer so that the content to be played back is transferred from one piece of equipment to the other.
- the SyncPlay function provides the Media Renderer with a presentation time for playing back content. This means that the Media Server and the Media Renderer share a wall clock, such as the reference clock for the home local area network 2.
- NTP Network Time Protocol
- RFC1375 Network Time Protocol
- Sharing a common clock among multiple pieces of equipment in a local area network is possible, for example, via the Network Time Protocol (NTP, described in the document RFC1305) for local and wide area networks.
- NTP is a protocol that makes it possible to use a network to synchronize the reference clock used by equipment connected to the network.
- This system for synchronizing equipment to a wall clock is based on a client/server principle.
- Converting a multicast data stream transporting audiovisual content with timestamps indicating the time elapsed since the start of the content to be played back in a content file that is available on a UPnP AV :4 Media Server therefore makes it possible, depending on the embodiment of the invention, to play back on a piece of playback equipment that is connected to the local area network and implements the Media Renderer functions described in UPnP AV :4.
- Converting the data stream in a file further allows synchronous playback with another component of audiovisual content that is available in another file on a UPnP AV :4 Media Server on the same local area network.
- the example of multiple views of the same concert, filmed and recorded by different cameras and then encoded by the encoder 8 and stored partly on the content server 6 and partly, including a full view of the concert, on a streaming storage space 7 illustrates one of the advantages of the method.
- the server 6 makes audiovisual content available showing, for example, only a view of the pianist throughout the concert.
- the content streamer 7 has audiovisual content for a main view that shows all of the musicians during the concert.
- converting the received multicast stream into a file available on a UPnP Media Server that is connected to the home local area network 2 allows for playback to be synchronized with the pianist view if it was previously downloaded from the content server 6 and made available on a UPnP AV :4 Media Server that is connected to the home local area network 2.
- the main view that is multicasted includes a series of timestamps indicating the time elapsed since the start of the concert.
- the client application executed by the personal computer 5 detects the multicast stream that is streamed from the content streamer 7 and displays the list of streams and the corresponding streamed content on the screen of the personal computer 5.
- the user of the client application can request to save a multicast stream in the form of a file from the UPnP AV :4 Media Server on the local server 13.
- the control and processing unit 134 receives configuration information from the client application executed by the personal computer 5.
- the control and processing unit 134 configures the network interface 131 to receive and save a multicast data stream on the local server 13.
- the network gateway sends a request to subscribe to the stream using the IGMP routing protocol (Internet Group Management Protocol, defined as a standard in RFC 1112), and the multicast stream is then transmitted to the network interface 131 on the network gateway 1.
- IGMP routing protocol Internet Group Management Protocol, defined as a standard in RFC 1112
- the audiovisual content from the multicast stream is transmitted by packets according to the Real-Time Transfer Protocol (RTP) on a User Datagram Protocol (UDP), which are common transmission protocols for transmitting audiovisual content over wide and local area networks.
- RTP Real-Time Transfer Protocol
- UDP User Datagram Protocol
- the usable data that encodes the audiovisual content uses a Motion Picture Expert Group (MPEG) data compression protocol. Timestamps, like those specified in the ETSI TS 102823 standard, are also transmitted at regular intervals in the stream corresponding to the audiovisual content, indicating the time elapsed since the start of the content for the purpose of its playback, referred to here at the Time Offset current value (TOc).
- TOc Time Offset current value
- the local area network interface 131 receives the packets from the multicast stream and filters them in order to transfer only the packets carrying usable data (ex. audio and video components of the transmitted audiovisual content) and packets including timestamps (TOc) to the calculator 132.
- usable data ex. audio and video components of the transmitted audiovisual content
- TOc timestamps
- the calculator 132 calculates, based on the reference time of the local area network 2 and a timestamp received in the multicast stream, indicating the current position relative to the start of the audiovisual content, temporal playback information in accordance with UPnP AV :4.
- This temporal playback information is an availability time for the playback, expressed in hours, minutes, seconds, and tenths of a second.
- This temporal information is then read by the control unit 134 in a calculator 132 log and written to an information field for the CDS associated with the save file that stores the data corresponding to the audiovisual content on the saving unit 133 for the local server 13.
- Figure 4 temporally illustrates multicasted streamed content being received and a file being downloaded for synchronized playback of its content.
- the multicast stream corresponds to the main view of the music concert transmitted from the streaming server 7.
- Receiving and reading the first timestamp TOc associated with the adjacent data encoding the corresponding audio and video for the view indicates that the concert started 11 minutes and 3 seconds ago.
- the reference time for the local network 2 (called "wall clock") is available to the calculator 132. It is transmitted to the calculator using, for example, the NTP protocol.
- the calculator 132 for the local server 13 that receives the multicast stream calculates the start time of the content (in terms of playback) relative to the ToD reference time for the home local area network 2.
- ToD0 ToD - TOc which corresponds to, for example:
- Calculating ToD0 can accurately position the transmitted content in time, with reference to the clock for the home local area network 2 (the wall clock), since the content is encoded in a system without a common clock with the network 2. It is thereby possible to synchronize the playback of the content with the playback of one or more other pieces of content on a local server belonging to the home local area network 2 by defining an exact playback time.
- a maximum processing time TBR (“Time Before Rendering") is defined for all of these operations. Playback on equipment implementing the UPnP Media Renderer function can start from ToD + TBR.
- the control and processing unit 134 reads the ToDRc, TOc, and ToD0 information in the calculator 132 logs and writes the ToDRc value to the information field called ScheduleStartTime for the UPnP AV :4 content description service (CDS) that is available on the saving unit of the local server 13.
- This information field is meant to indicate the scheduled start time for playing back content.
- control and processing unit 134 reads the ToDRc, TOc, and ToD0 information in the calculator 132 logs and then calculates and writes a value corresponding to a time after ToDRc to the information field called ScheduleStartTime for the UPnP AV :4 content description service (CDS) in order to delay the start of playback based on other synchronization criteria.
- CDS content description service
- the method therefore calculates temporal playback information (ex. ToDRc) from at least one timestamp (ex. TOc) and the reference clock for the home local area network 2, the reference clock being the local area network 2 time reference.
- Temporal playback information which can be ToDRc or a value corresponding to a later time, is saved to the information field for the content description service (CDS) indicating a scheduled start time (ScheduledStartTime).
- CDS content description service
- the UPnP AV :4 Control Point is executed by the personal computer 5.
- the SyncPlay command provides a parameter that indicates the location of the file on the network, along with the exact time to start playback and a position reference parameter.
- a second SyncPlay command tells the playback device 4, for example, to play back the content of the file for the pianist view, which was previously downloaded onto the UPnP AV :4 Media Server from the local server 13, indicating a playback start time calculated by the Control Point from the calculated start time for the main view and the TBR processing time.
- the UPnP AV :4 Control Point also tells the playback device 4 to locate the corresponding file.
- the playback device 4 that implements the UPnP AV :4 Media Renderer functions sends a request to transfer content to the UPnP AV :4 Media Server from the local server 13.
- the UPnP AV :4 Media Server for the local server 13 transfers the contents of the file to the UPnP AV :4 Media Renderer (the playback device 4), which plays back the contents synchronously with the other UPnP AV :4 Media Renderer (the playback device 9), the temporal parameters sent by the SyncPlay commands referring to the clock for the home local area network 2 and calculated according to the same principle by the UPnP AV :4 Control Point.
- FIG. 5 is a block diagram that illustrates the method according to one embodiment of the invention.
- the step S1 is the starting step.
- the multicast stream is streamed by the content streamer 7 over the wide area network 3, but it is not routed to the network gateway 1.
- a user of the personal computer on the home local area network 2 orders the multicast stream to be saved to the local server (UPnP Media Server) at the network gateway 1.
- a command is sent from the local computer 5 to the network gateway 1, which sends a subscription request to the multicast stream, using a routing protocol like IGMP.
- the multicast stream is routed to the network gateway and received by the interface 131 with the local server 13.
- the calculator is configured to detect the first timestamp in the multicast stream, indicating the time elapsed since the start of the streamed content (the time is the time elapsed during playback).
- the calculator detects and reads a timestamp and calculates the time difference between the current reference time for the home local area network 2 ToD (the "wall clock") and the value indicated by the TOc timestamp.
- the ToD0 time when the start of the content would have been received if the gateway had been subscribed to the multicast flow prior to the start of the stream, is therefore known, and it is possible to locate all of the streamed content based on the reference time for the home local area network 2 (the "wall clock"). This time is called the content start time.
- the calculator also updates a TBR log that indicates the total data processing time and conditions when the data can be begin to be presented for content playback.
- the audiovisual content is saved to the storage space for the saving unit 133.
- the storage space can be a hard drive, flash memory, RAM, or any other media capable of saving audiovisual content.
- the storage space is included with the saving unit 133 or connected to the saving unit 133.
- Content can be saved to a file containing the data since the start of receipt (from the subscription to the multicast stream) to the end of the content or until the end of receipt (end of the subscription to the multicast stream).
- saving can be done using a circular memory buffer technique, well known to those skilled in the art, that limits how much space is needed for saving.
- the content is then written to a file using a write pointer in circular memory, and playback is then positioned so that a read pointer used for playback follows the write pointer.
- the calculator 132 uses at least some of ToD0, ToD, TOc, and TBR to determine the first point from which content should be played back by a playback device that is compatible with a Media Renderer, as described in the UPnP standard.
- This point which can be expressed as a time, is written to the ScheduleStartTime field in the UPnP content descriptor, making it possible to give the content from the local server 13 to a device that is compatible with the Control Point, as defined in UPnP.
- the content playback start time is "forced" by writing the ScheduleStartTime field.
- the ScheduleStartTime field is initially set to indicate the start time for a program according to the UPnP standard.
- a UPnP Control Point can read temporal playback information, in the form of a playback start time, in the ScheduleStartTime field for the CDS on the media server. If the Control Point sends a synchronized playback command to a UPnP Media Renderer playback device, the method includes a synchronized playback step whose start time depends on the temporal playback information.
- the ToDRC value is calculated based on other temporal and non-temporal information. Calculating the ToDRC from the ToD (the wall clock) and the TOc (the first timestamp sent in the stream, indicating the time elapsed since the start of the content), along with the TBR processing time, determines the first moment when playback is possible.
- ToD the wall clock
- TOc the first timestamp sent in the stream, indicating the time elapsed since the start of the content
- the playback start time is calculated based on the available space on the saving unit for saving the content, considering that the data must be able to be stored without overflow and without disrupting the content playback.
- the playback start time happens before the circular buffer has filled with data from the multicast stream, for example.
- the method allows audiovisual content, for example, sent by a broadcast channel, to be made accessible from a media server that is connected to a local area network, like other content available on the media server and coming from another source.
- the media server that is connected to the local area network is, for example, a media server as defined in the UPnP AV standard.
- Content that has been streamed and saved to a file on a media server can then be played back on a playback device that is connected to the network at any moment (or time), as defined according to various criteria. This can be from the time when the first content data was received, to which the necessary processing time is added between receipt and playback and referenced according to the clock for the local area network. It can also be a specific time.
- a tip for the method is using an information field that tells a control device or playback device when the content can be played back. It is therefore possible to force this time for one or more playback devices.
- This moment (this time) can be delayed based on other criteria to conditionalize or to synchronize the playback of content.
- the method can force a time before which playback is not possible.
- content is synchronously played back from temporal playback information (such as the information field ScheduleStartTime defined in the UPnP standard) by multiple playback devices.
- temporal playback information such as the information field ScheduleStartTime defined in the UPnP standard
- playback is initiated by separate control requests transmitted to each of the playback devices or by one control request transmitted to all or some of the playback devices connected to the home local area network (such as devices belonging to the same group (based on their type, for example)).
- the requests are unicast, multicast, or broadcast.
- saving happens at the end of the transmitted audiovisual content or applies sequentially as content is streamed.
- the media server that contains the save file and the playback device for playing back the content are included in a single piece of equipment or shared among multiple separate pieces of equipment.
- the file is fully or partially transmitted from the media server (Media Server) to one or more playback devices (Media Renderer).
- the playback device is included on the same equipment as the media server.
- the broadcast network is suitable for transmitting streamed contend by a general or targeted broadcast mode to one or more destinations.
- the content includes audio and/or video data and/or interactive data.
- the media server is a dedicated piece of equipment, or it is included in devices such as a network gateway, a television receiver/decoder, a computer, a television, a personal assistant, or a mobile phone.
- the media server is a dedicated device, or it is included in one or more devices.
- the timestamps are transmitted as a content component or in packet headers received via the broadcast channel, or in content in a form using the audio or video watermarking technique that is already known as watermarking or fingerprinting.
- the reference clock for the local area network is transmitted from a server on the wide area network to all of the equipment connected to the home local area network.
- the media server is a local or remote server.
- the invention also applies to any device that is connected to a network, contains a file server where the files contain one or more audio/video components, and whose component playback is synchronized relative to a network reference clock.
- the invention applies to any device or system implementing the method described above, according to its variants.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Automation & Control Theory (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Information Transfer Between Computers (AREA)
- Television Signal Processing For Recording (AREA)
- Small-Scale Networks (AREA)
Description
- This invention relates to the domain of content distribution in general and more specifically to the availability of streamed content on a server in a local area network.
- Current local area networks allow various equipment to be interconnected, such as computers, data storage devices, audio and/or video playback devices, data receiver devices, printers, routers, landline and mobile telephones, personal assistants, and communications interfaces. This ability to interconnect nearby equipment and to implement sometimes complementary functions produces a growing need for interoperability between equipment connected to the same network.
- Interoperability allows equipment to communicate with other equipment and to make functionality available to a user while ensuring simplicity of implementation. For example, it is convenient to be able to connect a printer to a local area network and to be able to print a document from a computer that is connected to the same network without having to first configure a set of parameters used for printing. To do this, there are interoperability standards that, implemented, allow equipment to automatically be detected after being connected to a network and to communicate its characteristics to other equipment, such as its make, model, main function, capacity, or even services that can be implemented through it.
- One interoperability standard is UPnP ("universal plug and play", UPnP is defined by a set of documents available at http://upnp.org/). UPnP is a standard whose purpose is to allow equipment to easily connect to a local area network and to simplify its implementation on the network. UPnP-compatible equipment also supports file sharing, communication, and data interchange for utilities and entertainment applications. UPnP enables the interoperability of equipment with greater simplicity for the user by defining and publishing UPnP control protocols based on the usual communication standards for networks. UPnP AV is the part of the standard dealing with audio and video aspects. The current version of UPnP AV is UPnP AV :4.
- With UPnP, it is possible to see from a computer, for example, files stored on a storage space on remote equipment that has been connected to the same local area network as the computer, without having to configure either piece of equipment. Similarly, it is possible to begin playback of audio/video content stored in a file saved on a storage space of a remote device, such as a hard drive, from a playback device, such as a television that is compatible with the content format, as long as both pieces of equipment were connected to the same local area network without any other configuration.
- In the case wherein separate files, stored on a hard drive, contain separate components with the same audiovisual content, for example, UPnP can synchronize their respective playback on one or more playback devices. The same content can be a television program, and the components can be the audio and video or the videos corresponding to two different views of the same scene. Synchronization is possible if the components each contain temporal information related to their playback and the pieces of equipment share a common clock. Sharing a common clock applies to equipment connected to the same local area network. The common clock for a local area network is commonly called a "wall clock".
- Methods for saving content allow that content to be played back from one or more files stored on a local server, the playback being specified, for example, according to a UPnP standard. However, these methods have the disadvantage of not allowing access to streamed content, such as audiovisual content transmitted by DTT (Digital Terrestrial Television), by satellite, or even multicasted on a broadband network.
- Digital Video Broadcasting (DVB) published a specification for DVB services in the home network as an ETSI Technical specification (TS 102 905). This technical specification describes how DVB broadcast services are distributed over home networks. To that end the technical specification mentions UPnP Content Directory services and proposes a mapping of some UPnP functionalities in the DVB system. However this technical specification does not address synchronizing a play back of a broadcast content with some other content.
- Patent application
WO201102870A1 WO201102870A1 - This invention resolves at least one of the drawbacks of the prior art. Specifically, it aims to allow a data stream to be saved to a server, the saving being adapted to content playback, comprising playback of a content synchronized with another content playback, for example, according to the UPnP AV :4 standards.
- More specifically, the invention relates to a method for saving content to a server that is connected to a local area network, the server being adapted to the synchronized playback of audiovisual content by one or more playback devices connected to the local area network and by the use of an interoperability protocol between the server and the playback device(s).
- The method comprises the steps of:
- receiving, from a broadcast channel, by a piece of equipment that is connected to the local area network, transmitted content, the transmitted content including timestamps, the timestamps indicating the time elapsed since the start of the content,
- calculating at least one piece of temporal playback information for synchronized playback by a playback device that is connected to the local area network, from at least one of the timestamps and a wall clock for the local area network,
- saving at least a portion of the content to the server of said local area network,
- saving said temporal playback information to the network server, so that it is possible to synchronize the playback of the broadcasted content, received from the broadcast channel, with another content playback, both playbacks being achieved by equipment connected to the local area network.
- According to one embodiment of the invention, the method further includes a playback step whose start time depends on temporal playback information.
- According to one embodiment of the invention, the playback step cannot begin before a time specified by the temporal playback information, the content being unavailable before the specified time.
- According to one embodiment of the invention, the playback step begins at a time specified by the temporal playback information.
- According to one embodiment of the invention, the playback step includes a step for transmitting the file to at least one destination at a time specified by the temporal playback information.
- According to one embodiment of the invention, the playback step includes a step for transmitting the file to at least one destination upon receipt of a request to transmit at least a portion of the content.
- According to one embodiment of the invention, the step for calculating temporal playback information depends on the amount of space available for saving content.
- According to one embodiment of the invention, the step for saving at least a portion of the content to a file uses a buffer file according to a circular buffer method.
- According to one embodiment of the invention, the server file contains all of the content after it is received.
- According to one embodiment of the invention, the broadcast channel is a bidirectional broadband channel.
- According to one embodiment of the invention, the server is a local media server that is compatible with the Universal Plug and Play AV interoperability standard.
- According to one embodiment of the invention, the temporal playback information is written to a content description service information field indicating a scheduled start time parameter (ScheduledStartTime).
- According to one embodiment of the invention, the temporal playback information can be accessed by equipment connected to the server in a content description service.
- The invention also relates to a device for saving content to a file. The device includes an input interface to receive transmitted content that includes timestamps, the timestamps indicating the time elapsed since the start of the content; a module for calculating at least a temporal playback information of a possible playback start time from at least one timestamp and a wall clock, the wall clock being a local area network time reference; and a module for saving that can save transmitted content to a file and save temporal playback information.
- According to one embodiment of the invention, the device includes a playback module that can play the file, based on the temporal playback information. The playback of the file can then be synchronized with the playback of another content (or another view of the same content). Advantageously, the device includes a module for saving the at least one piece of temporal playback information to a content description service information field indicating a scheduled start time parameter.
- According to one embodiment of the invention, the device includes an interface for transmitting content at a moment based on the temporal playback information.
- The invention will be better understood and illustrated by means of the following embodiments and implementations, by no means limiting, with reference to the figures attached in the appendix, wherein:
-
Figure 1 is a block diagram of a home local area network connected to a wide area network by means of a network gateway, according to one embodiment of the invention. -
Figure 2 illustrates a network gateway that includes a media server according to one embodiment of the invention. -
Figure 3 illustrates the media server for the network gateway according to one embodiment of the invention. -
Figure 4 temporally illustrates multicasted streamed content being received and a file being downloaded for synchronous playback. -
Figure 5 is a diagram illustrating the steps of the method. - In
Figures 1 to 3 , the modules shown are functional units that may or may not correspond to physically distinguishable units. For example, these modules or some of them can be grouped together in a single component, or constitute functions of the same software. On the contrary, some modules may be composed of separate physical entities. - The embodiment falls within the scope of a home local area network connected to a wide area network by means of a network gateway, but the invention is not limited to this particular environment and can be applied within other environments.
-
Figure 1 shows a homelocal area network 2 connected to awide area network 3, by means of anetwork gateway 1. Thewide area network 3 is a bidirectional broadband network, commonly known as the Internet. The home local area network is an Ethernet (ISO/IEC 8802-3) network according to the embodiment of the invention that is well known to those skilled in the art and includes wired connections. According to another embodiment, the local area network contains wireless connections, such as those described in the wireless transmission standards IEEE802.11. Various equipment is connected to the homelocal area network 2. This equipment includes thenetwork gateway 1, apersonal computer 5, and twoplayback devices playback devices - The
network gateway 1 includes a modem and a router for connecting the homelocal area network 2 to the Internet and for accessing the Internet and related services such as telephony, the television, and video on-demand for the subscriber, a user of the homelocal area network 2. - A
content streamer 7 is also connected to thewide area network 3 and streams audiovisual content that has been encoded by theencoder 8 on thewide area network 3. Audiovisual content is streamed according to a mode of multicast transmission. Also, a receiver device that can receive a multicast data stream can subscribe to a multicast stream transmitted by thecontent streamer 7. Such a device can be, for example, thepersonal computer 5 or thenetwork gateway 1. - A
content server 6 is also connected to thewide area network 3. The content server provides access, through thewide area network 3, to audiovisual content files encoded by theencoder 8. Thepersonal computer 5 connected to the homelocal area network 2 can download a file and save it to a storage unit on thelocal network 2. - It is therefore possible, for a device connected to the home
local area network 2, to receive a multicast data stream transporting audiovisual content streamed by thecontent streamer 7 and to download audiovisual content in the form of a file from thecontent server 6 for the purposes of saving it to a storage unit that is connected to the homelocal area network 2. The audiovisual content is thus available in the form of files and can include common temporal information since the content was encoded by theencoder 8, regardless of whether it is stored on thecontent server 6 or streamed by thecontent streamer 7. -
Figure 2 shows the overall architecture of thenetwork gateway 1 according to one embodiment of the invention. Thenetwork gateway 1 includes a modem/router 12 for connecting thelocal area network 2 and thewide area network 3. Thegateway 1 further includes alocal server 13 that is connected to thelocal area network 2. Thelocal server 13 is compatible with the UPnP AV :4 interoperability standard and allows files to be stored and accessed from equipment that is connected to the homelocal area network 2. It is therefore possible to share content files on thelocal server 13. - The system architecture that is shown allows sites that are accessible over the Internet, represented by the
wide area network 3, to be viewed from apersonal computer 5 that is connected to the homelocal area network 2. It is then possible to access thecontent server 6 connected to thewide area network 3 from thepersonal computer 5. Therefore, a user of the
personal computer can download files that are available on thecontent server 6 to thepersonal computer 5 and save the downloaded files to thelocal server 13 forgateway 1. Thelocal server 13 includes a savingunit 133 shown inFigure 3 that is accessible from the homelocal area network 3. -
Figure 3 illustrates thelocal server 13 for thenetwork gateway 1. - The
local server 13 includes an interface for connecting to alocal area network 131 for the connection to the homelocal area network 2. The local server also includes acalculator 132 that is connected to theinterface 131 and a savingunit 133 that is connected to the calculator. Theinterface 131 is a bidirectional interface that can send and receive data to and from the savingunit 133 through thecalculator 132. All of the management operations, control operations, and configurations to thenetwork interface 131, thecalculator 132, and the savingunit 133 are performed by the control andprocessing unit 134 by means of the control bus 135. - The control and
processing unit 134 includes a microcontroller and related memory (not shown) that execute routines used for implementing functions on thelocal server 13. The savingunit 133 includes a hard drive and a hard disk controller that can be configured by thecontrol unit 134. - The
control unit 134 implements the services and functions specified by the UPnP AV:4 interoperability standard. Thelocal server 13 is compatible with the UPnP AV :4 specifications and is seen as a Media Server, as defined by the specifications and all of the protocols described in UPnP AV :4. - Depending on the embodiment of the invention, a client application executed by the
personal computer 5 can detect multicast streams streamed by thecontent streamer 7 and prompt the user to have thelocal server 13 store a multicast stream in the form of a file containing audiovisual content.
The stored file is available for a playback device that is connected to thelocal area network 2. - The UPnP AV :4 standard defines a Media Server, a Media Renderer, and a Control Point. According to UPnP, the Control Point communicates with the Media Servers using established protocols that allow them to identify files on the Media Server and access a Content Descriptor Service (CDS).
- According to the UPnP AV :4 standard, a piece of equipment that implements the UPnP Control Point function connected to a local area network can therefore request for a piece of equipment implementing the UPnP Media Renderer function and connected to the same local area network to play audiovisual content from a file that is accessible on the network and stored on equipment implementing the UPnP Media Server function. The synchronous playback of both streams of audiovisual content under UPnP is carried out by a SyncPlay function. The model for UPnP interaction between compatible equipment is such that the Control Point interacts with the Media Server(s) and the appropriate Media Renderer for playing back content. The Media Server and Media Renderer interact with one another directly via a protocol, depending on the type of local area network connecting them. The Control Point uses UPnP to initialize and configure the Media Server and the Media Renderer so that the content to be played back is transferred from one piece of equipment to the other. The SyncPlay function provides the Media Renderer with a presentation time for playing back content. This means that the Media Server and the Media Renderer share a wall clock, such as the reference clock for the home
local area network 2. - Sharing a common clock among multiple pieces of equipment in a local area network is possible, for example, via the Network Time Protocol (NTP, described in the document RFC1305) for local and wide area networks. NTP is a protocol that makes it possible to use a network to synchronize the reference clock used by equipment connected to the network. This system for synchronizing equipment to a wall clock is based on a client/server principle.
- Converting a multicast data stream transporting audiovisual content with timestamps indicating the time elapsed since the start of the content to be played back in a content file that is available on a UPnP AV :4 Media Server therefore makes it possible, depending on the embodiment of the invention, to play back on a piece of playback equipment that is connected to the local area network and implements the Media Renderer functions described in UPnP AV :4.
- Converting the data stream in a file further allows synchronous playback with another component of audiovisual content that is available in another file on a UPnP AV :4 Media Server on the same local area network.
- The example of multiple views of the same concert, filmed and recorded by different cameras and then encoded by the
encoder 8 and stored partly on thecontent server 6 and partly, including a full view of the concert, on astreaming storage space 7 illustrates one of the advantages of the method. Theserver 6 makes audiovisual content available showing, for example, only a view of the pianist throughout the concert. Thecontent streamer 7 has audiovisual content for a main view that shows all of the musicians during the concert. When the main view is multicasted, by a television operator, on thewide area network 3 and available to a local area network, such as the homelocal area network 2, converting the received multicast stream into a file available on a UPnP Media Server that is connected to the homelocal area network 2 allows for playback to be synchronized with the pianist view if it was previously downloaded from thecontent server 6 and made available on a UPnP AV :4 Media Server that is connected to the homelocal area network 2. This is possible when the main view that is multicasted (and whose content is "pushed" from thecontent streamer 7 to thenetwork gateway 1 after thenetwork gateway 1 has requested to be part of the multicast stream) includes a series of timestamps indicating the time elapsed since the start of the concert. - Depending on the embodiment of the invention, the client application executed by the
personal computer 5 detects the multicast stream that is streamed from thecontent streamer 7 and displays the list of streams and the corresponding streamed content on the screen of thepersonal computer 5. The user of the client application can request to save a multicast stream in the form of a file from the UPnP AV :4 Media Server on thelocal server 13. The control andprocessing unit 134 receives configuration information from the client application executed by thepersonal computer 5. The control andprocessing unit 134 configures thenetwork interface 131 to receive and save a multicast data stream on thelocal server 13. The network gateway sends a request to subscribe to the stream using the IGMP routing protocol (Internet Group Management Protocol, defined as a standard in RFC 1112), and the multicast stream is then transmitted to thenetwork interface 131 on thenetwork gateway 1. - Depending on the embodiment of the invention, the audiovisual content from the multicast stream is transmitted by packets according to the Real-Time Transfer Protocol (RTP) on a User Datagram Protocol (UDP), which are common transmission protocols for transmitting audiovisual content over wide and local area networks. The usable data that encodes the audiovisual content uses a Motion Picture Expert Group (MPEG) data compression protocol. Timestamps, like those specified in the ETSI TS 102823 standard, are also transmitted at regular intervals in the stream corresponding to the audiovisual content, indicating the time elapsed since the start of the content for the purpose of its playback, referred to here at the Time Offset current value (TOc).
- The local
area network interface 131 receives the packets from the multicast stream and filters them in order to transfer only the packets carrying usable data (ex. audio and video components of the transmitted audiovisual content) and packets including timestamps (TOc) to thecalculator 132. - Depending on the embodiment of the invention, the
calculator 132 calculates, based on the reference time of thelocal area network 2 and a timestamp received in the multicast stream, indicating the current position relative to the start of the audiovisual content, temporal playback information in accordance with UPnP AV :4. This temporal playback information is an availability time for the playback, expressed in hours, minutes, seconds, and tenths of a second. - This temporal information is then read by the
control unit 134 in acalculator 132 log and written to an information field for the CDS associated with the save file that stores the data corresponding to the audiovisual content on the savingunit 133 for thelocal server 13. -
Figure 4 temporally illustrates multicasted streamed content being received and a file being downloaded for synchronized playback of its content. The multicast stream corresponds to the main view of the music concert transmitted from the streamingserver 7. Receiving and reading the first timestamp TOc associated with the adjacent data encoding the corresponding audio and video for the view indicates that the concert started 11 minutes and 3 seconds ago. The reference time for the local network 2 (called "wall clock") is available to thecalculator 132. It is transmitted to the calculator using, for example, the NTP protocol. When the first timestamp detected in the stream is read, the reference time (wall clock) for the home local area network is ToD = 9hr 04min 3s. Thecalculator 132 for thelocal server 13 that receives the multicast stream calculates the start time of the content (in terms of playback) relative to the ToD reference time for the homelocal area network 2. - In other words, ToD0 = ToD - TOc
which corresponds to, for example: - ToD0 = 09
hr 04 min 03 s - 11 min 03 s - ToD0 = 08
hr 53 min - Calculating ToD0 can accurately position the transmitted content in time, with reference to the clock for the home local area network 2 (the wall clock), since the content is encoded in a system without a common clock with the
network 2. It is thereby possible to synchronize the playback of the content with the playback of one or more other pieces of content on a local server belonging to the homelocal area network 2 by defining an exact playback time. - The time from which a portion of received multicast stream can be played back, referenced relative to the reference time ToD of the
local area network 2, must account for processing delays related to receiving, saving, controlling, reading, and transmitting it for playback. A maximum processing time TBR ("Time Before Rendering") is defined for all of these operations. Playback on equipment implementing the UPnP Media Renderer function can start from ToD + TBR. - The possible playback start time for the content whose component is pushed by the streaming
server 7 and received by thelocal server 13 of thegateway 1 is ToDRc = ToD + TBR, referenced relative to clock for thelocal area network 2. - Depending on the embodiment of the invention, the control and
processing unit 134 reads the ToDRc, TOc, and ToD0 information in thecalculator 132 logs and writes the ToDRc value to the information field called ScheduleStartTime for the UPnP AV :4 content description service (CDS) that is available on the saving unit of thelocal server 13. This information field is meant to indicate the scheduled start time for playing back content. - According to another embodiment of the invention, the control and
processing unit 134 reads the ToDRc, TOc, and ToD0 information in thecalculator 132 logs and then calculates and writes a value corresponding to a time after ToDRc to the information field called ScheduleStartTime for the UPnP AV :4 content description service (CDS) in order to delay the start of playback based on other synchronization criteria. - The method therefore calculates temporal playback information (ex. ToDRc) from at least one timestamp (ex. TOc) and the reference clock for the home
local area network 2, the reference clock being thelocal area network 2 time reference. - Temporal playback information, which can be ToDRc or a value corresponding to a later time, is saved to the information field for the content description service (CDS) indicating a scheduled start time (ScheduledStartTime).
- If the file for the pianist view, available on the
content server 6, is downloaded and stored on the savingunit 133 so that it is available for reading as of time ToD = 9:10, for example, it is possible for a UPnP AV :4 Control Point to request the synchronized playback of the two views (the main view of the concert and the auxiliary pianist view) by executing SyncPlay commands using the UPnP AV :4 protocol. In one embodiment of the invention, the UPnP AV :4 Control Point is executed by thepersonal computer 5. - A first SyncPlay command tells, for example, the
playback device 9, which implements the UPnP AV :4 Media Renderer functions, to play back the content of the corresponding file upon receipt of the multicast stream available on the UPnP AV :4 Media Server for thelocal server 13 at the time whereby ToDRc = ToD + TBR. The SyncPlay command provides a parameter that indicates the location of the file on the network, along with the exact time to start playback and a position reference parameter. - A second SyncPlay command tells the
playback device 4, for example, to play back the content of the file for the pianist view, which was previously downloaded onto the UPnP AV :4 Media Server from thelocal server 13, indicating a playback start time calculated by the Control Point from the calculated start time for the main view and the TBR processing time. - The UPnP AV :4 Control Point also tells the
playback device 4 to locate the corresponding file. Theplayback device 4 that implements the UPnP AV :4 Media Renderer functions sends a request to transfer content to the UPnP AV :4 Media Server from thelocal server 13. - The UPnP AV :4 Media Server for the
local server 13 transfers the contents of the file to the UPnP AV :4 Media Renderer (the playback device 4), which plays back the contents synchronously with the other UPnP AV :4 Media Renderer (the playback device 9), the temporal parameters sent by the SyncPlay commands referring to the clock for the homelocal area network 2 and calculated according to the same principle by the UPnP AV :4 Control Point. -
Figure 5 is a block diagram that illustrates the method according to one embodiment of the invention. The step S1 is the starting step. The multicast stream is streamed by thecontent streamer 7 over thewide area network 3, but it is not routed to thenetwork gateway 1. - In the step S2, a user of the personal computer on the home
local area network 2 orders the multicast stream to be saved to the local server (UPnP Media Server) at thenetwork gateway 1. A command is sent from thelocal computer 5 to thenetwork gateway 1, which sends a subscription request to the multicast stream, using a routing protocol like IGMP. - In the step S3, the multicast stream is routed to the network gateway and received by the
interface 131 with thelocal server 13. The calculator is configured to detect the first timestamp in the multicast stream, indicating the time elapsed since the start of the streamed content (the time is the time elapsed during playback). - In the step S4, the calculator detects and reads a timestamp and calculates the time difference between the current reference time for the home
local area network 2 ToD (the "wall clock") and the value indicated by the TOc timestamp. The ToD0 time, when the start of the content would have been received if the gateway had been subscribed to the multicast flow prior to the start of the stream, is therefore known, and it is possible to locate all of the streamed content based on the reference time for the home local area network 2 (the "wall clock"). This time is called the content start time. - The calculator also updates a TBR log that indicates the total data processing time and conditions when the data can be begin to be presented for content playback.
- In the step S5, the audiovisual content is saved to the storage space for the saving
unit 133. The storage space can be a hard drive, flash memory, RAM, or any other media capable of saving audiovisual content. Depending on the variant, the storage space is included with the savingunit 133 or connected to the savingunit 133. - Content can be saved to a file containing the data since the start of receipt (from the subscription to the multicast stream) to the end of the content or until the end of receipt (end of the subscription to the multicast stream).
- In another embodiment, saving can be done using a circular memory buffer technique, well known to those skilled in the art, that limits how much space is needed for saving. The content is then written to a file using a write pointer in circular memory, and playback is then positioned so that a read pointer used for playback follows the write pointer.
- In the step S6, the
calculator 132 uses at least some of ToD0, ToD, TOc, and TBR to determine the first point from which content should be played back by a playback device that is compatible with a Media Renderer, as described in the UPnP standard. This point, which can be expressed as a time, is written to the ScheduleStartTime field in the UPnP content descriptor, making it possible to give the content from thelocal server 13 to a device that is compatible with the Control Point, as defined in UPnP. In other words, the content playback start time is "forced" by writing the ScheduleStartTime field. The ScheduleStartTime field is initially set to indicate the start time for a program according to the UPnP standard. - A UPnP Control Point can read temporal playback information, in the form of a playback start time, in the ScheduleStartTime field for the CDS on the media server. If the Control Point sends a synchronized playback command to a UPnP Media Renderer playback device, the method includes a synchronized playback step whose start time depends on the temporal playback information.
- In another embodiment, the ToDRC value is calculated based on other temporal and non-temporal information. Calculating the ToDRC from the ToD (the wall clock) and the TOc (the first timestamp sent in the stream, indicating the time elapsed since the start of the content), along with the TBR processing time, determines the first moment when playback is possible.
- It is possible to delay the start of playback by defining a later playback start time using criteria such as the subscriber's rights to services or viewing based on the content to be played back, for example. By forcing the playback time, it is possible to make audiovisual content unavailable for playback before a given time.
- In another embodiment, the playback start time is calculated based on the available space on the saving unit for saving the content, considering that the data must be able to be stored without overflow and without disrupting the content playback. When using a file with the "circular buffer" method with consideration for the defined buffer file size, the playback start time happens before the circular buffer has filled with data from the multicast stream, for example.
- The method allows audiovisual content, for example, sent by a broadcast channel, to be made accessible from a media server that is connected to a local area network, like other content available on the media server and coming from another source. The media server that is connected to the local area network is, for example, a media server as defined in the UPnP AV standard. Content that has been streamed and saved to a file on a media server can then be played back on a playback device that is connected to the network at any moment (or time), as defined according to various criteria. This can be from the time when the first content data was received, to which the necessary processing time is added between receipt and playback and referenced according to the clock for the local area network. It can also be a specific time.
- A tip for the method is using an information field that tells a control device or playback device when the content can be played back. It is therefore possible to force this time for one or more playback devices.
- This moment (this time) can be delayed based on other criteria to conditionalize or to synchronize the playback of content.
- The method can force a time before which playback is not possible.
- Of course, the invention is not limited to the examples described above. It also relates to variants.
- According to variants of the embodiment, content is synchronously played back from temporal playback information (such as the information field ScheduleStartTime defined in the UPnP standard) by multiple playback devices.
- In other variants, playback is initiated by separate control requests transmitted to each of the playback devices or by one control request transmitted to all or some of the playback devices connected to the home local area network (such as devices belonging to the same group (based on their type, for example)).
- In some variants, the requests are unicast, multicast, or broadcast.
- In variants of the embodiment, saving happens at the end of the transmitted audiovisual content or applies sequentially as content is streamed.
- In variants of the embodiment, the media server that contains the save file and the playback device for playing back the content are included in a single piece of equipment or shared among multiple separate pieces of equipment.
- In some variants, the file is fully or partially transmitted from the media server (Media Server) to one or more playback devices (Media Renderer). In one variant, the playback device is included on the same equipment as the media server.
- Specifically, in one variant, the broadcast network is suitable for transmitting streamed contend by a general or targeted broadcast mode to one or more destinations.
- In various embodiments, the content includes audio and/or video data and/or interactive data.
- In various embodiments, the media server is a dedicated piece of equipment, or it is included in devices such as a network gateway, a television receiver/decoder, a computer, a television, a personal assistant, or a mobile phone.
- In some variants, the media server is a dedicated device, or it is included in one or more devices.
- In some variants, the timestamps are transmitted as a content component or in packet headers received via the broadcast channel, or in content in a form using the audio or video watermarking technique that is already known as watermarking or fingerprinting.
- In various embodiments, the reference clock for the local area network is transmitted from a server on the wide area network to all of the equipment connected to the home local area network.
- In variants, the media server is a local or remote server.
- The invention also applies to any device that is connected to a network, contains a file server where the files contain one or more audio/video components, and whose component playback is synchronized relative to a network reference clock.
- More generally, the invention applies to any device or system implementing the method described above, according to its variants.
Claims (6)
- A method for saving at least a portion of a content to a network server (13) that is connected to a local area network (2), the network server being adapted to synchronize a playback of said at least a portion of the content, received from a broadcast channel, with another content previously downloaded and saved to a file on the network server on at least one playback device (4, 9) connected to said local area network by a use of the UPnP AV:4 protocol, said method including the steps of:- receiving (S2, S3), said at least a portion of the content from said broadcast channel, by a piece of equipment that is connected to said local area network, said at least a portion of the content including a timestamp (TOi) indicating a time offset of said at least a portion of the content relative to the start of the content,- saving (S5) said at least a portion of said content to said network server,- calculating a start time (ToD0) of said content relative to a reference clock of the local area network to position the received content in time, by subtracting the time offset of said at least a portion of the content to a time of reception of the at least a portion of the content, said time of reception being relative to the reference clock of said local area network;- calculating a possible playback start time (ToDRc) of said at least a portion of said content by adding an offset, corresponding at least to a processing time (TBR) to the time of reception of said at least a portion of said content,- saving (S6) said possible playback start time (ToDRc) of said at least a portion of said content in a ScheduledStartTime field of a UPnP Content Directory Service in said network server, said ScheduledStartTime field corresponding to the saved at least a portion of the content, and- synchronizing said playback of said at least a portion of said content with the other content on the at least one playback device by- sending a first SyncPlay command telling the at least one playback device (4, 9) to play back the at least portion of the content at the time of possible playback start time (ToDRc);- sending a second SyncPlay command telling the at least one playback device (4,9) to play back the other content at a time derived from the calculated start time (ToD0) and the processing time (TBR);where the temporal parameters of the SyncPlay commands refers to the reference clock of the local area network.
- The method according to claim 1, characterized in that said at least a portion of said content is transmitted to at least one destination at a time depending on said possible playback start time (ToDRc) of said content.
- The method according to any one of claims 1 to 2, characterized in that said at least a portion of said content is transmitted to at least one destination after receiving a request to transmit said at least a portion of said content.
- The method according to any one of claims 1 to 3 characterized in that the step for calculating said possible playback start time (ToDRc) of said content depends on an amount of available space for saving said content.
- The method according to any one of claims 1 to 4 characterized in that said broadcast channel is a bidirectional broadband channel.
- A device (1) for saving at least a portion of a content to a file, the device comprising:- a network interface (131) adapted to receive the at least a portion of the content from a broadcast channel, including a timestamp (TOi) indicating a time offset of said at least a portion of the content relative to the start of the content, the network interface (131) being further adapted to receive a reference clock of a local network;- a saving unit (133) being adapted to save said at least a portion of said content, and another content as a file; and- a processing module (134) configured to:where the temporal parameters of the SyncPlay commands refers to the reference clock of the local area network.- calculate a start time (ToD0) of said content relative to the reference clock of the local area network to position the received content in time, by subtracting the time offset of said at least a portion of the content to a time of reception of the at least a portion of the content, said time of reception being relative to the reference clock of said local area network;- calculate a possible playback start time (ToDRc) of said at least a portion of said content by adding an offset, corresponding at least to a processing time (TBR) to the time of reception of said at least a portion of said content,- save the possible playback start time (ToDRc) of said at least a portion of the content in a ScheduledStartTime field of a UPnP Content Directory Service of said device, said ScheduledStartTime field corresponding to the saved at least a portion of the content and- synchronize a playback of said at least a portion of said content with another content on at least one playback device (4,9) by a use of the UPnP AV:4 protocol comprising:∘ sending a first SyncPlay command telling the at least one playback device (4, 9) to play back the at least portion of the content at the time of possible playback start time (ToDRc);∘ sending a second SyncPlay command telling the at least one playback device (4,9) to play back the other content at a time derived from the calculated start time (ToD0) and the processing time (TBR);
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FR1158633A FR2980662A1 (en) | 2011-09-27 | 2011-09-27 | METHOD FOR RECORDING CONTENT IN A FILE ON A SERVER AND CORRESPONDING DEVICE |
PCT/EP2012/068890 WO2013045456A1 (en) | 2011-09-27 | 2012-09-25 | Method of saving content to a file on a server and corresponding device |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2761884A1 EP2761884A1 (en) | 2014-08-06 |
EP2761884B1 true EP2761884B1 (en) | 2018-03-21 |
Family
ID=46889062
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP12762294.2A Active EP2761884B1 (en) | 2011-09-27 | 2012-09-25 | Method of saving content to a file on a server and corresponding device |
Country Status (10)
Country | Link |
---|---|
US (1) | US9635082B2 (en) |
EP (1) | EP2761884B1 (en) |
JP (1) | JP2015501018A (en) |
KR (1) | KR20140066208A (en) |
CN (1) | CN103828383A (en) |
AU (1) | AU2012314513A1 (en) |
BR (1) | BR112014006695A2 (en) |
FR (1) | FR2980662A1 (en) |
HK (1) | HK1200998A1 (en) |
WO (1) | WO2013045456A1 (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9100687B2 (en) | 2013-05-31 | 2015-08-04 | Sonic Ip, Inc. | Playback synchronization across playback devices |
US9380099B2 (en) * | 2013-05-31 | 2016-06-28 | Sonic Ip, Inc. | Synchronizing multiple over the top streaming clients |
FR3021150B1 (en) * | 2014-05-19 | 2017-10-20 | Esii | METHOD AND SYSTEM FOR RESTITUTION OF A SOUND CONTENT ASSOCIATED WITH VISUAL CONTENT |
US10616085B2 (en) * | 2017-08-31 | 2020-04-07 | Zte Corporation | Residence time measurement for optimizing network services |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120219019A1 (en) * | 2009-08-19 | 2012-08-30 | Raumfeld Gmbh | Method and arrangement for synchronising data streams in networks and a corresponding computer program and corresponding computer-readable storage medium |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20030095897A (en) | 2002-06-15 | 2003-12-24 | 엘지전자 주식회사 | Synchronizing method for universal plug and play network |
US7685428B2 (en) * | 2003-08-14 | 2010-03-23 | Ricoh Company, Ltd. | Transmission of event markers to data stream recorder |
US20070110074A1 (en) | 2004-06-04 | 2007-05-17 | Bob Bradley | System and Method for Synchronizing Media Presentation at Multiple Recipients |
US20070260634A1 (en) * | 2006-05-04 | 2007-11-08 | Nokia Corporation | Apparatus, system, method, and computer program product for synchronizing the presentation of media content |
US20080250101A1 (en) * | 2007-04-05 | 2008-10-09 | Matsushita Electric Industrial Co., Ltd. | Multimedia data transmitting apparatus and multimedia data receiving apparatus |
FR2925814A1 (en) * | 2007-12-19 | 2009-06-26 | Thomson Licensing Sas | DEVICE AND METHOD FOR RECEIVING VIDEO DATA PACKETS. |
WO2009117645A2 (en) * | 2008-03-20 | 2009-09-24 | Verna Anthony F | Talking replay |
US20100208082A1 (en) * | 2008-12-18 | 2010-08-19 | Band Crashers, Llc | Media systems and methods for providing synchronized multiple streaming camera signals of an event |
US8705933B2 (en) * | 2009-09-25 | 2014-04-22 | Sony Corporation | Video bookmarking |
KR101046182B1 (en) | 2010-07-26 | 2011-07-04 | 주식회사 인스프리트 | Dlna media control apparatus and method for generating synchronized media in dlna network |
-
2011
- 2011-09-27 FR FR1158633A patent/FR2980662A1/en active Pending
-
2012
- 2012-09-25 US US14/346,809 patent/US9635082B2/en active Active
- 2012-09-25 KR KR1020147008216A patent/KR20140066208A/en not_active Application Discontinuation
- 2012-09-25 BR BR112014006695A patent/BR112014006695A2/en not_active IP Right Cessation
- 2012-09-25 CN CN201280047157.9A patent/CN103828383A/en active Pending
- 2012-09-25 WO PCT/EP2012/068890 patent/WO2013045456A1/en active Application Filing
- 2012-09-25 JP JP2014532355A patent/JP2015501018A/en not_active Withdrawn
- 2012-09-25 AU AU2012314513A patent/AU2012314513A1/en not_active Abandoned
- 2012-09-25 EP EP12762294.2A patent/EP2761884B1/en active Active
-
2015
- 2015-02-04 HK HK15101223.0A patent/HK1200998A1/en unknown
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120219019A1 (en) * | 2009-08-19 | 2012-08-30 | Raumfeld Gmbh | Method and arrangement for synchronising data streams in networks and a corresponding computer program and corresponding computer-readable storage medium |
Non-Patent Citations (1)
Title |
---|
"Digital Video Broadcasting (DVB); Technical Specification for DVB Services in the Home Network Phase 1", TECHNICAL SPECIFICATION, EUROPEAN TELECOMMUNICATIONS STANDARDS INSTITUTE (ETSI), 650, ROUTE DES LUCIOLES ; F-06921 SOPHIA-ANTIPOLIS ; FRANCE, vol. BROADCAS, no. V1.1.1, 1 May 2010 (2010-05-01), XP014047156 * |
Also Published As
Publication number | Publication date |
---|---|
JP2015501018A (en) | 2015-01-08 |
AU2012314513A1 (en) | 2014-03-13 |
BR112014006695A2 (en) | 2017-03-28 |
US9635082B2 (en) | 2017-04-25 |
US20140237086A1 (en) | 2014-08-21 |
KR20140066208A (en) | 2014-05-30 |
WO2013045456A1 (en) | 2013-04-04 |
HK1200998A1 (en) | 2015-08-14 |
FR2980662A1 (en) | 2013-03-29 |
EP2761884A1 (en) | 2014-08-06 |
CN103828383A (en) | 2014-05-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5624468B2 (en) | Synchronization of related data streams in interconnected networks | |
EP2690876A2 (en) | Heterogeneous network-based linked broadcast content transmitting/receiving device and method | |
US9621682B2 (en) | Reduced latency media distribution system | |
WO2014035645A1 (en) | Broadcast content to http client conversion | |
CN107819809B (en) | Method and device for synchronizing content | |
KR20120114016A (en) | Method and apparatus for network adaptive streaming user data in a outer terminal | |
KR20130005873A (en) | Method and apparatus for receiving contents in broadcast system | |
KR20110063654A (en) | Network device and method for setting up an iptv session | |
US20200021867A1 (en) | Broadcast signal transmitting and receiving method and device | |
WO2015035742A1 (en) | Method, terminal and system for audio and video sharing of digital television | |
EP2891323B1 (en) | Rendering time control | |
EP2761884B1 (en) | Method of saving content to a file on a server and corresponding device | |
EP2479984A1 (en) | Device and method for synchronizing content received from different sources | |
KR102166780B1 (en) | Method of synchronisation during the processing, by a multimedia player, of an item of multimedia content transmitted by an mbms service | |
EP2892225B1 (en) | Recording method, device and system | |
US20190191195A1 (en) | A method for transmitting real time based digital video signals in networks | |
KR101920051B1 (en) | Methods of Providing Timing Information for Synchronizing MMT Packet Streams in MMT Hybrid Delivery Service and Methods of Synchronizing MMT Packet Streams in MMT Hybrid Delivery Service | |
US20100031302A1 (en) | Stream distribution system, stream receiving device, and stream reproduction method | |
JP2008016905A (en) | Content transmission apparatus, content receiving apparatus, and content distribution method | |
Köbel et al. | Project Acronym HBB-NEXT |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20140113 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
DAX | Request for extension of the european patent (deleted) | ||
17Q | First examination report despatched |
Effective date: 20150409 |
|
REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1200998 Country of ref document: HK |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R079 Ref document number: 602012044225 Country of ref document: DE Free format text: PREVIOUS MAIN CLASS: H04N0021430000 Ipc: H04N0021262000 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: H04L 29/06 20060101ALI20170922BHEP Ipc: H04N 21/433 20110101ALI20170922BHEP Ipc: H04N 21/43 20110101ALI20170922BHEP Ipc: H04N 21/262 20110101AFI20170922BHEP Ipc: H04L 12/28 20060101ALI20170922BHEP Ipc: H04N 21/436 20110101ALI20170922BHEP Ipc: H04N 21/218 20110101ALI20170922BHEP |
|
INTG | Intention to grant announced |
Effective date: 20171020 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 982355 Country of ref document: AT Kind code of ref document: T Effective date: 20180415 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602012044225 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20180321 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180621 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 982355 Country of ref document: AT Kind code of ref document: T Effective date: 20180321 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180622 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180621 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 7 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180723 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602012044225 Country of ref document: DE |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 |
|
26N | No opposition filed |
Effective date: 20190102 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R081 Ref document number: 602012044225 Country of ref document: DE Owner name: INTERDIGITAL CE PATENT HOLDINGS SAS, FR Free format text: FORMER OWNER: THOMSON LICENSING, ISSY-LES-MOULINEAUX, FR |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20180925 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20180930 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180925 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180925 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180930 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180930 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180930 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180925 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180925 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180321 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20120925 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180321 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180721 |
|
REG | Reference to a national code |
Ref country code: HK Ref legal event code: WD Ref document number: 1200998 Country of ref document: HK |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20230926 Year of fee payment: 12 Ref country code: DE Payment date: 20230928 Year of fee payment: 12 |