WO2012018300A2 - Synchronized playback of media files - Google Patents

Synchronized playback of media files Download PDF

Info

Publication number
WO2012018300A2
WO2012018300A2 PCT/SE2011/050976 SE2011050976W WO2012018300A2 WO 2012018300 A2 WO2012018300 A2 WO 2012018300A2 SE 2011050976 W SE2011050976 W SE 2011050976W WO 2012018300 A2 WO2012018300 A2 WO 2012018300A2
Authority
WO
WIPO (PCT)
Prior art keywords
server
media stream
playback
media
stream
Prior art date
Application number
PCT/SE2011/050976
Other languages
French (fr)
Other versions
WO2012018300A3 (en
Inventor
Stefan Ytterborn
Original Assignee
Poc Sweden Ab
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Poc Sweden Ab filed Critical Poc Sweden Ab
Publication of WO2012018300A2 publication Critical patent/WO2012018300A2/en
Publication of WO2012018300A3 publication Critical patent/WO2012018300A3/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60Network streaming of media packets
    • H04L65/61Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
    • H04L65/612Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for unicast

Definitions

  • a method performed by a first User Terminal, UT, for synchronizing playback of media streams is provided.
  • the UT selects a media stream to be played synchronously by the first UT and at least one second UT.
  • the first UT provides an indicator associated with the selected media stream and an indicator of the at least one second UT to a coordination server.
  • the first UT requests the selected media stream from streaming storage and buffers the received data being associated with the selected media stream.
  • the first UT then receives synchronization instructions from the coordination server and thereafter initiates playback of the media stream based on the received synchronization instructions, and thereby enabling the first UTs to, with respect to the at least one second UT, synchronously playback the media stream.
  • a first User Terminal UT, adapted to perform synchronized playback.
  • the first UT comprises a communication interface which is adapted to be in communication with a communication server and at least one second UT.
  • the first UT further comprises a processing module, a memory module and a synchronization module.
  • the processing module is adapted to select a media stream to be played synchronously by the first UT and at least one second UT.
  • the processing unit is further adapted to provide, using the
  • the communication interface to a coordination server, an indicator associated with the selected media stream.
  • the communication interface is adapted to request the selected media stream from a streaming server.
  • the memory module is adapted to buffer received data being associated with the selected media stream.
  • the synchronization module is adapted to receive synchronization instructions from the coordination server and synchronization module is further adapted to instruct the processing unit to initiate playback of the media stream based on the received synchronization instructions, and thereby enabling the first UT to, with respect to the at least one second UT, synchronously playback the media stream.
  • the method and arrangement according to the aspects described above may have several advantages. For example, by using the coordination server, to which the UTs may be connected for other reasons than synchronization, new integrated media experiences may be provided. Further, the synchronization between UTs may be done in a faster and more robust manner. The method and arrangement also enables the first UT to command and determine which media streams that should be played back synchronously at the at least one second UT. [0008] The above method and arrangement may be configured and
  • the coordination server may be a web server, and the first UT and the at least one second UT may be individually connected to the web server.
  • the web server may be a social media service server and the first UT may be associated with a first user profile in the social media server and wherein the second UT may be associated with a second user profile in the social media server.
  • the media stream may be a stream of audio data.
  • the streaming storage may be an audio streaming server being separate from the coordination server.
  • the indicator may be an identifier of an audio file to be streamed to the first UT and the second UTs.
  • Fig. 1 is a block chart illustrating a plurality of UTs being in connection with a coordination server, according to one example embodiment.
  • Fig. 2 is a signaling scenario illustrating the synchronization
  • FIG. 3 is a signaling scenario illustrating the synchronization
  • Fig. 4 is a block chart illustrating a user terminal, according to one example embodiment.
  • Fig. 5 is a block chart illustrating a coordination server, according to one example embodiment.
  • Fig. 6 is a block diagram illustrating a UT streaming a media stream from two or more other UTs, according to one example embodiment.
  • Fig. 7 is a block diagram illustrating multiple users being in connection, according to one example embodiment.
  • Fig. 8 is a block diagram illustrating a graphical user interface, according to one example embodiment.
  • Fig. 9 is a flow chart illustrating a procedure for synchronously playback a media stream, according to one example embodiment.
  • Fig. 10 is a flow chart illustrating a procedure for synchronous playback of a media stream, according to one example embodiment.
  • Fig. 1 1 is a block diagram illustrating computer program modules in an arrangement, according to one example embodiment.
  • a method and arrangement for enabling synchronized playback of media files to a multiple of users is provided.
  • the users may connect, synchronize and stream the media files via a web server. Since a user nowadays may enjoy media files from the Internet, many of the previous constraints are removed. Hence, this new freedom to access media ad hoc may imply a change in user behavior towards adopting new social patterns for enjoying and consuming media.
  • the inventor has analyzed the current situation as well as the emerging social patterns of media consumption. The inventor has also studied the problems and drawbacks of the prior art in order to enable users to engage in shared media experiences.
  • Internet is generally used to mean the Internet as it is almost universally known, and will hereinafter be referred to as internet. Although, the term may additionally or alternatively refer to other types of public or private network which various embodiments of the present invention may operation, such as , for example Local Area Network ,LAN, metropolitan area networks, wide area networks, cellular networks etc.
  • stateless protocol as used herein is a protocol for
  • a server communicating over the internet between a client and a server/client, without storing session information on the server.
  • the action a server takes in response to a message from a client is based on the content of the current message, and not on any previous message. Examples of such protocols are Hypertext Transfer Protocol, HTTP, Real Time Messaging Protocol, RTMP, Real Time Streaming
  • RTSP Real-Time Transport Protocol
  • MMS Microsoft Media Server
  • file transfer protocol any appropriate protocol to transfer files, which exists in profusion in the art.
  • file transfer protocol is not limited to any particular layer according to the OSI-protocol stack.
  • file transfer protocol may include both stateful and stateless protocols.
  • media stream shall mean any type of media file being streamed from a server or node to a receiving UT.
  • media stream may, according to certain embodiments be exchanged with an audio stream.
  • audio stream as used hereinafter may mean any type of digital representation of audio.
  • an audio stream may comprise a file according to a known file format.
  • an audio stream may also be a stream over a packet switched network where the packets comprise a piece of the actual file or stream. Also known as streaming audio over internet.
  • FIG. 1 an example signaling scenario illustrating a procedure for synchronous playback of media is described.
  • This overview scenario comprise a master UT 200 associated with a user and one or more slave UTs 100a...100n which is normally also associated with a respective user.
  • the master UT 200 selects a media file subject to synchronous playback, i.e. the master UT 200, for example, selects an audio stream to be played by the UTs
  • the master UT 200 may select the audio stream by user input or by using playlists or by applying random or pseudorandom selection schemes. The user interaction will be further disclosed below in this description.
  • the signaling scenario of fig. 1 aims to provide the audio stream to one or more of the slave UTs 100a...100n and optionally also to the master UT 200. This can be done in many different ways which is known to the person skilled in the art. However, some examples will be incorporated in this section for reference.
  • the audio stream can be provided by various sources.
  • the audio stream is provided to each UT by a file transfer protocol, where the source of the music song normally is a memory module 310 arranged in a coordination server 300.
  • the music song is in a first action provided to the coordination server 300. In a subsequent action the content server provides the music song to the respective UT.
  • the master UT 200 determines if at least one of the slave UTs 100a...n are ready to begin synchronized playback. If the slave UTs 100a...n, or at least a sufficient number of slave UTs according to a threshold value, are ready to initiate synchronized playback the master UT 200 may start playback.
  • the master UT 200 provides the slave UTs 100a...n with a time reference to synchronize to.
  • a time reference may for instance be an internal system clock and/or local clock synchronization and a time offset.
  • the initiation is performed by broadcast messaging. Various examples of such methods will be incorporated below in this specification for reference.
  • the coordination server 300 is coordinating synchronization and/or providing the audio stream.
  • the master UT 200 only selects the audio stream subject to synchronous playback.
  • all signals between the UTs may be administered by the coordination according to the known client-server
  • the audio stream may also be provided by a streaming storage (not shown), such as a dedicated server for streaming.
  • a master UT 200 initiates the synchronous playback in a first action 1 :1 by sending a message to a slave UT 100.
  • the message can for example comprise information about the audio stream which is subject to playback.
  • the message of action 1 :1 also can be relayed by for instance the coordination server 300.
  • the slave UT 100 requests the audio stream which is selected by the master UT 200.
  • the coordination server 300 provides the audio stream to the slave UT and also alternatively to the master UT in action 1 :3.
  • the action 1 :3 may comprise to actually provide a stream of audio data from a memory module comprised in the coordination server 300. However, the action may also comprise providing a reference to another location where the audio data can be acquired. Another alternative is that the coordination server provides the UTs 100, 200 with references for means for so-called Peer-2-Peer, P2P, file transfer. Both centralized P2P, e.g. Napster style, and decentralized P2P, e.g. Gnutella style, may be used for enabling data transmission in action 1 :3.
  • centralized P2P e.g. Napster style
  • decentralized P2P e.g. Gnutella style
  • the action 1 :4 comprises buffering of the audio stream.
  • the first parts of the audio stream is assigned higher priority and consequently transmitted and buffered prior to the end part of the audio stream.
  • Such buffer is also known as a jitter-buffer to enable continuous playback at the UT even though network packets arrive non-sequential and with various latency.
  • an action 1 :5 of providing a "ready to play"-message is sent to the coordination server 300 from the slave UTs 100.
  • the coordination server 300 will then control whether or not the UTs 100, 200 which are subject to playback are ready to initiate playback.
  • a threshold value of ready UTs 100 is satisfied, e.g. a certain amount of UTs 100 has sent their "ready-to-play"-message, or a certain time has elapsed since action 1 :3, the coordination server 300 sends a message to the master UT 200 in action 1 :7.
  • the master UT 200 determines means for synchronization. This action will be further described below in this description and means in this signaling scheme of fig. 2 will be apparent from synchronization examples disclosed in this description.
  • the UTs 100 buffers the audio stream subject to synchronous playback. Once the UTs 100 have buffered a sufficiently large amount of data in their jitter-buffers, depending on transmission properties, the slave UT 100 issues a "start play"-message. Such "start play"- message may define an offset based on a common time reference, e.g. a timestamp provided by an appropriate header in a protocol layer of the OSI-stack.
  • coordination server 300 may issue a message defining an audio stream offset in combination with a time reference based on an offset as described above. Hence, the UTs may verify that they still are in synch and if necessary correct the playback to get back into synchronized playback.
  • the master UT 200 may commission a "start playback"-message to the slave UT 100 in action 1 :9 where the message may comprise further synchronization instructions.
  • the slave UT 100 and the master UT 200 may start playback of the audio stream synchronously, and normally after a predefined time after action 1 :9 has elapsed.
  • Fig. 3 a signaling scheme illustrating a scenario in which no coordination server is involved. Such interaction may be used when no internet connectivity is possible but only UT-to-UT connectivity.
  • Fig. 3 essentially comprises the same set of actions as the signaling scheme shown in fig. 2.
  • a master UT or a slave UT is not a static state. For instance, after a signaling scheme in accordance with fig. 2 or fig. 3 a slave UT may become a master UT and vice versa. Moreover, in certain embodiments a plurality of master UTs is in connectivity with a one or more slave UTs. In such a scenario the audio stream precedence will be determined according to selection algorithms and/or queue algorithms that exist in profusion in the art.
  • the example signal schemes of fig's 1 -3 may bring many benefits.
  • One effect is that a master UT 200 decides and orchestrates the media experience of one or more slave UTs 100a..n.
  • Another effect is that the slave UTs 100a..n, as well as the master UT 200 may enjoy synchronized playback.
  • a first user of a slave UT which is capable of presenting stereo playback and a second user of a slave UT which is capable of presenting stereo playback could exchange one each other's headphones and still enjoy a synchronized media experience.
  • Digital media of today comprises audio and video of different forms. In order to transfer this type of files in a user satisfying manner various codecs may be used.
  • the data representing the respective media file is compressed according to lossy data compression in order to satisfy limited bandwidth users. However, in other embodiments also lossless data compressions can be applied.
  • FIG. 4 illustrates another aspect of a UT 400 adapted to implement the signals and methods described in this specification as well as according to the accompanying drawings. It may also further be adapted to run software and provide graphical interfaces according the example interface illustrated in fig. 8.
  • the UT 400 comprises a communication unit 410 adapted to communicate over the internet.
  • the communication unit 410 is further adapted to decode received signals according to the protocols which exist in the art.
  • the UT 400 further comprises a processing module capable of storing and acquiring data from the memory module 450.
  • the processing unit is further adapted to prepare data received from the memory module 450 or from the communication module 410 in order to enable the media I/O module 440 to perform playback.
  • the processing module 420 is adapted to select a media stream to be played synchronously by the UT 400 and at least one second UT (not shown).
  • the processing unit 420 is further adapted to provide, using the communication interface an indicator associated with the selected media stream to a coordination server 480.
  • the communication interface is further adapted to request the selected media stream from a streaming storage 490 and the memory module 450 is adapted to buffer, using a buffer 460, the received data being associated with the selected media stream.
  • the synchronization module 430 in Fig. 4 is adapted to receive synchronization instructions from the coordination server via the communication interface 410.
  • the synchronization module 430 is hence adapted to instruct the processing unit 420 to initiate playback of the media stream based on the received synchronization instructions, and thereby enabling the UT 400 to, with respect to the at least one second UT (not shown), synchronously playback the media stream.
  • the UT 400 may further comprise a time module 470 adapted to provide time, such as time stamps, time operations or time synchronization operations to the synchronization module 430.
  • a time module 470 adapted to provide time, such as time stamps, time operations or time synchronization operations to the synchronization module 430.
  • the buffer 460 may according to one embodiment comprise a jitter buffer specially adapted to buffer audio streamed over Ethernet. Such jitter buffer may increase the chance for continuous playback of the audio stream when streaming the audio stream over the Internet and to prevent the effects of packet delivery variation.
  • fig. 4 merely illustrates various functional modules or units in the UT 400 in a logical sense, although the skilled person is free to implement these functions in practice using suitable software and hardware means.
  • the invention is generally not limited to the shown structures of the UT 400, while its functional modules may be configured to operate according to the features described for Fig's 1 -3 above and Fig's. 5-1 1 below, where
  • the coordination server 600 comprises a communication module 630 which is adapted to communicate according to internet or other data communication standards. As shown in fig. 5 the communication module receives and sends information to the UT 670.
  • the coordination server 600 further comprises a processing module 640 which is administering communications and normal calculations within the coordination server 600.
  • the processing module is further operatively connected to a synchronization module 620.
  • the synchronization module 620 is adapted to determine synchronization instructions to be provided to the UT(s) 670.
  • the synchronization module is further adapted to be in operatively connected to a time module 650. Said time module may for instance provide time operations.
  • the coordination server 600 further comprises a memory module 610 which also optionally comprises a dedicated streaming storage 660 wherein the audio streams are stored.
  • the streaming storage 660 is arranged separately from the coordination server 600.
  • the streaming storage 660 may in some examples comprise dedicated storage and streaming servers to enable large amounts of audio streams to be efficiently transmitted.
  • a block diagram illustrating the procedure of transmitting, or streaming, parts of a media stream from two or more other UTs is described.
  • the UTs may stream the media to subject to playback from a dedicated server as previous mentioned in this description.
  • a UT can stream audio from other UTs having said particular audio stream stored in a memory module.
  • a media file is divided into sections.
  • a downloading UT 103 is then streaming the sections of the media file from a plurality of UTs 104,105 wherein each of the plurality of UTs 104, 105 provides a unique section.
  • Such a streaming scenario is illustrated in fig 6.
  • the downloading UT 103 is provided segment A, C and E from a first UT 104, while the remaining segments of the file are provided by a second UT 105.
  • the first, second and downloading UT may initiate synchronized playback faster and without connection to internet or content servers.
  • FIG. 7 an example of an view comprising UTs engaging in synchronous playback in which the UTs are in connectivity according to a mesh- topology.
  • one or more slave UTs 100a, 100b, 100c, 100d are in connectivity to a master UT 200 and also a passive UT 101 which is not subject to synchronous playback.
  • Fig.7 aims to show an example where no coordination server or similar unit is present.
  • the audio stream may be streamed and/or be transmitted efficiently in a P2P manner.
  • the passive UT 200 contributes to transmit audio streams within the mesh-network of fig. 7.
  • a broadcast synchronization scheme may be applied in which the the synchronized start of audio playback may be initiated such that a master UT broadcasts a message to start audio playback.
  • this will not provide "true" synchronization but so-called plesiochronious playback.
  • the slave UTs has low latency connectivity to the master UT, the interrelated difference of the audio playback may be small, i.e. the playback will be perceived as synchronous to the user.
  • internal clock synchronization may be applied.
  • a master UT or alternatively a coordination server sets a reference time a corresponding time units.
  • the synchronized playback will then be orchestrated according to the time reference but according to an internal clock.
  • new reference times and/or reference audio stream positions are provided during the audio stream playback enabling the UTs to verify synchronization.
  • the precision time protocol may be used which is further specified in the standard IEEE 1588.
  • Accuracy in the sub-microsecond range may be achieved with low-cost implementations.
  • external clock synchronization may be possible.
  • the UT comprises connectivity to an external clock which is common to master UT as well as to slave UT.
  • an external clock is Global Positioning System, GPS which may provide a synchronized time to a plurality of UTs.
  • the master UT or alternatively the coordination may orchestrate the synchronous start according to the external clock. In some embodiments this is achieved by providing an offset to a given reference time.
  • NTP Network Time Protocol
  • UTC Coordinated Universal Time
  • the coordination server or the master UT may provide an absolute UTC reference or an offset to a given UTC to provide synchronous playback initiation.
  • a user interaction interface 500 displayed at a display at the UT is described.
  • Fig. 8 illustrates example elements in the user interface for enabling user-operability of a synchronized playback program.
  • the user interface may be displayed at a screen which is dependent of the nature of the UT. In fact, in one embodiment the screen is an external and provided by the UT though video I/O modules.
  • the user interface 500 comprises a menu item 51 1 which in turn may comprise several sub-menus for accessing various operations.
  • the menu item 51 1 is embedded in a top bar 510 further capable of displaying various information messages to the user.
  • the user interface 500 further comprises one or more playlists in a playlist bar 520.
  • the playlists 521 , 522, 523 may comprise sets of representations of audio streams to be subject to synchronous playback.
  • a selected playlist may occur to the track items element 570 showing one or more selectable audio stream IDs 571 , 572, 573.
  • the user interface further comprises a list 560 of user representation of UTs which are involved in the signaling scenarios according to fig. 1 to fig. 3.
  • the user representation 561 , 562, 563, 564 may comprise various information about the represented user. It is further, in some embodiments, possible to indicate the status of the user by displaying information in the user list 560. Moreover, in some embodiments a master UT may decide which of the listed users 560 which may be subject to synchronized playback.
  • the user interface 500 further comprises a set of control elements 530 to starting/stopping 532, rewinding 531 and forwarding 532 an audio stream.
  • the interface also comprises a volume control 540 to enable the user to adjust the volume of the synchronous playback.
  • the user interface 500 comprises a progress bar 550.
  • the progress bar enables the user to select a position within the audio stream in a continuous manner, while the control elements in 530 enable discrete forwarding/rewinding.
  • the user interface may be displayed on touch screen whereas the user interacts with the user interface through screen commandos.
  • the graphical user interface as described above may be integrated with other web services such as a social network service. However, it may also be an stand-alone graphical user interface
  • a flow chart illustrating a procedure for synchronous playback is described.
  • the procedure is performed by a first UT.
  • the first UT selects a media stream to be played synchronously by the first UT and at least one second UT.
  • the first UT provides, to a
  • the coordination server an indicator associated with the selected media stream and an indicator of the at least one second UT.
  • the selected media stream is requested from streaming storage and thereafter, in action 704,
  • the UT receives synchronization instructions from the coordination server in action 705.
  • the UT initiates, in action 706, playback of the media stream based on the received synchronization instructions, and thereby enabling the first UTs to, with respect to the at least one second UT, synchronously playback the media stream.
  • fig. 10 the method of fig. 9 is disclosed in a more detailed manner according to one example embodiment.
  • fig. 10 also comprises an action 713 wherein the UT is requesting the audio stream if the audio stream is not already stored in the UT's memory module.
  • the UT is buffering the audio stream such that the buffer comprises enough data to initiate playback without experiencing stuttering.
  • a conditional action 715 the UT will continue to add data into a buffer, e.g. a jitter buffer, until a threshold value is satisfied.
  • a next action 716 the UT is sending a so-called "ready to play"-message to the master UT and/or the coordination server indicating that enough data is added to the buffer.
  • the UT receives synchronization parameters, such as for example a reference timestamp, a start play offset.
  • the UT determines, based on the synchronization parameters, time of initiation of playback in order to achieve synchronicity, according to action 719.
  • the method may restart by letting a UT select a new audio stream for playback.
  • FIG. 1 1 schematically shows an embodiment of an arrangement 800 in a UT, as illustrated in Fig. 4 and described above.
  • a processing unit 806 e.g. with a DSP (Digital Signal Processor), a receiving module, a requesting module, a selecting module, a synchronizing module, an initiating module and a providing module.
  • DSP Digital Signal Processor
  • the processing unit 806 can be a single unit or a plurality of units to perform different actions of procedures described herein.
  • the arrangement 800 may also comprise an input unit 802 for receiving signals and information from other entities such as coordination server, streaming servers or other UTs, and an output unit 804 for providing signals, requests and responses or other type of information to entities such as
  • the input unit 802 and the output unit 804 may be arranged as an integrated entity.
  • the arrangement 800 comprises at least one computer program product 808 in the form of a non-volatile memory, e.g. an EEPROM (Electrically Erasable Programmable Read-Only Memory), a flash memory and a disk drive.
  • the computer program product 808 comprises a computer program 810, which comprises code means, which when run in the processing unit 806 in the arrangement 800 causes the arrangement to perform the actions of the procedures described earlier in conjunction with Fig. 9 and 10.
  • the computer program 810 may be configured as a computer program code structured in computer program modules.
  • the code means in the computer program 810 of the arrangement 800 comprises a receiving module 810a for selecting a media stream to be played back synchronously.
  • the computer program further comprises a providing module 810b for providing the necessary indicators to the coordination server.
  • the computer program 810 further comprises a requesting module 810c for requesting the selected media stream to be streamed.
  • the computer program 810 also comprises a receiving module 81 Od which receives synchronization instructions for enable synchronous playback which is performed by the initiating module 81 Oe.
  • the requests and indicators may be provided to the coordination server using the output unit 804. Likewise, the received signals may be received using the input unit 802.
  • the modules 810a-e could essentially perform the actions of the flow illustrated in Fig. 9, to emulate the arrangement in a UT illustrated in Fig 4.
  • the different modules 610a-d are run on the processing unit 806, they correspond to the modules and its functions 610-640 of Fig. 4.
  • Fig. 1 1 are implemented as computer program modules which when run on the processing unit causes the arrangement and/or the UT to perform the actions described above in the conjunction with Figs mentioned above, at least one of the code means may in alternative embodiments be implemented at least partly as hardware circuits.
  • the processor may be a single CPU (Central processing unit), but could also comprise two or more processing units.
  • the processor may include general purpose microprocessors; instruction set processors and/or related chips sets and/or special purpose microprocessors such as ASICs
  • the processor may also comprise board memory for caching purposes.
  • the computer program may be carried by a computer program product connected to the processor.
  • the computer program product comprises a computer readable medium on which the computer program is stored.
  • the computer program product may be a flash memory, a RAM (Random-access memory) ROM (Read-Only Memory) or an EEPROM, and the computer program modules described above could in alternative embodiments be distributed on different computer program products in the form of memories within the data receiving unit.
  • the coordination server may be integrated or a part of a social media platform or a social network service, such as Facebook.
  • Social media generally refers to web-based and mobile technologies to turn communication between users into an interactive dialogue.
  • a social network is generally a social structure comprising nodes representing an individual or a persona ficta. The nodes are connected by one or more interdependencies.
  • Using social media platforms as a coordination server may bring several advantages due to that an automatic social context may be added to the synchronous playback.
  • the predefined user profiles may be used to represent the first and the second UTs in the synchronization and playback of the media files.
  • the procedures and arrangements described above may also enable one or more users to synchronously listen to the same media file, e.g. a music song. This can be done by enabling one of the users associated with a UT to select which song to be subject for synchronous playback. This example enables a plurality of users to collectively enjoy a defined music song without the requirement of loudspeakers. Moreover, the song may be streamed over the internet but still be presented synchronously at the UTs. All users involved in synchronous playback may hear the same music note at a given time. These advantages may contribute to enable a plurality of persons to listen to the same songs without disturbing the surroundings in for instance public environments.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

A first User Terminal, UT, and a method performed by a first User Terminal, UT, for synchronizing playback of media streams is provided. The UT selects a media stream to be played synchronously by the first UT and at least one second UT. The first UT provides an indicator associated with the selected media stream and an indicator of the at least one second UT to a coordination server. The first UT requests the selected media stream from streaming storage and buffers the received data being associated with the selected media stream. The first UT then receives synchronization instructions from the coordination server and thereafter initiates playback of the media stream based on the received synchronization instructions, and thereby enabling the first UTs to, with respect to the at least one second UT, synchronously playback the media stream.

Description

SYNCHRONIZED PLAYBACK OF MEDIA FILES
BACKGROUND
[0001 ] Decades ago, portable analogous terminals for media playback, e.g. freestyles, were introduced to the consumer market. This was, at that time, indeed a novel way of enjoying audio without the constraint of being stationary. Such media technology enabled new social patterns. The analogous portable tape player introduced the ability of sharing audio from the same source, still in a portable mode. This was achieved by providing an additional second audio output which was streaming the exact same audio source as in the first audio output, e.g. a freestyle player with two audio output slots. Thus, this enabled two users to share a portable audio source with headphones without disturbing the ambiance of other physically present users. However, they were constrained in many ways. One limitation was, for example, that the two users had to share the same physical playback device. [0002] Since the introduction of portable terminals for audio playback, numerous developments have taken place in order to enhance the user experience. For example, it is now possible to store and playback many songs in a device fitting inside normal clothing pocket. Also new types of media and media combinations can be enjoyed by users portably. The quality of the media has also increased due to the digitalization.
[0003] Decades ago, the audio streams were stored on tapes. According to the more recent prior art, the user-operated portable media playback devices, hereinafter referred to as User Terminals, UTs, overcome the problem of limited memory for storing digital media content by enabling the UT connectivity. In the prior art, the connectivity is typically achieved according to Ethernet, General
Packet Radio Service, GPRS, Universal Mobile Telecommunication Systems, UMTS, High-Speed Downlink Packet Access, HSDPA or Institute of Electrical and Electronics Engineers, IEEE, standard 802.1 1 or other connectivity standards that exists in the art. By enabling the UT to send and receive data, the local memory ceases to be the limiting factor of the amount of media to be subject to playback at the UT. Instead virtually all media can be streamed provided by the different packet-switched network standards. This enables the user of a UT to decide ad hoc which audio stream to play. Having access to large media libraries regardless of the location of the UT may enable new ways of consuming media experiences. However, most of the devices still only support a fraction of the possible ways to consume played back media. It has also been noticed that services for consuming media, such as for example, streaming music and video, have become of importance in order to enable telecommunication operators to increase the subscribers Average Revenue Per User, ARPU. SUMMARY
[0004] It is an object of the invention to address at least some of the limitations, problems and issues outlined above. It is also an object to improve the process of playback media files to a plurality of users in a synchronized manner. It is possible to achieve these objects and others by using a method and an arrangement as defined in the attached independent claims.
[0005] According to a first aspect, a method performed by a first User Terminal, UT, for synchronizing playback of media streams is provided. The UT selects a media stream to be played synchronously by the first UT and at least one second UT. The first UT provides an indicator associated with the selected media stream and an indicator of the at least one second UT to a coordination server. The first UT requests the selected media stream from streaming storage and buffers the received data being associated with the selected media stream. The first UT then receives synchronization instructions from the coordination server and thereafter initiates playback of the media stream based on the received synchronization instructions, and thereby enabling the first UTs to, with respect to the at least one second UT, synchronously playback the media stream.
[0006] According to a second aspect, a first User Terminal, UT, adapted to perform synchronized playback. The first UT comprises a communication interface which is adapted to be in communication with a communication server and at least one second UT. The first UT further comprises a processing module, a memory module and a synchronization module. The processing module is adapted to select a media stream to be played synchronously by the first UT and at least one second UT. The processing unit is further adapted to provide, using the
communication interface, to a coordination server, an indicator associated with the selected media stream. The communication interface is adapted to request the selected media stream from a streaming server. The memory module is adapted to buffer received data being associated with the selected media stream. The synchronization module is adapted to receive synchronization instructions from the coordination server and synchronization module is further adapted to instruct the processing unit to initiate playback of the media stream based on the received synchronization instructions, and thereby enabling the first UT to, with respect to the at least one second UT, synchronously playback the media stream.
[0007] The method and arrangement according to the aspects described above may have several advantages. For example, by using the coordination server, to which the UTs may be connected for other reasons than synchronization, new integrated media experiences may be provided. Further, the synchronization between UTs may be done in a faster and more robust manner. The method and arrangement also enables the first UT to command and determine which media streams that should be played back synchronously at the at least one second UT. [0008] The above method and arrangement may be configured and
implemented according to different embodiments. In one example embodiment, the coordination server may be a web server, and the first UT and the at least one second UT may be individually connected to the web server.
[0009] According to another example embodiment, the web server may be a social media service server and the first UT may be associated with a first user profile in the social media server and wherein the second UT may be associated with a second user profile in the social media server.
[0010] According to another example embodiment, the media stream may be a stream of audio data. [001 1 ] According to another possible embodiment, the streaming storage may be an audio streaming server being separate from the coordination server.
[0012] According to another possible embodiment, the indicator may be an identifier of an audio file to be streamed to the first UT and the second UTs. BRIEF DESCRIPTION OF DRAWINGS
[0013] The invention is now described, by way of example, with reference to the accompanying drawings, in which:
[0014] Fig. 1 is a block chart illustrating a plurality of UTs being in connection with a coordination server, according to one example embodiment. [0015] Fig. 2 is a signaling scenario illustrating the synchronization
communication between a first UT, a second UT and a coordination server, according to one example embodiment.
[0016] Fig. 3 is a signaling scenario illustrating the synchronization
communication between a first and a second UT, according to one example embodiment.
[0017] Fig. 4 is a block chart illustrating a user terminal, according to one example embodiment.
[0018] Fig. 5 is a block chart illustrating a coordination server, according to one example embodiment. [0019] Fig. 6 is a block diagram illustrating a UT streaming a media stream from two or more other UTs, according to one example embodiment.
[0020] Fig. 7 is a block diagram illustrating multiple users being in connection, according to one example embodiment.
[0021 ] Fig. 8 is a block diagram illustrating a graphical user interface, according to one example embodiment. [0022] Fig. 9 is a flow chart illustrating a procedure for synchronously playback a media stream, according to one example embodiment.
[0023] Fig. 10 is a flow chart illustrating a procedure for synchronous playback of a media stream, according to one example embodiment. [0024] Fig. 1 1 is a block diagram illustrating computer program modules in an arrangement, according to one example embodiment.
DESCRIPTION
[0025] Briefly described, a method and arrangement for enabling synchronized playback of media files to a multiple of users is provided. The users may connect, synchronize and stream the media files via a web server. Since a user nowadays may enjoy media files from the Internet, many of the previous constraints are removed. Hence, this new freedom to access media ad hoc may imply a change in user behavior towards adopting new social patterns for enjoying and consuming media. The inventor has analyzed the current situation as well as the emerging social patterns of media consumption. The inventor has also studied the problems and drawbacks of the prior art in order to enable users to engage in shared media experiences.
[0026] In the following, a detailed description of a synchronized media playback by several UTs is disclosed. In particular, but not limited to, a detailed description of synchronized playback of an audio file or an audio stream which is selected by a master UT. The audio file may be synchronously played at the master UT and at a slave UT. Although the examples in this specification exemplify audio streams, the skilled person will appreciate that the arrangements and methods may apply to virtually any media type. For example video streams and/or combination of audio and video streams may also be subject to synchronous playback.
[0027] It must be noted that, as used in this specification and the appended claims, the singular forms "a", "an" and "the" include plural referents unless the context clearly dictates otherwise. [0028] This detailed description comprises arrangements, system, methods, clients, server, computer programs, etc. Even if each embodiment is not disclosed, it will become apparent to the person skilled in the art how to equivalently implement a method from an arrangement and so forth. [0029] As used in this description the following terms shall have the meanings indicated, unless the context otherwise requires. However, the terms are purely to be understood as descriptive and not in any way as to be understood as delimiting the scope of the description.
[0030] The term "internet" is generally used to mean the Internet as it is almost universally known, and will hereinafter be referred to as internet. Although, the term may additionally or alternatively refer to other types of public or private network which various embodiments of the present invention may operation, such as , for example Local Area Network ,LAN, metropolitan area networks, wide area networks, cellular networks etc. [0031 ] The term "stateless protocol" as used herein is a protocol for
communicating over the internet between a client and a server/client, without storing session information on the server. The action a server takes in response to a message from a client is based on the content of the current message, and not on any previous message. Examples of such protocols are Hypertext Transfer Protocol, HTTP, Real Time Messaging Protocol, RTMP, Real Time Streaming
Protocol, RTSP. Also other protocols are possible, such as for example, Microsoft Media Server, MMS. Analogously, the term "stateful protocol" shall mean that the server keeps a session with the client. Thus, the server's response is dependent on earlier responses or initiated by handshakes. [0032] The term "file transfer protocol" as used hereinafter shall mean any appropriate protocol to transfer files, which exists in profusion in the art. The "file transfer protocol" is not limited to any particular layer according to the OSI-protocol stack. Moreover, the term "file transfer protocol" may include both stateful and stateless protocols. [0033] The term "media stream" shall mean any type of media file being streamed from a server or node to a receiving UT. The term media stream may, according to certain embodiments be exchanged with an audio stream. The term "audio stream" as used hereinafter may mean any type of digital representation of audio. Thus, an audio stream may comprise a file according to a known file format. However, an audio stream may also be a stream over a packet switched network where the packets comprise a piece of the actual file or stream. Also known as streaming audio over internet.
[0034] With reference to Fig. 1 , an example signaling scenario illustrating a procedure for synchronous playback of media is described. This overview scenario comprise a master UT 200 associated with a user and one or more slave UTs 100a...100n which is normally also associated with a respective user. The master UT 200 selects a media file subject to synchronous playback, i.e. the master UT 200, for example, selects an audio stream to be played by the UTs
100a...100n;200. The master UT 200 may select the audio stream by user input or by using playlists or by applying random or pseudorandom selection schemes. The user interaction will be further disclosed below in this description.
[0035] In a second set of actions, the signaling scenario of fig. 1 aims to provide the audio stream to one or more of the slave UTs 100a...100n and optionally also to the master UT 200. This can be done in many different ways which is known to the person skilled in the art. However, some examples will be incorporated in this section for reference. The audio stream can be provided by various sources.
According to one example signaling scenario, the audio stream is provided to each UT by a file transfer protocol, where the source of the music song normally is a memory module 310 arranged in a coordination server 300. According to another alternative, the music song is in a first action provided to the coordination server 300. In a subsequent action the content server provides the music song to the respective UT.
[0036] In a subsequent action, the master UT 200 determines if at least one of the slave UTs 100a...n are ready to begin synchronized playback. If the slave UTs 100a...n, or at least a sufficient number of slave UTs according to a threshold value, are ready to initiate synchronized playback the master UT 200 may start playback. In one example the master UT 200 provides the slave UTs 100a...n with a time reference to synchronize to. An example of such time reference may for instance be an internal system clock and/or local clock synchronization and a time offset. In another example, the initiation is performed by broadcast messaging. Various examples of such methods will be incorporated below in this specification for reference.
[0037] According to another alternative embodiment, the coordination server 300 is coordinating synchronization and/or providing the audio stream. In such an example, the master UT 200 only selects the audio stream subject to synchronous playback. Moreover, it should be noted that all signals between the UTs may be administered by the coordination according to the known client-server
architectures. The audio stream may also be provided by a streaming storage (not shown), such as a dedicated server for streaming.
[0038] With reference to Fig. 2, a signaling scenario illustrating of the signal interaction in fig. 1 is now described. In Fig. 3, a master UT 200 initiates the synchronous playback in a first action 1 :1 by sending a message to a slave UT 100. The message can for example comprise information about the audio stream which is subject to playback. It should be noted that the message of action 1 :1 also can be relayed by for instance the coordination server 300. Regardless, in a second action 1 :2 the slave UT 100 requests the audio stream which is selected by the master UT 200. The coordination server 300 provides the audio stream to the slave UT and also alternatively to the master UT in action 1 :3. [0039] The action 1 :3 may comprise to actually provide a stream of audio data from a memory module comprised in the coordination server 300. However, the action may also comprise providing a reference to another location where the audio data can be acquired. Another alternative is that the coordination server provides the UTs 100, 200 with references for means for so-called Peer-2-Peer, P2P, file transfer. Both centralized P2P, e.g. Napster style, and decentralized P2P, e.g. Gnutella style, may be used for enabling data transmission in action 1 :3.
[0040] The action 1 :4 comprises buffering of the audio stream. According to one alternative, the first parts of the audio stream is assigned higher priority and consequently transmitted and buffered prior to the end part of the audio stream. Such buffer is also known as a jitter-buffer to enable continuous playback at the UT even though network packets arrive non-sequential and with various latency. When the buffer comprises enough audio stream data, according to a threshold, an action 1 :5 of providing a "ready to play"-message is sent to the coordination server 300 from the slave UTs 100. The coordination server 300 will then control whether or not the UTs 100, 200 which are subject to playback are ready to initiate playback. When a threshold value of ready UTs 100 is satisfied, e.g. a certain amount of UTs 100 has sent their "ready-to-play"-message, or a certain time has elapsed since action 1 :3, the coordination server 300 sends a message to the master UT 200 in action 1 :7.
[0041 ] According to action 1 :8 the master UT 200 determines means for synchronization. This action will be further described below in this description and means in this signaling scheme of fig. 2 will be apparent from synchronization examples disclosed in this description. [0042] According to one alternative of action 1 :8 the UTs 100 buffers the audio stream subject to synchronous playback. Once the UTs 100 have buffered a sufficiently large amount of data in their jitter-buffers, depending on transmission properties, the slave UT 100 issues a "start play"-message. Such "start play"- message may define an offset based on a common time reference, e.g. a timestamp provided by an appropriate header in a protocol layer of the OSI-stack.
[0043] The above mentioned procedure can further be applied in order to verify whether the UTs are in sync or not. Then, the master UT 200 and or the
coordination server 300 may issue a message defining an audio stream offset in combination with a time reference based on an offset as described above. Hence, the UTs may verify that they still are in synch and if necessary correct the playback to get back into synchronized playback.
[0044] When means for synchronous playback is established, the master UT 200 may commission a "start playback"-message to the slave UT 100 in action 1 :9 where the message may comprise further synchronization instructions. Thus, the slave UT 100 and the master UT 200 may start playback of the audio stream synchronously, and normally after a predefined time after action 1 :9 has elapsed.
[0045] With reference to Fig. 3, a signaling scheme illustrating a scenario in which no coordination server is involved. Such interaction may be used when no internet connectivity is possible but only UT-to-UT connectivity. Fig. 3 essentially comprises the same set of actions as the signaling scheme shown in fig. 2.
However, the actions previously performed by the coordination server are now instead performed by the Master UT 200.
[0046] It should be noted that being a master UT or a slave UT is not a static state. For instance, after a signaling scheme in accordance with fig. 2 or fig. 3 a slave UT may become a master UT and vice versa. Moreover, in certain embodiments a plurality of master UTs is in connectivity with a one or more slave UTs. In such a scenario the audio stream precedence will be determined according to selection algorithms and/or queue algorithms that exist in profusion in the art.
[0047] The example signal schemes of fig's 1 -3 may bring many benefits. One effect is that a master UT 200 decides and orchestrates the media experience of one or more slave UTs 100a..n. Another effect is that the slave UTs 100a..n, as well as the master UT 200 may enjoy synchronized playback. In the example of the music song, a first user of a slave UT which is capable of presenting stereo playback and a second user of a slave UT which is capable of presenting stereo playback could exchange one each other's headphones and still enjoy a synchronized media experience. [0048] Digital media of today comprises audio and video of different forms. In order to transfer this type of files in a user satisfying manner various codecs may be used. In one example, according to any of the above-described embodiments the data representing the respective media file is compressed according to lossy data compression in order to satisfy limited bandwidth users. However, in other embodiments also lossless data compressions can be applied.
[0049] In the event that the capacity of the data transmission capability of a slave UT is hampering the UT to buffer media data in order to stay in synchronized playback, down sampling of the media is possible. For example, in the case of a UT is about to fall out of synch due low data transmission capability the audio stream has represented by lesser bits. One possible solution is to switch to a lossier codec, another solution is simply to reduce the fidelity regarding bit representation. Methods and arrangements for performing such bandwidth adjustment exits in profusion in the art and will be apparent to the skilled person. [0050] Prior to action 1 :1 of figure 2, the user associated to the master UT may choose which media stream to be subject to synchronized playback according to the present invention. The selection of the audio stream may be performed according to a playlist which is based on availability, preferences or other selection criterions. [0051 ] Figure 4 illustrates another aspect of a UT 400 adapted to implement the signals and methods described in this specification as well as according to the accompanying drawings. It may also further be adapted to run software and provide graphical interfaces according the example interface illustrated in fig. 8. The UT 400 comprises a communication unit 410 adapted to communicate over the internet. The communication unit 410 is further adapted to decode received signals according to the protocols which exist in the art. The UT 400 further comprises a processing module capable of storing and acquiring data from the memory module 450. The processing unit is further adapted to prepare data received from the memory module 450 or from the communication module 410 in order to enable the media I/O module 440 to perform playback. [0052] The processing module 420 is adapted to select a media stream to be played synchronously by the UT 400 and at least one second UT (not shown). The processing unit 420 is further adapted to provide, using the communication interface an indicator associated with the selected media stream to a coordination server 480. The communication interface is further adapted to request the selected media stream from a streaming storage 490 and the memory module 450 is adapted to buffer, using a buffer 460, the received data being associated with the selected media stream.
[0053] The synchronization module 430 in Fig. 4, is adapted to receive synchronization instructions from the coordination server via the communication interface 410.The synchronization module 430 is hence adapted to instruct the processing unit 420 to initiate playback of the media stream based on the received synchronization instructions, and thereby enabling the UT 400 to, with respect to the at least one second UT (not shown), synchronously playback the media stream.
[0054] The UT 400 may further comprise a time module 470 adapted to provide time, such as time stamps, time operations or time synchronization operations to the synchronization module 430.
[0055] The buffer 460 may according to one embodiment comprise a jitter buffer specially adapted to buffer audio streamed over Ethernet. Such jitter buffer may increase the chance for continuous playback of the audio stream when streaming the audio stream over the Internet and to prevent the effects of packet delivery variation.
[0056] It should be noted that fig. 4 merely illustrates various functional modules or units in the UT 400 in a logical sense, although the skilled person is free to implement these functions in practice using suitable software and hardware means. Thus, the invention is generally not limited to the shown structures of the UT 400, while its functional modules may be configured to operate according to the features described for Fig's 1 -3 above and Fig's. 5-1 1 below, where
appropriate. [0057] With reference to fig. 5, a block diagram illustrating arrangements comprised in a coordination server 300 is described. The coordination server 600 comprises a communication module 630 which is adapted to communicate according to internet or other data communication standards. As shown in fig. 5 the communication module receives and sends information to the UT 670. The coordination server 600 further comprises a processing module 640 which is administering communications and normal calculations within the coordination server 600. The processing module is further operatively connected to a synchronization module 620. The synchronization module 620 is adapted to determine synchronization instructions to be provided to the UT(s) 670. The synchronization module is further adapted to be in operatively connected to a time module 650. Said time module may for instance provide time operations.
[0058] The coordination server 600 further comprises a memory module 610 which also optionally comprises a dedicated streaming storage 660 wherein the audio streams are stored. However, in Fig. 5, the streaming storage 660 is arranged separately from the coordination server 600. In fact, the streaming storage 660 may in some examples comprise dedicated storage and streaming servers to enable large amounts of audio streams to be efficiently transmitted.
[0059] With reference to Fig 6, a block diagram illustrating the procedure of transmitting, or streaming, parts of a media stream from two or more other UTs is described. The UTs may stream the media to subject to playback from a dedicated server as previous mentioned in this description. However, in the embodiment shown in Fig. 6, a UT can stream audio from other UTs having said particular audio stream stored in a memory module. In another scenario which is illustrated in fig.6, a media file is divided into sections. A downloading UT 103 is then streaming the sections of the media file from a plurality of UTs 104,105 wherein each of the plurality of UTs 104, 105 provides a unique section. Such a streaming scenario is illustrated in fig 6. The downloading UT 103 is provided segment A, C and E from a first UT 104, while the remaining segments of the file are provided by a second UT 105. By applying streaming protocols according to the example illustration of fig. 6 the first, second and downloading UT may initiate synchronized playback faster and without connection to internet or content servers.
[0060] With reference to fig. 7, an example of an view comprising UTs engaging in synchronous playback in which the UTs are in connectivity according to a mesh- topology. As an illustrative example one or more slave UTs 100a, 100b, 100c, 100d are in connectivity to a master UT 200 and also a passive UT 101 which is not subject to synchronous playback. Fig.7 aims to show an example where no coordination server or similar unit is present. According to the transmission procedure of fig. 6 in conjunction with the scenario of fig. 7 the audio stream may be streamed and/or be transmitted efficiently in a P2P manner. Moreover, according to some embodiments also the passive UT 200 contributes to transmit audio streams within the mesh-network of fig. 7.
[0061 ] Synchronization is mentioned herein above in several instances and embodiments. Therefore, a description of different synchronization embodiments will follow. It should be noted that these embodiments are only for the cause of being illustrative examples.
[0062] According to one example, a broadcast synchronization scheme may be applied in which the the synchronized start of audio playback may be initiated such that a master UT broadcasts a message to start audio playback. However, this will not provide "true" synchronization but so-called plesiochronious playback. If the slave UTs has low latency connectivity to the master UT, the interrelated difference of the audio playback may be small, i.e. the playback will be perceived as synchronous to the user.
[0063] According to another example, internal clock synchronization may be applied. In this example, a master UT or alternatively a coordination server sets a reference time a corresponding time units. The synchronized playback will then be orchestrated according to the time reference but according to an internal clock. In some embodiments, new reference times and/or reference audio stream positions are provided during the audio stream playback enabling the UTs to verify synchronization. [0064] According to another synchronization scheme for clock synchronization, the precision time protocol may be used which is further specified in the standard IEEE 1588. According to some examples, Accuracy in the sub-microsecond range may be achieved with low-cost implementations. [0065] In other embodiments, external clock synchronization may be possible. In such embodiment, the UT comprises connectivity to an external clock which is common to master UT as well as to slave UT. One such example of such external clock is Global Positioning System, GPS which may provide a synchronized time to a plurality of UTs. The master UT or alternatively the coordination may orchestrate the synchronous start according to the external clock. In some embodiments this is achieved by providing an offset to a given reference time.
[0066] Yet another example of synchronization according to an external clock is when the time reference is provided through the internet. This can also be implemented with timestamp and offset synchronization. According to several networks, distributed timestamps, i.e. a time indicator, can be used in order to enable synchronous playback. According to one example the Network Time Protocol, NTP, may be used as such time indicator. NTP may maintain time indication to within ±10 milliseconds (1/100 s) to Coordinated Universal Time, UTC, over the Internet. NTP may even achieve accuracies of ±200 microseconds (1/5000 s) to UTC or even better in local area networks under some condition. Accordingly, the coordination server or the master UT may provide an absolute UTC reference or an offset to a given UTC to provide synchronous playback initiation.
[0067] With reference to Fig. 8, a user interaction interface 500 displayed at a display at the UT is described. Fig. 8 illustrates example elements in the user interface for enabling user-operability of a synchronized playback program. The user interface may be displayed at a screen which is dependent of the nature of the UT. In fact, in one embodiment the screen is an external and provided by the UT though video I/O modules. [0068] The user interface 500 comprises a menu item 51 1 which in turn may comprise several sub-menus for accessing various operations. The menu item 51 1 is embedded in a top bar 510 further capable of displaying various information messages to the user. The user interface 500 further comprises one or more playlists in a playlist bar 520. The playlists 521 , 522, 523 may comprise sets of representations of audio streams to be subject to synchronous playback.
[0069] A selected playlist may occur to the track items element 570 showing one or more selectable audio stream IDs 571 , 572, 573. The user interface further comprises a list 560 of user representation of UTs which are involved in the signaling scenarios according to fig. 1 to fig. 3. The user representation 561 , 562, 563, 564 may comprise various information about the represented user. It is further, in some embodiments, possible to indicate the status of the user by displaying information in the user list 560. Moreover, in some embodiments a master UT may decide which of the listed users 560 which may be subject to synchronized playback.
[0070] The user interface 500 further comprises a set of control elements 530 to starting/stopping 532, rewinding 531 and forwarding 532 an audio stream. The interface also comprises a volume control 540 to enable the user to adjust the volume of the synchronous playback. To enable the user to select a position within the playback progress of the audio stream, the user interface 500 comprises a progress bar 550. The progress bar enables the user to select a position within the audio stream in a continuous manner, while the control elements in 530 enable discrete forwarding/rewinding.
[0071 ] In one embodiment, the user interface may be displayed on touch screen whereas the user interacts with the user interface through screen commandos.
According to another embodiment, the graphical user interface as described above, may be integrated with other web services such as a social network service. However, it may also be an stand-alone graphical user interface
With reference to Fig. 9, a flow chart illustrating a procedure for synchronous playback is described. The procedure is performed by a first UT. In action 701 , the first UT selects a media stream to be played synchronously by the first UT and at least one second UT. Then, in action 702, the first UT provides, to a
coordination server, an indicator associated with the selected media stream and an indicator of the at least one second UT. In action 703, the selected media stream is requested from streaming storage and thereafter, in action 704,
buffering the received data being associated with the selected media stream. The UT receives synchronization instructions from the coordination server in action 705. The UT initiates, in action 706, playback of the media stream based on the received synchronization instructions, and thereby enabling the first UTs to, with respect to the at least one second UT, synchronously playback the media stream.
[0072] In fig. 10 the method of fig. 9 is disclosed in a more detailed manner according to one example embodiment. In addition to the first and second action of fig. 9, fig. 10 also comprises an action 713 wherein the UT is requesting the audio stream if the audio stream is not already stored in the UT's memory module. In a subsequent action 714, the UT is buffering the audio stream such that the buffer comprises enough data to initiate playback without experiencing stuttering. In a conditional action 715 the UT will continue to add data into a buffer, e.g. a jitter buffer, until a threshold value is satisfied. In a next action 716 the UT is sending a so-called "ready to play"-message to the master UT and/or the coordination server indicating that enough data is added to the buffer. In action 717 the UT receives synchronization parameters, such as for example a reference timestamp, a start play offset. In action 718 the UT determines, based on the synchronization parameters, time of initiation of playback in order to achieve synchronicity, according to action 719. In a last and optional action 720 the method may restart by letting a UT select a new audio stream for playback.
[0073] It is to be understood that even though the actions are presented in a certain sequential order in this description, they may be performed in any mutual order. Moreover, one or several steps may be omitted without limiting the scope of the possibilities of the solution. It is to be understood as indicating the concept of the solution. [0074] Fig. 1 1 schematically shows an embodiment of an arrangement 800 in a UT, as illustrated in Fig. 4 and described above. Comprised in the arrangement 800 are here a processing unit 806, e.g. with a DSP (Digital Signal Processor), a receiving module, a requesting module, a selecting module, a synchronizing module, an initiating module and a providing module. The processing unit 806 can be a single unit or a plurality of units to perform different actions of procedures described herein. The arrangement 800 may also comprise an input unit 802 for receiving signals and information from other entities such as coordination server, streaming servers or other UTs, and an output unit 804 for providing signals, requests and responses or other type of information to entities such as
coordination servers, streaming servers or other UTs. The input unit 802 and the output unit 804 may be arranged as an integrated entity.
[0075] Furthermore, the arrangement 800 comprises at least one computer program product 808 in the form of a non-volatile memory, e.g. an EEPROM (Electrically Erasable Programmable Read-Only Memory), a flash memory and a disk drive. The computer program product 808 comprises a computer program 810, which comprises code means, which when run in the processing unit 806 in the arrangement 800 causes the arrangement to perform the actions of the procedures described earlier in conjunction with Fig. 9 and 10. [0076]The computer program 810 may be configured as a computer program code structured in computer program modules. Hence in the example embodiments described, the code means in the computer program 810 of the arrangement 800 comprises a receiving module 810a for selecting a media stream to be played back synchronously. The computer program further comprises a providing module 810b for providing the necessary indicators to the coordination server. The computer program 810 further comprises a requesting module 810c for requesting the selected media stream to be streamed. The computer program 810 also comprises a receiving module 81 Od which receives synchronization instructions for enable synchronous playback which is performed by the initiating module 81 Oe. The requests and indicators may be provided to the coordination server using the output unit 804. Likewise, the received signals may be received using the input unit 802.
[0077] The modules 810a-e could essentially perform the actions of the flow illustrated in Fig. 9, to emulate the arrangement in a UT illustrated in Fig 4. In other words, when the different modules 610a-d are run on the processing unit 806, they correspond to the modules and its functions 610-640 of Fig. 4.
[0078] Although the code means in the embodiment disclosed above in
conjunction with Fig. 1 1 are implemented as computer program modules which when run on the processing unit causes the arrangement and/or the UT to perform the actions described above in the conjunction with Figs mentioned above, at least one of the code means may in alternative embodiments be implemented at least partly as hardware circuits.
[0079]The processor may be a single CPU (Central processing unit), but could also comprise two or more processing units. For example, the processor may include general purpose microprocessors; instruction set processors and/or related chips sets and/or special purpose microprocessors such as ASICs
(Application Specific Integrated Circuit). The processor may also comprise board memory for caching purposes. The computer program may be carried by a computer program product connected to the processor. The computer program product comprises a computer readable medium on which the computer program is stored. For example, the computer program product may be a flash memory, a RAM (Random-access memory) ROM (Read-Only Memory) or an EEPROM, and the computer program modules described above could in alternative embodiments be distributed on different computer program products in the form of memories within the data receiving unit.
[0080] According to one embodiment, the coordination server may be integrated or a part of a social media platform or a social network service, such as Facebook. Social media generally refers to web-based and mobile technologies to turn communication between users into an interactive dialogue. A social network is generally a social structure comprising nodes representing an individual or a persona ficta. The nodes are connected by one or more interdependencies. Using social media platforms as a coordination server may bring several advantages due to that an automatic social context may be added to the synchronous playback. Moreover, in an embodiment using a social media platform or a social network, the predefined user profiles may be used to represent the first and the second UTs in the synchronization and playback of the media files.
[0081 ]The procedures and arrangements described above may also enable one or more users to synchronously listen to the same media file, e.g. a music song. This can be done by enabling one of the users associated with a UT to select which song to be subject for synchronous playback. This example enables a plurality of users to collectively enjoy a defined music song without the requirement of loudspeakers. Moreover, the song may be streamed over the internet but still be presented synchronously at the UTs. All users involved in synchronous playback may hear the same music note at a given time. These advantages may contribute to enable a plurality of persons to listen to the same songs without disturbing the surroundings in for instance public environments.
[0082] While the invention has been described with reference to specific exemplary embodiments, the description is generally only intended to illustrate the inventive concept and should not be taken as limiting the scope of the invention. For example, the terms "synchronization parameter", "master UT", "slave UT", "audio stream", "synchronization module", "jitter buffer", and "coordinator server", have been used throughout this description, although any other corresponding functions, parameters, nodes and/or units could also be used having the
functionalities and characteristics described here. Abbreviations
• ARPU - Average Revenue Per User
• GPRS - General Packet Radio Service
• GPS - Global Positioning System • HSDPA - High-Speed Downlink Packet Access
• HTTP - Hypertext Transfer Protocol
• IEEE - Institute of Electrical and Electronics Engineers
• NTP - Network Time Protocol
• P2P- Peer To Peer
• UMTS - Universal Mobile Telecommunication Systems
• UT - User Terminal
• UTC - Coordinated Universal Time

Claims

1 . A method performed by a first User Terminal, UT, for synchronizing playback of media streams, said method comprising: - selecting (701 ) a media stream to be played synchronously by the first UT and at least one second UT;
- providing (702), to a coordination server, an indicator associated with the selected media stream and an indicator of the at least one second UT; - requesting (703) the selected media stream from streaming storage and buffering (704) received data being associated with the selected media stream;
- receiving (705) synchronization instructions from the coordination server; and
- initiating (706) playback of the media stream based on the received
synchronization instructions, and thereby enabling the first UTs to, with respect to the at least one second UT, synchronously playback the media stream.
2. A method according to claim 1 , wherein said coordination server is a web server, wherein the first UT and the at least one second UT is individually connected to the web server.
3. A method according to claim 2, wherein said web server is a social media service server and wherein the first UT associated with a first user profile in the social media server and wherein the second UT is associated with a second user profile in the social media server.
4. A method according to any one of the previous claims, wherein said media stream is a stream of audio data.
5. A method according to claim 4, wherein the streaming storage is an audio streaming server being separate from the coordination server.
6. A method according to any one of the previous claims, wherein the indicator is an identifier of an audio file to be streamed to the first UT and the second UTs.
7. A first User Terminal ,UT, (400) adapted to perform synchronized playback, the first UT comprising:
- a communication interface (410) adapted to be in communication with a communication server and at least one second UT;
- a processing module (420);
- a memory module (450);
- a synchronization module (430), wherein the processing module is adapted to select a media stream to be played synchronously by the first UT and at least one second UT, wherein said processing unit is further adapted to provide, using the communication interface, to a coordination server, an indicator associated with the selected media stream, and wherein the communication interface is adapted to request the selected media stream and wherein the memory module is adapted to buffer received data being associated with the selected media stream, and wherein the synchronization module is adapted to receive synchronization instructions from the coordination server, and wherein the synchronization module is adapted to instruct the processing unit to initiate playback of the media stream based on the received synchronization instructions, and thereby enabling the first UTs to, with respect to the at least one second UT, synchronously playback the media stream.
8. A first UT according to claim 7, wherein said communication interface is adapted to be in communication with a web server acting as a coordination server, and wherein the first UT and the at least one second UT is individually connected to the web server.
9. A first UT according to claim 8, wherein said communication interface is adapted to be in communication web server being a social media service server and wherein the first UT associated with a first user profile in the social media server and wherein the second UT is associated with a second user profile in the social media server.
10. A first UT according to any one of the claims 7-9, wherein said media stream is a stream of audio data.
1 1 . A first UT according to any one of the claims 7-10, wherein the communication interface is adapted to be in connection with a streaming storage being an audio streaming server being separate from the coordination server.
12. A first UT according to any one of the claims 7-1 1 , wherein the processing unit is further adapted to provide an indicator being an identifier of an audio file to be streamed to the first UT and the second UT(s).
PCT/SE2011/050976 2010-08-03 2011-08-03 Synchronized playback of media files WO2012018300A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US37024710P 2010-08-03 2010-08-03
US61/370,247 2010-08-03

Publications (2)

Publication Number Publication Date
WO2012018300A2 true WO2012018300A2 (en) 2012-02-09
WO2012018300A3 WO2012018300A3 (en) 2012-06-07

Family

ID=45559943

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/SE2011/050976 WO2012018300A2 (en) 2010-08-03 2011-08-03 Synchronized playback of media files

Country Status (1)

Country Link
WO (1) WO2012018300A2 (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130013740A1 (en) * 2011-07-05 2013-01-10 Te-Chuan Liu Media Sharing Method and Non-transitory Machine Readable Media thereof
EP2680524A1 (en) * 2012-06-27 2014-01-01 Watchever Group System for displaying content
EP3041244A1 (en) * 2015-01-01 2016-07-06 Muvix Media networks Ltd. Methods and devices for distributed audio/video synchronization and playback using an a/v orchestrator
WO2016119792A1 (en) * 2015-01-28 2016-08-04 Dynastrom Aps Self-organizing audio synchronization
EP3509312A1 (en) * 2018-01-08 2019-07-10 MySyncster Holding OÜ System for real-time synchronization
US10614856B2 (en) 2015-01-28 2020-04-07 Roku, Inc. Audio time synchronization using prioritized schedule
EP3515083A4 (en) * 2016-09-14 2020-04-15 Boe Technology Group Co. Ltd. Method and apparatus for performing synchronization operation on contents
US10679673B2 (en) 2015-01-28 2020-06-09 Roku, Inc. Synchronization in audio playback network independent of system clock
EP3544308A4 (en) * 2016-11-11 2020-11-18 Alibaba Group Holding Limited Play control method and device
WO2022049210A1 (en) * 2020-09-02 2022-03-10 Effing Simon Method for synchronizing the playback of an audio file with an associated video file

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030061371A1 (en) * 2001-08-31 2003-03-27 Deshpande Sachin G. System and method for simultaneous media playout
WO2005101411A2 (en) * 2004-04-12 2005-10-27 Tivo Inc. Multicasting multimedia content distribution system
US20060002681A1 (en) * 2004-07-01 2006-01-05 Skipjam Corp. Method and system for synchronization of digital media playback
US20060149850A1 (en) * 2005-01-05 2006-07-06 Control4 Corporation Method and apparatus for synchronizing playback of streaming media in multiple output devices
EP1940111A2 (en) * 2006-12-25 2008-07-02 Sony Corporation Content playback system, playback device, playback control method and program
US20090106357A1 (en) * 2007-10-17 2009-04-23 Marvin Igelman Synchronized Media Playback Using Autonomous Clients Over Standard Internet Protocols

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030061371A1 (en) * 2001-08-31 2003-03-27 Deshpande Sachin G. System and method for simultaneous media playout
WO2005101411A2 (en) * 2004-04-12 2005-10-27 Tivo Inc. Multicasting multimedia content distribution system
US20060002681A1 (en) * 2004-07-01 2006-01-05 Skipjam Corp. Method and system for synchronization of digital media playback
US20060149850A1 (en) * 2005-01-05 2006-07-06 Control4 Corporation Method and apparatus for synchronizing playback of streaming media in multiple output devices
EP1940111A2 (en) * 2006-12-25 2008-07-02 Sony Corporation Content playback system, playback device, playback control method and program
US20090106357A1 (en) * 2007-10-17 2009-04-23 Marvin Igelman Synchronized Media Playback Using Autonomous Clients Over Standard Internet Protocols

Non-Patent Citations (7)

* Cited by examiner, † Cited by third party
Title
ESPINOZA, J.M.: 'Histone H2B ubiquitination: the cancer connection' GENES & DEVELOPMENT vol. 22, 2008, pages 2743 - 2749 *
HAHN, M.A. ET AL.: 'The tumor suppressor CDC73 interacts with the ring finger proteins RNF20 and RNF40 and is required for the maintenance of histone 2B monoubiquitination' HUMAN MOLECULAR GENETICS 2011, *
KIM, J. ET AL.: 'RAD6-Mediated Transcription-Coupled H2B Ubiquitylation Directly Stimulates H3K4 Methylation in Human Cells' CELL vol. 137, 2009, pages 459 - 471 *
MINSKY, N. ET AL.: 'Monoubiquitinated H2B is associated with the transcribed region of highly expressed genes in human cells' NATURE CELL BIOLOGY vol. 10, 2008, pages 483 - 491 *
SHEMA, E. ET AL.: 'The histone H2B-specific ubiquitin ligase RNF20/hBRE 1 acts as a putative tumour suppressor through selective regulation of gene expression' GENES & DEVELOPMENT vol. 22, 2008, pages 2664 - 2676 *
WANG, P. ET AL.: 'Parafibromin, a Component of the Human PAF Complex, Regulates Growth Factors and Is Required for Embryonic Development and Survival in Adult Mice' MOLECULAR AND CELL BIOLOGY vol. 28, 2008, pages 2930 - 2940 *
ZHU, P. ET AL.: 'Monoubiquitination of Human Histone H2B: The Factors Involved and Their Roles in HOX Gene Regulation' MOLECULAR CELL vol. 20, 2005, pages 601 - 611 *

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130013740A1 (en) * 2011-07-05 2013-01-10 Te-Chuan Liu Media Sharing Method and Non-transitory Machine Readable Media thereof
US9848037B2 (en) 2012-06-27 2017-12-19 Watchever Group System for displaying content
EP2680524A1 (en) * 2012-06-27 2014-01-01 Watchever Group System for displaying content
WO2014001480A1 (en) * 2012-06-27 2014-01-03 Watchever Group System for displaying content
EP3041244A1 (en) * 2015-01-01 2016-07-06 Muvix Media networks Ltd. Methods and devices for distributed audio/video synchronization and playback using an a/v orchestrator
US9679610B2 (en) 2015-01-01 2017-06-13 Muvix Media Networks Ltd Methods and devices for distributed audio/video synchronization and playback using an A/V orchestrator
US10614856B2 (en) 2015-01-28 2020-04-07 Roku, Inc. Audio time synchronization using prioritized schedule
US10231008B2 (en) 2015-01-28 2019-03-12 Roku, Inc. Self-organizing audio synchronization
US11922976B2 (en) 2015-01-28 2024-03-05 Roku, Inc. Audio time synchronization using prioritized schedule
US11437075B2 (en) 2015-01-28 2022-09-06 Roku, Inc. Audio time synchronization using prioritized schedule
WO2016119792A1 (en) * 2015-01-28 2016-08-04 Dynastrom Aps Self-organizing audio synchronization
US10679673B2 (en) 2015-01-28 2020-06-09 Roku, Inc. Synchronization in audio playback network independent of system clock
EP3515083A4 (en) * 2016-09-14 2020-04-15 Boe Technology Group Co. Ltd. Method and apparatus for performing synchronization operation on contents
EP3544308A4 (en) * 2016-11-11 2020-11-18 Alibaba Group Holding Limited Play control method and device
US10999645B2 (en) 2016-11-11 2021-05-04 Alibaba Group Holding Limited Playing control method and apparatus
JP2021510491A (en) * 2018-01-08 2021-04-22 マイシンクスター・ホールディング・オサウヒングMySyncster Holding OU System for real-time synchronization
WO2019134859A1 (en) * 2018-01-08 2019-07-11 Mysyncster Holding Oü System for real-time synchronization
US11706477B2 (en) 2018-01-08 2023-07-18 Mysyncster Holding Oü System for real-time synchronization
JP7330208B2 (en) 2018-01-08 2023-08-21 マイシンクスター・ホールディング・オサウヒング System for real-time synchronization
EP3509312A1 (en) * 2018-01-08 2019-07-10 MySyncster Holding OÜ System for real-time synchronization
WO2022049210A1 (en) * 2020-09-02 2022-03-10 Effing Simon Method for synchronizing the playback of an audio file with an associated video file

Also Published As

Publication number Publication date
WO2012018300A3 (en) 2012-06-07

Similar Documents

Publication Publication Date Title
WO2012018300A2 (en) Synchronized playback of media files
US10334207B2 (en) Audio video streaming system and method
CA2976437C (en) Methods and apparatus for reducing latency shift in switching between distinct content streams
CN107431860B (en) Audio system based on location service
US8233648B2 (en) Ad-hoc adaptive wireless mobile sound system
US20150256587A1 (en) Network Connection Servers And Related Methods For Interactive Music Systems
US8943215B2 (en) Distributed smooth streaming utilizing dynamic manifests
US9787523B2 (en) Managing data in a data queue including synchronization of media on multiple devices
US9843825B1 (en) Distributed and synchronized media switching
US10856018B2 (en) Clock synchronization techniques including modification of sample rate conversion
GB2479272A (en) Setting playlist file and media file duration and timing
CN108293145A (en) Video distribution synchronizes
AU2022252735B2 (en) Method and apparatus for synchronizing applications' consumption of remote data
EP3868043B1 (en) Wireless audio synchronization
EP2695389B1 (en) Processing media streams for synchronised output at multiple end points
EP3107261B1 (en) System, method and devices for low latency transmission
CN108696762A (en) A kind of synchronous broadcast method, device and system
WO2014100384A1 (en) Audio video streaming system and method
US11259069B1 (en) Synchronized video player
JP2010183237A (en) System and method of content synchronous reproduction, reproduction terminal, method of controlling the same, and control program
KR101810883B1 (en) Live streaming system and streaming client thereof
JP6361012B2 (en) Playback system and playback method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11814872

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC OF 020513

122 Ep: pct application non-entry in european phase

Ref document number: 11814872

Country of ref document: EP

Kind code of ref document: A2