WO2011079477A1

WO2011079477A1 - Method and device for providing comments on multimedia contents

Info

Publication number: WO2011079477A1
Application number: PCT/CN2010/000002
Authority: WO
Inventors: Jun Li; Jun Xu; Xiaojun Ma
Original assignee: Thomson Licensing; Thomas Inc.
Priority date: 2010-01-04
Filing date: 2010-01-04
Publication date: 2011-07-07

Abstract

Provided is a method for providing comments on multimedia contents among a plurality of user devices. At the side of a first user device, it receives an input of a comment when a requested multimedia content is being displayed on the first user device; in response to the input of the comment, it determines an offset value relative to the requested multimedia content, wherein the offset value indicates a start position of a segment of the requested multimedia content; and it sends a message containing the comment and the offset value to a comment server for distribution to other user devices among the plurality of user devices, wherein the other user devices are watching the requested multimedia content. So it allows the building of a relationship between users watching the same multimedia content.

Description

METHOD AND DEVICE FOR PROVIDING COMMENTS ON MULTIMEDIA

CONTENTS

TECHNICAL FIELD

The present invention relates to data communication, and more particularly, relates to a method and a device for providing comments on multimedia contents among a plurality of users.

BACKGROUND

Generally, multimedia contents, such as video, audio, text and animation data, are provided to end users as synchronized media, which means multiple media objects, such as a video and its corresponding audio, share a common timeline, or each has a timeline and their timelines are correlated . Sometimes, for example, at the side of a multimedia server, a video and its corresponding audio are encoded as two separate data streams, each having its own data structure; and at the side of an end user, the two data streams are playbacked in synchronization with each other by, for example, using their time stamps in data streams. Herein, the term "streaming" means the data representing the various media types is provided over a network to a client computer on a real-time, as- needed basis, rather than being pre-delivered in its entirety before playback. Correspondingly, the end user device renders/presents/displays streaming data as it is received, rather than waiting for an entire file to be delivered.

Unicast, multicast and broadcast are possibly used to deliver the multimedia contents to end users. Among them, multicast is often used by media content providers to deliver their media contents, e.g. a TV program, a video of video on demand (VOD). The multicast takes use of multicast addressing for the delivery of data to a group of destinations simultaneously by using the most efficient strategy to deliver data over each link of the network only once, creating copies only when the links to the multiple destinations split. Generally, the implementation of the multicast concept occurs at the I P routing level, where routers create optimal distribution paths for datagrams sent to a multicast destination address spanning tree in real-time. At data link layer, multicast describes one-to-many distribution such as Ethernet multicast addressing, Asynchronous Transfer Mode (ATM) point-to-multipoint virtual circuits, etc.

The Internet applications for, for example, providing online VOD or video broadcast/multicast service, normally only allow users to watch multimedia contents. Among them, some allow users, after watching multimedia contents, to provide comments and/or rank scores to watched multimedia contents. However, there is no inner relationship between the comments and the multimedia contents. Therefore, a method for providing comments on multimedia contents among a plurality of users is desired.

SUMMARY

According to an aspect of present invention, it is provided a method for providing comments on multimedia contents among a plurality of user devices, wherein, the steps performed at the side of a first user device comprises: receiving an input of a comment when a requested multimedia content is being displayed on the first user device; in response to the input of the comment, determining an offset value relative to the requested multimedia content, wherein the offset value indicates a start position of a segment of the requested multimedia content; and sending a message containing the comment and the offset value to a comment server for distribution to other user devices among the plurality of user devices, wherein the other user devices are watching the requested multimedia content.

According to another aspect of present invention, it is provided a device for providing comments on multimedia contents among a plurality of devices of this kind, wherein the device is currently watching a requested multimedia content, the device comprises: a display interface configured to display multimedia contents; an input interface configured to receive user inputs; and a processing module configured to determine an offset value relative to the requested multimedia content when the input interface receives an input of a comment, wherein the offset value indicates a start position of a segment of the requested multimedia content; and to send a message containing the comment and the offset value to a comment server for distribution to other devices among the plurality of devices, wherein the other devices watching the requested multimedia content.

According to the aspects of present invention, it builds a relationship among users watching the same multimedia content. Further, it allows a user quickly browsing a segment of the multimedia content corresponding to a comment to which he is interested.

It is to be understood that more aspects and advantages of the invention will be found in the following detailed description of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a further understanding of embodiments of the invention together with the description which serves to explain the principle of the invention. Therefore, the invention is not limited to the embodiments. In the drawings:

Fig. 1 is a diagram schematically illustrating a system for providing comments on multimedia contents according to an embodiment of present invention;

Fig. 2 is a message sequence chart illustrating a method for providing comments to a requested multimedia content according to the present embodiment of the invention;

Fig. 3 is a flow chart illustrating actions performed by user end device upon a click on the comment according to the present embodiment of the invention.

DETAILED DESCRIPTION An embodiment of the present invention will now be described in detail in conjunction with the drawings. In the following description, some detailed descriptions of known functions and configurations may be omitted for clarity and conciseness.

Fig. 1 is a diagram schematically illustrating a system for providing comments on multimedia contents according to an embodiment of present invention. A multimedia content comprises at least one synchronized object selected from a group comprising video, audio, text, animation, photo etc. The system comprises one or more video server 101 , one or more comment server 103 and one or more user end device 102 which is comprised of a video client 104 and a comment client 105 in view of functions they (102, 104) perform. Their functions are as follows:

-The video server 101 is configured to provide data streams of multimedia contents to the video client 104 in the form of data packet. That is, the data packets are used to carry the multimedia streams. Each data packet contains a time code indicating the time offset/time location of the data carried in the data packet relative to a timeline of a multimedia content. The time code here is defined as a coded signal of the multimedia content giving information about time relative to a timeline of the multimedia content, the coded signal indicates when the data to which the coded signal is associated will be displayed. Normally the time code is used for synchronization, e.g. between a video, its corresponding audio and subtitle. Herein, the time code is the time stamp of the multimedia content.

-The video client 104 is configured to subscribe to video server for a multimedia content selected by a user, receive and display the data stream of the selected multimedia content. In addition, the video client is further configured to cache or buffer the data of the selected multimedia content from a time point when the video client begins to receive the data of the selected multimedia content. And when the user switches to other multimedia content, the video client stops caching/buffering the data of old multimedia content and starts to cache/buffer the data of the newly switched multimedia content. Moreover, the video client 104 is also configured to carry out the instructions received from the comment client 105, and the details will be described below.

-The comment client 105 is configured to receive comment(s) from the user, and at the same time, to determine a time stamp of the multimedia content corresponding to a time when the comment client 105 senses that the user inputs the comment and an identifier uniquely indentifying the multimedia content currently being displayed on the video client 104 (The determination of the time stamp can be realized by video client's periodically reporting its currently used time stamp to the comment client or comment client's query and video client's reply approach). It then sends a message containing the identifier, the time stamp and the comment to the comment server 103 for record and distribution, and displays the comment in a display region to the user. Herein, the "sense" of the comment client 105 can be implemented by user's clicking on a send button after inputting his comment, or trigging an event by the comment client 05 when the first letter is inputted into a comment input area. In addition, when the user clicks on a comment displayed in a comment display area of the comment client 105, the comment client 105 instructs the video client 104 to playback a segment of the multimedia content starting from the time stamp corresponding to the clicked comment and lasting for a predetermined length of period, such as 10 seconds. This can be implemented because the comment client 05 maintains a database storing comments and their corresponding time stamps. It should be noted that the comments and time stamps associated with the multimedia content currently being displayed on the video client 104 are maintained until, for example, the user switches to another multimedia content or the user turns off the user end device 102. In other words, once the user switches to another multimedia content, the comment client will delete the comments and time stamps associated with former multimedia content and start to store the comments and time stamps associated with current multimedia content. --The comment server 103 is configured to receive comments from and distribute comments to comment client(s). It should note that although the Fig. 1 shows only one user end device, two or more user end devices are possibly connected to the video server 101 and comment server 103, and consequently, these user end devices form a social network system among users watching a same multimedia content and each user among users watching the same multimedia content can watch other's comments and review segments of the multimedia content relative to the comments. Moreover, each time a user switches to a multimedia content, his comment client 105 determines the identifier of the switched multimedia content, and then registers (i.e. the corresponding relationship between the comment client and the multimedia content) to the comment server 103. Correspondingly, the comment server shall maintain a database storing the relationship between multimedia contents and comment clients. For example, the database may be in a form of a table containing two fields of multimedia content identifier and comment client identifier. Besides, the comment server shall also maintain a database for storing comments, time stamps and multimedia content identifiers received from comment clients. A person skilled in the art will appreciate that the structures of tables may vary depending on the concrete implementation of the present invention.

Moreover, it should note that the example of the present invention described above uses time stamp to establish an association between comments and their corresponding segments of multimedia contents. But a person skilled in the art will appreciate that any other identification that can be used to uniquely position/location the segment in the whole time line is also applicable. For example, the sequence number can be used to substitute the time stamp for marking the offset of segments to the start point of the multimedia content. Each video frame has a unique sequence number which can be used to obtain the offset by, for example, querying in video frames of the multimedia content. Fig. 2 is a message sequence chart illustrating a method for providing comments to a requested multimedia content according to the present embodiment. As in this example, the multimedia content is a video, and the video is transmitted by means of multicast.

--At the step 201 , a user end device sends a request for a video to the video server. The request contains a video ID (identifier) uniquely identifying the requested video.

-At the step 203, the video server sends a data stream of the requested video to the user end device. A person skilled in the art will appreciate that: given the fact that this example uses multicast, the user end device may join a already-existed multicast group for receiving the data stream when at least one other user end device is receiving this video (i.e. the multicast group for the requested video exists), or join a newly established multicast group for receiving the data stream when no multicast group for the requested video exists before. The data stream is carried by data packets, and each data packet contains a data portion and its corresponding time stamp (or sequence number as mentioned above).

-At the step 205, the user end device records the video ID and starts buffering the received data stream of the requested video. Herein, the user end device does not delete the buffered data stream until the user ends watching the requested video by, e.g. tuning to another video or shutting down the user end device etc. In other words, the received data stream is kept being buffered from user's tuning to the video and till user's closing the video.

--At the step 207, the user end device sends user end device I D (identifier) and video I D of the requested video to the comment server. Herein, the user end device ID is used to uniquely indentify a user end device.

-At the step 208, the comment server records the received user end device I D and video I D. They can be recorded in a device table containing two data fields of user end device ID and video ID. -At the step 209, the user end device receives a comment from the user. The comment can be inputted by an input interface of the user end device, such as a textbox etc.

-At the step 21 1 , the user end device determines a time stamp of the requested video. For example, it can be determined by the video client 104 of the user end device 102, the video client checks current playing position of the video, and returns the time stamp of current playing position relative to the start point (or the sequence number of current playing video). Herein, the determined time stamp relates to the time point (e.g. the time stamp returned by the video client) when the user starts to input a comment or finishes the comment. The 'relate' herein means the determined time stamp precedes, equals or after a time stamp corresponding to the time point when the user starts or finishes the comment. Preferably, the determined time stamp precedes a predetermined time period, e.g. 10s, to the time stamp corresponding to time stamp when the user starts or finishes the comment. This is advantageous because the scene of the video after the determined time stamp is likely the reason causing the user to input the comment. Besides, the predetermined time period can be set to other value that the actual user deems appropriate. This can be implemented by user end device's providing an interface for user to reset his own value of the time length of segment.

-At the step 213, the user end device sends a message containing the received comment, the determined time stamp and the video I D to the comment server.

-At the step 215, the comment server records the received information contained in the message. As an example, the comment server maintains a table containing at least data fields of comment, time stamp and video ID for recording such information.

-At the step 217, the comment server distributes received information to relevant user end devices. It can be implemented by, for example, a scheme that the comment server first refers to the device table and finds which end user devices are receiving the video having the same video ID as that in the received message, and then it sends the comment and the time stamp contained in the received message to those end user devices. At the side of each end user device, upon reception of the comment and the time stamp, it stores the information, e.g. in a user end comment table, and updates its display of the comments.

Fig. 3 is a flow chart illustrating actions performed by the user end device upon a click on the comment according to the present embodiment. As in this example, each user end device maintains a user end comment table recording comments and their corresponding time stamp received from the comment server 103.

--At the step 301 , the user end device displays comments associated with the requested video in the comment display area. Herein, comments are from all users watching the same video through comment server's distribution. According to the present embodiment, after a user end device registers to the comment server as indicated in the step 217, the comment server 103 sends all comments and time stamps relative to the requested video to the user end device. According to a variant, the comment server only sends comments and time stamps received after the registration of the user end device without sending those received before the registration of the user end device.

--At the step 303, the user end device receives an input relating to a comment, e.g. a click on the comment. Besides, it could also be a selection among the comments through a drop down list.

--At the step 305, the user end device determines a time stamp corresponding to the clicked comment. As abovementioned, the user end device maintains relationship information between time stamps and comments associated with the requested video, the user end device can refers to the end user table for determining the time stamp. According to a variant, the time stamp can be determined by querying the comment server with video ID and clicked comment. A person skilled in art will appreciate that the information contained in querying message and/or data fields of user end comment table may vary dependent on different implementations.

-At the step 307, the user end device playbacks a segment of the requested video starting from the determined time stamp and lasting a predetermined length of period, e.g. 10s. Sometimes, the segment corresponding to the clicked comment may not be buffered in the user end device because the clicked comment is contributed by other user end device at a time point before this user end device starts to receive data stream of the requested video. In that case, it may be advantageous to add a judgment step for determine whether the segment has already been buffered in the user end device before playback. The judgment step can be implemented by comparing the time stamp of the clicked comment to the earliest time stamp stored in the user end device. And if buffered, the user end device playbacks by using buffered data, or otherwise, the use end device retrieves it from the video server by establishing a unicast connection. According to a variant of the present embodiment, the judgment step isn't added. Each time a comment is clicked, the user end device firstly tries to fetch data from its stored data, and if it fails, it then directs to the video server for the segment of the requested video. According to another variant of the present embodiment, the user end device does not store the data stream received from the video server as performed in the step 205, and upon a click on a comment, it always directs to the video server for retrieval of the corresponding segment.

According to a variant of the present embodiment, the video is transmitted by means of unicast. Once a user end device requests for a video, the comment server distributes all comments relating to the requested video to the user end device. Moreover, the unicast mode may coexist with multicast mode. For example, the user may select a mode from unicast and multicast modes before sending request for a video. No matter which mode is selected, the comment server will distribute all comments relating to the requested video to the user end device. According to another variant, the comment server only distributes the comments received at a time point later than the time when the user end device starts to receive the requested video.

A person skilled in the art will appreciate that the actual implementations of the invention by different engineers may be different, e.g. some parameter in a message may be omitted and looks different from the embodiment/example described above. However, a person skilled in the art could make different implementations of the invention as long as that they establish an association between comments and their corresponding segments of multimedia contents. The start time is relative to the multimedia contents and relates to a time point when the user inputs or finishes the comment.

According to a variant of present embodiment, the video client 104 and comment client 105 are not within a same user end device. In other words, they are separate and a user end device may only include a comment client. Moreover, according to another variant, several comment clients share a common video client. The video client or other control device may collect users' clicks on comments from the several comment clients within a predetermined time period, and after the time period expires, the segment corresponding to the comment with most clicks will be playbacked.

A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made. For example, elements of different implementations may be combined, supplemented, modified, or removed to produce other implementations. Additionally, one of ordinary skill will understand that other structures and processes may be substituted for those disclosed and the resulting implementations will perform at least substantially the same function(s), in at least substantially the same way(s), to achieve at least substantially the same result(s) as the implementations disclosed. Accordingly, these and other implementations shall fall in the scope of the invention.

Claims

1. A method for providing comments on multimedia contents among a plurality of user devices, characterized by the steps performed at the side of a first user device,

receiving an input of a comment when a requested multimedia content is being displayed on the first user device;

in response to the input of the comment, determining an offset value relative to the requested multimedia content, wherein the offset value indicates a start position of a segment of the requested multimedia content; and

sending a message containing the comment and the offset value to a comment server for distribution to other user devices among the plurality of user devices, wherein the other user devices are watching the requested multimedia content.

2. The method of claim 1 , characterized by further comprising,

displaying comments contributed by all user devices watching the requested multimedia content; and

in response to an input upon a particular comment, playbacking a segment of the requested multimedia content starting at an offset value corresponding to the particular comment.

3. The method of claim 1 or 2, characterized in that each segment corresponds to a time length value indicating the time length of the segment.

4. The method of claim 2, characterized in that the playbacked segment is retrieved from a content server containing the requested multimedia content by using the offset value.

5. The method of claim 2, characterized by further comprising,

buffering data of the requested multimedia content since the first user device begins to receive data of the requested multimedia content.

6. The method of claim 5, characterized in that in response to the input upon the particular comment, it further comprises before playback step,

if determining the segment to be playbacked exists within the buffered data, using the buffered data to playback the segment; or otherwise,

retrieving the segment from a content server containing the requested multimedia content.

7. A device for providing comments on multimedia contents among a plurality of devices of this kind, characterized in that the device is currently watching a requested multimedia content, by comprising,

a display interface configured to display multimedia contents;

an input interface configured to receive user inputs; and

a processing module configured to determine an offset value relative to the requested multimedia content when the input interface receives an input of a comment, wherein the offset value indicates a start position of a segment of the requested multimedia content; and to send a message containing the comment and the offset value to a comment server for distribution to other devices among the plurality of devices, wherein the other devices watching the requested multimedia content.

8. The device of claim 7, characterized by further comprising,

a comment interface configured to display comments contributed by all devices watching the requested multimedia content; and in response to an input upon a particular comment via the input interface,

the processing module is further configured to instruct the display interface to playback a segment of the requested multimedia content starting at an offset value corresponding to the particular comment.

9. The device of claim 8, characterized by further comprising,

a storage module configured to buffer data of the requested multimedia content since the device begins to receive data of the requested multimedia content.

10. The device of claim 9, characterized in that in response to the input upon the particular comment via the input interface, the processing module is further configured to determine if the segment to be playbacked exists within the storage module, and if exists, it is configured to instruct the display interface to playback the segment by using data from the storage module, or otherwise, it is configured to instruct the display interface to playback the segment by using data retrieved from a content server containing the requested multimedia content.