WO2005104557A1

WO2005104557A1 - Method for recording of interactive information in interactive digital television and playback thereof

Info

Publication number: WO2005104557A1
Application number: PCT/IB2005/051275
Authority: WO
Inventors: Liang Gan; Philip S. Newton; Jingwei Tan; Jun Shi
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2004-04-26
Filing date: 2005-04-19
Publication date: 2005-11-03

Abstract

When recording interactive television an interactive application (12) is stored separate from the audio-visual content of a transport stream (20). During recording correct synchronization between the audio/video content and the interactive application (12) must be guaranteed. The present invention uses a unique timing based validating attribute being recorded together with the application objects for synchronizing said audio-visual content with the thereto related interactive application objects. According to embodiments timestamps for when an item related to the interactive application is recorded, such as the AIT table, stream events etc. are used. During playback these timestamps are used to guarantee correct synchronization for stream events, DSMCC module version control and for application validation timing.

Description

Method for recording of interactive information in interactive digital television and playback thereof

This invention relates in general to the field of interactive television and more particularly to the recording of interactive television content and playback thereof, and even more particularly to handling of applications data and associated audio-visual data in the field of interactive television content recording and playback.

Interactive television (iTV) is becoming more and more popular. An example of interactive television is the Multimedia Home Platform (MHP), which is a digital video broadcasting (DVB) standard intended to combine digital television (DTV) with interactivity and access to the Internet and the World Wide Web resulting in interactive digital television (iDTN). iDTN service providers offer a large variety of audio-visual (A/V) television programs and also of applications that allow the interaction of the viewer/user with the TN set and its contents. One type of applications are service-bound applications, which are strongly related to the content, wherein content generally is the audio/video transmitted together with the application. This type of applications only has relevance if the associated content is available. Therefore, service-bound applications in general will be recorded together with the associated content. The behavior of the application has timing relationship with the A/V content. In interactive Digital TN (iDTN) systems, the content is broadcasted using a MPEG-2 Transport Stream protocol. The interactive program is composed of audio/video and application, etc. This application can be linked to the program content and it allows the user to have some basic interaction with the content. Typical applications are game shows, and sports, wherein the applications may also comprise supplementary interactive services to the AN content. Interactive television content, such as MHP, transmits applications in a Transport Stream (TS) inside modules through a Digital Storage Media Command and

Control (DSMCC) object carousel. The DSMCC-Modules are transmitted down a broadcast channel in the same TS together with the Audio/video content. The DSMCC data/object carousel defines how and when to send modules/files containing applications in the TS with the audio-visual content. There is no connection to the server for a receiving device to ask for wanted files. All files are repeatedly sent all the time, e.g. once every 10 seconds. Interactive television terminals, e.g. MHP terminals, such as a Set Top Box (STB), look for the files they need as they come around. In the upcoming new version of the MHP 2.0 Personal Digital Recorder (PDR) functionality will be integrated. Similar to today's video recorders for analogue television broadcasts using video tapes for recording broadcast streams, these digital video recorders for interactive television are either using fixed storage media, such as a hard disk, or removable media, such as optical discs, for storing recorded broadcasts. PDR's for interactive television record both A/V television contents and applications for playback at a later point in time. For MHP applications that will make use of this new functionality this means that they will be able to make recordings, obtain information on stored programs and obtain background information on the content. When recording interactive television, ilN application modules/files are extracted from the Transport Stream. These modules are generally stored outside the Transport Stream in a separate file on the above-mentioned storage medium. In some cases the stored modules are re-inserted into the Transport Stream for playback over a digital interface to an external STB with an MHP stack. Under certain circumstances, this might lead to a reconstructed Transport Stream, of which the iTV application cannot be reconstructed properly by e.g. a STB, because the correct application data cannot be identified and read from the storage medium and be broadcast in the reconstructed Transport Stream. For instance one A/V stream may be related to several applications, events or data during the length of the A/V stream, e.g. the score data of a football game changes during the game or the different applications are run at different times of a game show reflecting the actual A V content of the game show. Moreover, even in playback cases where it is not necessary that the recorded objects and streams are to be reconstructed for playing back over a digital interface to an external STB with an MHP stack, the recorded MHP program from a local storage will not be played back properly either. The correct application data related to a specific section of a stored A/V stream cannot be identified and read from the storage medium. For instance, one A/V stream may be related to several applications, events or data during the length of the A/V stream, e.g. the score data of a football game changes during the game or the different applications are run at different times of a game show reflecting the actual A/V content of the game show. Therefore the object to be achieved by the invention is how to ensure that an iDTV application is synchronized with the related A/V content when recorded, such that a correct playback of the application with the related AN content is guaranteed. The present invention provides a method, a computer readable medium and a storage medium according to the appended patent claims. The general solution according to the invention is that validating information, such as the validating timing and validating period, is recorded together with the application. Generally, there are three application related timing controls in MHP. 1. One is the application validating timing, which is signalled in the

Application Information Table (AIT). The AIT is also repeated in the Transport Stream, wherein there is a field to indicate whether the application is validated or not in the AIT. When the AIT indicates the application is validated, the arrival time of this AIT is taken as the application validating timing. 2. Another one is the stream event validating timing. When a Stream event is used to trigger the application, then the application will respond to the stream event. For example, when a goal occurs in a match, the broadcast will insert a new score stream event into the Transport Stream. 3. The other timing control is for version changing of the application component (application data). If a FileObject has two versions, each of them should have its validating period. In live broadcasting, all the three timing controls mentioned above depend on the arrival time in the Transport Stream. When recording iDTN the application and the AIT, the stream event etc are filtered out and recorded into files or modules. As mentioned above, the synchronization between application and AV content has to be guaranteed. The present invention has the advantage over the prior art that it enables a reliable synchronization between application and A/V content of iDTV by detecting a version change of the data object and timestamping it, or when each object has a validating time point, using this validating time point as a timestamp, or if the stream events are stored separately from the A/V stream, attaching the stream events with a timestamp.

Preferred embodiments of the present invention will be described in the following detailed disclosure, reference being made to the accompanying drawings, in which Fig. 1 shows a schematic overview over an iDTV recording system in recording mode, Fig. 2 is a schematic overview over an iDTV recording system in playback mode, Fig. 3 is a schematic illustration of a transport stream with two PCR discontinuities, Fig. 4 illustrates in a flowchart an embodiment of the method according to the invention, Fig. 5 is a flowchart of another embodiment of the method according to the invention, Fig. 6 is a flowchart of yet another embodiment of the method according to the invention, Fig. 7 is a schematic illustration of a computer readable medium comprising code segments for executing the method of the invention, and Fig. 8 is a schematic illustration of a storage medium being recorded by means of the method of the present invention.

Fig. 1 shows a schematic overview over an iTV recording system 1 in recording mode. As mentioned above, interactive television content, such as MHP, transmits applications in a Transport Stream (TS) inside modules through a Digital Storage Media Command and Control (DSMCC) object carousel. Audio-visual content is transmitted in the same TS. Interactive television content, such as MHP, is transmitted over a digital interface 20, such as in a Transport Stream (TS), inside modules 12 through a Digital Storage Media Command and Control (DSMCC) object carousel 2. Audio- visual content is transmitted in the same TS 20. Arrow 10 indicates the rotation of objects 11 in the carousel over time. The TS 20 is received by a recording device, such as an MHP recorder. Recording device 30 comprises a storage medium 31 on which files/objects 32 are recorded from the TS. Fig. 2 is a schematic overview over a corresponding iTV recording system 3 in playback mode. Files/modules 32 from the recorded interactive television content, are transferred from storage medium 31, to a DSMCC object carousel 4. As shown in Fig. 2, the files/modules and audio-visual content are broadcast via a digital interface 45, such as in a Transport Stream (TS), inside modules through the DSMCC object carousel 4. Arrow 10 indicates the rotation of objects 11 in the carousel over time. The TS 45 is received by a receiving device 50, such as a Set Top Box (STB). Hence, during the transmission of an interactive TV program, at least one data object is transmitted in a carousel for interactivity. The data object may get updated during the course of transmission. When the program is not stored as the received transport stream, but as a stream with A V data with separately stored data objects, a timestamp is needed to know about the time of use, i.e. the update of the data object. In order to meet these requirements, the following inventive embodiments are provided: 1. Version change of the data object is detected and timestamped. 2. Each object has a validating time point and shall also have a timestamp.

This scenario is true for the carousel appearing after the transport stream is tuned. 3. If the stream events are stored separately from the A/V stream, the stream events shall be attached with a timestamp. For the case of stream events, the timestamp is preferably the trigger time of the stream event. In the DSMCC standard, stream events are classified into two types, i.e. the 'do it now' stream event and the 'scheduled' stream event. For the 'do it now' stream event, the timestamp may be the PCR or arrival time of the first data packet that contains this stream event. For the 'scheduled' stream event, preferably the trigger time or validation time of the stream event is used. This also covers the case when a 'scheduled' stream event is sent in advance at any time before the stream event is to be triggered or fired.

Hereinafter, embodiments of the inventive method, which guarantee the timing relationship, are described in more detail. A timestamp of validating timing is added to every application component, i.e. file, stream event, or module, based on: 1. The Program Clock Reference (PCR) timing when a component appears in the Transport Stream (TS) is used; 2. The ATC (arriving time clock, which may be based on any natural clock time, such as UTC (Universal Time, Co-ordinated time) timing when a component arrives in the TS is used; or 3. The Normal Play Time (NPT) time base is used, i.e. the NPT time value when the component arrives is recorded as the component's timing attribute. Fig. 4 illustrates in a flowchart an embodiment 6 of the method according to the invention, corresponding to the above-mentioned case 1. The Program Clock Reference (PCR) timing when a component appears in the Transport Stream (TS) is used for deriving a time stamp in step 60. In step 61 the time stamp is recorded together with the application. Fig. 5 is a flowchart of another embodiment 7 of the method according to the invention, corresponding to the above-mentioned case 2. The ATC (arriving time clock, which may be based on any natural clock time, such as UTC (Universal Time, Co-ordinated time) timing when a component arrives in the TS is used for deriving a time stamp in step 70. In step 71 the time stamp is recorded together with the application. Fig. 6 is a flowchart of yet another embodiment 8 of the method according to the invention, corresponding to the above-mentioned case 3. The Normal Play Time (NPT) time base is used, i.e. the NPT time value when the component arrives is used for deriving a time stamp in step 80. In step 81 the time stamp is recorded together with the application. A sample FileObject with two versions recorded is given as follows: <TreeNode> <Index>50</Index> <Name>football.gif</Name> <ObjectInfo> <Version>0</Version> <Timing>2357943884435</Timing> <LocationPointer>385047</LocationPointer> <ObjectSize>12373</ObjectSize> </ObjectInfo> <ObjectInfo> <Version> 1 </Version> <Timing>1993742296802</Timing> <LocationPointer>830733</LocationPointer> <ObjectSize>12373</ObjectSize> </ObjectInfo> </TreeNode> The sample uses PCR as a timing attribute. In this sample, a file named

"footballgif ' is recorded, which has two versions. Each version is recorded with the PCR value when it first appears in the transport stream as its timing attribute, thus the validating period is also derived as the time difference between the PCR values. When playing back the recorded iDTV contents, the recorded objects are validated at suitable timing according to timestamps. The interactive component is fed back into the playback system's IRD (Integrated receiver decoder) with the same timing relationship as in the live broadcast, when it originally was recorded. However, if the transport stream's PCR is not continuous, the PCR discontinuity has to be taken into account, in order to avoid timing problems during playback. The section below will in detail explain an alternative PCR solution in case of at least one PCR discontinuity occurring in the recorded transport stream. The PCR may be discontinuous when broadcast. For example, when user chooses to record a program lasting for a longer period of time, such as two hours, maybe several discontinuities occur. Fig. 3 is an illustration of a transport stream 38 with two PCR discontinuities, wherein in this illustrative example two discontinuities D (a) and D (b) are assumed. When recording, in this case the recorder records the contents into three parts, each part being internally PCR continuous: • Part I: from the recording start point S to the discontinuity D (a),

• Part II: from the first discontinuity D(a) to the second discontinuity D(b),

• Part III: from the second discontinuity D(b) to the recording end point E. Each part is composed of A/V content and corresponding interactive content. For the interactive content, each component is attached by its internal PCR value when it arrives as its attribute. For an interactive component that has only one version and is valid within the whole exemplary two hour period, for example "fifa.gif . Its instances in part II and III may is a symbolic link to its corresponding part I, and the instances in part II and III need not to be recorded, thus saving recording space on a storage medium. Fig. 7 is a schematic illustration of a computer readable medium (9) comprising code segments for executing the method of the invention. The computer-readable medium (9) has embodied thereon a computer program for processing by a computer (90), for recording interactive information of interactive digital television (iDTV) content from a transport stream (TS) on a storage medium, wherein said content comprises audio-visual content and at least one thereto related interactive application object. The computer program comprises a code segment (91) for recording a unique timing based validating attribute together with the application objects for synchronizing said audio-visual content with said thereto related interactive application objects. Fig. 8 is a schematic illustration of a storage medium (100) being recorded by means of the method of the present invention. The medium according to the embodiment is a digital storage medium (100) and comprises information (101) recorded according to the method of the invention. Further, the medium (100) is correspondingly adapted for playback. Preferably the digital storage medium (100) is an optical disc or a hard disk. Preferably the optical disc is a Blu-Ray DVD or a SFFO disc. As explained above, MHP uses DSMCC Object Carousel to transport the MHP content in the Transport Stream. The solution according to the invention is not specific for MHP but also works for other iTV content broadcast in an Object Carousel or a Data Carousel. The present invention has been described above with reference to specific embodiments. However, other embodiments than the preferred above are equally possible within the scope of the appended claims, e.g. performing the above method by hardware or software, etc. For the sake of completeness, it is specifically pointed out that the present method differs substantially from known methods of simply adding unique timestamps to each single data packet in a stream of data packets. The interactive application data objects maybe transmitted in several data packets or several application data objects may be transmitted in one single data packet. For example, when an application data object is transmitted in several data packets, the timestamp may be the arrival time of the first data packet. In this case, there is no need of marking up every data packet with a timestamp. However, the all data packets of the same application may be marked with the arrival time of the first data packet. Furthermore, the term "comprises/comprising" when used in this specification does not exclude other elements or steps, the terms "a" and "an" do not exclude a plurality and a single processor or other units may fulfill the functions of several of the units or circuits recited in the claims.

Claims

CLAIMS:

1. A method of recording interactive information of interactive digital television

(iDTV) content from a transport stream (TS) on a storage medium, said content comprising audio-visual content and at least one thereto related interactive application object, said method comprising the step of recording a unique timing based validating attribute together with said application objects for synchronizing said audio-visual content with said thereto related interactive application objects.

2. A method according to claim 1, wherein said validating attribute being the validating timing and validating period of said application object in said transport stream (TS).

3. A method according to claim 1 or 2, wherein said validating attribute being a timestamp.

4. A method according to claim 3, comprising deriving said timestamp from the Program Clock Reference (PCR) timing when a application object component appears firstly in the Transport Stream (TS).

5. A method according to claim 3, comprising deriving said timestamp from the arriving time clock (ATC) timing when a component arrives in the transport stream (TS).

6. A method according to claim 3, comprising deriving said timestamp from the Normal Play Time (NPT) time base.

7. A method according to claim 6, comprising recording said NPT timestamp at that point in time when the component being recorded arrives from the transport stream (TS).

8. A method according to any of the preceding claims, comprising storing of said attribute on the same storage medium as said audio-visual content.

9. A method according to any of claims 1 to 8, comprising detecting a version change of an application object being recorded from the transport stream (TS), said timestamp being the time of version change, and timestamping the recorded application object with the time of version change.

10. A method according to any of claims 3 to 8, wherein each object having a validating time point and said timestamp being said validating time point.

11. A method according to any of claims 3 to 8, comprising recording of stream events separately from the A/V stream, and attaching a timestamp to each stream event.

12. A method according to claim 11, wherein the timestamp being the trigger time of the stream event.

13. A method of playing back iDTV contents recorded by the method according to any of the preceding claims, wherein the recorded objects are validated at suitable timing according to said attributes and re-inserted into a playback transport stream.

14. A computer-readable medium (9) having embodied thereon a computer program for processing by a computer (90), for recording interactive information of interactive digital television (iDTV) content from a transport stream (TS) on a storage medium, said content comprising audio-visual content and at least one thereto related interactive application object, said computer program comprising a code segment (91) for recording a unique timing based validating attribute together with said application objects for synchronizing said audio-visual content with said thereto related interactive application objects.

15. A digital storage medium (100) comprising information (101) recorded according to the method of any of claims 1-12 adapted for playback according the method of claim 13.

16. A digital storage medium (100) according to claim 15, wherein said medium is an optical disc.

17. A digital storage medium (100) according to claim 16, wherein said optical disc is a Blu-Ray DVD or a SFFO disc.

18. A digital storage medium (100) according to claim 15, said storage medium being a hard disk.