WO2005057931A2

WO2005057931A2 - Method and system for generating highlights

Info

Publication number: WO2005057931A2
Application number: PCT/IB2004/052687
Authority: WO
Inventors: Jun Shi; Jingwei Tan; Liang Gan; George H. Canevit Jr
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2003-12-09
Filing date: 2004-12-07
Publication date: 2005-06-23
Also published as: WO2005057931A3; CN1627813A; TW200621021A

Abstract

The present invention provides a new method for generating a mark of a specific fragment in a digital program. Firstly, determining the position of a specific fragment in a digital program, and then integrating a corresponding mark into the digital program. The invention also provides a method for extracting a specific fragment from a digital program. Firstly, obtaining a mark of a specific fragment from a digital program, then extracting program content of the fragment of the digital program according to the content of the mark. The position of the specific fragment in the digital program is determined according to the position of its corresponding mark. The mark of the specific program needs neither specific generating means, nor additional channel to be sent; so that the audiences can locate and record the specific fragment more conveniently and accurately.

Description

METHOD AND SYSTEM FOR GENERATING HIGHLIGHTS

Technical Field

The present invention relates to a method of generating highlights and the

apparatus therefor, and particularly, to a method of generating highlights in digital

programs and the apparatus therefor.

Technical Background of the Invention

Digital programs are gradually entering the public life and will replace analog

programs in the near future. This is an irreversible trend in the broadcasting

industry, thus appearing a plurality of standards concerning digital programs and

its broadcasting.

MPEG-2 (Motion Picture Experts Group) is one of current standards which plays a

relatively dominant role. ISO/IEC13818 is a standard providing digital data

program stream for digital devices such as Set Top Box (STB), Digital TV (DTV),

interactive Digital TV (iDTV), personal computer, portable device and other

interactive devices. Digital Video Broadcasting (DVB) is the international digital

broadcasting standard based on MPEG-2 standard, using MPEG-2 packet as data

container, thus being able to effectively transmit program data. When playing the program (including analog programs and digital programs),

there are usually certain specific fragments in which the audience is especially interested, namely the so-called highlights, for example, the scoring highlights of a

soccer match, which the audience hopes to view repeatedly during or after the

playing. To satisfy the audience's demand, the program broadcaster usually

makes a collection of selected highlights in the montage manner during or after the

playing for the audience, thus audience could record the collection to watch at any

moment.

Just as a large number of audiences possess analog program recording device

nowadays, a large number of audiences will possess Digital Video Recorder (DVR)

in the near future. To convenient for the audiences to find the program fragments

in which they are interested in the recorded digital programs more quickly, the

international application (international application date: February 27, 2000) under

international application number of PCT/US02/05984 brings forward the following

technical solution:

At the same time of playing a normal digital TV program, the broadcasting end

generates a data stream parallel to the TV program, e.g. IP contents stream,

whose contents are associated with the TV program being played and could be

transmitted through a special data path.

The IP contents stream could include one or more Event Log Indexes (here the

"Event" is equivalent to the "highlight" in the present application), and the contents and format of the Event Log Indexes could be described in XML language as

follows:

<period id="1"> <possession id="1" pos="Cou"> <event id="10" play="2" kind="touchdown" yards="27" highlight="y" clock="9:34" start="815"> <description> WSU Touchdown </description> </event> </prosession> </period>

When recording the above TV program, at the same time the audience could

record the contents of the above IP contents stream including the Event Log

Indexes, and then associate the Event Log Indexes with the related fragments in

the TV program, so that the audience could perform "intelligent" search in the

program more conveniently according to these Event Log Indexes to find the

fragments they are interested in.

However, the technical solution in the above application has the following defects:

first, the IP contents stream is parallel to the TV program instead of a part of the

TV program stream, thus increasing the technical complexity and cost of transmission, storage and playback, for example, the contents of the IP program

steam might be lost during transmission; second, the defining format of the IP

contents stream dose not conform to the MPEG-2 standard, so the expandability

of the solution is inferior and needs a special device to generate the contents

stream; and last, the positioning time for the event position in the technical solution

is a "relative time", i.e. a time interval relative to a program beginning or a certain

time point, e.g. "9:34"; if the TV program contents of the fragment to which the

time interval corresponds are lost when recording, the event will be misplaced or

lost in the recorded TV program.

Therefore, there is needs for a new method and apparatus for generating

highlights in digital program, thus enable program broadcaster to better generate

highlight mark so that the audience could generate highlights more conveniently

and accurately.

Contents of the Invention

One of the objects of the present invention is to eliminate the above defects in the

existing technical solution so that the program broadcaster could generate and

transmit highlight mark more conveniently and the audience could find highlights

more conveniently and accurately. The present invention provides a new method of generating a specific fragment

mark in digital program. First, determining a specific fragment in digital program,

i.e. a highlight; then, integrating a corresponding mark into the digital program. The

digital program is in conformance with the MPEG-2 standard or the digital

programs compatible with the MPEG-2 standard, and the inserted specific

fragment mark is the datum conformance with the MPEG-2 standard, and is

multiplexed with the digital program in an integration manner.

The invention further provides a method of extracting a specific fragment

from a digital program. First; obtaining a mark of the specific fragment of the digital

program; then extracting content of the fragment of the digital program according

to content of the mark. The position of the specific fragment is associated with the

position of the specific fragment mark in the digital program and the extracted

highlight could be stored in a storage device as a part or whole of highlight

collection. This digital program is one of the digital program in conformance with

the MPEG-2 standard and the digital programs compatible with the MPEG-2

standard, and the specific fragment mark is the datum in conformance with the

MPEG-2 standard.

The invention further provides a digital program playing apparatus, which

has the function of extracting a highlight from one digital program according to the above method. The digital program playing apparatus comprises: obtaining means

for obtaining a digital program, wherein the digital program is integrated at least a

mark of a specific fragment; playing means for playing the digital program; storage

means for storing the digital program; and fragment extracting means, comprising:

mark obtaining means for obtaining a mark of the specific fragment from the digital

program; and content obtaining means for extracting content of the fragment of

the digital program according to content of the mark.

The specific program mark of the present invention is the datum in conformance

with the MPEG-2 standard, being a part of the digital program. Therefore, the

program broadcaster needs neither special apparatus to generate the specific

program mark, nor additional path for transmission, whereas the program recorder,

e.g. the audience, could locate the specific fragment, record the specific fragment

and generate highlight collection more conveniently and accurately because the

position of the specific fragment of the present invention is associated with the

position of the specific fragment mark in the digital program stream and the

position where the specific fragment is in digital program is determined based on

the position where the mark is in the digital program. The other objects and achievements of the present invention will be obvious, and

the present invention could be better understood when reference is made to the

following illustration of the drawings and the claims.

Brief Description of the Drawings

The present invention is explained in details with reference to the drawings

through embodiments, wherein:

Fig. 1 is a system schematic diagram of a digital program stream producing

apparatus having the function of generating highlight mark according to an

embodiment of the present invention;

Fig. 2 is a flow schematic diagram of generating highlight mark according to an

embodiment of the present invention;

Fig. 3 is a system block diagram of a digital program playing apparatus having the

function of generating highlight according to an embodiment of the present

invention;

Fig. 4 is a flow schematic diagram of extracting a highlight from a digital program

according to an embodiment of the present invention.

In all the drawings, the same reference numbers represent the same or similar

features and functions. Detailed description of the Preferred Embodiments

Fig. 1 is a system schematic diagram of a digital program stream producing

apparatus having the function of generating highlight mark according to an

embodiment of the present invention. The highlight is a specific fragment. The

video signal and the audio signal of a program, after being received by video

encoder 112 and audio encoder 114, are respectively compressed into video

Elementary Stream (ES) and audio Elementary Stream (ES) according to a

predetermined algorithm; these elementary streams, together with data such as

Teletext, RBG and the like, are respectively converted by packager 118 into

Packetized video Elementary Stream (PES), Packetized audio Elementary Stream

and Packetized data Elementary Stream under the control of the Program Clock

Reference (PCR) sent from clock device 130; these PES and Private Sections

form all the contents of the program.

After the contents of the program and the Program Specific Information Sections

(PSI sections) are inputted to and multiplexed by multiplexer 140, the multiplexer

140 outputs to standard MPEG-2 program Transport Stream (TS). The Transport

Stream is formed of many TS packets. PSI sections could comprise Program

Associated Table (PAT) and Program Mapping Table (PMT) and the like. In this embodiment, the digital program stream producing apparatus further

comprises a positioning device 150, which could identify a position that needs to

generate highlights from among video signal, audio signal or data according to a

predetermined rule, and provides this position information to multiplexer 140, thus

inserting a Highlight Mark (HCK) in the corresponding position of the program

stream. The detailed process of the positioning is as below.

The contents of the highlight mark could be arranged in the private data field

specified by MPEG-2 standard, and the form of the contents could be a descriptor,

or the private data field of a descriptor, e.g. the contents of the highlight mark

could be defined in the private data field of a stream event (see DSM-CC,

ISO/IEC13818-6). The details of the contents of the highlight mark are as below.

Fig. 2 is a flow schematic diagram of generating highlight mark according to an

embodiment of the present invention. The highlight is a specific fragment. First,

starting to receive a digital program (step S210), which could be formed of video

signal, audio signal or data shown in Fig. 1 and produced by producers on site; it

could also be a digital program transport stream in conformance with MPEG-2

standard, or a digital program transport stream compatible with MPEG-2 standard. Next, in the received digital program, determining a position that needs to generate

Highlight Mark (HCK) according to a predetermined rule (step S220); if not found,

directly going to step S250 to determine if the program is ended or not; if not

ended, returning to step S210 to continue receiving the digital program.

If a position that needs to generate Highlight Mark (HCK) is found in step S220,

receiving the program producer's setting of the highlight mark (step 230). As stated

above, the contents of the highlight mark could be arranged in the private data

field specified by MPEG-2 standard, and the form of the contents could be a

descriptor and the like.

And next, multiplexing the highlight mark into the digital program (step S240),

making it being a part of the digital program. The position of the highlight mark

comes from step S220, while its contents from step S230. In this embodiment, the

contents exist in program mapping table in the form of descriptor. When the

position of the highlight mark needs to be inserted, This program mapping table

and the one before it change in version, adding a descriptor of highlight mark.

In the end, determining if the program is end (step S250). If not ended, returning to

step S210.

During generating the highlight mark, the positioning in step S220 could be

implemented by receiving a signal instruction from the program producer by the positioning device 150. The signal instruction needs to generates a highlight mark

at a specific program position, e.g. when watching a scoring scene, the program

producer could send a request demanding the creation of a highlight mark at this

time point.

The positioning in step S220 could also be automatically implemented by

identifying the program contents by positioning device 150 according to a

predetermined rule, e.g. the rule could be an analysis of scene, i.e. if there is a

scene of football being shot to the goal, the position of this scene in the program

will act as the position requesting for creation of a highlight mark; or through

analysis of sound, i.e. if the sheer of audiences in a scene exceeds a

predetermined decibel, this position will act as the position requesting for creation

of a highlight mark; or through analysis of subtitles, i.e. if the contents of score in

the subtitle change, this position will act as the position requesting for creation of a

highlight mark.

In step S230, this embodiment arranges the contents of the highlight mark in the

private data field of program mapping table (PMT). See HCKdeschptor() in Fig. 1.

Table 1 : Program Mapping Table Syntax No. of Mnemonic bits TS_program_map_section() { Table id 8 uimsbf Section_syntax_indicator 1 bslbf O' 1 bslbf Reserved 2 bslbf Sectionjength 12 uimsbf Program_number 16 uimsbf Reserved 2 bslbf Version_number 5 uimsbf Current_next_indicator 1 bslbf Section_number 8 uimsbf Last_section_number 8 uimsbf Reserved 3 bslbf PCR_PID 13 uimsbf Reserved 4 bslbf Program_info_length 12 uimsbf For (i=0; i<N; i++) { descriptor ) } For (i=0;i<N1 ;i++) { stream_type 8 uimsbf reserved 3 bslbf elementary_PID 13 uimsnf reserved 4 bslbf ESjnfoJength 12 uimsbf for (i=0; i<N2; i++) {

HCKdescriptor( ) } } CRC_32 32 rpchof }

During editing and transmitting the whole program, the program mapping table is

amended continuously and transmitted continuously. In the table above (Table 1), there is a field being "version_number". Each time the program mapping table is

amended, e.g. inserting a HCKdescriptorQ, the contents of the field will be

amended accordingly. The method of inserting a HCKdescriptor() to program

mapping table is identical with the method of inserting other type of descriptor() to

program mapping table, so the unnecessary details will be omitted here.

The contents format of the highlight mark could be defined in the following syntax

(Table 2):

Table 2: Syntax Format of HCKdescriptor( ) Syntax Number of Bits Mnemo nic HCKDescriptorO { Descriptor Tag 8 uimsbf DescriptorLength 8 uimsbf Mode 8 uimsbf If (Mode!=STANDARD PRESET MODE) 16 uimsbf { Highlight duration value } }

The numerical value of the field "Descriptor Tag" in table 2 indicates that this

descriptor is the descriptor of a highlight mark, e.g. "0x85" (the prefix Ox indicates

that this number is of hexadecimal system, and similarly hereinafter), all the

descriptors being "0x85" in this field are descriptors of highlight mark. The

numerical value at field "DescriptorLength" represents the length of this descriptor; the numerical value at field "Mode" represents the type of the highlight mark

descriptor, and there are altogether three types of field "Mode" in this embodiment,

including Standard Preset Mode (SPM), Duration Mode (DM) and Index Mode (IM);

the numerical value of field "Highlight duration value" could be either one or many,

for representing the duration of the highlight.

In the highlight descriptor of Standard Preset Mode (SPM), the numerical value of

field "Mode" could be "0x01", while the value of field "Highlight duration value"

could be unnecessary or defaulted as a time period. Taking the position where the

highlight mark appears as the reference point, The time period could be one

predetermined time value forward, i.e. the starting position information of the

highlight, e.g. 5 seconds, and could be one predetermined time value backward,

i.e. the ending position information of the highlight, e.g. 6 seconds, and the two

predetermined numerical values could be identical. In this way, at the playing end,

the program playing apparatus could automatically associate the default numerical

value with the descriptor.

In the highlight descriptor of Duration Mode (DM), the numerical value of field

"Mode" could be "0x02", while the value of field "Highlight duration value" could

comprise two numerical values taking the position where the highlight mark

appears as the reference point, one forward time value, i.e. the starting position information of the highlight, e.g. 5 seconds, and the other backward time value, i.e.

the ending position information of the highlight, e.g. 6 seconds, and the two

numerical values could be identical, or equal to zero, but not equal to zero

simultaneously. Certainly, the value of field "Highlight duration value" could

comprise only one numerical value taking the position where the highlight mark

appears as the reference point, e.g. a backward time value (e.g. 6 seconds). In

this way, the program playing apparatus automatically defaults the position where

the highlight mark appears as the starting position of the highlight.

In the highlight descriptor of Index Mode (IM), the numerical value of field "Mode"

could be "0x03", while the value of field "Highlight duration value" could be an

index value, and based on this index value, the duration time of the highlight could

be found in a sheet of index table. The index table could be preset in a standard

duration time table by program producer based on different program contents such

as soccer, US football, hockey or basketball, like table 3.

Table 3. Highlight Duration Index Table

The index value in the index column in table 3, e.g. "06", correspond to the

numerical value in the duration time column, e.g. "X". The "X" could be any

predetermined numerical value, while "*" indicates that the duration time to which

the index corresponds is applicable for the type of program.

As for the program stream in conformance with DVB standard, the form of the

contents of the highlight mark could also exist in the form of Stream Event, e.g.

"Do-lt-Now" stream event. See table 4 for the syntax format of the stream event.

Table 4: Syntax Format of Stream Event as Highlight Mark Syntax Number of Bits Mnemon ic Private data of Stream Event (ID=) ( ) { Length 8 uimsbf Mode 8 uimsbf Highlight duration value 16 uimsbf }

The "ID" value in table 4, e.g. "100", is for distinguishing the stream event from

other ones; the value of field "Length" represents the length of the stream event;

the value of field "Mode" represents the type of the highlight mark, and there are

two types of field "Mode" here, one is Duration Mode (DM), the other is Index

Mode (IM); the field "Highlight duration value" could be one or many numerical

values, for representing the duration time of the highlight. As for Standard Preset

Mode (SPM), the stream event format of its highlight mark comprises only ID value,

without the stream event of private data portion, e.g. the ID value is 100, but there

is no stream event of private data portion.

The editing of the stream event acting as highlight mark could be the same as

editing other type of stream event, e.g. editing by means of a THLAS Coral MHP

server.

Certainly, certain new fields could be added to the syntax structure of the

descriptor and stream event as mentioned above, e.g. field "Category" , for representing the contents category of the highlight mark. For example, field

"Category" being "0xA1" indicates that the contents of the highlight mark are

shooting at the goal.

function of generating highlight according to an embodiment of the present

invention. The highlight is a specific fragment. The digital program playing

apparatus comprises: a program obtaining device 310, for obtaining a digital

program transport stream and at least one highlight mark is integrated in the digital

program; a program playing device 320, for playing the digital program; a storage

device 330, for storing the digital program; and a highlight extracting device 340,

for extracting the highlight from the digital program according to the audience's

instruction.

The program playing device 320 comprises a de-multiplexer 322, a cache 324 and

a decoder 326. The de-multiplexer 322 obtains a digital program stream from

program obtaining device 310 and sends it to cache 324 after de-multiplexing; the

contents of cache 324 could be further sent to decoder 326 to be decoded for

presentation, or could also be stored in storage device 330. Certainly, the digital

program obtained by program obtaining device 310 could also be directly stored in

storage device 330. Highlight extracting device 340 further comprises a mark obtaining device 342, for

obtaining a highlight mark from the digital program; and a contents obtaining

device 344, for extracting the program contents of the corresponding fragment in

the digital program as a highlight according to the contents of the mark; the

contents obtaining device comprises a cache 345.

During the working process, the digital program transport stream, after being de¬

multiplexed by de-multiplexer 322, is sent to cache 345 at the same time of being

sent to cache 324. When the mark obtaining device 342 obtains a highlight mark

from de-multiplexer 322, the corresponding contents in cache 345 are extracted

and stored in storage device 330 according to the mark's contents and its position

in the program, and the process of extracting is shown in Fig. 4. The size of cache

345 depends on the "Highlight duration value" of the highlight mark. Usually, a

program fragment with sufficient length could be stored. The time span of this

fragment should be larger than the backward duration time value that takes the

position where the highlight mark appears as the reference point.

according to an embodiment of the present invention. The highlight is a specific

fragment. First, receiving a digital program (step S410), such as a digital program

stream compatible with MPEG-2 standard, and playing the program (step S420), and at the time of playing the program, detecting if there is highlight mark in the

received digital program (step S430), if none, then going to step S470, further

determining if the program is ended, if not ended, then returning to step S410 to

continue receiving the digital program.

The detecting in step S430 in this embodiment could be the continuously detecting

of the program mapping table received; if its version information

"version_number" changes, then sifting the HCKdescriptor() of highlight mark in

the table; if the descriptor is found, it indicates that a highlight program mark is

found.

As for the digital program transport stream that uses stream event to mark a

highlight, a section filter in the de-multiplexer 322 in Fig. 3 could filter the stream

event that acts as highlight mark.

If a highlight mark is detected in step S430, obtaining the contents of the highlight

mark (step S440); if the highlight mark is one of a standard preset mode, the

default value of the duration time pre-stored in local is taken directly as the

duration time of the highlight mark; if the highlight mark is a mark of duration mode,

the value at the field "Highlight duration value" is taken as the duration time of

the highlight mark; if the highlight mark is a mark of index mode, the duration time

of the corresponding highlight mark value detected in the local index table is pre- stored according to the value at the field "Highlight duration value", taking this

duration as the duration of the highlight mark.

Next, extracting the corresponding program fragment from the digital program

(step S450) based on the position of the highlight mark detected in step S430 in

the digital program and the contents of the highlight mark obtained in step S440.

The extracted highli ght could be stored for future playing (step S460), and a

plurality of highlights could be stored together to form a highlight collection. During

the extracting and storing, a prompt could also be given to the user for him or her

to determine if the highlight is to be extracted or stored according to the description

information of the highlight, e.g. the information at the field "Category".

Another solution for producing a highlight collection is to generate a highlight index

file based on the position where the highlight mark detected in step S430 is in the

digital program and the contents of the highlight mark obtained in step S440.

When playing back the digital program in the future, the audience could quickly

find the highlight in the index file.

In the end, determining if the program is ended (step S470), if not ended, returning

to step S410 to continue receiving the digital program.

Although the present invention is described through specific embodiments, many

substitutions, amendments and variations made according to the above text will be obvious to those ordinarily skilled in the art, and all these substitutions,

amendments and variations shall be included in the present invention when they

fall within the spirit and scope of the appended claims.

Claims

WHAT IS CLAIMED IS:

1. A method for generating a mark of a specific fragment in a digital program,

comprising the steps of: determining a specific fragment in the digital program; and integrating a corresponding mark to the digital program.

2. The method according to claim 1 , wherein the determining step includes: receiving an instruction containing information associated with the specific

fragment.

3. The method according to claim 1 , wherein the determining step includes: detecting the specific fragment in the digital program according to a

predetermined rule.

4. The method according to claim 1 , wherein the digital program is in conformance

with MPEG-2 standard or compatible with MPEG-2 standard.

5. The method according to claimed 1, wherein the mark is a datum in

conformance with the MPEG-2 standard.

6. The method according to claim 5, wherein the integrating step is arranged to

inserting the mark into a private data field of the digital program.

7. The method according to claim 1, wherein the mark contains a start position

information of the specific fragment, the start position information being associated

with position of the mark in the digital program.

8. The method according to claim 1 , wherein the mark contains an end position

information of the specific fragment, the end position information being associated

with the position of the mark in the digital program.

9. The method according to claim 1, wherein the integrating step is arrange to

multiplexing the mark with the digital program.

10. A apparatus for generating a mark of a specific fragment in a digital program,

comprising: determining means for determining a specific fragment in the digital program;

and integrating means for integrating a corresponding mark into the digital

program.

11. The apparatus according to claim 10, wherein the digital program is a digital

program transmitting stream in conformance with MPEG-2 standard, and the mark

is a datum in conformance with MPEG-2 standard.

12.The apparatus according to claim 10, wherein integrating means is arranged to

multiplex the mark with the digital program.

13. A method for extracting a specific fragment from a digital program, comprising

the steps of: obtaining a mark of the specific fragment of the digital program; and extracting content of the fragment of the digital program according to

content of the mark.

14. The method according to claim 13, further comprising the step of: storing the extracted content.

15.The method according to claim 13, wherein position of the specific fragment in

the digital program is determined according to position of the mark in the digital

program.

16.The method according to claim 13, wherein the digital program is a digital

program transmitting stream in conformance with MPEG-2 standard, and the mark

is a datum in conformance with MPEG-2 standard.

17.The method according to claim 13, wherein the mark is inserted into the digital

program in an integration manner.

18 A apparatus for extracting a specific fragment from a digital program,

comprising: mark obtaining means for obtaining a mark of the specific fragment from the

digital program; and content obtaining means for extracting content of the fragment of the digital

program according to content of the mark.

19.The apparatus according to claim 18, further comprising storage means for

storing the extracted program content.

20.The apparatus according to claim 18, wherein position of the specific fragment

in the digital program is determined according to position of the mark in the digital

program.

21. The apparatus according to claim 18, wherein the mark is inserted in the digital

program in an integration manner.

22. The apparatus according to claim 18, wherein the digital program is a digital

program transmitting stream in conformance with MPEG-2 standard, and the mark

is a datum in conformance with MPEG-2 standard.

23.A apparatus for playing the digital program, comprising: obtaining means for obtaining a digital program, wherein the digital program

is integrated at least a mark of a specific fragment; playing means for playing the digital program; storage means for storing the digital program; and fragment extracting means, comprising: mark obtaining means for obtaining a mark of the specific fragment from the

program according to content of the mark.