CA2439737A1 - Extented time-code for multimedia presentations - Google Patents

Abstract

The present disclosure relates to providing extended time-code numbers to multimedia elements that make up multimedia presentation content. A linear time-code is used to properly sequence the multimedia elements for proper presentation. Prefix and suffixes are attached to the linear time-code to different between multimedia elements having the same linear time-code. Multimedia elements having the same linear time-code are used for different presentation variations, such as scene variations for a movie. The unique extended time-code may be provided in a time map table such as used in a medium such as DVD, and may be provided separate from the multimedia elements. Such a time map is appended to a pre-existing information file or provided as a replacement information extension file that includes a legacy IFO file.

Description

PRIORITY TO PROVISIONAL APPLICATION

2 This application claims priority to provisional application serial number 60/416,976 filed October 7, 2002.

TEC~INICAL FIELD
The systems and methods described herein relate to associating time references to multimedia presentations and specifically to systems and methods that associate unique time-code values to audio video elements and interspersed elements.
io BACKGROUND
~2 Audio video (A/V) presentations are made up of discrete elements (A/V
i3 elements) that include video frames, pictures, video clips, and audio clips. A
is grouping or sequencing of A!V elements (i.e., video and audio) provides a ;s particular scene or chapter in an A/V presentation. Examples of A/V
presentations ~s include video tape movies, television broadcast, and multiple titles in a digital versatile disc (DVD). DVD disc titles may include a trailer title, a promotional ~a title, and a movie title. In certain DVD discs, there may be multiple movie titles.
To provide the proper viewing experience, specific video and audio 2o elements are sequenced to play at a particular time. To accomplish this, various 2~ elements are associated with time values which are in turn compared with a time 22 reference or other linearly increasing clock value to determine when to initiate the 23 elements. For example, audio and video elements for a first scene of a movie 2a might be played beginning at a time 141 and audio and video elements for a ~eet~hayeswk sos.na..,us I G:IMSI-111444usIMSl-1444USpal.app.doc 1 second scene of the movie might be played beginning at a subsequent time 160, 2 wherein both times are specified in relation to the common time reference.
s Prior methods of providing time values for audiovisual elements include a a scheme known as "control track." Videotape presentations make particular use of s the control track scheme. The control track scheme attaches nondescript pulses to s each frame of a videotape. As the videotape is played, the pulses trigger a counter, which in turn generates a monotonically increasing time value corresponding to s the currently playing point in the presentation. This time value is compared to the s time reference to govern playback speed, location, and synchronization. A
shortcoming in the control track scheme is that different time values are generated 1 i depending on the point in the presentation at which the counter is initiated-the 1z point at which playback begins. This can be problematic when searching for and t3 accurately identifying particular frames, since there is no unique value that can be is guaranteed to be consistently associated with any particular frame.
15 "Time-code" schemes are alternatives to the control track scheme. A time-isll code scheme associates a mufti-bit time-code number to each A/V element or scene. A time-code number can be structured as "HH:MM:SS:FF" where "HH"
represents hours, "MM" represents minutes, "SS" represents seconds, and "FF"
1 s represents frames within each second. Corresponding video and audio elements zo have the same time-code number, so that the particular video and corresponding audio begin at the same time. Again, the time-code values are compared with the 22 time reference to determine when the elements are to be initiated.
Zs The Society of Motion Picture and Television Engineers (SMPTE), a z4 professional organization chartered to developed standards for film and video, 2s standardized the use of time-code. Evolving SMTPE time-code standards include lee~lhayesp~e sosam.sxs' 2 G:IMSl-1I1444urIMSI-1444US.pat.app.doc 1 SMTPE 12M and SMTPE 328M. Since SMTPE time-code standards are widely 2 implemented in industry, it is desirable that AN presentations support or are backwards compatible with SMPTE time-code.
In many cases, an A/V element is made up of a computer or machine-s readable data file. In this case, a time-code number may be placed in a header of s each AN element or file. Each element might correspond to a particular scene in a movie, and a movie therefore might comprise a number of scenes or elements.
Movies and their corresponding AlV elements are often stored and distributed on a DVD. A DVD contains audiovisual data that is organized as video object units (VOBUs). A VOBU is a physical unit of DVD storage, and is » the smallest access unit of a DVD disc. A contiguous (physically sequential) ~2 group of VOBUs may make up a unit known as a "cell". A cell is the smallest 13 portion of video addressable by an index. Each cell corresponds to some portion is of A!V content, for instance a scene. Cells may vary in temporal length from _fr_actions of a second to hours.
A group of cells may logically make up a video object (VOB). A VOB is 17 single file that may include multiple multiplexed audio and video streams.
A
1s collection of VOBs may make up a video object set (VOBS).
A DVD might have numerous "titles". Each title is a potentially different zo arrangement or sequencing of cells (and VOBUs). In many cases, these titles 2i might correspond to different versions of a movie. For example, a movie might 22 have versions with different ratings, in different languages, and so on.
When 2s viewing a DVD, the user typically selects which version is to be rendered.
In 2a many cases, different titles will share or make use of many of the same cells or leel~hayes w~ sosam.emss 3 G: lMS!-lll444usIMS!-1444US.pal.app.doc VOBUs [from the same VOBS, Video Object Set; possibly from the same VOB
file in the set].
s Data on a DVD is physically formatted as a sequence of logical sectors a identified by logical sector numbers (LSNs). Each sector contains a sequence of s 2048 bytes of data. It is conventional for cells and VOBs on the DVD to be s specified in terms of their sectors' LSN and byte offset into a file or sector.
Each Video Title Set on a DVD has an information or IFO file that s describes the playback behavior and user control options for "Titles" on the DVD.
In particular an IFO file provides pointers to cells (i.e., LSN) on the DVD.
The 1o IFO file specifically is used in the playing of titles on the DVD providing 11 navigation and presentation information.
12 Because of the DVD's organization, its content can be played in different 13 ways to result in the different DVD titles. Titles in a DVD-Video disc may be 1a made up of up to 999 program chains (PGC). A program chain contains up to is programs (PG). A PGC is a list of cells. Each PGC has a program control block is (PCB) that includes an ordered set of pointers to cells indicating a sequence in which programs and cells are to be played. Different PGCs allow different sequences of cells. IFO files provide the indexes to resolve the cells listed in a is PGC to VOBUs and LSNs that allow playing of different titles with different zo video sequences.
Fig. 1 illustrates two types of DVD titles and how they might be played.
z2 DVD titles start at a "Relative Zero Time" on a timeline 102, where timeline 102 z3 represents a linear time reference. A DVD title that is to be played straight 24 through is referred to as an "one sequential PGC title". A DVD title that has 25 potential variations in its sequencing is referred to as a "multi~PGC
title", a lee~hayes pk sosa=.azss 4 G: IMSI-111444usIMSl-1444US.pal.app.doc 1 "multi~angle title", or a "parental block title". A PGC 1 105 is shown that provides for a title to be played straight through or provides for linear play. Non-s linear play exists when a "multi~PGC title", "mufti angle title", or a a "parental block title" exists. A particular title may start with a PGC 2 110 then s branches off to either a PGC 3 115 or a PGC 4 120. PGC 3 115 branches to a s PGC 5 125. PGC 4 120 branches to a PGC 6 130. PGC 5 125 branches to PGC 7 135. PGC 6 130 may branch to either a PGC 8 140 or a PGC 9 145. Finally, all s titles in this example end by playing PGC 10 146 in locations 150, 152, or 154.
s The version of the title that branches to PGC 7 135 might be a NC-17 rated 1 o version of the movie. The version of the title that branches to PGC 8 140 might be > > an R rated version of the movie. The version of the title that branches to 12 145 might be a G rated version of the movie.
i3 Different variations (i.e. titles having "mufti PGC title" or "parental block is title") will have scenes that start at different times. In other words different 1s variations will have corresponding scenes that may start at different times on timeline 102, which is the duration of time from the start of PGC 2 110. This is because titles defined by "mufti PGC title" may have different videos for the same scene, and/or different languages for the same video. Based only on a time-code ~s number, there is no way to differentiate which video and/or which audio of the Zo different variations to play for a particular scene. In this example, PGC

could begin playing at three different times on timeline 102, depending on the title 22 playing and the path taken to get there.
Zs Time-code numbers may also apply to broadcasts or streaming of AN
za presentations. Fig. 2 illustrates a prior art streaming of scenes interspersed with 25 elements, such as a hypertext markup language (HTML) document that represents IeeQlhayeso.a ws.axau' 5 GrIMSI-111444us1MS!-1444US.pat.app.doc a commercial. An element is distinguished from a scene that is composed of audio z and video components. The scenes and elements may come from different sources or broadcast points. A user receives a broadcast 200 that includes a streams) of a scenes and elements. The scenes may be made up of a number of discrete A/V
components that include audio and video clips. In this example broadcast 200 s broadcasts in the following order: scene 205, element 210, scene 21 S, scene 220, element 225 and scene 230. Each A/V component (clip) of each scene is identified by a particular linear time-code (LTC) number. The particular LTC
s number is placed in the file header of and identifies each scene. The example shown illustrates a LTC header 205-A that is associated with audio clip 205-B
and > > video clip 205-C of scene 205. LTC header 215-A is associated with audio clip 1z 215-B and video clip 215-C of scene 215. LTC header 220-A is associated with audio clip 220-B and video clip 220-C of scene 220. LTC header 230-A is is associated to audio clip 230-B and video clip 230-C of scene 230.
15 Elements 210 and 225 may be computer (machine) readable files such as 1s files written in HTML or a similar language. In this example elements 210 and 225 do not have particular LTC numbers associated with them. Broadcast and is received streams of scenes have LTC numbers describing when the scenes are to ~s be played; however, elements without LTC numbers do not have information to zo indicate when they are to be played. This is particularly problematic when scenes 2~ and interspersed elements are received from different sources and there is no way z2 to determine their sequence of play.
23 An evolving area is web based multimedia playback that incorporates DVD
2a video from a local player, interspersed with elements from a remote web site.
zs Such elements may include scene updates, scene changes (e.g., new plot twists in lee~hayesure soo-nzvaxss 6 G:IMS7-I1I444usIMSl-l444USpat.app.doc 1 a movie), different language (audio) tracks, overlays of text and graphics (e.g., z subtitles), and/or promotional content. Since a DVD relies on an IFO file, new or s interspersed web based elements without time-code can not be sequenced with a existing A/V elements or randomly accessed based on the existing IFO file.
s S LrMMARY
The systems and methods described herein include identifying multimedia s elements that have linear time-code numbers associated with them and adding s prefix and suffix values to the linear time-code numbers to create unique extended ~o time-code numbers that differentiate the particular multimedia elements.
i i In certain embodiments linear time-code numbers with prefix and suffix 1z values are added to multimedia elements without a linear time-code.
13 In particular embodiments, a time map table may be used to correlate is extended time-code numbers with particular multimedia elements.
1 s BRIEF DESCRIPTION OF THE DRAWINGS
1~ Fig. 1 is a block diagram illustrating titles defined by a linear play "one sequential PGC title" and a non-linear play "mufti PGC tile".
7s Fig. 2 is a block diagram illustrating prior art streaming of scenes and zo interspersed elements from a broadcast point or broadcast points.
zi Fig. 3A is a block diagram illustrating receiving a stream of interspersed zz elements from one source and a stream of primary elements from another source at z3 a player.
za Fig. 3B is a block diagram illustrating an extended time-code header that is z5 part of interspersed and primary elements.
lee~hayesmk sosaz..szss 7 G.'IMSI-II1444ut'IMSI-1444US.paf.app.doc 1 Fig. 4 is a block diagram illustrating an IFX file that includes an IFO file z and an appended time map table, and interrelationship to a DVD.
Fig. 5 is a block diagram illustrating a system of multimedia devices and a broadcasters to provide multimedia presentations.
s Fig. 6 is a block diagram illustrating an architecture of a multimedia device.
Fig. 7 is a block diagram illustrating an architecture of a multimedia player.
Fig. 8 is a flowchart illustrating a process to create or add extended time code values to A/V elements and/or elements.
s Fig. 9 is a flowchart illustrating a process which searches for an element 1o based on an extended time-code number.

1z DETAILED DESCRIPTION
13 Unigue Time-code Identifier Number 14 The techniques described utilize prefixes and suffixes in combination with 1s lir_ear time-code numbers in order to provide a unique identification of any 1s particular multimedia element. In some embodiments, the prefixes and suffixes 1~ are appended to the SMPTE time-code structure "HH.MM.SS.FF" in order to 1s preserve backwards compatibility with relevant precursor standards, such as 1s SMPTE 12M and 328M. The SMPTE time-code structure HH:MM:SS:FF" is zo defined as follows: "HH" represents hours, "MM" represents minutes, "SS"
z1 represents seconds, and "FF" represents frames within each second.
zz Fig. 3A shows receiving interspersed elements and primary elements.
zs Interspersed elements 300(1), 300(2), ..., 300(N) include for each particular 2a interspersed element 300 a header 305(1), 305(2), ..., 305(N), and actual content is 310(1), 310(2), ..., 310(N). Interspersed elements 300 are received at a player iee~nayes wk so.a,..szs' 8 G: IMSI-111444uslMSl-1444US.pat.app.doc 1 315. Interspersed elements 300 may be received from a source such as an Internet z source (e.g., a website).

3 Player 315 further receives primary elements 320(1), 320(2), ..., 320(N) a that includes for each particular primary element 320 a header 325(1), 325(2), ..., s 325(N), and actual content 330(1), 330(2), ..., 330(N). Primary elements 320 may s be received from a separate source, such as an Internet source (e.g., a second website). When actual contents 310 and 330, are authored, the relative timings of s actual contents 310 and 320 relative to one another may not be known. By s attaching a time-code or timestamps to headers 305 and 325, each of the 1o interspersed elements 300 may be properly sequenced amongst each of the 11 primary elements 320.
i2 For example, the player 315 may be playing primary elements 320 from a i3 DVD disc, while interspersed elements 300 are received from (i.e., streamed from) ~a a website where the interspersed elements 300 may be director's comments related to particular primary elements 320. With a time-code attached to each 1s interspersed element 300, a proper sequencing is performed with the primary i~ elements 320. In other words, the director's comments of interspersed elements 300 match particular primary elements 320, which may be scenes of a movie.
1s Other examples include properly sequencing promotional elements such as Zo commercials as interspersed elements 300 with primary elements 320 which may 2i be a movie. This allows commercials to be appropriately placed between scenes z2 of a movie. Presentation of "interspersed elements" may alternate with the 2s primary audiovisual elements, or be simultaneous (e.g., foreign language dialog 2a synchronized to actors' lips, and graphical or text subtitles presented z5 superimposed on the video image or underneath it).
lee~lhayesauc sos~sx~ezss 9 G:IMSI-111444usIMSI-1444USpot.app.doc Fig. 3B illustrates pertinent details of a header 335 associated with a particular piece of content 300 and 320. The header 335 comprises an LTC
number 340, a prefix 345, and suffixes 350, 355, and 360. LTC number field 340 a specifies a presentation time in the format "HH.MM.SS.FF", to indicate when content 300 and 320 should begin. LTC number field 340 may be a pre-existing s SMPTE number that has previously been associated with a particular A/V
element, or may be a derived value that is attached to a multimedia element that s previously did not have an SMPTE number. Sequencing of interspersed elements s with corresponding audio/video (A/V) elements may be performed as a production 1o process by when the audio and video elements are created or edited for a particular > > A/V presentation. Alternatively, corresponding time-code numbers may be ~z attached to interspersed elements separately delivered (i.e. combining web v3 delivered content with audiovisual content fram a local DVD-Video disc), and the 1a separately delivered elements linked to existing A/V elements with mutually ,5 consistent time-code numbers. This process is referred to as "late-bound"
or "late-is binding" synchronization. Time-code itself may be "late-bound" to existing » audiovisual content by downloading a time-code index that references audiovisual content that was previously without time-code (e.g.., a time map table that 1s synchronizes time-code to audiovisual content using an LSN index).
zo Prefix 345 precedes LTC number field 340 and comprises an epoch or title z~ field. Title field 345 may describe and differentiate particular titles in a DVD
2z disc, or may describe or differentiate between a broadcast program or streamed zs movie and an interspersed element such as a commercial. For example, on a za particular DVD disc, there may be a title that contains a special video feature 25 about how the movie was made, and a title that contains the movie. It may be IeeQ~hayes wk sosaz~axss 10 G:IMS!-l1I444us1MSl-I444USpat.app.doc 1 desirable to sequence particular multimedia elements from the special feature with 2 scenes from the movie title. In accordance with the scheme described herein, a particular special feature video segment will be sequenced with a particular movie a scene, and may have the same time-code value in its respective LTC number field s 340. The prefix title field 345 is used to differentiate between the comment and s movie scene. For example, the respective headers might contain "making the 7 movie.00:53:33:13" and "movie.00:53:33:13" where a LTC value of "00:53:33:13" is used to synchronize a particular comment with a particular movie s scene.
~o Prefix title field 345 itself may be formatted in accordance with one of several standards including International Standard Audiovisual Number (ISAN).
12 ISAN is intended to provide a global database that uniquely labels each video 13 work, such as a movie or television show. The ISAN numbering convention is defines an identifier of 16 hexadecimal digits divided into two segments: a ~s digit root segment followed by a 4-digit segment for the identification of episodes is or parts when applicable.
17 As discussed there may be different variations that include different is combinations of video and audio describing particular scenes. Variations include, 19 but are not limited to, language, camera angle, and rating (i.e., parental block).
2o Header 335 contains three suffix value fields: language fteld 350, angle field 355, 21 and parental block (rating) field 360. Language field 350 differentiates a 2z particular title by the language used in audio, video, and subpictures (e.g, Spanish, zs English, and French). Angle field 355 differentiates further differentiates a title by 2a camera perspective (e.g., front, back, first person). Parental block field 25 differentiates a particular title by rating (e.g., "G", "R", "PG").
IeE'~hayesWC sosaz~etss 11 G:IMSI-III444us1MSl-1444US.pat.app.doc Different A/V elements (i.e. different scenes that are played for different variations) may make use of the same value for LTC number 340 (i.e., different scenes played at the same time or sequence); however, they are differentiated by particular values for language field 350, angle field 355, and parental block field 360. For example, if a particular combination of prefixes and suffixes allow, there s may be a scene described by "movie.1:23:02.15.span.front.NC-17" where a movie scene starts at a relative time value of 1 hour, 23 minutes, 2 seconds, and 15 s frames, content in Spanish, front view angle, and having an NC-17 rating.
s Another element may have the same time value of 1 hour, 23 minutes, 2 seconds, io and 15 frames; however, this element may be an interview with the director.
The i 1 particular element may be identified as "interview. ) :23:02.15.english.null.null"
1z Exemplary applications making use of the extended time-code naming convention include playing a particular movie based on a selection of particular is suffix values; skipping over particular elements that are defined by a ,s "commercial" prefix title field 345; and searching for a particular scene based on a particular suffix(s) played at a particular LTC number.
m An extended time-code number such as contained in header 335 may be described as "TT.HH.MM.SS:FF.LL.AA.PB", where "TT" is title, "LL" is language, "AA" is angle, and "PP" is parental block.
2o Multimedia elements are described by particular file formats. For example, 21 an audio file may be described by "wave" audio file format as defined by the 22 Microsoft Corporation. A wave audio file is defined by a ".wav" extension.
An z3 example of a particular video file format is the audio/video interleaved format or AVI. An AVI file is defined by an ".avi" extension. It is contemplated that lee~hayeswk soamx~ozss 12 G:IMSI-I1I444usIMS!-I444US.pal.app.doc regardless of the file format, each particular A/V element may include a header that describes a particular extended time-code number.
In certain cases, particular fields may not apply. In other words, there may a only be one or a limited number of variations for a particular scene, and for other scenes there may be a greater number of variations. A "NULL" value may be set s for fields that do not apply. Different delimiters may be used (e.g., period, comma, colon, semicolon, slash, etc.), and delimiters may be omitted in the case s where leading or trailing nulls can be represented without ambiguity.
s Information Extension (IFX) file ~o A DVD disc can be described as a sequence of data structures referenced by 11 logical sector numbers (LSN) and byte offsets which are identified in index data 1z structures. Index data structures occupy a defined order of sequential and ,3 contiguous LSNs. Video objects (VOB) are arranged sequentially and is contiguously in order to provide known locations by LSN for access by program 1s chains (PGC) and other video index data structures. An information (IFO) file on 1s a DVD contains the index data structures and provides pointers to physical disc » locations (i.e. LSN) which contain the video referenced by a particular PGC.
For i8 a title that is mufti PGC title, multiple PGCs in the IFO file are linked by title is information, selected by user actions, or selected by programming commands, zo resulting in playback variations of the title that is selected.
z, Data structures in the IFO file include Video Manager Information (VMGI) zz that provides information as to arrangement of the disc; Video Title Set 23 Information (VTSI) that provides information as to navigation and presentation of za a particular title set on the disc; and Video Object Sets (YOBS) that contain a z5 grouping of video cells in VOBs. One of the functions of the IFO file is to ~ee6~nayeswk so..m..~~ I3 G:IMSl-111444us1MS1.1444USpal.app.doc i describe the location of VOBUs, cells, and VOBs of particular titles by providing the appropriate pointers to particular LSNs that contain the data.
Fig. 4 shows an IFX file 400 that includes an IFO file 405, a time map (TMAP) data structure or TMAP 410, and an extended TMAP data structure or s extended TMAP 415.
s TMAP 410 associates LTC values to VOBUs in its Video Title Set. In this example, TMAP 410 is made up of entry 1 415(1), entry 2 415(2) to entry N
s 415(N). Each entry of entries 415 has a linear time code value 420 and a pointer s 425. Pointers 425 of entries 41~ point to a VOBU table 430 and in particular to 1o entries 435(1) to 435(N) that provide video content and are defined by VOBU

> > and LSN 442. TMAP 410 defines defines one sequential PGC titles. VOBU
12 table 430 may be included in IFO file 405 and links each VOBUs to storage locations on the disc expressed and indexed by the first LSN 442 of each VOBU
i4 440.
15 I Typically IFO file 405 includes only TMAP 410. Extended TMAP 415 is appended to TMAP 410 and IFO file 405 to create IFX file 400. Extended TMAP
415 provides an implementation of extended time-code format that adds extended i8 time-code for mufti-PGC titles while retaining normal DVD-Video IFO file 19 functionality on legacy playback devices that only read the IFO file 405.
In this 2o example, the extended TMAP 415 is shown providing pointers 445 to some entries z7 435 of VOBU table 430. In certain cases the same entries of entries 435 as z2 referenced by TMAP 410 are pointed to; however extended TMAP 41 S may not Zs necessarily reference (point to) all or the same entries 435 referenced by TMAP
2a 410.
Zs lee~lhayesqk sosameZSe 14 G:IMSI-I11444usIMSI-I444US.pat.app.doc 1 Extended TMAP 415 is used by players that are equipped to read extended z time-code numbers. In particular, extended TMAP 41 S associates VOBUs with extended time-code numbers. Legacy players that are not equipped to read a extended time-code numbers, use IFO file 425 (TMAP 410) and ignore extended s TMAP 415.
s Extended TMAP 415 includes entry 1 450(1), entry 2 450(2), ..., entry N
450(N). Entries 450 associate particular elements with particular linear time-code s numbers as provided by column linear time-code 455. Each of the entries 450 are s further identified by an element column 460. Entries 450 may be differentiated ~o based on a prefix or title value provided by title (prefix) 465 describing the applicable DVD title that the particular entry supports. Further a suffix 470 ~z differentiates entries based on suffix values such as "language, camera angle, and i3 parental block". Title (prefix) 465, linear time-code 455, and suffix 470 make up 1a the extended time-code structure. For each entry 450 a pointer is provided by ~s pointer 445. Pointers of entries 450 point to particular VOBUs in table 430.
Physical access to the video data corresponding to the extended time-code is completed by accessing the LSN 442 associated with the VOBU 440 in table 430 is and its linked extended time-code in table 415.
1s IFX file 405 is structured so that IFO 405 (TMAP 410) is made available to zo legacy players that are not capable of reading extended time-code numbers.
21 Extended TMAP 415 is appended to IFO file 405 to provide extended time-code zz numbers, providing additional functionality. The extended time-code numbers za provide functionality to present and search for (i.e., navigate) DVD
content based za on extended time-code numbers. Extended time-code numbers allow DVD titles 25 to be particularly identified by the title prefix of the structure of the extended time-~ee~hayes wk somz.ezss I5 G: IMSI-III444usIMSl-1444US.pat.app.doc 1 code number, and refer to elements of particular titles by the linear time-code that z is part of the extended time-code number. Further, additional elements may be s included (i.e., interspersed with) with DVD content of a particular title described a by extended time-code, where the additional elements include extended time-code numbers that identify where they are to be played in relation to elements of a s particular title.
Exemnlary System s Fig. 5 shows a system 500 of multimedia devices and broadcasters to s provide multimedia presentations. A network 505 includes local area, wide area, 1o private, and public/private networks, including the Internet. Hardwired (e.g., 1i cable) and/or wireless networks may make up network 505. Network SOS is 12 designed to provide multimedia communications from and to various entities.
i 3 A television broadcast point S 10 provides "multimedia presentation i4 content" in system 500. A television broadcast point 510 plays multimedia elements from an A/V source 515. A number of multimedia elements such as A/V
1s elements make up a total A/V presentation from A/V source 515. A/V elements may be associated with a particular time-code number. Television broadcast point is 510 may intersperse A/V elements from A/V source 515 with elements such as is commercials or promotions (i.e., interspersed elements) from an added content 2o source 520. The interspersed elements may not have time-code numbers 2~ associated with them. Collectively the A!V elements and the added elements are 22 referred to as multimedia presentation content or MPC. A/V source 515 and zs added content source 520 may stream at different times, therefore may be received 2a at different times at various multimedia devices. An extended time-code number lee~hayescue sosomans 16 G:VMSI-J11444usIMSI-1444US.pat.app.doc i as defined in Fig. 3B may be appended to each AlV element and interspersed z element as they are streamed from television broadcast point 510.
Television broadcast point 510 is connected to a satellite dish 525 which a uplinks to a satellite 530. Satellite dish 525 transmits signals carrying MPC to s satellite 530. A satellite dish 535 downlinks the signals carrying MPC from s satellite 530. Network 505 is directly connected to satellite dish 535 to receive the MPC. Alternatively network 505 may be directly connected to television 8 broadcast point 510. Television broadcast point 510 may also transmit radio s frequency (RF) signals carrying the MPC through an antenna 540. Such RF
yo signals may be received by various multimedia devices that include televisions.
i ~ A multimedia device 545 is a standalone unit not coupled to network 505, ~z but is able receive the MPC from television broadcast point 510. Multimedia is device 545 includes an A/V display unit 545-A which may be a television set or 1a monitor with a tuner and A!V inputs and outputs. Multimedia device 545 may 1s further include a multimedia player 545-B such as a DVD player, and a s multimedia recorder 545-C. Multimedia device 545 is connected to an antenna 550 that receives RF signals from sources such as antenna 540, where the RF
~a signals carry MPC. MPC may further be received by multimedia device 545 1s through a satellite dish 555, where satellite dish 555 receives signals from satellite zo 530.
zi In certain cases, multimedia device 545 receives A/V elements and zz interspersed elements without any associated time-code number. An extended z3 time-code number may be attached to each A/V element and interspersed element za as they are received at multimedia device 545. The A/V elements and interspersed leefs~hayes wa soa.m..szss 17 G:IMSI-lll444ttsIMSl-1444US.pat.app.doc 1 elements may be recorded by multimedia recorder 545-C, and accessed by z reference to unique extended time-code numbers.
Broadcast of MPC may also be performed through the Internet. Typically, a this is known as a "web broadcast" and provides digital transmission of MPC.
A
s web broadcast point 560 may include an interactive web site accessible through s network 505. Web broadcast 560 receives A/V presentation from an AIV source 565 with interspersed elements that can include text, graphics, audio/video clips, s and/or HTML or SMIL files from an added content source 570. Alternatively s such elements may come from other sources.
io Web broadcast point 560 delivers MPC (i.e., A/V elements and interspersed 1 ~ elements) to network 505 for access by various multimedia devices connected to 1z network 505. Alternatively, web broadcast point 560 may deliver MPC through a 1s satellite as described above.
1a A multimedia device 575 is coupled to network 505 and receives MPC
is from various sources including television broadcast point 510 and web broadcast ~s point 560. Multimedia device 575 includes an A/V display unit 575-A which may i~ be a television set or monitor with a tuner and A/V inputs and outputs.
is Multimedia device 575 may further include a multimedia player 575-B such as a ~s DVD player, and a multimedia recorder 575-C.
zo In certain cases, multimedia device 575, similar to multimedia device 545, z~ may receive A/V elements and interspersed elements without any time-code zz number associated. An extended time-code number is attached to each A/V
z3 element and interspersed element as they are received at multimedia device 575.
za The AlV elements and interspersed elements may be recorded by multimedia zs recorder 575-C, and accessed by reference to unique extended time-code numbers.
~eephayes wk sosaa'sus 1 8 G:IMSI-III444usIMS!-I444USpa(.app.doc 1 In other cases, multimedia devices 545 and 575 receive A/V elements and/or z interspersed elements that have extended time-code numbers. Sequencing of and 3 playing the presentation at multimedia devices 545 and 575 are based on the a extended time-code numbers.
s Multimedia devices and standalone units include WebTV units, and s personal video recorder (PVR) units. In this example a PC 580 is coupled to network 505 and receives MPC. Like multimedia devices 545 and 575, PC 580 a may attach extended time-code numbers to A/V elements and interspersed s elements that are received without extended time-code numbers.
1o Various sources may provide other content, including updates to files and i1 programs, used by multimedia devices. This content may not necessarily be 1z multimedia elements or A/V elements and can include IFX files as described in is Fig. 4. In this example, a web site 585 illustrates a source from which a ~a multimedia device accesses additional content such as updated files.
Further, i5 broadcast points such as television broadcast point 510 and web broadcast point is 560 may stream new A/V elements to be sequenced with pre-existing content at ~ various multimedia devices.
is Exemplary Multimedia Device is Fig. 6 shows an architecture for an exemplary multimedia device 600.
zo Multimedia devices 545, 575, and 580 of Fig. 5 are representative of the z1 architecture of multimedia device 600. Multimedia device 600 may be a distinct zz self contained unit such as PC 580, or may be comprised of various units such as zs multimedia devices 545 and 575. As will be evident below, multimedia 600 za provides functionality to receive MPC, process MPC, and display MPC. In certain z5 cases, multimedia device 600 also records MPC.
tee~nayes w~ sosaz,ezss 19 G: IMSI-1I1444uslMSl-1444USpat.app.doc Multimedia device 600 may include a receiver module 605 that is z configured to receive RF signals from a television source. Other signals and data 3 may also be received from various sources, including devices and networks, a through an audio video input module 610.
Multimedia device 600 includes one or more processors 615 (e.g., s microprocessors) which process various instructions to control the operation of multimedia device 600 and to communicate with other electronic and computing a devices.
When A/V elements and interspersed elements are received without time code numbers, such numbers may be attached to each through the use of an 11 extended time-code module 620. Extended time-code module 620 is configured to 1z generate and attach a unique extended time-code number to each A/V element and 1s each interspersed element as they are received by multimedia device 600.
is Multimedia device 600 can be implemented with one or more memory i5 components, examples of which include a non-volatile memory 630 (e.g., ROM, is Flash, EPROM, EEPROM, etc.). Memory 630 is configured to store an operating system module 635 that contains an operating system to be used on multimedia is device 600. Application programs may be stored in an applications program module 640 of memory 630. Random access memory (RAM) may also be zo implemented in multimedia device 600 for temporary storage of files and 2i programs used by multimedia device 600. A RAM module 645 is illustrated.
zz MPC may be stored in a multimedia storage module 650. Multimedia zs storage module 650 may make use of various media such as optical disks, za magnetic tape, and magnetic disk. Multimedia storage module 650 may be zs accessed by other modules and devices that are part of multimedia device 600, and lee6°~shayes ~s~ sos.u.azss 20 G:IMSl-I11444uslMSl-1444US.pal.app.doc ~ devices external to multimedia device 600. Multimedia recorder module 655 z provides recording capability to multimedia storage 650. A multimedia player 3 module 660 may either directly receive MPC from processors) 615 or access

4 MPC from multimedia storage 650 for MPC. Processors) 615 may access AlV
elements and elements in multimedia storage module 650 by identifying particular s extended time-code numbers.
An audio/video output module 665 provides direct output to a display or to s an intermediate device. Output module 665 may split audio and video information s and stream them to individual channels. Processors) 615 controls output module ~0 665 and the A/V streams sent from output module 665.
1 ~ Exemplary Multimedia Player Fig. 7 shows an architecture for an exemplary multimedia player 700.
is Multimedia player 700 includes various devices, but in particular includes DVD
i4 players that utilize industry standards, in particular standards set by the National i5 Televisions Systems Committee (NTSC). The architecture of multimedia player is 700 may be used by multimedia player 660 of Fig. 6. Multimedia player 700 may ~ read from prerecorded media such as a commercially produced DVD, or from is media that may be written to such as recordable DVDs, or receive A/V
element is and interspersed element streams.
2o One or more processors 705 may be used to process signals received from a z1 medium (e.g. DVD), and to communicate with various modules of multimedia 22 player 700. In cases where multimedia player 700 is part of multimedia device 23 700, processors) 615 illustrated in Fig. 6 may be used for processors) 705.

IeeNylhayes y sosamexss 21 G: IMS!-lll444uslMS!-l444US.pat.app.doc i Processor(s) 705 process and sequence A/V elements appended extended z time-code numbers. In the case of DVD, IFX files are used by processors) 705 to sequence A/V elements.
Multimedia player 700 may include a memory module 710 which stores an s operating system module 715 that contains an operating system to be used on s multimedia player 700. Memory module 710 may further include a fileslprograms module 720 that contains applicable files and/or programs for multimedia player a 700.
s An AIV element or element synchronizer module 725 is used to synchronize separate streams of audio, video, and/or other data based on extended time-code numbers. Synchronizer module 725 is particular used to perform late 1z binding of streams of A/V elements and/or elements that were previously not 13 synchronized. For example, multimedia player may be receiving a stream is containing a new language track that is to be synchronized with video that is either 15 resident at multimedia player 700 or received from another source.
Synchronizer 1s module 725 synchronizes the audio and video based on the extended time-code » number. In particular, a linear time-code number of the extended time-code 1s number is used to synchronize the audio and video.
1s Multimedia player 700 may further include a presentation engine module z~ 730 that uses presentation data from the media to control what is displayed from z1 multimedia player 300 as derived from information in an IFO or IFX file. A
zz navigation manager module 735 uses navigation data to provide an interface, z3 create menus, and control any branching of A/V elements or scenes which are also za derived from an IFO or IFX file. Further, an audiolvideo output module 740 is zs IeeQ~hayes one sos.msezss 22 G: IMSI-111444usIMSI-1444US.pat.app.doc provided in multimedia player 700 to output signals to a display device, or an z intermediate device prior to the display device.
3 Adding extended time-code a Fig. 8 shows a process 800 which creates or adds extended time-code s values. Process 800 may be performed at a broadcast point such as television s broadcast point 510 or web broadcast point 560 of Fig. 5. In certain embodiments, process 800 is performed at a multimedia device such as multimedia devices 545, s 575 and 580 as shown in Fig. 5, and described by the architecture of multimedia s device 600 as shown in Fig. 6. In other cases, process 800 is performed when a 1o DVD or other media is created.
At block 805, a decision is made if an AN element or interspersed element has a linear time-code number appended to it. The linear time-code number may 13 be defined by a SMPTE standard. If the A/V element or interspersed element has is a linear time-code number (following the YES branch of block 805), the linear 15 time-code number is kept. If the A/V element or interspersed element does not is have a linear time-code number (following the NO branch of 805), a linear time-code is attached at block 810. At block 810, the linear time-code that is attached js describes when the particular A!V element or interspersed element is to be played.
~s At block 815, a title value is added to the linear time-code of a particular zo A/V element or interspersed element. An extended time-code number is created.
zi The title value differentiates among different MPC presentations.
zz At block 820, a decision is made if applicable suffixes (i.e., suffix fields) 23 exists that further define an A/V element or interspersed element. Such suffixes include angle, language, and parental block (i.e. rating). If a suffix applies (the zs YES branch of block 820), block 825 is performed, and a suffix value is added to lee~hayesyk sosazsezss 23 G:IAfSl-l1I444us1MSl-I444USpat.app.doc i the extended time-code number. Block 820 is performed until all applicable suffixes are determined. Each applicable suffix is concatenated to the extended time-code number, until all suffix values have been determined (the NO branch of d block 820). In certain cases, particular suffix fields will not apply to certain A/V
elements or interspersed elements.
s Searching based on extended time-code Fig. 9 shows a process 900 which searches for an element based on an a extended time-code number. Process 900 may be performed at a multimedia s device such as multimedia devices 54~, 575 and 580 as shown in Fig. 5, and io described by the architecture of multimedia device 600 as shown in Fig. 6.
11 Multimedia player ?00 of Fig. 7 may also use process 900 to search for particular ~2 AIV elements andlor interspersed elements based on title prefix and/or suffix 13 values.
is I At block 905, the title value of the extended time-code number is 1511 determined. The title value distinguishes A/V elements and interspersed elements that are part of a particular MPC from other MPCs. For example, a search may be i7 made as to a particular movie title, or to particular commercials.
1B Several suffixes may apply in the selection of a particular A/V element 19 and/or an interspersed element. For example, it may be desirable to search based 20 on a particular language, a particular angle, a particular parental block, or a 2i combination of the three.
22 At block 910, a decision is made as whether a particular suffix applies.
23 Although suffixes may proceed or follow other suffixes in an extended time-code 2a number structure, suffixes have the same hierarchy. Therefore searches may be 25 made for any particular suffix value.
lee~hayesou~ sasaz,9zss 24 G:IMSl-111444usIMSI-1444USpar.app.doc 1 If the search is based on a particular suffix value (the YES branch of block 2 910), block 915 is performed. At block 915 the particular suffix value is 3 determined. If all suffix field search values have been presented (the NO
branch 4 of block 910), then block 920 is performed. At block 920, all A/V elements and s interspersed elements are selected based on the determined title and suffix values.
s It may be desirable to complete process 900 at this point and provide all A/V
elements and interspersed elements that meet the value determinations.
A further step may be taken to select a particular AN element or s interspersed element, which can include a particular scene, based on a linear time-1o code number. At block 925, the searching is narrowed down to a particular linear 11 time-code number. If further search is performed based on the linear time-code 1z (following the YES branch of block 925), then block 930 is performed identifying 13 the particular element. For example, process 900 may be used to perform a search 14 for a particular element based on a particular scene of a particular variation of a 15 particular movie title.
is Although the invention has been described in language specific to structural 1~ features andlor methodological acts, it is to be understood that the invention 18 defined in the appended claims is not necessarily limited to the specific features or 1s acts described. Rather, the specific features and acts are disclosed as exemplary 2o forms of implementing the claimed invention.

lee(~hayeso~ sosamsus 25 G:IMSI-Ii1444usIMSl-I444USpat.app.doc

Claims (42)

1. A method comprising:
identifying multimedia elements having a linear time-code number;
adding a prefix value to linear time-code numbers of each identified multimedia element; and adding suffix values to the linear time-code numbers of each identified multimedia element.

2. The method of claim 1 wherein the multimedia elements comprise audio video elements and interspersed elements.

3. The method of claim 1 wherein the prefix value comprises a title value.

4. The method of claim 1 wherein the suffix values comprise language value, angle value, and parental block value.

5. The method of claim 1 wherein linear time-code numbers with prefix and suffix values are resident in a time map table that associates the multimedia elements to linear time-code numbers.

6. The method of claim 5 wherein the time map is used by a DVD
player to point to particular sectors on a DVD disc containing content representing the multimedia elements.

7. The method of claim 5 wherein the time map table is part of a file that provides pointers to data structures in a medium.

8. A DVD player that implements the method of claim 7, and wherein the medium is a DVD disc.

9. A broadcast point that performs the method of claim 1.

10. A multimedia device that performs the method of claim 1.

11. The method of claim 1 further comprising identifying elements without a linear time-code number, and adding a linear time-code to the identified elements without a linear time-code.

12. The method of claim 11 wherein the elements are interspersed between audio video elements that have linear time-code numbers.

13. A method comprising:
identifying a title value describing a particular multimedia presentation content comprised of multimedia elements described by extended time-code numbers; and searching for particular multimedia elements based on their extended time-code numbers.

14. The method of claim 13 wherein the extended time-code numbers comprise a prefix and suffixes.

15. The method of claim 13 wherein the searching is performed based on the prefix and on one or more of the suffixes.

16. The method of claim 13 wherein the suffixes comprise angle, language, and parental block.

17. The method of claim 13 wherein the searching is performed based on a time map table that associates multimedia elements with extended time-code numbers.

18. A DVD player that implements the method of claim 17.

19. A personal video recorder that implements the method of claim 17.

20. The method of claim 17 wherein the time map table is part of an information file that provides navigation and presentation information for titles in a medium.

21. A DVD player that implements the method of claim 20, wherein the medium is a DVD disc.

22. A personal video recorder that implements the method of claim 20.

23. A multimedia device that performs the method of claim 13.

24. A multimedia player that performs the method of claim 13.

25. A method comprising:
receiving multimedia elements with a linear time-code;
receiving multimedia elements without a linear time-code;
attaching linear time-codes; and synchronizing all multimedia elements based the unique linear time-codes.

26. A broadcast point that performs the method of claim 23.

27. A multimedia device that performs the method of claim 23.

28. A multimedia player that performs that method of claim 23.

29. A method comprising:
extending an information file in a DVD to provide added functionality in describing content stored on the DVD, and providing navigation and presentation information from the information file for playback of various titles.

30. The method of claim 29 wherein the extending the information file is performed by extended time-code numbers that describe sequencing of the content.

31. The method of claim 29 wherein the extending the information files adds a time map table that describes sequencing of the content.

32. The method of claim 29 wherein the extending the information files is performed by receiving information describing the added functionality from a separate source from the DVD.

33. A multimedia device comprising:
a processor;
a multimedia storage module executable on the processor and configured to store multimedia presentation content comprising of multimedia elements; and an extended time-code number module executable on the processor configured to append extended time-code numbers to multimedia element without a time-code number.

34. The multimedia device of 33 further comprising a multimedia player module executable on the processor and configured to play the audio/video element and interspersed elements.

35. The multimedia device of claim 33 wherein the multimedia player is configured to search for audio/video element and interspersed elements based on extended time-code numbers.

36. The multimedia device of claim 33 further comprising a multimedia player.

37. The multimedia device of claim 33 wherein the multimedia device is a DVD player.

38. The multimedia device of claim 33 wherein the multimedia device is a personal video recorder.

39. A computer-readable medium having stored thereon a data-structure comprising:
a first data field containing a title value;
a second data field containing a time-code value; and a third data field containing a suffix value.

40. A system comprising:
a broadcast point providing multimedia elements; and a multimedia device that receives the multimedia elements, wherein the multimedia device further receives extended time-code numbers associated with each multimedia element.

41. The system of claim 40 wherein the extended time-code numbers are provided by the broadcast point.

42. The system of claim 40 wherein the extended time-code numbers are provided by a web site.