WO2002011428A2 - Visualization and playback of television shows at a sub-show level - Google Patents

Visualization and playback of television shows at a sub-show level Download PDF

Info

Publication number
WO2002011428A2
WO2002011428A2 PCT/US2001/023800 US0123800W WO0211428A2 WO 2002011428 A2 WO2002011428 A2 WO 2002011428A2 US 0123800 W US0123800 W US 0123800W WO 0211428 A2 WO0211428 A2 WO 0211428A2
Authority
WO
WIPO (PCT)
Prior art keywords
content
segments
segment
user
segmenting
Prior art date
Application number
PCT/US2001/023800
Other languages
French (fr)
Other versions
WO2002011428A3 (en
Inventor
George Marmaropoulos
Clive R. Van Heerden
John Zimmerman
Original Assignee
Koninklijke Philips Electronics N.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics N.V. filed Critical Koninklijke Philips Electronics N.V.
Priority to EP01959295A priority Critical patent/EP1308028A2/en
Priority to JP2002515823A priority patent/JP2004521525A/en
Publication of WO2002011428A2 publication Critical patent/WO2002011428A2/en
Publication of WO2002011428A3 publication Critical patent/WO2002011428A3/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/233—Processing of audio elementary streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234336—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by media transcoding, e.g. video is transformed into a slideshow of still pictures or audio is converted into text
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
    • H04N21/2353—Processing of additional data, e.g. scrambling of additional data or processing content descriptors specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • H04N21/25866—Management of end-user data
    • H04N21/25891—Management of end-user data being end-user preferences
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/426—Internal components of the client ; Characteristics thereof
    • H04N21/42661—Internal components of the client ; Characteristics thereof for reading from or writing on a magnetic storage medium, e.g. hard disk drive
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440236—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/4508—Management of client data or end-user data
    • H04N21/4532—Management of client data or end-user data involving end-user characteristics, e.g. viewer profile, preferences
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/482—End-user interface for program selection
    • H04N21/4828—End-user interface for program selection for searching program descriptors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/84—Generation or processing of descriptive data, e.g. content descriptors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/454—Content or additional data filtering, e.g. blocking advertisements

Definitions

  • the present invention relates generally to a user interaction paradigm related to the customization of the TV watching process.
  • Digital television and digital information has changed and will continue to change home entertainment by offering a vast amount of digital content.
  • the number of channels and variety of content available will become so large that it will be impossible for any viewer to browse the content using conventional channel surfing methods.
  • Electronic program guides such as those described in U.S. Patent No. 4,706,121 and 5,353,121, herein incorporated by reference, provide schedule information processing systems which provide a viewer with a convenient way to select programs based on viewer supplied selection criteria.
  • EPGs even EPGs, despite their current developments will be insufficient to provide a level of personalization for the copious information which will soon become available.
  • a method of displaying content at a segment level includes several steps .
  • One step is segmenting the content into at least two segments.
  • Another step is displaying a representation of at least one of the at least two segments.
  • An additional step is indicating how the at least one of the at least two segments relate to the content.
  • content relates to any type of programming or information that can be broadcast, stored by a user, or otherwise transmitted to a user.
  • the segmenting step further includes segmenting based on pixel color analysis in a sampled frame, dialogue analysis, object recognition techniques, written content descriptions, closed captioning, speech to text, MPEG 7, or audio analysis. Segmenting can be performed according to any known method.
  • the indicating step may include symbolically indicating the length of each segment or the length of a segment in relation to the length of the content in which it is contained.
  • the indicating step also may include indicating the temporal position of a segment within the content in which it is contained.
  • the indicating step may include generating symbols that may be pictures or text. A user can search for content using the symbols.
  • a user can select the representation of a segment to display (i.e., play) the segment in a full, or partial format .
  • the method includes storing a segment in a storage medium. A user may also store and display more than one segment.
  • a system for displaying content at a segment level includes at least one source of content a means for segmenting the content, and a display means for displaying at least one representation of segmented content.
  • the display means is a television or monitor.
  • the content is at least one source of television programming, or source of linear media.
  • the source of television programming or streaming media is a satellite source, a land based source, or a broadcast source.
  • the means for segmenting content operates at a content head end, at a user's location, or both a content head end and a user's location.
  • the means for segmenting content segments content automatically based on a user's individual viewing habits. Further, the display means displays at least one of the at least two segments .
  • a program stored on a processor-readable medium for displaying content at a segment level includes portions for carrying out steps.
  • the program segment includes portions for segmenting content into at least two segments, portions for displaying at least one of the at least two segments, and portions for indicating how the at least one of the at least two segments relates to the content.
  • Fig. 1 illustrates a preferred embodiment of a user interface incorporating segment level representations
  • Fig. 2 illustrates a preferred embodiment of a user interface on which a segment representation contains additional information
  • Fig. 3 illustrates a preferred embodiment of a user interface that allows a user to search content by segments
  • Fig. 4 illustrates a preferred embodiment of a system for displaying content at a segment level
  • Fig. 5 illustrates a block diagram of a program stored on a processor-readable medium for displaying content at a segment level .
  • the proposed method, system, and program provides a television viewer with the ability to select content at a sub-show level, hereinafter called “segment level.”
  • Fig. 1 illustrates a preferred embodiment of the proposed user interface incorporating segment level representations.
  • a user begins by selecting from category menu 10 on user interface 100.
  • the user can search for content within a category.
  • the user selects talk shows using a conventional input device, such as a remote control or a keyboard. This reveals content (i.e., television programs represented by icons) which fit the selected talk show category. Each portion of content that fits the category may be selected.
  • the user selects icon 11 using a conventional input device, such as a remote control or a keyboard. Icon represents a different portion content (i.e., different program) .
  • Icon 11 corresponds to the television program "The Late Show with David Letterman, " hereinafter "The Late Show.” This program includes various segments where guests are interviewed, musicians play, comedians perform, etc.
  • One method breaks content down into segments by perusing closed captioning associated with content to identify cues where segments begin and end.
  • a cue to identify where a segment begins can be the words "and my next guest tonight is ⁇ ” .
  • a cue to identify the end of a segment can be where sustained applause accompanies the words "Thank youg. "
  • Another method identifies segments from the inherent structure of a program. This is achieved, for example, by identifying a musical performance in a talk show by recognizing a sustained portion where music is played. The cues would be when the music begins and ends.
  • Another method recognizes sounds associated with segments, such as a whistle being blown at the beginning and end of a quarter of a football game, or portions where applause meets a threshold volume (i.e., notable plays in a sporting event) .
  • sounds associated with segments such as a whistle being blown at the beginning and end of a quarter of a football game, or portions where applause meets a threshold volume (i.e., notable plays in a sporting event) .
  • a threshold volume i.e., notable plays in a sporting event
  • segment display region 12 displays segment representations 13-17.
  • Segment representations 13-17 represent various segments of The Late Show.
  • segment representation 13 corresponds to the segment of The Late Show where Dolly Parton is the guest .
  • - Segment representation 14 corresponds to the segment of The Late Show where Drew Carey appears . If a user wishes to watch one or more segments of The Late Show, they can select the corresponding segment representations. Once selected, the segment may be displayed or stored. As an example, assume a user selects segment representation 13 from segment display region 12. The user can then watch that segment, compile a series of selected segments and watch them all as an editable montage, or store the segment for later viewing.
  • the area of a segment representation is proportional to the temporal length of a segment.
  • a brief viewing of segment display region 12 reveals that segment representations 13 and 14 are longer than segment representations 15 and 16.
  • a user can quickly and easily compare the length of a segment representation to the length of an entire portion of content (i.e., a program) .
  • a user may approximate that segment representation 13 is about 1/5 the length of segment display region 12. Consequently the user can determine since The Late Show is 1 hour long, the segment corresponding to segment representation 13 is 1/5 of 1 hour, or 12 minutes long.
  • a user highlights a segment representation they can be provided with the segment length and the segment length in comparison to the entire content portion.
  • FIG. 2 illustrates a user interface 101 on which a segment representation provides a user with detailed information about the segments of a content portion.
  • Display region 12 again contains segment representations 13-17 that are indicated by icons, such as icon 21.
  • information may be provided, such as information regarding the celebrity, the cues that form the segment, the theme, etc.
  • Fig. 2 when a user highlights segment representation 13 corresponding to Dolly Parton, information 20 appears which aides a user in deciding whether to store or view the highlighted segment.
  • Displaying segment representations as indicated above is useful for an additional reason.
  • a content portion places greater emphasis on certain segments contained therein. For example, a news program will place the greatest emphasis on the "top stories” segment. Further, the "top stories” will usually occur early in the content portion. Displaying content at a segment level allows a user to understand where a segment exists with regard to the entire content portion and how significant the segment is based on its relation to the entire content portion.
  • Fig. 3 displays a user interface that allows a user to search content by segments.
  • a user searched for content portions relating to finance, i.e., from category menu 10 in Fig. 1.
  • Content portions represented by icons 31-33 were found according to the user's search.
  • the user can then further search content portions 31-33 on a segment level.
  • the user performs this segment level search by selecting a specific segment 37 from search engine 36.
  • the user only wishes to find segments relating to Nokia within content portions relating to finance.
  • segment representations 38 and 39 were found.
  • the user may observe that no segments in the content portion represented by icon 33 contain th ' e desired segments.
  • the user can visualize that the content portion corresponding to icon 32 includes a segment (i.e., segment representation 39) regarding Nokia which is likely more detailed than the content portion regarding Nokia corresponding to icon 31.
  • the user may wish to see or store both the segments found in their search, only the shorter segment (i.e., corresponding to segment representation 38), or only the longer segment (i.e., corresponding to segment representation 39) .
  • Fig. 4 displays a system for displaying content at a segment level.
  • Display screen 401 displays the user interface 101.
  • Segment representations and menus may be selected, edited, and otherwise manipulated using a cursor (not shown) .
  • the cursor may be moved left, right, up, and down using a conventional input device such as a remote control 47 or any other customary input device known in the art .
  • microprocessor 45 scans content source 44 for the selected content portions. Microprocessor 45 then segments the selected content portions according to the user's input. This can occur, for example, if the user selects an icon representing a content portion. Then microprocessor 45 will segment the program as . indicated above, and display the segments as segment representations. Alternatively, microprocessor 45 can scan content source 44, and segment programs across more than one category.
  • microprocessor 45 may direct display 401 to display the segment.
  • Microprocessor '45 may also store a selected segment in storage medium 46.
  • Microprocessor 45 can also suggest segments automatically based on the user's individual viewing habits.
  • Segments level displaying relates more generally to all forms of streaming media, such as radio, and streaming feeds from the Internet.
  • a user interface for a radio device such as a digital cable radio feed.
  • The. radio content portions can be segmented, searched, and displayed for selection by a user.
  • the block diagram of Fig. 5 depicts a program stored on a processor- eadable medium for displaying content at a segment.
  • the program queries whether the user selected a category of content in step 500. If not, the program directs microprocessor 45 to generate a category menu in step 502. Once generated, the program directs microprocessor 45 to request and receive input in steps 503 and 504, respectively. If the user selected a category in step 500, or once input is received in step 503, the program directs microprocessor 45 to search and/or filter content based on the selected category in step 501.
  • the program queries the user whether they wish to search segments within the category results. This occurs in step 505. If the user does not wish to search segments within the selected category, the program queries whether an icon representing a content portion has been selected. This occurs in step 506. If not the program awaits icon selection in step 507. When the user selects an icon, the content portion the icon represents is segmented in step 508. If the user wishes to search segments within the selected category, the program directs microprocessor 45 to segment all content portions within the category in step 509. In step 510, the program queries whether the user selected a segment category. If not, the program directs microprocessor 45 to generate a segment category menu in step 511.
  • the program directs microprocessor 45 to request and receive input in steps 512 and 513, respectively. If the user selected a segment category in step 510, or once input is received in step 513, the program directs microprocessor 45 to search and/or filter segments based on the selected segment category in step 514. Once the content portion the icon represents is segmented in step 508, or the program directs microprocessor 45 to search and/or filter segments based on the selected segment category in step 514, the program directs display 401 to display segments as segment representations in step 515.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Computer Graphics (AREA)
  • Library & Information Science (AREA)
  • Human Computer Interaction (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • User Interface Of Digital Computer (AREA)
  • Indexing, Searching, Synchronizing, And The Amount Of Synchronization Travel Of Record Carriers (AREA)
  • Television Systems (AREA)

Abstract

A method and system display content, such as television programs, at a segment level. This includes segmenting the content into at least two segments, displaying a representation of at least one of the at least two segments, and indicating how the at least one of the at least two segments relate to the content. The term 'content' relates to any type of programming or information that can be broadcast, stored by a user, or otherwise transmitted to a user.

Description

VISUALIZATION AND PLAYBACK OF TELEVISION SHOWS AT A SUB-SHOW LEVEL
CROSS REFERENCE TO RELATED APPLICATIONS This application is a nonprovisional application of U.S. provisional and parent application "VISUALIZATION AND PLAYBACK OF TELEVISION SHOWS AT A SUB-SHOW LEVEL," U.S. Serial No. 60/221,404, filed July 28, 2000 having George Marmaropoulos and John Zimmerman as the inventors and is hereby incorporated by reference in its entirety.
BACKGROUND OF THE INVENTION The present invention relates generally to a user interaction paradigm related to the customization of the TV watching process. Digital television and digital information has changed and will continue to change home entertainment by offering a vast amount of digital content. The number of channels and variety of content available will become so large that it will be impossible for any viewer to browse the content using conventional channel surfing methods. Electronic program guides (EPGs) , such as those described in U.S. Patent No. 4,706,121 and 5,353,121, herein incorporated by reference, provide schedule information processing systems which provide a viewer with a convenient way to select programs based on viewer supplied selection criteria. However, even EPGs, despite their current developments will be insufficient to provide a level of personalization for the copious information which will soon become available. Various devices and systems, such as personal TV recorders and agent technologies offer a certain level of personalization. These systems operate at the meta-data level using EPGs and user preferences. Such devices normally have a hard disk with the ability to store several hours of video content in order to facilitate the process of viewing the preferred programs. However, new paradigms that allow the users to intuitively search and find the specific content that they would like to watch or hear are becoming necessary. Currently, users conceptualize content in terms of individual, indivisible shows. Consequently, users can only chose to watch or hear content at a show level .
SUMMARY OF THE INVENTION According to one aspect of the invention, a method of displaying content at a segment level includes several steps . One step is segmenting the content into at least two segments. Another step is displaying a representation of at least one of the at least two segments. An additional step is indicating how the at least one of the at least two segments relate to the content.
The term "content" relates to any type of programming or information that can be broadcast, stored by a user, or otherwise transmitted to a user.
In some embodiments of the invention the segmenting step further includes segmenting based on pixel color analysis in a sampled frame, dialogue analysis, object recognition techniques, written content descriptions, closed captioning, speech to text, MPEG 7, or audio analysis. Segmenting can be performed according to any known method.
In some other embodiments of the invention, the indicating step may include symbolically indicating the length of each segment or the length of a segment in relation to the length of the content in which it is contained. The indicating step also may include indicating the temporal position of a segment within the content in which it is contained.
In some other embodiments, the indicating step may include generating symbols that may be pictures or text. A user can search for content using the symbols.
In another embodiment, a user can select the representation of a segment to display (i.e., play) the segment in a full, or partial format . In another embodiment, the method includes storing a segment in a storage medium. A user may also store and display more than one segment.
In another aspect of the invention, a system for displaying content at a segment level includes at least one source of content a means for segmenting the content, and a display means for displaying at least one representation of segmented content.
In some embodiments, the display means is a television or monitor. The content is at least one source of television programming, or source of linear media. Further, the source of television programming or streaming media is a satellite source, a land based source, or a broadcast source. In other embodiments, the means for segmenting content operates at a content head end, at a user's location, or both a content head end and a user's location.
In other embodiments the means for segmenting content segments content automatically based on a user's individual viewing habits. Further, the display means displays at least one of the at least two segments .
In another aspect of the invention, a program stored on a processor-readable medium for displaying content at a segment level includes portions for carrying out steps. The program segment includes portions for segmenting content into at least two segments, portions for displaying at least one of the at least two segments, and portions for indicating how the at least one of the at least two segments relates to the content. The invention provides several advantages. For example purposes only, the system, method, and program segment are capable of segmenting content and displaying segmented content. Users can select only the segments that they wish to watch, and can search vast amounts of content for only specific segments. This will make television viewing easier and more personalized as the number of channels and volume of information continues to grow. Further, copious information can be organized and viewed quickly and simply, without requiring excess scanning. These and many other advantages are elucidated with reference to the embodiments below.
BRIEF DESCRIPTION OF THE DRAWING Fig. 1 illustrates a preferred embodiment of a user interface incorporating segment level representations;
Fig. 2 illustrates a preferred embodiment of a user interface on which a segment representation contains additional information; Fig. 3 illustrates a preferred embodiment of a user interface that allows a user to search content by segments;
Fig. 4 illustrates a preferred embodiment of a system for displaying content at a segment level;
Fig. 5 illustrates a block diagram of a program stored on a processor-readable medium for displaying content at a segment level .
DETAILED DESCRIPTION OF THE PRIMARY EMBODIMENT The proposed method, system, and program provides a television viewer with the ability to select content at a sub-show level, hereinafter called "segment level."
Fig. 1 illustrates a preferred embodiment of the proposed user interface incorporating segment level representations. A user begins by selecting from category menu 10 on user interface 100. Here the user can search for content within a category. In Fig. 1, the user selects talk shows using a conventional input device, such as a remote control or a keyboard. This reveals content (i.e., television programs represented by icons) which fit the selected talk show category. Each portion of content that fits the category may be selected. In Fig. 1, the user selects icon 11 using a conventional input device, such as a remote control or a keyboard. Icon represents a different portion content (i.e., different program) .
Icon 11 corresponds to the television program "The Late Show with David Letterman, " hereinafter "The Late Show." This program includes various segments where guests are interviewed, musicians play, comedians perform, etc.
There' are several known methods of segmenting content for display at a segment level. One method breaks content down into segments by perusing closed captioning associated with content to identify cues where segments begin and end. As an example, a cue to identify where a segment begins can be the words "and my next guest tonight is§" . A cue to identify the end of a segment can be where sustained applause accompanies the words "Thank youg. " Another method identifies segments from the inherent structure of a program. This is achieved, for example, by identifying a musical performance in a talk show by recognizing a sustained portion where music is played. The cues would be when the music begins and ends. Another method recognizes sounds associated with segments, such as a whistle being blown at the beginning and end of a quarter of a football game, or portions where applause meets a threshold volume (i.e., notable plays in a sporting event) . Many additional methods of segmenting content are available and may be used in the present invention.
Icon 11 is selected in Fig. 1, segment display region 12 displays segment representations 13-17. Segment representations 13-17 represent various segments of The Late Show. For example, segment representation 13 corresponds to the segment of The Late Show where Dolly Parton is the guest .- Segment representation 14 corresponds to the segment of The Late Show where Drew Carey appears . If a user wishes to watch one or more segments of The Late Show, they can select the corresponding segment representations. Once selected, the segment may be displayed or stored. As an example, assume a user selects segment representation 13 from segment display region 12. The user can then watch that segment, compile a series of selected segments and watch them all as an editable montage, or store the segment for later viewing.
The area of a segment representation is proportional to the temporal length of a segment. As an example, in Fig. 1, a brief viewing of segment display region 12 reveals that segment representations 13 and 14 are longer than segment representations 15 and 16. Further, a user can quickly and easily compare the length of a segment representation to the length of an entire portion of content (i.e., a program) . As an example, a user may approximate that segment representation 13 is about 1/5 the length of segment display region 12. Consequently the user can determine since The Late Show is 1 hour long, the segment corresponding to segment representation 13 is 1/5 of 1 hour, or 12 minutes long. Alternatively, when a user highlights a segment representation, they can be provided with the segment length and the segment length in comparison to the entire content portion. Fig. 2 illustrates a user interface 101 on which a segment representation provides a user with detailed information about the segments of a content portion. Display region 12 again contains segment representations 13-17 that are indicated by icons, such as icon 21. When a user highlights a segment representation, information may be provided, such as information regarding the celebrity, the cues that form the segment, the theme, etc. In Fig. 2, when a user highlights segment representation 13 corresponding to Dolly Parton, information 20 appears which aides a user in deciding whether to store or view the highlighted segment.
Displaying segment representations as indicated above is useful for an additional reason. Generally, a content portion places greater emphasis on certain segments contained therein. For example, a news program will place the greatest emphasis on the "top stories" segment. Further, the "top stories" will usually occur early in the content portion. Displaying content at a segment level allows a user to understand where a segment exists with regard to the entire content portion and how significant the segment is based on its relation to the entire content portion.
Fig. 3 displays a user interface that allows a user to search content by segments. According to user interface 102, a user searched for content portions relating to finance, i.e., from category menu 10 in Fig. 1. Content portions represented by icons 31-33 were found according to the user's search. The user can then further search content portions 31-33 on a segment level. The user performs this segment level search by selecting a specific segment 37 from search engine 36. In Fig. 3, the user only wishes to find segments relating to Nokia within content portions relating to finance. As indicated by display regions 34 and 35, segment representations 38 and 39 were found. The user may observe that no segments in the content portion represented by icon 33 contain th'e desired segments. Further, the user can visualize that the content portion corresponding to icon 32 includes a segment (i.e., segment representation 39) regarding Nokia which is likely more detailed than the content portion regarding Nokia corresponding to icon 31. The user may wish to see or store both the segments found in their search, only the shorter segment (i.e., corresponding to segment representation 38), or only the longer segment (i.e., corresponding to segment representation 39) . Fig. 4 displays a system for displaying content at a segment level. Display screen 401 displays the user interface 101. Segment representations and menus may be selected, edited, and otherwise manipulated using a cursor (not shown) . The cursor may be moved left, right, up, and down using a conventional input device such as a remote control 47 or any other customary input device known in the art .
Once a user selects a category of content portions, microprocessor 45 scans content source 44 for the selected content portions. Microprocessor 45 then segments the selected content portions according to the user's input. This can occur, for example, if the user selects an icon representing a content portion. Then microprocessor 45 will segment the program as .indicated above, and display the segments as segment representations. Alternatively, microprocessor 45 can scan content source 44, and segment programs across more than one category.
When a user selects a segment representation, microprocessor 45 may direct display 401 to display the segment. Microprocessor '45 may also store a selected segment in storage medium 46. Microprocessor 45 can also suggest segments automatically based on the user's individual viewing habits.
Segments level displaying relates more generally to all forms of streaming media, such as radio, and streaming feeds from the Internet. For example, a user interface for a radio device (such as a digital cable radio feed). The. radio content portions can be segmented, searched, and displayed for selection by a user.
The block diagram of Fig. 5 depicts a program stored on a processor- eadable medium for displaying content at a segment. The program queries whether the user selected a category of content in step 500. If not, the program directs microprocessor 45 to generate a category menu in step 502. Once generated, the program directs microprocessor 45 to request and receive input in steps 503 and 504, respectively. If the user selected a category in step 500, or once input is received in step 503, the program directs microprocessor 45 to search and/or filter content based on the selected category in step 501.
Once the content searched / and filtered based on the selected category, the program queries the user whether they wish to search segments within the category results. This occurs in step 505. If the user does not wish to search segments within the selected category, the program queries whether an icon representing a content portion has been selected. This occurs in step 506. If not the program awaits icon selection in step 507. When the user selects an icon, the content portion the icon represents is segmented in step 508. If the user wishes to search segments within the selected category, the program directs microprocessor 45 to segment all content portions within the category in step 509. In step 510, the program queries whether the user selected a segment category. If not, the program directs microprocessor 45 to generate a segment category menu in step 511. Once generated, the program directs microprocessor 45 to request and receive input in steps 512 and 513, respectively. If the user selected a segment category in step 510, or once input is received in step 513, the program directs microprocessor 45 to search and/or filter segments based on the selected segment category in step 514. Once the content portion the icon represents is segmented in step 508, or the program directs microprocessor 45 to search and/or filter segments based on the selected segment category in step 514, the program directs display 401 to display segments as segment representations in step 515.
The preceding expressions and examples are exemplary and are not intended to limit the scope of the claims that follow.

Claims

What is claimed is:
1. A method of displaying content at a segment level comprising: segmenting the content into at least two segments; displaying a representation of at least one of the at least two segments; and indicating how the at least one of the at least two segments relate to the content.
2. The method of Claim 1, wherein the segmenting step further comprises segmenting based on one of: pixel color analysis in a sampled frame, dialogue analysis, object recognition techniques, written content descriptions, closed captioning, speech to text, MPEG 7, and audio analysis.
3. The method of Claim 1 wherein the indicating step further comprises symbolically indicating the length of each segment.
4. The method of Claim 1 wherein the indicating step further comprises indicating the length of the at least one of the at least two segments in relation to the length of the content in which it is contained.
5. The method of Claim 1 wherein the indicating step further comprises indicating the temporal position of the at least one of at least two segments within the content.
6. The method of Claim 1 wherein the representation comprises at least one picture or text portion.
7. The method of Claim 6 wherein a user can search- content using the representation.
8. The method of Claim 1, wherein a user can select the representation of at least one of the at least two segments to display the segment.
9. The method of Claim 1, wherein the method further comprises storing a segment represented by the representation of at least one of the at least two segments on a storage medium.
10. The method of Claim 9, wherein a user can store and display more than one segment.
11. A system for displaying content at a segment level comprising: at least one source of content; means for segmenting the content into at least two segments; and a display means for displaying a representation of at least one of the at least two segments.
12. The system of Claim 11, wherein the display means comprises a television or monitor.
13. The • system of Claim 11, wherein the content f rther comprises at least one source of television programming, or source of linear media.
14. The system of Claim 11, wherein the at least one source of television programming or streaming media is a satellite source, a land based source, or a broadcast source.
15. The system of Claim 11, wherein the means for segmenting content operates at a content head end, at a user's location, or both a content head end and a user's location.
16. The system of Claim 11, wherein the means for segmenting content segments content automatically based on a user's individual viewing habits .
17. The system of Claim 11 further comprising a storage medium.
18. The system of Claim 11, wherein the display means displays at least one of the at least two segments.
19. A program stored on a processor-readable medium for displaying content at a segment level comprising: portions for segmenting content into at least two segments; portions for displaying at least one of the at least two segments; and portions for indicating how the at least one of the at least two segments relates to the content.
PCT/US2001/023800 2000-07-28 2001-07-27 Visualization and playback of television shows at a sub-show level WO2002011428A2 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP01959295A EP1308028A2 (en) 2000-07-28 2001-07-27 Visualization and playback of television shows at a sub-show level
JP2002515823A JP2004521525A (en) 2000-07-28 2001-07-27 Visualization and playback of TV show programs at a lower level than the programs

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US22140400P 2000-07-28 2000-07-28
US60/122,404 2000-07-28

Publications (2)

Publication Number Publication Date
WO2002011428A2 true WO2002011428A2 (en) 2002-02-07
WO2002011428A3 WO2002011428A3 (en) 2002-04-18

Family

ID=27623146

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2001/023800 WO2002011428A2 (en) 2000-07-28 2001-07-27 Visualization and playback of television shows at a sub-show level

Country Status (4)

Country Link
EP (1) EP1308028A2 (en)
JP (1) JP2004521525A (en)
CN (1) CN1218567C (en)
WO (1) WO2002011428A2 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003069621A1 (en) * 2002-02-14 2003-08-21 Koninklijke Philips Electronics N.V. Visual summary for scanning forwards and backwards in video content
US8225355B2 (en) 2006-05-01 2012-07-17 Canon Kabushiki Kaisha Program search apparatus and program search method for same

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102137300A (en) * 2011-03-24 2011-07-27 深圳创维数字技术股份有限公司 Set top box and program watching method and device based on set top box
CN103796060B (en) * 2012-11-26 2017-07-28 深圳Tcl新技术有限公司 Television set and its CC pattern-recognitions display methods, device

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5682326A (en) 1992-08-03 1997-10-28 Radius Inc. Desktop digital video processing system

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0332286A3 (en) * 1988-03-09 1990-02-28 Hewlett-Packard Company Programmable protocol analyzer
US5353121A (en) * 1989-10-30 1994-10-04 Starsight Telecast, Inc. Television schedule system
US5987211A (en) * 1993-01-11 1999-11-16 Abecassis; Max Seamless transmission of non-sequential video segments
US5835667A (en) * 1994-10-14 1998-11-10 Carnegie Mellon University Method and apparatus for creating a searchable digital video library and a system and method of using such a library
JPH08153350A (en) * 1994-11-29 1996-06-11 Toshiba Corp Switch signal generation device, reproducing device and tv receiver
JPH09139913A (en) * 1995-11-16 1997-05-27 Matsushita Electric Ind Co Ltd Video display method and device therefor
US6263507B1 (en) * 1996-12-05 2001-07-17 Interval Research Corporation Browser for use in navigating a body of information, with particular application to browsing information represented by audiovisual data

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5682326A (en) 1992-08-03 1997-10-28 Radius Inc. Desktop digital video processing system

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
AHANGER G; LITTLE T D C: "A SURVEY OF TECHNOLOGIES FOR PARSING AND INDEXING DIGITAL VIDEO", JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION, vol. 7, no. 1, 1 March 1996 (1996-03-01), pages 28 - 43
BORECZKY J S; ROWE L A: "COMPARISON OF VIDEO SHOT BOUNDARY DETECTION TECHNIQUES", JOURNAL OF ELECTRONIC IMAGING, 1 April 1996 (1996-04-01), pages 122 - 128
JONGJIANG ZHANG; WEI-YING MA: "Structured and content-based video browsing", CONFERENCE RECORD OF THE THIRTY-SECOND ASILOMAR CONFERENCE ON SIGNALS, SYSTEMS & COMPUTERS, 1 November 1998 (1998-11-01)
See also references of EP1308028A2

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003069621A1 (en) * 2002-02-14 2003-08-21 Koninklijke Philips Electronics N.V. Visual summary for scanning forwards and backwards in video content
US8225355B2 (en) 2006-05-01 2012-07-17 Canon Kabushiki Kaisha Program search apparatus and program search method for same

Also Published As

Publication number Publication date
CN1218567C (en) 2005-09-07
EP1308028A2 (en) 2003-05-07
JP2004521525A (en) 2004-07-15
CN1393097A (en) 2003-01-22
WO2002011428A3 (en) 2002-04-18

Similar Documents

Publication Publication Date Title
US9998792B2 (en) Electronic program guide for indicating availability of past programs
US7451469B2 (en) Electronic program guide displayed simultaneously with television programming
US7031596B2 (en) Digital video reproduction method, digital video reproducing apparatus and digital video recording and reproducing apparatus
US8010979B2 (en) Electronic program guide using markup language
KR101027466B1 (en) Electronic program guide with digital storage
EP1099338B1 (en) Interactive system for selecting television programmes
KR101111537B1 (en) Apparatus and method for supporting viewing and listening
US8505050B2 (en) Electronic program guide for indicating availability of past programs in the future
US20060013557A1 (en) Suppression of trick modes in commercial playback
US20060013555A1 (en) Commercial progress bar
US20060013554A1 (en) Commercial storage and retrieval
US20060013556A1 (en) Commercial information and guide
KR20070086558A (en) Improved method and apparatus for managing tv channel lists
JP2011072033A (en) Systems and methods for providing promotions with recorded programs
US20030084445A1 (en) Method of enhancing awareness of a data cell in a grid
US7519269B2 (en) Video recording/reproducing apparatus and method based on program information
US20030219226A1 (en) Method and system for accessing video data
EP1308028A2 (en) Visualization and playback of television shows at a sub-show level
KR100481539B1 (en) system and method for referring to information by one click using the caption informations
WO2005089451A2 (en) Interactive multimedia system and method
MXPA01008022A (en) Electronic program guide with support for rich program content

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): CN JP KP US

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR

ENP Entry into the national phase

Ref country code: JP

Ref document number: 2002 515823

Kind code of ref document: A

Format of ref document f/p: F

121 Ep: the epo has been informed by wipo that ep was designated in this application
AK Designated states

Kind code of ref document: A3

Designated state(s): CN JP KP US

AL Designated countries for regional patents

Kind code of ref document: A3

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR

WWE Wipo information: entry into national phase

Ref document number: 018028616

Country of ref document: CN

WWE Wipo information: entry into national phase

Ref document number: 2001959295

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 2001959295

Country of ref document: EP