Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
• • G—PHYSICS
  • G11—INFORMATION STORAGE
  • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
  • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
  • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
  • G11B27/19—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier
  • G11B27/28—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording
  • G11B27/32—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording on separate auxiliary tracks of the same or an auxiliary record carrier
  • G11B27/327—Table of contents
  • G11B27/329—Table of contents on a disc [VTOC]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
  • G06F17/40—Data acquisition and logging
• • G—PHYSICS
  • G11—INFORMATION STORAGE
  • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
  • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
  • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
  • G11B27/19—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier
  • G11B27/28—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording
• • G—PHYSICS
  • G11—INFORMATION STORAGE
  • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
  • G11B2220/00—Record carriers by type
  • G11B2220/20—Disc-shaped record carriers

Definitions

• the present invention generally relates to a method, system and program product for generating a content-based table of contents for a program. Specifically, the present invention allows keyframes from sequences of a program to be selected based on video, audio, and textual content within the sequences .
• no existing system allows a table of contents to be generated based on the content of the program.
• no existing system allows a table of contents to be generated from keyframes that are selected based on the determined genre of the program and classification of each sequence. For example, if a program is a "horror movie" having a "murder sequence,” certain keyframes (e.g., the first frame and the fifth frame) might be selected from the sequence due to the fact it is a "murder sequence' within a "horror movie.” To this extent, the keyframes selected from the "murder sequence” could differ from those selected from a "dialogue sequence" within the program. No existing system provides such functionality.
• the present invention provides a method, system and program product for generating a content-based table of contents for a program. Specifically, under the present invention the genre of a program having sequences of content is determined.
• each sequence is assigned a classification.
• the classifications are assigned based on video content, audio content and textual content within the sequences .
• keyframe also known as keyelements or keysegments
• a method for generating a content-based table of contents for a program comprises: (1) determining a genre of a program having sequences of content; (2) determining a classification for each of the sequences based on the content; (3) identifying keyframes within the sequences based on the genre and the classification; and (4) generating a content-based table of contents based on the keyframes.
• a method for generating a content-based table of contents for a program comprises: (1) determining a genre of a program having a plurality of sequences, wherein the sequences include video content, audio content, and textual content; (2) assigning a classification to each of the sequences based on the video content, the audio content, and the textual content; (3) identifying keyframes within the sequences based on the genre and the classifications by applying a set of rules; and (4) generating a content-based table of contents based on the keyframes.
• a system for generating a content-based table of contents for a program comprises: (1) a genre system for determining a genre of a program having a plurality of sequences of content; (2) a classification system for determining a classification for each of the sequences of a program based on the content; (3) a frame system for identifying keyframes within the sequences based on the genre and the classifications; and (4) a table system for generating a content-based table of contents based on the keyframes.
• a program product stored on a recordable medium for generating a content-based table of contents for a program When executed, the program product comprises: (1) program code for determining a genre of a program having a plurality of sequences of content; (2) program code for determining a classification for each of the sequences of a program based on the content; (3) program code for identifying keyframes within the sequences based on the genre and the classifications; and (4) program code for generating a content-based table of contents based on the keyframes.
• the present invention provides a method, system and program product for generating a content-based table of contents for a program.
• Fig. 1 depicts a computerized system having a content processing system according to the present invention.
• Fig. 2 depicts the classification system of Fig. 1.
• Fig. 3 depicts an exemplary table of contents generated according to the present invention.
• Fig. 4 depicts a method flow diagram according to the present invention.
• the present invention provides a method, system and program product for generating a content-based table of contents for a program.
• the genre of a program having sequences of content is determined. Once the genre has been determined, each sequence is assigned a classification. The classifications are assigned based on video content, audio content and textual content within the sequences .
• keyframe(s) e.g., also known as keysegments or keyelements
• keysegments or keyelements are selected from the sequences for use in a content-based table of contents.
• Computerized system 10 is intended to be representative of any electronic device capable of "implementing" a program 34 that includes audio and/or video content .
• Typical examples include a set-top box for receiving cable or satellite television signals, or a hard-disk recorder (e.g., TIVO) for storing programs.
• the term "program” is intended to mean any arrangement of audio, video and/or textual content such as a television show, a movie, a presentation, etc.
• program 34 typically includes one or more sequences 36 that each has one or more frames or elements 38 of audio, video and/or textual content.
• computerized system 10 generally includes central processing unit (CPU) 12, memory 14, bus 16, input/output (I/O) interfaces 18, external devices/resources 20 and database 22.
• CPU 12 may comprise a single processing unit, or be distributed across one or more processing units in one or more locations, e.g., on a client and server.
• Memory 14 may comprise any known type of data storage and/or transmission media, including magnetic media, optical media, random access memory (RAM) , read-only memory (ROM) , a data cache, a data object, etc.
• RAM random access memory
• ROM read-only memory
• memory 14 may reside at a single physical location, comprising one or more types of data storage, or be distributed across a plurality of physical systems in various forms.
• I/O interfaces 18 may comprise any system for exchanging information to/from an external source.
• External devices/resources 20 may comprise any known type of external device, including speakers, a CRT, LED screen, hand-held device, keyboard, mouse, voice recognition system, speech output system, printer, monitor, facsimile, pager, etc.
• Bus 16 provides a communication link between each of the components in computerized system 10 and likewise may comprise any known type of transmission link, including electrical, optical, wireless, etc.
• additional components such as cache memory, communication systems, system software, etc., may be incorporated into computerized system 10.
• Database 22 may provide storage for information necessary to carry out the present invention. Such information could include, among other things, programs, classification parameters, rules, etc. As such, database 22 may include one or more storage devices, such as a magnetic disk drive or an optical disk drive. In another embodiment, database 22 includes data distributed across, for example, a local area network (LAN) , wide area network (WAN) or a storage area network (SAN) (not shown) . Database 22 may also be configured in such a way that one of ordinary skill in the art may interpret it to include one or more storage devices.
• LAN local area network
• WAN wide area network
• SAN storage area network
• content processing system 24 Stored in memory 14 of computerized system 10 is content processing system 24 (shown as a program product) .
• content processing system 24 includes genre system 26, classification system 28, frame system 30 and table system 32.
• content processing system 24 generates a content-based table of contents for program 34.
• content system 10 has been compartmentalized as shown for in a fashion for readily describing the invention. The teachings of the invention, however, should not be limited to any particular organization, and functions illustrated as being part of any particular system, module, etc., may be provided via other systems, modules, etc.
• genre system 26 will determine the genre thereof. For example, if program 34 were a "horror movie,” genre system 26 would determine the genre to be “horror.” To this extent, genre system 26 can include a system for interpreting a "video guide" for determining the genre of program 34. Alternatively, the genre can be included as data with program 34 (e.g., as a header) . In this case, genre system 26 will read the genre from the header. In any event, once the genre of program 34 has been determined, classification system 28 will classify each of the sequences 36. In general, classification involves reviewing the content within each frame, and assigning a particular classification thereto using classification parameters stored in database 22.
• classification system 28 includes video review system 50, audio review system 52, text review system 54 and assignment system 56.
• Video review system 50 and audio review system 52 will review the video and audio content of each sequence, respectively, in an attempt determines each sequence's classification.
• video review system 50 could review facial expressions, background scenery, visual effects, etc.
• audio review system 52 could review dialogue, explosions, clapping, jokes, volume levels, speech pitch, etc. in an attempt to determine what is transpiring in each sequence.
• Text review system 54 will review the textual content within each sequence.
• text review system could derive textual content from closed captions or from dialogue during the sequence.
• text review system 54 could include speech recognition software for deriving/extracting the textual content
• the video, audio, and textual content (data) gleaned from the review would be applied to the classification parameters in database 22 to determine a classification for each sequence.
• program 34 is a "horror movie.”
• a particular sequence in program 34 has video content showing one individual stabbing another individual and audio content comprised of screams.
• the classification parameters generally correlate genres with, video content, audio content, and classifications.
• the classification parameters could indicate a classification of "murder sequence.”
• the classification parameters could resemble the following:
• frame system 30 (Fig. 1) will access a set of rules (i.e., one or more rules) in database 22 to determine the keyframes from each sequence that should be used for table of contents 40.
• table of contents 40 will typically include representative keyframes from each sequence.
• frame system 30 will apply a set of rules that maps (i.e., correlates) the determined genre, with the determined classifications and the appropriate keyframes. For example, a certain types of segment within a certain genre of program could be best represented by keyframes taken from the beginning and the end of the segment .
• the rules provide a mapping function between the genre, the classifications and the most relevant parts (keyframes) of the sequences . Shown below is an exemplary set of mapping rules that could be applied if program 34 is a "horror movie.”
• program 34 is a "horror movie,” and one of the sequences was a “murder sequence,” the set of rules could dictate that the beginning and the end of the sequence are the most important. Therefore, keyframes A and Z are to be retrieved (e.g., copied, referenced, etc.) for use in the table of contents. It should be understood that, similar to the classification parameters shown above, the set of rules depicted above are for illustrative purposes only and not intended to be limiting.
• the keyframes are selected based upon sequence classification (type), audio content (e.g., silence, music, etc.), video content (e.g., number of faces in a scene), camera motion (e.g., pan, zoom, tilt, etc.) and genre.
• keyframes could be selected by first determining which sequences are the most important for a program (e.g., a "murder sequence” for a "horror movie” ) , and then by determining which keyframes are the most important for each of those sequences . In making these determinations, the present invention could implement the following Frame Detail calculation:
• thresholdl (# of edges + texture + # of objects ) > threshold 2
• Motion importance 1 for first and last frame in case of zooming and zoom out, 0 for all other frames.
• table system 32 will use the keyframes to generate a content-based table of contents.
• table of contents 40 could include a listing 60 for each sequence.
• Each listing 60 includes a sequence title 62 (which could typically include the corresponding sequence classification) and corresponding keyframes 64.
• the keyframes 64 are those selected based on a set (i.e., 1 or more) of rules as applied to each sequence in light of the genre and classifications.
• the keyframes for "SEQUENCE II - Murder of Jessica” would be frames one and five of the sequence (i.e., since the sequence was classified as a "murder sequence.”
• a remote control, or other input device a user could select and view the keyframes 64 in each listing. This would present the user with a quick synopsis of the particular sequence.
• Such a table of contents 40 could be useful to a user for many reasons such as browsing a program quickly, jumping to a particular point in a program and viewing highlights of a program. For example, if program 34 is a x horror movie" showing on a cable television network, user could utilize the remote control for the set -top box to access table of contents 40 for program 34.
• table of contents 40 depicted in Fig. 3 is intended to be exemplary only. Specifically, it should be understood that table of contents 40 could also include audio, video and/or textual content .
• first step 102 of method 100 is to determine a genre of a program having sequence of content .
• Second step 104 is to determine classifications for each of the sequences based on the content.
• Third step 106 is to identify keyframes within the sequences based on the genre and the classifications.
• Fourth step 108 is to generate a content- based table of contents based on the keyframes.
• the present invention can be realized in hardware, software, or a combination of hardware and software. Any kind of computer/server system(s) - or other apparatus adapted for carrying out the methods described herein - is suited.
• a typical combination of hardware and software could be a general purpose computer system with a computer program that, when loaded and executed, controls computerized system 10 such that it carries out the methods described herein.
• a specific use computer containing specialized hardware for carrying out one or more of the functional tasks of the invention could be utilized.
• the present invention can also be embedded in a computer program product, which comprises all the features enabling the implementation of the methods described herein, and which - when loaded in a computer system - is able to carry out these methods.
• Computer program, software program, program, or software in the present context mean any expression, in any language, code or notation, of a set of instructions intended to cause a system having an information processing capability to perform a particular function either directly or after either or both of the following: (a) conversion to another language, code or notation; and/or (b) reproduction in a different material form.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• Databases & Information Systems (AREA)
• Software Systems (AREA)
• Mathematical Physics (AREA)
• General Physics & Mathematics (AREA)
• Data Mining & Analysis (AREA)
• Computer Hardware Design (AREA)
• Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
• Processing Or Creating Images (AREA)
• Television Signal Processing For Recording (AREA)

Priority Applications (3)

Applications Claiming Priority (2)

Publications (1)

Family

ID=31187358

Family Applications (1)

Country Status (7)

Families Citing this family (13)

Citations (2)

Family Cites Families (17)

• 2002
  • 2002-08-01 US US10/210,521 patent/US20040024780A1/en not_active Abandoned
• 2003
  • 2003-07-17 CN CNB038177641A patent/CN100505072C/zh not_active Expired - Fee Related
  • 2003-07-17 KR KR1020057001755A patent/KR101021070B1/ko not_active Expired - Fee Related
  • 2003-07-17 EP EP03766557A patent/EP1527453A1/en not_active Withdrawn
  • 2003-07-17 WO PCT/IB2003/003265 patent/WO2004013857A1/en not_active Ceased
  • 2003-07-17 AU AU2003247101A patent/AU2003247101A1/en not_active Abandoned
  • 2003-07-17 JP JP2004525681A patent/JP4510624B2/ja not_active Expired - Fee Related

Patent Citations (2)

Non-Patent Citations (3)

Also Published As

Similar Documents

Legal Events