JP5951184B2 - Method and apparatus for configuring a media product - Google Patents

Description

  The present invention relates to a method for configuring a media product and an apparatus for configuring a media product.

  Media articles draw content for a person's sensation (whether it is real, imaginary, or computer generated). Media products can be presented to a person via a wide variety of media including text, audio, sound, pictures, or animation.

  As recording technology has improved, the amount of recorded media products available to consumers has increased exponentially. Media products are often recorded in media files (here the plural “media” is used, but “media file” is only given to the user by one media) Note that it should be understood to include both files that are intended to be communicated (eg, text and speech) and “multimedia” files whose meaning is communicated across multiple media. ). The Internet is the most recent communication network, providing a global transmission of recorded digital media files that represent text, sound, pictures, video, or combinations thereof. Since the number of media files accessible via the Internet is so large, media files need to be labeled with some description of their contents. Thus, for example, an HTML (Hypertext Markup Language) file includes a “meta” tag that includes a keyword that indicates what subject is being handled on the web page presented to the user.

  Labeling media files with metadata is more useful when a group of users agrees on how the metadata should be built and what elements it should contain. In many cases, XML (extensible Mark-up Language) is used to define such structures and the elements contained within those structures. In effect, XML can be used to define a metadata “language”. An example of such a metadata “language” is P. 903 to P. 902 of the minutes of Intelligent Tutoring Systems Conference 2002. As described in Gerdt et al., “StoryML: XML Extension for Woven Stories”, StoryML is a StoryML. It is a metadata language designed for, so it includes elements that provide contributors to the author and other contributors.

  Proposals for adding metadata to video files (more widely known as “multimedia content description interface”, known as MPEG-7) are being considered by the Moving Pictures Expert Group (Moving Pictures Expert Group).

  International Patent Application No. 02/057959 discloses computer software that provides a user with tools for organizing media files. Various metadata can be associated with these files. Files and metadata are stored in a “relational” database (“relational” as used in representational relational databases is the relationship between database entries or what is represented by these entries. (Note that there is little relation)) Instead, it refers to "relation" in the sense that words are used in mathematical set theory.

  One method of configuring a media product includes grouping multiple components. For example, the film is composed of a plurality of scenes as described in the screenplay. There have been several attempts to automatically create media products in this way. For example, the creation of a waltz in accordance with a musical dice game can be found at http: // sunsite. univie. ac. at / Mozart / dice /. Similarly, an automatic story generation program, Romance Writer, can be found at http://familygames.com. com / features / humor / romance. Available in html.

According to a first aspect of the invention,
Analyzing digital metadata associated with a first set of stored media data, the digital metadata comprising:
Related set identity data identifying a second set of stored media data;
Relationship data indicative of a relationship between what is represented by the first set of stored media data and what is represented by the second set of stored media data;
Including
Configuring the first set and the second set of media data stored in a media product according to the analysis;
A method for automatically configuring a media product is provided.

  Analyzing the digital data associated with the first set of stored media data and representing the second set of identifiers of the stored media data and the first set of the digital data; Configuring the first set and the second set of media data stored in accordance with the analysis into a media product, including an indication of the relationship between those represented by the second set By doing so, a method is provided for configuring a media product that obviates the need for another source of sequencing information provided when the media product is configured.

  The expression “set of stored media data” includes a set of pointers to media data files, streams, or files or databases.

  The digital collection of stored media data is stored in various formats. The representation set of stored media data is not intended to be limited to any particular format, and therefore rarely contains data regarding the location of components that the user sees when playing computer games, for example. Includes files not included. The data in the file is processed by rendering software to generate an image for later display to the user.

  In some embodiments, the method further includes generating the aggregate identity data and the relationship data.

  Preferably, the metadata further includes content data indicating what is represented by the set of stored media data, and the method depends on the content data from a plurality of sets of stored media data. Selecting one or more sets of stored media data, the one or more sets including the first set and the second set of stored media data.

  a) searching to provide a plurality of potential components of the media product, or a portion of the media data; and b) a plurality of the selected components according to metadata associated with those components. This combination with subsequent configuration of the media enables automatic configuration of media products that are more easily directed to a particular theme than previously possible.

  Preferably, the method further comprises making a plurality of such selections and concatenating the results of said selections.

  This also allows for an approach to media product construction that follows a wide range of established patterns, but allows for a degree of flexibility within that pattern that is not found in conventional systems. Therefore, a wide variety of pattern types known for stories and films, for example, StoryCraft Corporation's StoryCraft Corporation's StoryCraft Corporation's StoryCraft Corporation (CraftCraft Corporation) storycraft program (560 Roll Drive, Norfolk, VA23509, USA) The hero and the adversary in the export program, a collision between them followed by the return of the hero who won the victory (still usable, but changes in that pattern are easily introduced by changing the nature of the choice Help the author write the story by asking them to introduce before.

  This enables cost-effective creation of different versions of movies or computer games that follow different artistic or qualitative ambitions, for example, while maintaining a subjectively high level of story, movement, and audio continuity To do.

According to a second aspect of the invention,
Each of a plurality of sets of stored media data is represented in the content represented in said set of stored media data and in one or more other sets of stored media data One or more storage devices that store metadata including relationship data indicating one or more relationships between content;
One or more configured to operate to configure a media product by communicating with the one or more storage devices and configuring the set of media data or identifiers thereof stored according to the relationship data A processor;
Is provided.

  In a preferred embodiment, an object-oriented database is used to store objects that contain metadata associated with a set of stored media data identified by the metadata. As a result, the relationship metadata can be expressed by the relationship between objects in the object-oriented database.

  The collection of media data may instead be stored in a file system.

  By way of example only, specific embodiments of the present invention will now be described with reference to the accompanying drawings.

1 is a schematic diagram of components of a personal computer operable to provide a media assembly system according to a first embodiment of the present invention. FIG. FIG. 2 is an architecture diagram of a computer program used in controlling the personal computer of FIG. 1 according to the first embodiment of the present invention. FIG. 3 illustrates metadata associated with media elements following use of the media markup tool of FIG. It is a figure which shows the cause object memorize | stored in the object-oriented database of FIG. FIG. 4 is a diagram showing how an editor constructs relationship data included in the metadata of FIG. 3. FIG. 4 is a diagram showing how an editor constructs relationship data included in the metadata of FIG. 3. FIG. 4 is a diagram showing how an editor constructs relationship data included in the metadata of FIG. 3. FIG. 4 is a diagram showing how an editor constructs relationship data included in the metadata of FIG. 3. FIG. 4 is a diagram showing how an editor constructs relationship data included in the metadata of FIG. 3. FIG. 4 is a diagram showing how an editor constructs relationship data included in the metadata of FIG. 3. FIG. 4 is a diagram showing how an editor constructs relationship data included in the metadata of FIG. 3. FIG. 4D is a diagram of hierarchical representation of relationship data input by an editor as shown in FIGS. 4A to 4H. It is a figure which shows the template data produced | generated using the template creation tool of FIG. It is a figure which shows the user profile data produced | generated using the user profile creation tool of FIG. It is a flowchart which shows operation | movement of the component of the template populator of the 1st Embodiment of this invention. FIG. 8 is a diagram illustrating the operation of the template populator related to the first section of the template depicted in FIG. 7. FIG. 8 is a diagram illustrating the operation of the template populator related to the second section of the template depicted in FIG. 7. FIG. 8 is a diagram illustrating the operation of the template populator related to the second section of the template depicted in FIG. 7. FIG. 8 is a diagram illustrating the operation of the template populator related to the second section of the template depicted in FIG. 7. FIG. 8 is a diagram illustrating the operation of the template populator related to the second section of the template depicted in FIG. 7. It is a figure which shows the edit decision list | wrist which may be produced | generated by the template populator module. FIG. 6 is a diagram showing a display presented to a user when executing a computer program that provides a preferred embodiment of the present invention. FIG. 6 shows the same display at a later stage of the program operation. FIG. 6 illustrates an additional sub-window that opens when the user selects one of the media bins. FIG. 6 shows an additional sub-window presented to the user when the user selects one of the media elements contained within the selected media bin. FIG. 6 shows an application window after indicating that the user wishes to view relationship data associated with selected (or all) media elements contained within a media bin. FIG. 5 shows an additional sub-window that allows a user to define a vocabulary. FIG. 5 is a diagram showing a display editing a template after the user has selected a template for use in generating a media product. FIG. 4 is a diagram showing a display in which a template is enlarged after a user has selected a template for use in generating a media product. FIG. 6 is a diagram showing a sub-window that is displayed to show the result of a user executing a template populator component of a program. FIG. 20 depicts an additional functionality provided to the user via a query builder window (as seen in FIGS. 18 and 19), with an improved version of the preferred embodiment of the present invention. FIG. 7 illustrates additional controls provided to a user when previewing a media product that uses an improved version of the preferred embodiment of the present invention.

  FIG. 1 shows a personal computer with known hardware components connected together in a conventional manner. The known hardware components include a central processing unit 10, a random access memory 12, a read only memory 14, a hard disk 16, and input / output devices 18, 20, 22, 24 and 26. The hardware components are interconnected via one or more data buses and address bus 28. The input / output device includes a monitor 18, a keyboard 20, a mouse 22, a CD ROM drive 24 and a network card 26. The network card is connected to the server computer 30 via the public Internet 32.

  FIG. 2 shows the software and data components of the system according to the first embodiment of the present invention. The software includes three input program modules, namely a media markup tool program 40, a template creation tool program 42, a user profile creation tool program 44, an object-oriented database management software 45, and two output program modules, ie, a template populator program module. 46 and a content synthesis program module 48.

  The system also includes two storage devices for fixed storage of data. The first of these is a content storage device 50 comprising a number of media files stored on the hard disk 16 using a file system provided by a computer operating system program. A second storage device for fixed storage of data comprises an object oriented database 54 known as ObjectStore® supplied by Exelon Corporation of Burlington, Massachusetts. ing. The database includes three different categories of objects: media objects 51 (each containing metadata describing one of the media files stored in the content storage device 50), template objects 52, and user profile objects 53. Is remembered. Objects in the database are again stored on the hard disk 16 of the computer. Permanent storage for the object-oriented database and / or content storage device may be provided by a removable data carrier such as a DVD, CD, CD-ROM or by the network card 26 instead of a hard disk. May be provided on different computers accessible via a network connection (e.g. accessed via a URI).

  The three input program modules control a computer (FIG. 1) to provide a user interface that allows for the generation of objects that include data structures constructed according to a pre-defined structured data model. This increases the ease with which different program modules can exchange data. When the constructed data model is made public, different parties can write different components of the program that can nevertheless exchange data with each other. Also, objects created by one editor can be used at another location or by another editor working later.

  In the present embodiment, the three input program modules provide the following functionality.

  The media markup tool provides an interface for an editor to update the content storage device 50 and the object oriented database 54. In practice, editors using this embodiment are free to use media elements generated by other editors, rushes from a variety of sources, sections of created programs, still photos and (all Access various other parts of the media (represented in electronic form). These media elements are stored in a suitable directory structure within the content storage device. Each directory is known in the art as a “bin” (see a bin with a label containing several films related to a particular project).

  Media elements are often recorded digitally in a file that may contain many elements in a linear stream. Thus, such a file includes one or more media elements associated with it. With the media markup tool, the editor previews a file, and if there must be multiple media elements in the file, the start and end points that delimit each media element in the file Can be set. If there is only one media element associated with the file, the start point is only zero and the end point is the media length (duration). In this way, the editor can generate multiple files, each forming one media element. The editor names each file when it determines the start and end points of the media element.

  However, for the purposes of this description, it is assumed that the editor starts only with a file containing an electronic sequence of unedited film recorded in a football game, and an introductory sequence for a football program or the like. The unedited portion of the film is known in the art as “rush”. The editor may use the media markup tool 40 to select various sections of the rush and store each as a media element in a shorter file in a directory in the content storage device 50. is there.

  The media markup tool also provides a tool that allows an editor to generate or edit metadata for media elements stored in the content storage device 50. The tool stores this metadata as an object in the object oriented database 54.

  Upon selecting a directory in the content storage device, the editor is provided with a graphical user interface that presents a collection of icons each representing one of the media elements held in the directory (FIG. 4A). These icons may only be still photos from a video sequence contained within the file, for example. In the example below, they are drawn as boxes containing media element identifiers entered by the editor in the upper right corner and descriptions of what happens at the media element in the center of the box. The user interface allows the editor to select media elements using the mouse 22 and move the media elements around the screen using the mouse.

  After selecting one of the media elements, the editor enters metadata associated with this media element in two stages. In the first stage, the editor can double click on one of the pictures to bring up a format in which the values of the parameters contained in the schema can be entered.

  An example of the metadata generated in the first stage is shown in the second to twelfth lines of FIG. 3 (the first line information is generated when the editor gives the media element identifier to the file) Was).

  It will be appreciated that the metadata is organized according to a structured data model. In each row, the entry in the rightmost column represents the property value entered by the user. The structured data model is a unit term at the first level of a plurality of properties, called a collection (here called the collection of properties (the second column from the left of these rows with 4 columns)). It may be provided that it must be labeled as (members). A structured data model is that multiple sets are at the second level of a set (here called the superset of properties (the leftmost column of these rows with 3 or 4 columns)) It may also be provided that it must be labeled as a unit term. One skilled in the art will appreciate that additional levels of aggregation may be provided.

  The hierarchical arrangement is affected by the multimedia content description interface described above. The intent is not to enforce the use of a complete data model across all possible applications, but allows the reuse of content within the subject domain of a specific set of production companies or projects (eg wildlife documentary) Is to do. The provided data model is intended to provide a maximum set of elements and interfaces that assist in their use and the vocabulary that can be applied to them.

  The metadata contains a variable number of parameters (but nevertheless must fit into a given structured data model). In the example shown in FIG. 3, the editor entered values for 18 properties. These include the following:

  i) Media Element ID (which identifies the media element)-In this example, the editor is set to 0. A numerical value xx is given, where xx reflects the position of the media element in the original rush.

This is followed by a “media” superset with two properties and a “position” set of properties. The two properties are as follows.
ii) URI—Universal resource identifier of the file containing the media element.
iii) Format—This shows an indication of the format of the data that makes up the file.

The “Position” set contains two properties as follows:
iv) In-Indication of the time elapsed since the start of the rush at the beginning of the media element.
v) Out--Indication of the time that has elapsed since the start of the rush at the beginning of the media element.

The “medium” superset is followed by a superset of four “structured” properties. Its superset begins with:
vi) Description-description of the contents of the file.

It is followed by another set (called “event”) that includes the following three properties:
vii) Nature—The type of event found in the video sequence recorded in this file.
viii) Performer—Person performing the main action of the event.
ix) Recipient—A person who depends on the main behavior of the event.

  These properties are followed by a special superset of property domains that are reasonably applied to media elements related to material derived from a two-sided sport event.

The first two properties belong to a set of two properties (called “teams”).
x) Home Team-The name of the team playing on the home ground during the football game characterized by the original rush.
xi) Away Team—Another football name of a football game characterized by an original rush.

This set is followed by the following two properties:
xii) Leader Allegiance-if present, the side on which the lead actor is loyal.
xiii) Recipient loyalty-the side on which the recipient is loyal (if any).

These two properties are followed by a set (named “conceptually”) that includes the following two properties:
xiv) Interest value-This value between 0 and 1 indicates how important the editor thinks this media element.
xv) Rating-This value indicates the suitability of the media element for showing to people (in a manner similar to the classification given to film) based on age criteria.

  Once the editor has entered this data, the picture is replaced with a description of the media element given as the value of the “description” property. The format is then removed from the display and returned to the multiple picture display representing the media element selected by the editor from the original rush (FIG. 4A).

  The media element metadata is then stored as a media object in the object oriented database 54.

  A second stage of metadata creation that generates one or more “relationship” properties will now be described with respect to FIGS. 4A-4H. The user is provided with a graphical user interface that allows the user to move and click on the icons representing those media elements on the PC screen 18 (FIG. 1) to show the relationship properties between the media elements. Like that.

  One relationship that the editor may show is causality. To do this, the editor clicks a button element presented on a screen (not shown) that changes the shape of the cursor. The user then moves the cursor to the media element that is determined to be the cause of another media element. The user then clicks on the media element to identify it as the media element representing the cause, moves the cursor to the media element representing the cause effect, and clicks again. Once the user does this, an arrow is drawn from the first media element to the diamond shape representing the causal object, and then a second arrow is drawn from the diamond to the second media element. If the editor thinks that a viewer viewing the media element that represents the effect will also want to see the cause, he may wish to do this kind of cause association. In the example illustrated in FIG. 4B, the editor has shown:

i) Media element 0.13 is caused by media element 0.12, both media elements 0.14 and 0.15 are caused by media element 0.13,
ii) Media element 0.53 is caused by media element 0.52, which is then caused by media element 0.51.

  In response to the cause related input, a cause object (FIG. 3B) having a unique media element identifier is added to the object oriented database 54. The metadata associated with the media element representing the impact is updated to include a parameter indicating the media element identifier of the cause object. More specifically, a media object has a linked list of pointers to any cause object that causes or is achieved by it. Cause objects also maintain two lists of pointers that return to the media object with which they are associated.

  One example can be seen in the last parameter illustrated in FIG. The cause object also includes metadata indicating the media element identifier that causes the effect.

  Multiple different media objects can cause the same effect (e.g., the hero would have been poisoned or killed because of a clash) and could cause multiple different effects from one cause (e.g., the Due to the death of the protagonist, the evil queen survived to the age of 103, and a griefful princess swore a single life and became a nun). This is why causal relationships are expressed using cause objects.

  A second type of relationship that the editor may indicate is a sequence relationship. To show such a relationship, the editor arranges the media elements that it wants to classify into the same rectangular area of the screen, orders them from left to right according to the order they must follow, Clicking the add button (not shown) causes the cursor to change format and move the cursor to a position relative to one corner of the rectangular area. The editor then clicks the button with the mouse 22 and keeps pressing the button while moving to the opposite corner of the rectangular area, where it releases the button. As a result, a thick solid rectangular line is drawn around the medium element included in the rectangular area, and a thick short arrow is drawn at the upper left corner of the area defined by the rectangular line. In the example of FIG. 4C, the editor represents a media element representing Team B's 16th pass, followed by a media element representing Team B's 15th pass, followed by a media element representing Team B's first goal. Showed that it continues.

  The editor may wish to show an ordering relationship of this nature that feels that the media elements must be illustrated in the order shown (when multiple of the media elements are selected by the template populator module). is there. For example, media elements that represent gardening at different times of the year may be arranged in order, such as an element representing a spring garden preceding an element representing a summer garden.

  When an order is created in this manner, the order object is created in the object-oriented database as a container object that includes media objects associated with media elements included in the order. As can be seen below, an order can be generated that includes the order itself. This hierarchical property is reflected in the first number of the identifier whose cause is in order. If the order includes only individual media elements, the order identifier is 1. It will be in the form of x, where x is only incremented each time a new order or group (described below) of the first level of the hierarchy is formed. Accordingly, the identifier 1.1 is given in the order shown in FIG. 4C.

  The media object (ie, metadata) associated with each media element in the order has the position of the media element in the order appended to it. The object-oriented database also records the fact that each media object is a child of (ie, contained within) a newly created sequence object. An example of ordinal position metadata can be seen in the penultimate row of FIG.

  In this embodiment, there is no metadata that gives a description of the group or order, but there may be some. Such metadata may be entered, for example, by right-clicking and selecting a property and entering text in the description field of the resulting dialog.

  A third type of relationship that an editor may wish to show between media elements is a group membership relationship. The editor may do this if multiple media elements in the group are selected and then want to indicate that they should be shown together. Groups are formed in the same way as the order, except for the order of media elements on the screen that are not important, and the editor represents the group before defining a rectangular area that contains the media elements that must be included in the group. Click a third button (not shown).

  This action creates a group object, a container object that includes media objects associated with media elements in the group. Group objects are also stored in the object-oriented database 54.

  4D shows that the editor assigns media elements 0.13, 0.14, and 0.15 to the first level group 1.2, 0.39, 0.40, and 0.41 to the second first level order 1. 3 shows the display that the editor will see after media elements 0.52 and 0.53 have been formed in the second first level group 1.4.

  FIG. 4E shows the editor associating the media element 0.51 and the third first level order 1.4 to the second level group 2.1. This creates another group object in the database that is a container object for another group object. In this way, a hierarchy of container objects can be constructed in the object-oriented database 54.

  FIG. 4F shows an order of two first levels, 1.1 and 1.3, a first level group 1.2 and a second level group 2.1 into a third level group 3.1. Shows the editor to put in.

  FIG. 4G shows an editor that defines the order of the fourth level.

  FIG. 5 shows the relationships entered in a hierarchical format by the editor, thus reflecting the object hierarchy constructed by the editor in the object-oriented database 54 on the hard disk 16.

  Returning to FIG. 2, the template creation tool 42 provides an interface for the editor and / or producer to specify the desired characteristics of the media product. Therefore, it becomes an interface for creating a template object (FIG. 6). The tool stores the metadata that makes up this template object in the object-oriented database 54.

  Like media objects, template objects for use in this embodiment conform to a comprehensive predefined data model. As can be seen from FIG. 6, this predefined data model includes a title field and a plurality of sections. Each section is a set with a name field, a query field, and optional constraint fields. The template populator tool prompts the user to enter the name of the template and show the section structure (the top level section may itself contain a section). The editor shows the section structure using components of a similar graphical user interface, for example, in a folder list provided in Microsoft Windows® Explorer. In the example shown in FIG. 6, the template has a flat structure of three sections.

  Because templates encode media product features using complex queries and potentially deep structures, the editor interface divides the task of creating a template among multiple roles. The person assigned the editor role defines the top-level structure of the template, and the person assigned the producer role (the producer has closer control of the actual product that is usually created) refines the structure and queries ( Request for information from an object-oriented database). In particular, as described below, producers specify linkages to user profiles, thereby defining a “force balance” between themselves and the consumer.

  The template creation tool provides an object browser that can be used to search for existing media objects and template objects. Existing templates can be modified and parts of the template can be copied into a new template.

  Having defined the section structure using the graphical user interface described above, perhaps using the media object browser, the editor / producer is provided with a graphical user interface that facilitates the process of query formation.

  The editor uses this graphical user interface to enter a query string for each of the sections. The query string in the first section of FIG. 6 indicates that candidate media objects for filling this slot in the template must have the string “Football Intro” in their URI. ing.

  The query string for a section may be significantly more complex, as seen in the “Main” section of the template of FIG. Here, the query is created in the XPath language. The way in which the query is evaluated will be described after the user profile metadata is described below.

  The editor also enters constraints for those sections that wish to impose any constraints on the media elements represented by the media objects retrieved from the database in response to the introduction. The constraints are intended to limit the way in which media objects are assembled by the template populator. Possible examples of constraints are time (eg, this section must be 5 minutes long), space (eg, this presentation must be displayed on a 640 × 480 pixel display) or number (“ The "Headline" section must contain 5 news items).

  The user profile creation tool 44 provides a user interface for generating a user profile such as the user profile seen in FIG.

  A user profile represents a particular user's preferences. As with media objects and template objects, data must be faithful to a given data structure or schema. The schema indicates that each user is identified by an identifier (in this case, the number display “1”). The “structural” elements of the user profile indicate what the user likes (in this case, football team B, especially Paulo Di Canio and Trevor Sinclair, Essex cricket team, especially Nasser Hussain, actress Milla Jovovich and actor Jimmy Nail Yes.

  The template populator program module (FIG. 8) provides a process for automatic assembly of an edit decision list into a consumer personal media product in preparation for compositing a collection of media objects. At start (step 60), the template populator takes as input its specific template, specific user profile, and media object storage indicators (in this case, an indication of the location of the object-oriented database 54).

  Once specified template, user profile, and media object storage is specified, the template populator examines the template (FIG. 6) for any query written in the XPath language (step 61). When such a query is detected, it is resolved using the Apache Project's Xalan processor to evaluate the XPath.

For example, a query read as follows (team is “! Profile (profile / sports // team)”) and (action is “goal”)
Is decomposed below when the user profile of FIG. 7 (specified at the time of starting the template populator program module) is considered.

(Team is “B Team” or “Essex”) and (Action is “Goal”)
The template populator then identifies the first section of the template (FIG. 8) and iterates through the template hierarchy (steps 62-75).

  Each iteration (steps 62-75) includes the next section in the template being detected, and any queries in that section are executed in the object-oriented database 54 (step 62) to select the associated media object. To return.

  The first iteration relates to the section named “Intro” in FIG. The iteration begins with the execution of a query included with the section (step 62) (ie, the URI includes a “football intro”). FIG. 9A depicts the choices that give rise to this example.

  Next, in step 64, a tree is built that includes the media object selected as the “leaf”. This construction is done as follows. The parent object of the first selected media object is searched and so on until it reaches an object that does not have a parent object associated with it (since the "introduction" object in this example has no parent object, The only object contained in the tree). At this point, a single linked list from the leaf object to the top level container has been rebuilt. An object whose another selected leaf object is examined (if more than one object is selected as a result of a query) and whose genealogy of that leaf object already exists in the linked list representing the genealogy of the first object Or until another top-level container is encountered. By repeating this process for all other objects in the selection, the minimal tree containing those objects is reconstructed.

  As indicated above, in the first iteration, the resulting tree contains only the “introduction” media object 0.1.

  Subsequent steps in the loop of instructions in the template populator program that modify the tree data structure stored and used in creating the edit decision list (steps 66-72) are performed in the second section of the template. As will be described below with respect to the second iteration of the instruction loop being executed, there is no effect on the first section of the template.

  Through iteration, the tree structure is stored in the volatile memory 12 of the PC.

  At the end of each iteration of the group of instructions, a determination is made as to whether the last section in the template has been considered (step 74). If not, the next section is identified (step 75) and the next iteration is performed.

  A second iteration is performed on the central section of the football goal highlight template (FIG. 6). In this case, a section query is performed using the user profile (FIG. 7) to decompose the query as described above to take into account user preferences.

  Query (step 62) results in selection of media elements 0.12 and 0.53 (FIG. 10A). A new tree data structure is created as described above and stores data representing the tree depicted in FIG. 10B.

  The media object selection is then expanded to take into account the cause / effect relationship specified by the user (step 66). Specifically, this step includes an examination of the metadata of each selected media object to find out how many causal objects are associated with this media object. If no cause object is detected, the media object is moved to the list of decomposed media objects. If only one cause object is detected, the cause object is moved to the list of cause objects and the media object is moved to the list of disassembled media objects. If more than one cause object is detected, each possible cause object is added to the list of possible cause objects (if it is not already in the list) and the media object is not disassembled Added to the list of objects.

  In this example, only one causal object is detected (as depicted in FIG. 3B), so media object 0.53 is moved to the list of disassembled media objects (as illustrated in FIG. 10C). To the tree data structure associated with this iteration of the instruction loop), the cause object is moved to the list of cause objects.

  When a list of possible cause objects is created, the cause object that causes the least decomposed media object is detected. This cause object is moved into the list of cause objects and the media object it causes is added to the list of decomposed media objects described above. This process is repeated until all lists of unresolved media objects are empty. In this example, this step does not apply.

  Each cause object in the list of cause objects is then examined to find out how many media objects it has caused. If only one media object causes it, the cause object is moved to the list of decomposed cause objects and the media object is added to the list of unresolved media objects. If cause objects are caused by multiple media objects, they are added to the list of possible media objects if they do not already exist.

  In this example, since only one cause object (CO1) in the list of cause objects is caused by only one media object (0.52), the cause object is moved to the list of decomposed cause objects and the media Object (0.52) is moved to the list of undissolved media objects.

  When a list of possible media objects is obtained, the media objects that cause the most causal objects are detected. The cause object is then moved into the list of decomposed cause objects and the media object is added to the list of media objects that have not been decomposed. This is repeated until the cause object list is empty.

  The procedure is repeated for every unresolved media object (so that the causal relationship is traced back to the original cause). Thus, in this case, the procedure is repeated for media element 0.52, resulting in the addition of media object 0.51 to the tree associated with this iteration (FIG. 10D).

  The tree construction (steps 62 to 66) is followed by the reordering of objects in the tree (steps 68 and 70).

  The first stage of reordering (step 68) takes into account the order information entered by the user. This is done using a known “Quicksort” algorithm to position the nodes of the tree in the correct order as identified by the ordinal position metadata associated with the object. This is done by starting at the top of the tree and then moving towards the leaves of the tree (ie, the media object).

  The second stage of reordering takes into account the causal / influence link between group members or group member descendants (ie, objects further down the tree). If groups do not have such a cause / effect link, this reordering step need not be performed for those groups.

  The second stage of reordering is a group in the tree with all causes and effects attached to it (if it is a media object) or to any of its descendants (if it is a container). Start by labeling each member of.

  An additional label is then added to the object metadata to reflect the logical relationship that if a causes b and b causes c, then a causes c. The same is done to reflect the logical relationship that if f is caused by e and if e is caused by d, then f is caused by d.

  The known Quicksort algorithm is then used to ensure that the cause is shown before the impact. As understood by those skilled in the art, the Quicksort implementation allows the user to define a function that indicates the order of the two objects passed to it. In this case, a function is provided that indicates that d comes after c if a goes before b is cause b and d is caused by c.

  Thus, at the end of the second iteration reordering step (steps 68 and 70), media elements 0.12, 0.51, 0.52, and 0.53 are defined in the football goal highlight template (FIG. 6). Form a leaf node of the tree associated with the central section.

  The template populator then evaluates any constraints and updates the tree accordingly (step 72). To evaluate the time constraints, the duration of each media object contained in the tree is calculated by subtracting the “Out” property from the “In” property, and these durations are then calculated as actual durations. Added together to reach. If this duration is found to be longer than the target duration, the media object is deleted from the tree. If this duration is shorter than the target duration, the media object is added to the tree.

  In this embodiment, this pruning or growth of the tree is performed according to the metadata of interest values associated with the media object.

  If the actual duration is longer than the target duration, the following process is performed.

  1) If the difference between the target duration and the actual duration is less than the duration of the shortest media element in the tree, the process ends.

  2) Otherwise, a list of media objects in the tree is created and this list is sorted in the order of the “interest value” property.

  3) The media object with the lowest value is deleted from the tree (assuming this object is not one or more of the other media objects in the list) and the actual duration is recalculated. The difference between the target duration and the actual duration is recalculated and the above steps are repeated until the difference is shorter than the duration of the shortest media element remaining in the tree.

  If the actual duration is shorter than the target duration, the following process is performed.

  A) The query in the section is such that “interest value”> 0.6 by deleting its last “AND” operator and the subsequent condition, or if there is no “AND” operator It is modified by adding an “OR” operator followed by the condition. A new tree of the current section is then created in the same way as the original tree. This process is repeated until the actual duration is longer than the target duration. Thereafter, the steps 1) to 3) are executed.

  Once this pruning or growth is performed, the second iteration ends.

  It will be apparent that the third iteration will only generate a tree with media objects 0.99.

  When all sections are entered with media object metadata and arranged in order according to the query, constraints and user preferences are provided, and the template populator is detected in the leaves of the tree generated in three iterations of the loop. Then, the edit decision list (FIG. 11) is output (step 78).

  The edit decision list (FIG. 11) created by the template populator program module (46) is passed to the content synthesizer module (48). In this example, the content synthesizer module outputs scenes one by one in a streamed video presentation or concatenates scenes together to create a program file.

  Content synthesizers provide a process for automatically synthesizing a collection of media elements into a consumer-oriented media product. A synthesizer is similar to a conventional non-linear editing system in that it uses an edit decision list (FIG. 11) to locate a set of media elements and incorporate them into a finished media product. However, traditional editing tools are used only after production and reside in the editor's facility, while content synthesizers are at the edge of the network or, in fact, on consumer devices (such as PCs or personal digital recorders). It is a portable lightweight application that can be deployed inside. This means that the media product can be synthesized at the point of delivery with considerable advantages in terms of the level of personalization that can be achieved. Furthermore, any metadata changes including templates, media objects or user profiles can be dynamically reflected in the resulting media product.

  When invoked by the consumer, the content synthesizer passes the user profile and template to the template populator that processes them as described above and returns the edit decision list to the synthesizer. The edit decision list is then parsed to find out what media elements are needed and how they interact with each other. A timeline with various references to various media elements in the content storage device 50 is then assembled according to this information. Transition effects (examples are defined in the ANSI / SMPTE 258M / 1993 standard) apply to the media when needed. Finally, the caption text and graphical overlay are analyzed and then rendered into a static image that is added to the timeline with high priority ensuring that they are visible on any other graphic element .

  Once the timeline is assembled, the personal presentation is rendered in real-time using a suitable media compositor technology such as Apple Quicktime or Microsoft DirectShow. The modular nature of the compositor means that a content synthesizer could be more easily implemented within an embedded architecture such as a set top box.

  The graphical user interface provided by the preferred embodiment of the present invention will now be described with reference to FIGS.

  FIG. 12 shows an application window displayed when the media product application program is executed on the computer of FIG. It will be appreciated that it provides a number of well-known graphical user interface components.

  FIG. 13 shows the display after the user selects “Project” by using the file menu or the “Open” button to select a project to open. In this preferred embodiment, the directory structure in content storage device 50 is organized into the highest level project directories, each one or more “media bins”, ie, a collection of media elements, and one or more It includes a directory that holds “template bins”, that is, a directory that holds a collection of templates. In FIG. 13, when the user opens the “Paper Mill, 03.db” project, a project sub-window is displayed, two media bins called “paper media” and “juriamark”, and “paper template” A “paper mill 03.db” project directory is displayed indicating that it contains one template bin called.

  FIG. 14 shows an additional sub-window that opens when the user double-clicks the “paper media” media bin. This subwindow displays a plurality of rows, each containing a number of column entries. As can be seen, moving the mouse pointer over the sub-window title bar gives the user a list of all attributes contained in the structured data model for describing media elements. By clicking with the mouse, the user can select which elements in the structured data model are displayed in the “paper media” media-bin subwindow. Note that the structured data model closely corresponds to the structured data model under the metadata illustrated in FIG.

  When one of the thumbnails provided in the first column of each row is double-clicked, the metadata associated with the media element that the thumbnail represents is shown in an additional sub-window (FIG. 15). The value given to each attribute can be edited. In addition, some of the attributes have a vocabulary associated with them (ie, a limited number of predetermined values for the attribute). If this is the case, right-clicking on the value field will display a drop-down list containing those predetermined values. The user can select one of those predetermined values from this list in the usual way.

  When the user selects multiple (and possibly all) media elements contained in the subwindow depicted in FIG. 14, right-clicks and selects the “Relationship” option, an additional subwindow is included in the selection. Added to the display (FIG. 16) showing the relationship between various media elements. A boundary line in which a group of media elements is depicted in a manner similar to the method illustrated in FIGS. 4A-4G by enclosing the corresponding thumbnail within a solid line boundary, including an arrow symbol near the upper left corner. The sequence is shown similarly except for. Also depicted are two diamonds representing the cause object as described with respect to the first embodiment. The cause object on the left indicates that a cause / influence relationship exists between the media elements JKY08a and JKY08b.

  In the improved version of the first embodiment described above, the user can right click on the diamond shape representing the causal object, and then options: “backward”, “forward”, “bidirectional”, “disconnect all” And a list of “deletes”. The first three refer to how the tree expansion process described with respect to the first embodiment expands the tree when it selects a media element that has a causal object attached to it. If “forward” is selected, the selection of JKY08a means that JKY08b has been added to the tree, and vice versa. “Backward” has the opposite effect. That is, if JKY08b is selected, JKY08a is added to the tree, but vice versa. When “bidirectional” is selected, selecting either one increases tree expansion on the other. The “front”, “back”, and “bidirectional” options are stored as additional rows in the metadata of FIG. 3B.

  FIG. 17 shows the “Vocabulary” sub-window that the user can open by selecting the “Vocabulary” option from the edit menu. The underlying structured data model for the media object is again seen in the upper left box of the subwindow. The user may define a vocabulary, that is, a list of acceptable values for any attribute selected by the user in the data model. In FIG. 17, the user has selected the attribute “place” and defined the values “canal side”, “paper mill”, and “Sainsbury” as possible values for that attribute. This vocabulary is associated with the currently selected project and stored in the object-oriented database (FIGS. 2 to 54).

  FIG. 17 also illustrates the extensibility of the structured data model. This extensibility is enabled by allowing the user to add specific attributes to the domain (recall that examples of such attributes are also included in FIG. 3). Domain specific attributes are shown in the bottom left box of the subwindow depicted in FIG. 17, such as the “Add” and “Delete” buttons that may be used to add or remove specific attributes to the domain. Has been. Certain attributes of the domain are added to the data model associated with the currently open project.

  FIG. 18 depicts the steps followed by the user to generate template metadata similar to the template metadata illustrated in FIG. As with the generation of media object metadata, the user first opens the project subwindow. The user then double-clicks on one of the displayed template bins to launch the “Query Builder” sub-window (in the example shown, the user double-clicked on the “Paper Template” template bin) ). It can be seen that the tree structure on the left side of the sub-window is similar to the table of FIG. 6 where each “new scene” corresponds to a section.

  Circles in the tree structure represent filters that select media objects from media bins. There are three types of filters as follows:

i) Circles containing “=” or> or <, or ≠ These filters are for attributes selected by the user, such as equal to or greater than the value shown along the circle. Select all media objects in the media bin that have a value.

ii) A circle containing (*) representing a filter that selects all media objects in the media bin containing the strings shown along the circle. The logical operator used to combine the results of the filtering operation is a combiner. Known as and is displayed as a triangle. There are three types of combiners called “Random”, “Sequential” and “Either / or”.

  Sequential combiners simply display the results of the filtering operation immediately below the tree in the order in which they are positioned in the tree. Each sequential combiner has a straight arrow passing through it. Examples are found in the first and third sections of the “Machine (860)” template shown in FIG.

  A random combiner displays the results of one or more filters below it in the tree in a random order. The random combiner has a zigzag arrow through it. An example can be found in the second section of the machine (860) template found in FIG.

  Either / or the combiner (not shown) selects the result of the two filter branches below it. Either / or the combiner has a split arrow through it.

  FIG. 19 shows how a user can modify the tree to create template data similar to the template data shown in FIG. Once the filter is added, the user is given a choice of attributes and can enter the desired value.

  Another element that can be added to the tree is a “funnel”. This behaves like a filter and allows only a user-defined number of randomly chosen media objects up to a higher level in the tree.

  If filters are nested, each applies instead to the result of the filter further down the tree.

  To create an edit decision list, the user clicks a button labeled “!” In the toolbar while a template is selected. The edit decision list is then added to the edit decision list history subwindow that is displayed when the appropriate button on the toolbar is pressed. When the user right-clicks on the edit decision list in the window, the user has the option to play the video according to the edit decision list, the option to preview the storyboard (results illustrated in FIG. 20), and the edit decision list. You are given the option to import or export and the option to edit the edit decision list in its text format or delete the edit decision list.

  Additional functionality is provided in an improved version of the preferred embodiment of the present invention. A graphical user interface that provides this functionality to the user is depicted in FIG. When the word “Jacky, problem” is selected and a filter is inserted, the user is presented with a check box. When the check box is checked, the user is given a list of drop-down controllers as shown. By selecting one of those controllers, the user can apply a “controller” to the filter. The controller can be provided multiple times (it will be seen that the first controller has been applied twice in the template depicted in FIG. 21).

  FIG. 22 shows that a set of slider control devices can be moved by the user. For example, the “Conceptual_PlotValue” associated with controller # 1 can be reduced by dragging the top slider further to the left position (or vice versa). By changing the position of the slider, the user can change the template generated by the query builder directly from the media product preview window (FIG. 22) rather than having to redefine the filter. This not only provides an easier way to update the filters in the template, but also forms the basis for a simplified user interface that does not require access to a query builder or the like. In other words, non-technical users are provided with a means to generate a video presentation of a custom item.

  The present invention may be embodied in many different ways. For example, the embodiments described above may be modified in one or more of the ways listed below to provide alternative embodiments of the present invention (this list is by no means exhaustive).

  i) In the embodiment, the input program module generates a data structure having a predetermined structure. In some embodiments, the predetermined structure is provided by a document that conforms to XML schema guidelines.

  ii) The constraint terms in the template may be variable by the user. For example, the user may be provided with a graphical user interface that allows the user to select the duration of the media product that he wishes to see. The corresponding value can then be added to the template object by the template populator program.

  iii) One constraint that may be added to each selection or combination of selections would be a constraint on the number of times a specified media element appears in the finished media product. For example, it may be indicated that any media element must appear only once in a specified media product.

Claims (10)

A template for identifying a plurality of desired features of a media product, wherein the media product is automatically configured according to the template having a plurality of sections where each section may contain a query, comprising:
The media product includes a plurality of media elements;
Each of the plurality of media elements includes a digital collection of stored media data;
The stored digital collection of media data represents content that can be drawn to the human sense, eg, text, audio, sound, picture, or video,
The method
Iterating, by a digital processor, detecting each section in the template and returning a plurality of selected media objects as a result of executing a query in the section;
Wherein each of the plurality of media objects, and the corresponding media element associated with the digital metadata about each media element,
Each of the digital metadata associated with the selected plurality of media objects is
Identification data for the associated media object that identifies the associated media object;
Relationship data indicating the relationship between the content represented by the media element corresponding to the current media object and the content represented by the media element corresponding to the associated media object;
including,
As indicated by the digital metadata associated with said selected plurality of media objects, among a plurality of media objects the associated form some or all of the plurality of media objects, wherein the selected Depending on the relationship , arranging the plurality of media elements or the identifiers of the plurality of media elements corresponding to the selected plurality of media objects in a media product;
How to automatically configure media products, including The digital metadata associated with each media object further includes content data indicating content represented by a corresponding media element of the media object;
The selected media objects returned by the step of detecting each section in the template and performing a query to return the selected media objects are dependent on the content data The method of claim 1, wherein The step of arranging includes
3. The method according to claim 1, wherein the plurality of media elements are arranged by the digital processor to determine the order in which a user views or listens to content represented by the selected plurality of media elements. The method according to item. 4. A method according to any one of claims 1 to 3 , wherein the plurality of media elements comprises video data. A media product configuration device for configuring a media product,
The media product includes a plurality of media elements;
Each of the plurality of media elements includes a digital collection of stored media data;
The stored digital collection of media data represents content that can be drawn to the human sense, eg, text, audio, sound, picture, or video,
The media product component device is:
One or more storage devices for storing digital metadata associated with each of the plurality of media elements;
Wherein the digital metadata is a relationship indicating one or more relationships between the content represented in each of the media elements and the content represented in one or more other related media elements. Including data,
One or more digital processors in communication with the one or more storage devices;
Comprising
The one or more digital processors are:
A template identifying a plurality of desired characteristics of the media product, each section having a plurality of sections that may contain a query, and detecting each section in the template and executing the query in the section; Iterating back multiple selected media objects,
Here, each of the plurality of media objects, which are associated with the digital metadata for the corresponding media element and each media element,
As indicated by the digital metadata associated with said selected plurality of media objects, among a plurality of media objects the associated form some or all of the plurality of media objects, wherein the selected Depending on the relationship , by arranging in the media product the media elements or identifiers of the media elements corresponding to the selected media objects,
A media product configuration device operable to configure a media product according to the template. The relationship data, and content that is represented by one of the media element may indicate the type of causal relationship between the content represented by the associated media element device according to claim 5. 7. The apparatus of claim 5 or claim 6, wherein the one or more digital processors are operable to provide a user with an interface that allows the user to input the relationship data. The metadata is stored in a database that stores the plurality of media objects ;
Further, the one or more processors query the database to obtain a plurality of identifiers of a plurality of media elements,
The apparatus according to any one of claims 5 to 7 , wherein the associated metadata of the plurality of media elements satisfies one or more conditions specified in the query. The database includes an object-oriented database,
The apparatus of claim 8 , wherein metadata for each media element is stored as an object in the object-oriented database. A content storage device for storing a plurality of media elements;
The apparatus of claim 9 , wherein the metadata of each media element includes a pointer to the location of the media element in the content storage device.

Images