AU2003220747B2 - Multimedia Template Representation - Google Patents

Description

S&F Ref: 640608

AUSTRALIA

PATENTS ACT 1990 COMPLETE SPECIFICATION FOR A STANDARD PATENT Name and Address of Applicant: Actual Inventor(s): Address for Service: Invention Title: Canon Kabushiki Kaisha 30-2, Shimomaruko 3-chome, Ohta-ku Tokyo 146 Japan Lena Qian Spruson Ferguson St Martins Tower Level 31 Market Street Sydney NSW 2000 (CCN 3710000177) Multimedia Template Representation ASSOCIATED PROVISIONAL APPLICATION DETAILS [33] Country [31] Applic. No(s) AU 2002950446 [32] Application Date 26 Jul 2002 The following statement is a full description of this invention, including the best method of performing it known to me/us:- 5815c -1- MULTIMEDIA TEMPLATE REPRESENTATION Copyright Notice This patent specification contains material that is subject to copyright protection.

The copyright owner has no objection to the reproduction of this patent specification or related materials from associated patent office files for the purposes of review, but otherwise reserves all copyright whatsoever.

Technical Field of the Invention The present invention relates to multimedia documents and, in particular, to multimedia templates adapted for use in auto-editing systems.

Background The introduction of digital video cameras has lead to increased demand for software applications able to edit, or otherwise make use of the raw captured video content. Consumer users of digital video cameras are not versed in professional editing techniques, and are consequently faced with significant difficulties when attempting to produce high quality video productions from their raw captured material.

One technique for overcoming this problem is to use auto-editing methods which utilise auto-editing templates that have been designed and compiled by experienced graphic artists or video professionals. Auto-editing templates automatically guide autoediting systems, thereby enabling the consumer to simply select desired raw video content clips, as well as a desired auto-edit template, and the auto-editing system then produces a high quality video production.

Auto edit templates are characterised by a particular theme or style, which is typically dictated to a large extent by a cutting rule incorporated into the template.

Current templates can typically only make use of a single cutting rule. Auto editing systems and software applications typically have a fixed number of different templates incorporated therein, and additional templates cannot be added at will by the consumer.

180703 640608.DOC -2- Furthermore, the templates are typically one-off special purpose code modules that cannot be easily changed or manipulated by the user. As a consequence, although the auto editing approach enables the user to produce high-quality output, the limited number of available themes significantly constrain the user.

Summary of the Invention It is an object of the present invention to substantially overcome, or at least ameliorate, one or more disadvantages of existing arrangements.

According to a first aspect of the invention, there is provided a tree-structured template for directing an auto-editor to auto-edit media items into a production, said template comprising data elements including: a relational data element at a non-leaf node of the tree structure, said relational data element being adapted to establish relationships between the data elements; an authored media data element at a leaf node of the tree structure, said authored media data element referencing one of the media items designated at template authoring time; and an input media data element at another leaf node of said tree structure, said input media data element being adapted to be replaced by at least one of a sub-set of the media items after both the template and the sub-set of media items have been input to the auto-editor process and (ii) being arranged to accommodate a cutting rule according to which the designated media item and the sub-set of said media items are processed into the production.

According to another aspect of the invention, there is provided a method of autoediting media items using a tree-structured template comprising data elements including a relational data element, and media data elements including an authored media data element and an input media data element, said method comprising the steps of: 180703 640608.DOG -3establishing the tree structure according to relationships defined by the relational data element, wherein the relational data element is at a non-leaf node of the tree structure and the media data elements are at leaf nodes of said tree structure; (ii) traversing the tree structure to thereby arrive at a leaf node of said tree structure, wherein: if the authored media data element is located at the leaf node, the method comprises fetching a designated one of the media items to replace the authored media data element, said designation having been established at template authoring time; if the input media data element is located at the leaf node, the method comprises fetching a sub-set of the media items to replace the input media data element; and (iii) invoking a cutting rule to thereby process at least one of the designated media item and the sub-set of media items into the production.

According to another aspect of the invention, there is provided a data structure for a tree-structured template adapted to direct an auto-editor to auto-edit media items into a production, said data structure comprising data elements including: a relational data element at a non-leaf node of the tree structure, said relational data element adapted to establish relationships between the data elements; an authored media data element at a leaf node of the tree structure, said authored media data element referencing one of the media items designated at template authoring time; and an input media data element at another leaf node of said tree structure, said input media data element adapted to be replaced by at least one of a sub-set of the media items after both the template and the sub-set of media items have been input to the autoeditor process and (ii) being arranged to accommodate a cutting rule according to which 180703 640608.DOC -4- N the designated media item and the sub-set of said media items are processed into the Sproduction.

According to another aspect of the invention there is disclosed a method of producing a tree-structured template instantiation for directing an auto-editor to auto-edit media items into a production, the method comprising the steps of: establishing a root template comprising data elements including at least one of authored media data elements, input data elements and relational data elements, wherein Sthe authored media data elements are established when the template is authored and the authored media data elements are not changed at run time; the input data elements are placeholders that are replaced by user selected media at run time; and the relational data elements establish the relationship among media items associated with child data elements of the relational data elements, as well as the relationship among the child data elements; and assigning a value to a variable parameter of the root template to thereby form the desired template instantiation.

According to another aspect of the invention there is disclosed a method of producing a tree-structured template instantiation for directing an auto-editor to auto-edit media items into a production, the method comprising the steps of: 140206 640608 140206 640608 4a IN establishing a template hierarchy comprising templates comprising data elements

O

including at least one of authored media data elements, input data elements and relational data elements, wherein the authored media data elements are established when the template is authored and the authored media data elements are not changed at run time; the input data elements are placeholders that are replaced by user Sselected media at run time; and Othe relational data elements establish the relationship among media items associated with child data elements of the relational data elements, as well as the relationship among the child data elements; and assigning a value to a variable parameter of a root template in the template hierarchy to thereby form the desired template instantiation.

According to another aspect of the invention there is disclosed a method of producing a tree-structured template for directing an auto-editor to auto-edit media items into a production, the method comprising the steps of: selecting data elements including at least one of authored media data elements, input data elements and relational data elements, wherein the authored media data elements are established when the template is authored and the authored media data elements are not changed at run time; the input data elements are placeholders that are replaced by user selected media at run time; and the relational data elements establish the relationship among media items associated with child data elements of the relational data elements, as well as the relationship among the child data elements; and 140206 640608 140206 640608 -4b IND arranging the selected data elements in a hierarchical tree structure to thereby form the template.

SThe disclosed tree-structured template data structure can be used to define a tree- _structured hierarchical multi-media auto-editing template which provides flexibility in template authoring and manipulation. The data structure .also facilitates generation of a multi-dimensional GUI representation that is particularly advantageous for template editing and manipulation.

C",

Other aspects of the invention are also disclosed.

Brief Description of the Drawings One or more embodiments of the present invention will now be described with reference to the drawings, in which: Fig. 1 is a schematic block diagram of a general purpose computer upon which arrangements described can be practiced; Fig. 2 shows a functional block diagram for a video editor system; Fig. 3 shows a GUI for a video-editing module in Fig. 2; Fig. 4 shows a generic functional block diagram of an auto-editing template; Fig. 5 shows a data structure for media/relational data elements in a hierarchical template (hereinafter referred to as an HTEMPLATE); 140206 640608 Fig. 6 shows functional characteristics of an example of an attribute of a media data element; Fig. 7 depicts functional relationships between media data elements in an example of an HTEMPLATE; Fig. 8 shows functional characteristics of an example of a method definition of a media data element; Fig. 9 shows an example of a layout view associated with an HTEMPLATE; Fig. 10 shows an example of a temporal view associated with an HTEMPLATE; Fig. 11 shows an example of a logical representation of an HTEMPLATE; Fig. 12 shows an example of a temporal view of a multi-cutting HTEMPLATE; Fig. 13 shows an example of an HTEMPLATE class hierarchy; Fig. 14 shows a method for building a playlist at play time using an

HTEMPLATE;

Fig. 15 shows a functional block diagram of a system used to construct a playlist from an HTEMPLATE; and Fig. 16 depicts data elements and their graphical representations in Figs. 10 and 11.

Detailed Description including Best Mode Where reference is made in any one or more of the accompanying drawings to steps and/or features, which have the same reference numerals, those steps and/or features have for the purposes of this description the same function(s) or operation(s), unless the contrary intention appears.

Fig. 1 shows a general-purpose computer system 800 upon which a new "Hierarchical template" (hereinafter referred to as an HTEMPLATE) having a tree structure, and a method of auto editing a video production using the HTEMPLATE, are preferably practiced. This approach allows the process described in Fig. 14 to be 180703 640608.DOC -6implemented as software, such as an application program executing within the computer system 800. In particular, the steps of the method of auto editing a video production using the HTEMPLATE are effected by instructions in the software that are carried out by the computer. The instructions may be formed as one or more code modules, each for performing one or more particular tasks. The software may also be divided into two separate parts, in which a first part performs the auto editing of a video production using the HTEMPLATE, and a second part manages a user interface between the first part and the user. The software may be stored in a computer readable medium, including the storage devices described below, for example. The software is loaded into the computer from the computer readable medium, and then executed by the computer. A computer readable medium having such software or computer program recorded on it is a computer program product. The use of the computer program product in the computer preferably effects an advantageous apparatus for auto editing a video production using the

HTEMPLATE.

The computer system 800 comprises a computer module 801, input devices such as a keyboard 802 and a mouse 803, output devices including a printer 815 and a display device 814. A Modulator-Demodulator (Modem) transceiver device 816 is used by the computer module 801 for communicating to and from a communications network 820, for example connectable via a telephone line821 or other functional medium. The modem 816 can be used to obtain access to the Internet, and other network systems, such as a Local Area Network (LAN) or a Wide Area Network (WAN).

The computer module 801 typically includes at least one processor unit 805, a memory unit 806, for example formed from semiconductor random access memory (RAM) and read only memory (ROM). The module 801 also includes input/output (I/O) interfaces including a video interface 807, and an I/O interface 813 for the keyboard 802 and a mouse 803 and optionally a joystick (not illustrated), and an interface 808 for the 180703 640608.DOC -7modem 816. A storage device 809 is provided and typically includes a hard disk drive 810 and a floppy disk drive 811. A magnetic tape drive (not illustrated) may also be used. A CD-ROM drive 812 is typically provided as a non-volatile source of data. The components 805 to 813 of the computer module 801, typically communicate via an interconnected bus 804 and in a manner which results in a conventional mode of operation of the computer system 800 known to those in the relevant art. Examples of computers on which the described arrangements can be practised include IBM-PC's and compatibles, Sun Sparcstations or alike computer systems evolved therefrom.

Typically, the application program is resident on the hard disk drive 810 and read and controlled in its execution by the processor 805. Intermediate storage of the program and any data fetched over the network 820 may be accomplished using the semiconductor memory 806, possibly in concert with the hard disk drive 810. In some instances, the application program may be supplied to the user encoded on a CD-ROM 822 or a floppy disk 824 and read via the corresponding drives 812 or 811 respectively, or alternatively may be read by the user via the network 820 using the modem device 816.

Still further, the software can also be loaded into the computer system 800 from other computer readable media. The term "computer readable medium" as used herein refers to any storage or transmission medium that participates in providing instructions and/or data to the computer system 800 for execution and/or processing. Examples of storage media include floppy disks, magnetic tape, CD-ROM, a hard disk drive, a ROM or integrated circuit, a magneto-optical disk, or a computer readable card such as a PCMCIA card and the like, whether or not such devices are internal or external of the computer module 801.

Examples of transmission media include radio or infra-red transmission channels as well as a network connection to another computer or networked device, and the Internet or Intranets including email transmissions and information recorded on websites and the like.

180703 640608.DOG -8- The method of auto editing a video production using the HTEMPLATE may alternatively be implemented in dedicated hardware such as one or more integrated circuits performing the functions or sub functions of auto editing a video production using the HTEMPLATE. Such dedicated hardware may include graphic processors, digital signal processors, or one or more microprocessors and associated memories.

Fig. 2 shows a functional block diagram for a video editor system that can be implemented using the computer system 800 of Fig. 1. Media content from a media store 102 is selected and input to a video editing module 114 which in turn produces a playlist 112. The video editing module 114 can be implemented as a software application running on the processor 805 in the PC module 801 in Fig. 1. The playlist 112 is composed of a number of video and audio tracks. A video track contains a list of video clips in sequence, each of which is defined by a reference to a media file path, in and out points, effects and so on. A sound track references a number of sound files. The user is able to edit the playlist 112 by changing a video clip contained in the playlist, for example by adding an effect thereto, adjusting in and out points thereof, adding a background sound track and so on.

The editing function performed by the editing module 114 can be performed in a manual fashion by entering the media from the media store 102 as depicted by an arrow 132 to a manual editing process 108 which outputs, as depicted by an arrow 110 the playlist 112. Alternately, an auto editing technique can be applied, in which the media from the store 102 is input, as depicted by an arrow 106, to an auto editing process 116, which then outputs, as depicted by an arrow 134 the playlist 112. In order to perform the auto editing technique, a template must be selected from a template database 120, and the selected template is input, as depicted by an arrow 118 to the auto editing process 116.

The auto editing technique provides a higher quality output production (this production being defined by the playlist 112) than is produced by a typical consumer user who uses 180703 640608.DOC -9the manual editing process 108. The auto editing technique requires no user intervention apart from requiring the user to input clips from the media store 102 and a template from the template database 120.

The media content in the store 102 can have associated metadata in a metadata store 104 which describes key features of the media content, such as highlight points, feature tracking information and so on. The metadata in the store 104 is used by the auto editing process 116 in a manner determined by the template selected from the database 120.

A template authoring process 128 can be used to produce templates for the template database 120. The template authoring process 128 can be implemented as a separate software application from the video editing module 114 and is also run on the processor 805 of the PC module 801 of Fig. 1. The template authoring process 128 supports a Graphic User Interface (GUI) and can have special-purpose authoring functions which make template authoring convenient for the user. In this arrangement, the template authoring software application 128 provides the GUI on the display 814 (see Fig. 1) thereby enabling either complete authoring of a template, or merely adjustment of template parameters. Alternately, the template authoring process 128 can be included within the video editing module 114 in which case, both the editing module 114 and the template authoring tool 128 are implemented in a single software application running on the processor 805. In this arrangement the authoring process typically has a more limited set of options for template authoring including, for example, an ability to change a background effect for a selected media clip from a "black/white" filter to an "old movie" effect being a coloured production exhibiting a high noise content.

A bilateral arrow 126 indicates that the template authoring process 128 can both create a new template and deposit the template into the template database 120, or alternately, can edit a template from the template database 120 and then store the edited 180703 640608.DOC template back into the template database 120. The template authoring process 128 can also import/export templates from an external template database 122, as depicted by a bidirectional arrow 130.

When a template is downloaded from the template database 120 to the template authoring process 128, template parameters can be adjusted by the template authoring process 128, and the edited template can then be provided, as depicted by an arrow 133, to the auto editing process 116. This can be used for template testing, for example.

The media store 102 and the metadata store 104 can be realised as database structures on the hard disk drive 810 in the computer system 800. The external template database 122 is typically stored in a corresponding database structure on the hard disk drive 810, or alternately, on an external database (not shown) which is accessible by the computer system 800 over the network 820. The template database 120 is also typically implemented as a database structure on the hard disk drive 810. The editing module 114 and the template authoring module 128 are realised by corresponding software applications running on the processor 805. The playlist 112 is a data structure that is typically stored in the dynamic memory 806, and is graphically represented by the playlist summary bar 220 which forms part of the GUI 234 (see Fig. 3) that is displayed on the video display 814 (see Fig. 1).

Fig. 3 shows a GUI 234 for the video-editing module 114 of Fig. 2. The GUI 234 is presented on the video display 814 in a graphical window 204 which contains a browser pane 230, a template pane 200, a display region 206, a clip summary bar 216, and a playlist summary bar 220. The user typically drags a template such as 202 from the template pane 200 into the playlist summary bar 220. Thereafter, the user drags a video clip 232 from the browser pane 230 into the playlist summary bar 220. Other video clips can also be dragged from the browser pane 230 into the playlist summary bar 220. The auto-editing process 116 can be initiated by another control (not shown).

180703 640608.DOC -11- When the production depicted by the playlist summary bar 220 is previewed in the display area 206, a scrubber 226 on the playlist summary bar 220 traces over the playlist summary 220 in a progressive fashion, indicating which clip is being shown in the display area 206. As each particular clip 224 in the playlist summary bar 220 is shown in the display area 206, the clip summary bar 216 shows a representation of the clip 224' including corresponding in and out points 228' and 222' respectively, as well as a synchronised scrubber control 210 that is synchronised with the scrubber 226. The user can adjust the amount of the clip 224' present in the production, ie., in the playlist summary bar 220 by adjusting the in and out points 228' and 222'.

Fig. 4 shows a generic functional block diagram of an auto-editing template 300 from the template database 120. The template 300 comprises media data elements 302, which fall into two categories, namely "authored" media data elements 304, and "input" media data elements 308 (hereinafter referred to as INPUT media data elements). The authored media data elements 304 include, for example, start and end titles. Authored media data elements are typically designed when the template 300 is created or authored.

The authored media data elements 304 are not changed at run time (which is also referred to as play time). The INPUT media data elements 308 are placeholders that are replaced (the replacement operation being referred to as SETTING), as depicted by an arrow 314, by user selected media 312 at play time. The user- selected media 312 are generally selected by the user from the media store 102, part of which is depicted in the browser window pane 230 (see Fig. A template structure 310 defines relationships among the various data elements in the template 300, these relationships being depicted by dashed lines 316 to 326. The relationships 316-326 can include spatial relationships, temporal relationships, and logical relationships. The nature of these relationships are described in more details in relation to Fig. 7.

180703 640608.DOG -12- The template 300 also contains cutting rules 306 that determine which segments of the media items 312 selected from the media store 102 are retained in the playlist 112.

Exemplary cutting rules can produce, for example, a 30 second summary production that summarises highlights from the user selected media 312 which may be a collection of holiday video clips. Alternately, the template 300 may be designed to produce a silent movie style from the user-selected media 312.

The cutting rules 306 require certain parameters to be SET at template authoring time. Examples of such parameters include cropping time, skip time and so on. The user selected media 312 may have associated metadata 322 (stored in the metadata store 104) which includes highlight points, feature tracking periods and so on. The cutting rules 306 can make use of this metadata 322, to exclude video content lying outside of the aforementioned highlight and feature tracking periods, and (ii) compile the retained material into the playlist or timeline 112. Highlight and other metadata often reflect segments of video content that are considered to be of interest by the user. Accordingly, this metadata 322 is used by the cutting rules 306 in a manner that reflects the interest displayed by the user. The template 300 can also contain other rules 324 which can, for example, couple sprite animation to tracked objects which are identified by feature tracking metadata. It is, therefore, apparent that the metadata 322 can be used for various purposes, such as for guiding the cutting rules 306 and/or for guiding the other rules 324 used for animation incorporation.

The template depiction in Fig. 4 is generic, and does not show how the template 300 itself is structured, nor does it show the data structures from which the template 300 is constructed. Current templates are typically hard-coded and hand-crafted on an individual basis by experienced template authors working together with software programmers. Clearly this approach to template construction does not lend itself to manipulation of template parameters by users, or to re-use of template code.

180703 640608.DOC -13- Fig. 5 shows a flexible data structure arrangement 400, usable in relation to both authored media data elements 304 and INPUT media data elements 308. The data structure 400 is a building block used for provision of the flexible auto editing templates referred to as HTEMPLATES. The data structure 400 relates to a generic media/relational data element 402 which includes or is associated with a set of "attributes" 406, and a set of"method definitions" 408, as depicted by solid lines 412 and 414 respectively. When the generic media/relational data element 402 is directed to an INPUT media data element, it further includes or is associated with metadata 404, as depicted by a dashed line 410. Having regard to terminology, when the media/relational data element 402 is directed to general media data elements, it will be referred to as the "media data element" 402. When the media/relational data element 402 is particularly directed to INPUT media data elements, it will be referred to as the "INPUT media data element" 402. When the media/relational data element 402 is particularly directed to relational data elements, it will be referred to as the "relational data element" 402.

HTEMPLATES comprise, from a data structure perspective, a set of media/relational data elements exemplified by the media/relational data element 402.

From a functional perspective, HTEMPLATES contain all the functional data elements described in relation to Fig. 4, however construction of HTEMPLATES using the data structures described in relation to Figs. 5 to 6 and 8 result in a template whose data structure is regular, modular and flexible.

Fig. 6 shows functional characteristics of an exemplary attribute 416 that is a member of the set of attributes 406. The exemplary attribute 416 establishes spatial behaviour 502 of the media item associated with the media/relational data element 402, and spatial relationships between the media item associated with the media/relational data element 402 and media items associated with the other template media/relational data elements in an HTEMPLATE (see Fig. Spatial behaviour relates to such matters as 180703 640608.DOC -14the size and location of the media item associated with the data element 402 in the display area 206, as well as relative positions and sizes of the media item associated with the media data element 402 with respect to media items associated with the other HTEMPLATE media data elements in the display area 206.

The exemplary attribute 416 also establishes temporal behaviour 504 of the media item associated with the media/relational data element 402, as well as temporal relationships between the media item associated with the media/relational data element 402 and media items associated with the other HTEMPLATE media/relational data elements. The temporal behaviour relates to aspects such as duration of a video clip, transition time between two clips and so on. If the data element 402 is a relational data element, then one example of a temporal relationship established by the attribute 416 can be to require all media items associated with the media data elements that are dependent upon the relational data element to play in sequence, or in parallel. Another temporal relationship established by the attribute 416 in regard to media items associated with a relational data element can be to set the overall duration of the production to be seconds, this being defined at a "root" relational node of the HTEMPLATE. This is described in more detail in regard to Fig. 11.

The exemplary attribute 416 also establishes logical behaviour and relationships 506 between the media item associated with the media/relational data element 402 and media items associated with the other media/relational data elements in the HTEMPLATE. This is described in more detail in relation to Fig. 11.

Fig. 7 illustrates, on an exemplary GUI 614 associated with the template authoring process 128, functional relationships between the media item associated with the media/relational data element 402 (which is associated with the attribute 416) and media items associated with the other media/relational data elements in an exemplary HTEMPLATE. Having regard to an exemplary GUI window 602 shown on the display 180703 640608.DOC 814, the spatial behaviour 502 finds expression, as depicted by a dashed arrow 610, in a spatial view pane 612 in the GUI window 602, and in particular relates to a position of a title media item 616 associated with the media/relational data element 402. The temporal function 504 produces, as depicted by a dashed arrow 600, a temporal view pane 604 in the GUI window 602, and in particular relates to a position of a track media item 614 associated with the media/relational data element 402. The logical function 506 produces, as depicted by a dashed arrow 606, a logical view pane 608 in the GUI window 602, and in particular relates to a position of a data item 618 associated with the media/relational data element 402. The spatial, temporal and logical view panes 612, 604 and 608 are described in more detail with reference to Figs. 9, 10 and 11 respectively.

Fig. 8 shows functional characteristics of an exemplary method definition 418 from the set of method definitions 408 for the media/relational data element 402. The method definition 418 describes dynamic behaviour of the media item associated with the media/relational data element 402 in the context of the HTEMPLATE. The method definition 418 is a process that can be implemented in a script, or a programming language like C++ or alternatively by a script like ICIe or JavaScript®. The method definition 418 can include parameters 708, which are both internal and/or relate to media items associated with the other media/relational data elements and/or attributes in the HTEMPLATE. The method definition 418 can also incorporate a media item transformation 702, a cutting rule 704, and other rules 706. The parameters 708 can relate to either, or both the media item associated with the media/relational data element 402 with which the method definition 418 is associated, and media items associated with the other media/relational data elements in the HTEMPLATE and their associated attributes.

180703 640608.DOC -16- The data structures and functional relationships that have been described in relation to Figs. 5-8 provide a basis for defining HTEMPLATES in a general form given by the following logical formulation: T= E M) [1] where: T is an HTEMPLATE indicates a set of media/relational data elements included in the

HTEMPLATE;

indicates a set of attributes and/or method definitions included in a media/relational data element; E is a media/relational data element (exemplified by the data element 402); A is a set of attributes (eg. 406), each of which is associated with the corresponding media/relational data element; and M is a set of method definitions (eg. 408), each of which is associated with the corresponding media/relational data element.

Data elements E, if directed to media data elements, can be either visual or audio, and can be either authored, or INPUT. Authored visual data elements typically relate to titles, animations or scene mattes, while authored audio data elements typically relate to background sound tracks. An INPUT visual data element can accept, at template run time as depicted by the arrow 314, a random list of movie clips (ie. associated visual media items), and an INPUT audio data element can similarly accept, at template run time, a list of user selected music items (ie. Associated audio media items).

Examples of the attributes that the data element 402 can have, are: S Identification Default duration of the media item associated with the data element 180703 640608.DOC -17- 0 Synchronization (when to start and end, event-driven) 0 Speed Source file name if the data element is authored Position and dimension if the data element is a visual data element 0 Volume control if the data element is a audio data element The general template representation described by equation enables the HTEMPLATE to be represented by the three separate complimentary visual GUI representations in the layout view pane 612, the temporal view pane 604 and the logical view pane 608 in the GUI 614 of Fig. 7. The data structure 402 (see Fig. 5) described in relation to Figs. 5, 6, and 8, coupled with the general template representation depicted in Equation supports the tree (otherwise referred to as "logical") representation, the layout representation and the temporal representation, depicted in Fig. 7.

Fig. 9 shows an expanded layout view 800 in the full sized viewer display area 206 referred to in Fig. 3. Text, graphics, animations and so on can be defined in the template data structure by different media data elements in an HTEMPLATE and can consequently be specified to be at particular positions and orientations etc. within the boundary of the display area 206. In the absence of specific spatial definitions, media items associated with all visual data elements in the HTEMPLATE will be displayed in the viewer 206 in a default position which typically uses the entire display area 206. A title 914 in Fig. 9 is specified as occupying the bottom left hand section of the display area 206. For multi-view templates, different video clips can be displayed on different sub-divisions of the display area 206 at the same time. The layout (spatial) view 800 depicts spatial information associated with the media items associated with the data elements in the HTEMPLATE, but does not display any temporal information about these media items.

180703 640608.DOC -18- Fig. 10 shows a temporal view 1099 of the auto edit HTEMPLATE being considered. The temporal view 1099 is reminiscent, to some extent, of the playlist 112 that results from application of the HTEMPLATE to the user selected media items 312 (see Fig. 4) that are chosen by the user at run time in order to produce an output production. The media item associated with the media data element 402 and the media items associated with the other media data elements specified in the HTEMPLATE in accordance with equation are depicted in Fig. 10, in time sequence, in different tracks against a time axis 1000, and a track axis 1002. Different tracks can be created for movie clips, animations, text and audio and so on. The temporal view 1099 shows relative time relationships between the media items associated with the various media data elements in the HTEMPLATE as well as the time behaviour of each particular media item associated with a corresponding media data element. The temporal view 1099 depicts both visual media items 1004 associated with the visual media data elements in the HTEMPLATE, and audio media items 1006 associated with the audio media data elements in the HTEMPLATE. Media items associated with INPUT media data elements are shown in a shaded form (for example 1080 and 1098), while media items associated with authored media data elements are shown without shading (for example 1074, 1078, 1082, 1090 and so on).

Since the media items 312 associated with the INPUT media data elements 308 (see Fig. 4) are not defined at template authoring time, the INPUT media elements 308 are thus merely placeholders at authoring time, and thus the specific play duration for each media item associated with the INPUT media data elements is undefined in the temporal view 1099. Accordingly, the physical length of each media item (such as 1080) in the temporal view associated with a corresponding INPUT media data element does not represent a relative time duration, but only a general time-of-occurrence relationship between the media items associated with the various media data elements. The unshaded 180703 640608.DOC -19rectangles 1074 and 1078 represent media items such as animations, text, graphics, images and movies, associated with the authored visual media data elements. The single boundary shaded rectangle 1080 represents a media item associated with the INPUT visual media data element. The unshaded double boundary rectangle 1090 represents a media item associated with the authored audio media data element while the double boundary shaded rectangle 1098 represents a media item associated with the INPUT audio media data element.

Fig. 11 shows a logical (tree) representation 900 of the HTEMPLATE. The tree representation 900 shows media data elements at leaf nodes of the tree structure. Non-leaf nodes are relational data elements indicating whether their subordinate media data elements (children) are, for example, in parallel or series relationships with each other (this being indicated by a or respectively) in the respective relational nodes. A relational data element defines the relationship among media items associated with its child data elements, as well as the relationship among the child data elements themselves.

Since the tree representation 900 shows, in a 1:1 fashion, both the logical relationship between data elements, and the corresponding logical relationship between media items associated with said data elements, only the former relationships will be referred to, with the latter relationships being understood to be inferred unless explicitly stated otherwise.

From a practical perspective, a relational data element has subordinate child data elements playing in sequence, in parallel, or in other arrangements, in an auto-edited production.

The data elements and their legends in the graphic representation as used in Figs.

and 11 are summarised in Fig. 16.

The tree structure is established by the logical functional block 506 as described in relation to Fig. 6.

Fig. 11 commences with a root node 910, that is a relational data element designated as a data element, indicating that the children deriving therefrom are 180703 640608.DOC presented or played in parallel. A group of data elements 932 flowing from the root node 910 depicts visual media data elements in the presentation, while a group of data elements 934 flowing from the relational data element 910 represents audio media data elements in the presentation. The group 932 of visual media data elements commences with a relational node 912 that is designated as an node, indicating that children derived therefrom proceed in sequence. Accordingly, a start title 914 is followed timesequentially by a relational data element 916 which is, itself, then followed timesequentially by a title 918.

The relational data element 916, designated as a data element, has two children that proceed in parallel with each other. Accordingly, a user defined INPUT media data element 920 proceeds in parallel with a relational element 915 that in turn defines two authored media data elements 922 and 924 proceeding sequentially. It is noted that the parallel progression of the visual data element 920, with the sequentially proceeding visual data elements 922 and 924, form a time-sequential step in the sequential progression of the visual data elements 914, 916 and 918. The relational data elements, shown as ovals at non-leaf nodes of the tree structure are not limited to defining sequential or parallel relationships, but can also describe other types of relationships between the media data elements. Thus, for example, the data elements 920, and the sequentially proceeding data elements 922 and 924, which are presently, in Fig. 11, defined as playing once only in parallel by the relational node 916, could alternately be played in parallel over and over again if the relational node 916 so specifies.

Turning to the group 934 of audio data elements, a user provided (INPUT) audio media data element 928 proceeds in parallel with an authored audio media data element 930, as dictated by the relational data element 926, which is designated as a data element.

180703 640608.DOC -21- Returning to Fig. 10, it is recalled that the temporal representation 1099 of the HTEMPLATE is shown, with media items associated with various media data elements plotted against a time axis 1000 and a track axis 1002. The various data elements in the tree representation 900 (see Fig. 11) map to the temporal representation 1099 (see Fig.

10) as follows: 1074), (920-->1080), (918->1078), (922- 1082), (924-> 1084), (928->1098), and (930->1090): where: "914--1074" indicates that the authored visual media data element 914 (see Fig.

11) is mapped to the authored visual media data element 1074 (see Fig. It will be apparent that the three views described in relation to Figs. 9-11, when considered together in the GUI 614 of Fig. 7, provide a comprehensive and multidimensional picture of the HTEMPLATE being considered. Furthermore, the data structures defined by Figs. 5, 6 and 8 which are used to define the HTEMPLATE facilitate provision of these multi-dimensional views, as well as providing flexibility in template authoring and manipulation which is not available in hard-coded templates.

These data structures enable the user to manipulate attributes or method definitions associated with any HTEMPLATE data element, and accordingly, the resultant HTEMPLATE structure can be changed easily, and at virtually any time.

Furthermore, the cutting rule 704 is no longer associated with the HTEMPLATE itself, but instead is associated on a per-media data element basis, as described in relation to Figs. 5 and 8. Accordingly, an HTEMPLATE constructed using these data structures can, unlike currently available templates, incorporate more than one visual media data element, each of which can have a separate cutting rule.

180703 640608.DOC -22- Fig. 12, for example, shows a temporal view 1100 of an HTEMPLATE having three authored media data elements 1102, 1106 and 1110, and two INPUT media data elements 1104, 1108. The two INPUT media data elements 1104 and 1108 can each have a separate, and different cutting rule, and this can lead to a multi-theme video presentation.

In the event that media/relational data elements, along with their associated attributes and method definitions are defined in a database, an HTEMPLATE can easily be constructed therefrom when required. One example of an HTEMPLATE in Extended Mark-up Language (XML) format is illustrated in Appendix A. Table 1 indicates the functional relationship between data elements (DEs) in Fig. 11 and the code in Appendix A. It should be noted that the code in Appendix A uses tag pairs (such as <template> and </template> on lines 4 and 37 respectively.

Code Line No. Data Element in Fig. 11 5-36 910 6-27 912 28-35 926 7 914 8-25 916 26 918 29-31 928 32-34 930 9-16 920 17-24 915 18-21 922 22-23 924 Table 1.

180703 640608.DOC -23- In a practical setting, this HTEMPLATE can be stored in the external template database 122, and uploaded to the template database 120 for use in the editing system 114 described in relation to Fig. 2. The HTEMPLATE can be used by the auto editing module 116 in order to auto edit selected input media from the media database 102, thereby to produce the output playlist 112.

The general method of HTEMPLATE representation depicted by Equation together with the data structures described in relation to Figs. 5, 6 and 8 support definition of an HTEMPLATE class hierarchy. In this hierarchy, HTEMPLATE components can be reused, thereby significantly reducing the amount of effort required to produce, or author, HTEMPLATES that have common or similar characteristics. This derives from the fact that although all attributes and method definitions need to be SET, ie., specific values need to be ascribed to the attributes and method definitions, when a particular media data element is used in a specific HTEMPLATE, nonetheless the same generic media data element can be used in other HTEMPLATES, in which different values are ascribed to the aforementioned attributes and method definitions.

Thus, for example, in a first HTEMPLATE the title duration may be set to 6 seconds, and two other variables, "period 1" and "period 2" may be set to 4 seconds and 6 seconds respectively. The aforementioned variables, being used in the cutting rules associated with this first HTEMPLATE, would have a definitive effect in producing the characteristic style of the first HTEMPLATE. Accordingly, if the aforementioned variables were set to different values, say 2 and 4 seconds respectively, a subtly different style would be achieved. A more discernible effect would be achieved by, for example, changing from a black/white filter to an "old movie" filter that has added white noise. It is thus apparent that the same generic HTEMPLATE can achieve different styles by changing the values of attributes and method definitions therein.

180703 640608.DOC -24- Fig. 13 shows an exemplary HTEMPLATE class hierarchy 1200 in which two values, namely A and B determine the style achieved by HTEMPLATE instantiations.

The hierarchy 1200 can provide four separate HTEMPLATE instantiations 1208-1214 having different values for A and B. The first pair of HTEMPLATES 1208 and 1210 share a common value for A, but differ in terms of their values for B. The second pair of HTEMPLATES 1212 and 1214 share a common different value for A and have distinct values for B. In this manner, a root HTEMPLATE 1202, having a particular style, can spawn the HTEMPLATES 1208-1214 which share certain common characteristics with the root HTEMPLATE 1202 but which differ in terms of more fine grain type of effects.

The template author can set desired values for the variables A and B in the node 1202 by entering the desired values into a dialog box or drop down menu that is provided for that purpose in the GUI 234 (see Fig. Alternately, the user can set a desired value for the variable B in the nodes 1204 or 1206 by entering the desired values into the dialog box or the drop down menu.

It is noted that the root HTEMPLATE 1202 also sets general parameters (not shown) such as speed, audio volume, trimming head/tail time, and so on which are not presented as parameters to be changed in the nodes 1204 and 1206. The child HTEMPLATES 1208-1214 can, however, have different parameter values relating to different cutting rules for example. Thus, for example, "rhythmic" cutting rules basically alternate two cutting periods, generally entitled "period 1" and "period skipping a few seconds between cutting periods. Variation of the relative and absolute time values associated with the "period 1" and "period 2" variables can effect the resultant style of the production output by the auto editor. Segments that are cut (ie. extracted) from the raw media content are then inserted into the playlist 112.

The HTEMPLATES 1208-1214 are thus instances of the HTEMPLATE class represented by the root HTEMPLATE 1202. Each instantiated HTEMPLATE, say the 180703 640608.DOC HTEMPLATE 1208, has cutting rule variables A and B set to specific values. These values are set at the template authoring time. The root node 1202 and two intermediate nodes 1204 and 1206 represent HTEMPLATE classes, and the instantiated HTEMPLATES 1208-1214 are derived from these classes.

Although the description relating to Fig. 13 has been couched in terms of cutting rule variables, it is apparent that HTEMPLATE class hierarchies can be constructed in regard to parameters other than those associated with cutting rules.

HTEMPLATES can generally be saved as software data files, to the memory 810 in Fig. 1 these files being in any convenient format such as XML. If an HTEMPLATE is based upon a template class in a class hierarchy, it can be written in a particularly simple format, as exemplified by a code fragment in Appendix B. Line 7 of Appendix B indicates that the template to be defined, which is entitled XYZ (see line 6 of Appendix B) is based upon a pre-existing root template class entitled "SceneCluster" (which may in turn be depicted illustratively as 1204 in Fig. 13). Lines 7-10 in Appendix B depict how respective parameters "StartTitle", "EndTitle" and "Periodl time" in the template XYZ are to be set to values "filel", "file2", and "12 sees". These definitions result in a particular instantiation (see 1208-1214 in Fig. 13) of the pre-defined template SceneCluster. When the HTEMPLATE of Appendix B is loaded into the template authoring process 128, the instantiated HTEMPLATE in Appendix B will be added to the appropriate template class entitled "SceneCluster" (for example 1204 in Fig. 13) as a child node (for example 1210 in Fig. 13).

When auto-editing a production, the user selects an HTEMPLATE from the template database 120 (see Fig. as well as a number of media data elements from the media store 102. Thereafter, the user loads the selected HTEMPLATE and media data elements into the auto editor 116 as depicted by the arrows 118 and 106. Authored media data elements have their references to corresponding media items SET at template 180703 640608.DOC -26authoring time. INPUT media data elements in contrast have their references to corresponding media items SET at template run time, ie immediately prior to commencement of the auto-edit process.

When the auto-edit process commences, (ie at run time) the auto editor process 116 starts traversing the hierarchical template data element tree (see Fig. 11) in order to build the playlist 112. As each node in the template tree 900 is visited, its attributes are SET, and its method definitions are invoked in respect to the media items referenced by the node.

Consider a simple scenario as follows. An HTEMPLATE is defined, at authoring time, to have a number of authored media that have only attributes associated therewith no method definitions are associated with the authored media). The HTEMPLATE has, in addition, a single INPUT media data element. Turning to the authored media data element, in order to incorporate a start title, a reference to the relevant authored media is simply added to the HTEMPLATE at template authoring time, and the attribute of the authored media data element defining the duration of the production is SET. Turning to another authored media data element, a sprite animation contained in another authored media data element may have an associated transformation method definition to be executed. If, for example, the animation in the authored media data element is shorter than a required displaying period in the production, a temporal attribute associated with the animation media data element can dictate that the animation be repeated as much as necessary in order to conform to the required display period. Alternately, the animation can be presented at a slower speed, in order to conform to the required displaying period.

The media data element relating to the animation can have a transformation method definition associated therewith in order to describe how the animation should be played.

The aforementioned transformation definitions are made at template authoring time.

Considering run time, when the INPUT media data element is visited on the 180703 640608.DOC -27- HTEMPLATE tree (see Fig. 11) by the auto-edit process 116 (see Fig. 2) attributes of the INPUT media data element are SET, and the cutting rule method definition) is invoked. This scenario is illustrated in Fig. 14.

Fig. 14 shows a method 1600 for building the playlist 112 (see Fig. 2) the method 1600 being invoked at "run time", or what is alternately referred to as "play time". Preparatory to commencement of the method 1600, the user selects a desired HTEMPLATE from the template database 120 (see Fig. As a result of this selection, the authored media data elements in the selected HTEMPLATE, which were incorporated into the HTEMPLATE at template authoring time, will reference their corresponding media items in the media store 102. The user then selects desired media items (see 312 in Fig. 4) from the media store 102, also preparatory to commencing the method 1600. The INPUT media data elements in the HTEMPLATE, which were incorporated into the HTEMPLATE at template authoring time, act as placeholders for media clips 312 subsequently selected by the user (see Fig. 4) at play time. The method 1600 then commences with a start step 1602, after which the process proceeds to a step 1604 which visits the "next" template node (having regard to the tree representation of the HTEMPLATE in Fig. 11). The aforementioned "next" template node is, in the first instance, the HTEMPLATE root node (such as the node 910 in Fig. 11), after which other tree nodes are visited. A subsequent testing step 1606 determines whether the node presently being considered relates to an INPUT media data element, or an authored media data element.

If the node relates to an INPUT media data element, then the method 1600 is directed in accordance with an "INPUT" arrow to a step 1608. The step 1608 fetches, according to the aforementioned placeholder, the user selected media items 312 as depicted by the arrow 314 in Fig. 4, from the media store 102 (see Fig. Thereafter, the method 1600 proceeds to a step 1610 that invokes the method definitions and applies the 180703 640608.DOC -28attributes associated with the media data element. Returning to the step 1606, if the node presently being visited is associated with an authored media data element, then the method 1600 proceeds in accordance with an "AUTHORED" arrow to a step 1616. The step 1616 fetches, according to the aforementioned reference to the authored media item, the corresponding authored media item from the media store 102 (see Fig. After the method definitions associated with the present node have been invoked, and the appropriate attributes have been applied, the method 1600 proceeds to a step 1612 that determines if further nodes in the template tree have yet to be visited. If this is, the case, then the method 1600 is directed in accordance with a "YES" arrow back to the step 1604. If, on the other hand, the tree has been completely traversed, then, the method 1600 is directed in accordance with a "NO" arrow to a termination step 1614, at which point the playlist 112 has been constructed.

Fig. 15 shows a data flow diagram for the system described in relation to Fig. 2.

In the first instance, the user selects an HTEMPLATE 1418 from the template database 120 (see Fig. 2) as depicted by an arrow 118, and selects a number of media items from the media store 102. The selected template 1418 and the selected media items are conveyed, as depicted by the arrows 118 and 106 respectively, to the auto-editing module 116. The auto editor 116 traverses the tree structure associated with the template 1418 in order to build the playlist 112 as described in relation to Fig. 14. As each node in the tree is visited, its attributes are applied, and its method definitions are invoked, and in this manner, each node contributes incrementally to building the playlist 112. The INPUT media data elements are replaced (ie. SET), by user selected media at play time as depicted by the arrow 314 in Fig. 4.

As noted, INPUT media data elements have attributes associated therewith, and also have method definitions, in particular a specific cutting rule, also associated therewith. The cutting rule is applied to the INPUT media item when the node associated 180703 640608.DOC -29with the associate INPUT media data element is visited by the auto editor. In one arrangement, if a number of INPUT media data elements are selected, and only one of these data elements has an associate cutting rule, then as a default, the cutting rule will be applied to all the INPUT media data elements. The cutting rule operates on the INPUT media data element(s) adding cut segments from the INPUT media data element(s) to the playlist in accordance with the cutting algorithm used.

The cutting rule 1420 can, in practice, be implemented as a script in any software language, as long as the language used can support the interface to invoke primitive functions by means of the Application Programming Interfaces (APIs) 1404. A number of these primitive functions are illustrated in Appendix C. The primitive functions on lines 6-14 in Appendix C can be regarded as a library of functions specific to cutting rules. Thus, for example, Get-ighlighlntervals(ClipList) returns segments of a video clip that have been highlighted during capture of the video material. The cutting rule 1420 has function calls, depicted by the arrow 1412, to the APIs 1404. When the auto editor 116 visits the particular node in the tree 900 (see Fig. 11) relating to the cutting rule 1420, the APIs 1404 are executed in order to build the playlist 112. Extended APIs 1406 can also be incorporated, as depicted by an arrow 1408. Only extended cutting rules can invoke extended APIs.

An exemplary cutting rule is given in Appendix D. The cutting rule has been written in the ICI language. The code on lines 10 and 11 fetch a number of input media clips, after which a programming loop commences on line 12. The code on line 14 defines "trim" settings for both the head and the tail of each clip, after which line 16 returns intervals within which faces have been detected in the video clips. The code on lines 18 to 20 expand the detected intervals containing faces by 2 seconds on either side of the time instant at which faces are detected. The code on lines 21 and 22 add titles to the clips at each point that faces have been detected. The code on lines 23 to 31 then 180703 640608.DOC extract the video segments that have been constructed around each detected face, and add these extracted intervals to the playlist presently being constructed.

Three methods are now described for authoring an HTEMPLATE, namely reinstantiate an HTEMPLATE from the HTEMPLATE class, (ii) derive an HTEMPLATE class from a node in HTEMPLATE class hierarchy, or (iii) construct a HTEMPLATE class.

Method is a simple way to adjust the settings of the attributes in any HTEMPLATE from existing HTEMPLATE in the HTEMPLATE database 120. This can be performed as part of a video editor application. For instance in one arrangement, when the user selects, using a right-hand control button on the mouse 803 in Fig. 1, the desired HTEMPLATE 202 (see Fig. a corresponding dialog box is presented to the user. The appropriate HTEMPLATE attributes for adjustment are displayed in the dialog box together with all the facilities required to change the attributes settings, such as a scroll bar for volume, speed, trim head/tail time, and so on. Furthermore, a selected filter, transition and start/end title file names can also be made editable. Certain attributes are typically preset in the dialog box, since making too many attributes editable tends to be confusing to the users. The new HTEMPLATE is then added to the entry table in HTEMPLATE database.

Method (ii) derives an HTEMPLATE class from an HTEMPLATE class hierarchy as described in relation to Fig. 13. Fig. 13 is described in terms of simple static setting of parameters, however more complex arrangements can be used. Thus, for example, having regard to the template instance 1212 in Fig. 13, a condition can be defined in the template wherein the value of a parameter is checked. If, for example, the value of the parameter in question is greater than or equal to the numerical value "one", then a particular function which is one of a number of different possible functions can be 180703 640608.DOC -31 invoked. Thus, for example, one of a number of different filters can be applied to the production depending on the value of that particular parameter.

To be able to use the method the GUI 234 (see Fig. 3) displays a list of HTEMPLATE names, their attributes and cutting rule parameters. Some parameters are fixed so that a user cannot make changes thereto. Thus for example, one existing HTEMPLATE may set a minimum duration at four minutes, while another allows variable duration. The user desiring a 5 minute presentation can not use the first HTEMPLATE for this purpose, but can use the second by adjusting the duration parameter.

Method (iii) involves constructing an HTEMPLATE class in tree view. The user can use the template authoring function 128 (see Fig. 2) to create data elements of different kinds (such as authored media data elements, INPUT media data elements, and relational media data elements). The user can define attributes of the data elements, and write the method definitions (including cutting rules) of the data elements. If necessary, new APIs may be added to the auto-editor 116. During the construction of the HTEMPLATE, the default settings of the data element attributes are initialised. Then the new template class can be attached to the template hierarchy.

A particularly convenient template authoring tool is a text editor, which allows the user to create an HTEMPLATE in text format such as XML as shown in Appendix A.

A more sophisticated template authoring tool such as 128 in Fig. 2 supports a GUI such as that described in relation to Fig. 7. This GUI allows simpler creation of different kinds of data elements using the tree, spatial and temporal views, and enabling data element attributes to be set, method definitions to be added and so on.

The three methods to (iii) enable a range of HTEMPLATE creation options.

These options range from a simple authoring dialog box in a video editor to a separate 180703 640608.DOC -32template authoring tool with comprehensive GUI. This is a clear advantage over existing video editor applications.

Industrial Applicability It is apparent from the above that the arrangements described are applicable to the image processing industry.

The foregoing describes only some embodiments of the present invention, and modifications and/or changes can be made thereto without departing from the scope and spirit of the invention, the embodiments being illustrative and not restrictive.

In the context of this specification, the word "comprising" means "including principally but not necessarily solely" or "having" or "including" and not "consisting only of'. Variations of the word comprising, such as "comprise" and "comprises" have corresponding meanings.

180703 640608.DOC 33 Appendix A External template by XML description <template name "test"> <P id=pl> <title id="start" src="start_title.fls"> <inputMedia id=usermedia type=visual> <cuftingRule method definition SceneCluster <Period 1 time 8sec <Period2 time 4sec <speed percent= 120% <transition type=WIPE> </cuttingRule> </inputMedia>

<S>

<title type=BANNER event="zoom" text="Object Detected" fontSize=1 6pt color=red </title> <graphic begin=2s src="radar.swf" </graphic>

<IS>

<P

<title id="end" src="end title.fls"> <inputMedia type=audio> <default src="background_music.wav" </inputMedia> <sound> <default src="dingdong.wav" </sound>

<P

<IP>

</template> 180703 10703640608.DOC 34 Appendix B Example of a template defined in a file <Template Name XYZ> <BaseTemplate name SceneCluster> <StartTitle src filel <EndTitle src =file2> <Periodl time =12 secs> </Template> 180703 10703640608.DOC 35 Appendix C List of primitive functions.

Double GetDuration(clip); I ntervalList GetHighlighlntervals(CipList); IntervalL-ist GetFeatureTrackinglntervals(ClipList); I nse rtSce ne Matte (fi le na me); SetFilter(name); SetTransition(clip, name); SetTrim(clip, head, tail); SetDuration(clip, time); SetCutting Rule(name); 180703 10703640608.DOC 36 Appendix D New cutting rule <cutting rule name "Face Detection"> <script lang ici IntervalList CRFaceDetection(lnputMedia input, double start-time) n GetNumber~finputMediao; clip GetlnputArrayo; for (I 1 n; clip[I].SetTrim(1, trim head and tail by 1 second each p1l Get Face DetectedlIntervaIs I* returns a list of intervals 1, 2, each interval from p1l by 2 seconds on both beginning and end obj CreateTitle("c:\tmp\obj image. mov", "Face Detected"); m GetNumberOflntervals(pl); t start-time; for j interval pilo]; AddClipToVideoTrack(clip[l], t, interval); track GetAvailableVideoTracko; AddObject(obj, track, t); t t interval; /script> </cutting rule> 180703 10703640608.DOC

Claims (18)

1. A tree-structured template for directing an auto-editor to auto-edit media items into a production, the template comprising data elements including: a relational data element at a non-leaf node of the tree structure, the relational data element being adapted to establish relationships between the data elements; an authored media data element at a leaf node of the tree structure, the authored media data element referencing one of the media items designated at template authoring time; and an input media data element at another leaf node of the tree structure, the input media data element being adapted to be replaced by at least one of a sub-set of the media items after both the template and the sub-set of media items have been input to the auto-editor process and (ii) being arranged to accommodate a cutting rule according to which the designated media item and the sub-set of said media items are processed into the production.

2. A template according to claim 1, wherein the relationships between the data elements include at least one of spatial, temporal and logical characteristics of, and relationships between, the media items associated with at least one of the authored media data element and (ii) the input media data element.

3. A template according to claim 2, wherein the spatial characteristics include at least one of a size and a location, in a display area in which the production is presented, of at least one of the designated media item and (ii) a member of the sub-set of said media items. 180703 640608.DOC -38-

4. A template according to claim 2, wherein the spatial relationships include at least one of relative positions and relative sizes, in a display area in which the production is presented, of the designated media item and said sub-set of said media items.

5. A template according to claim 1, wherein the input media data element has at least one of a data element transformation and a rule establishing dynamic behaviour of at least one of the designated media item and (ii) a member of the sub-set of said media items in a display area in which the production is presented.

6. A method of auto-editing media items using a tree-structured template comprising data elements including a relational data element, and media data elements including an authored media data element and an input media data element, the method comprising the steps of: establishing the tree structure according to relationships defined by the relational data element, wherein the relational data element is at a non-leaf node of the tree structure and the media data elements are at leaf nodes of said tree structure; (ii) traversing the tree structure to thereby arrive at a leaf node of the tree structure, wherein: if the authored media data element is located at the leaf node, the method comprises fetching a designated one of the media items to replace the authored media data element, the designation having been established at template authoring time; if the input media data element is located at the leaf node, the method comprises fetching a sub-set of the media items to replace the input media data element; and (iii) invoking a cutting rule to thereby process at least one of the designated media item and the sub-set of media items into the production. 180703 640608.DOC -39-

7. A method according to claim 6 comprising a further step of: (iv) repeating the steps (ii) to (iii) until the entire tree has been traversed, thereby building the production.

8. A method according to either claim 6 or claim 7, wherein the relationships determine at least one of spatial, temporal and logical characteristics of, and relationships between, the media items associated with at least one of the authored media data element and (ii) the input media data element.

9. A method according to claim 8, wherein the spatial characteristics include at least one of a size and a location, in a display area in which the production is presented, of at least one of the designated media item and (ii) a member of the sub-set of said media items.

A method according to claim 8, wherein the spatial relationships include at least one of relative positions and relative sizes, in a display area in which the production is presented of the designated media item and said sub-set of said media items.

11. A data structure for a tree-structured template adapted to direct an auto-editor to auto-edit media items into a production, the data structure comprising data elements including: a relational data element at a non-leaf node of the tree structure, the relational data element adapted to establish relationships between the data elements; 180703 640608.DOC N an authored media data element at a leaf node of the tree structure, the authored media data element referencing one of the media items designated at template authoring time; and _an input media data element at another leaf node of said tree structure, the input media data element adapted to be replaced by at least one of a sub-set of the media items after both the template and the sub-set of media items have been input to the auto- editor process and (ii) being arranged to accommodate a cutting rule according to which Sthe designated media item and the sub-set of said media items are processed into the production.

12. A method of producing a tree-structured template instantiation for directing an auto-editor to auto-edit media items into a production, the method comprising the steps of: establishing a root template comprising data elements including at least one of authored media data elements, input data elements and relational data elements, wherein the authored media data elements are established when the template is authored and the authored media data elements are not changed at run time; the input data elements are placeholders that are replaced by user selected media at run time; and the relational data elements establish the relationship among media items associated with child data elements of the relational data elements, as well as the relationship among the child data elements; and assigning a value to a variable parameter of the root template to thereby form the desired template instantiation. 140206 640608 140206 640608 -41-

13. A method of producing a tree-structured template instantiation for directing an auto-editor to auto-edit media items into a production, the method comprising the steps of: establishing a template hierarchy comprising templates comprising data elements including at least one of authored media data elements, input data elements and relational data elements, wherein Sthe authored media data elements are established when the template is (Ni Sauthored and the authored media data elements are not changed at run time; the input data elements are placeholders that are replaced by user selected media at run time; and the relational data elements establish the relationship among media items associated with child data elements of the relational data elements, as well as the relationship among the child data elements; and assigning a value to a variable parameter of a root template in the template hierarchy to thereby form the desired template instantiation.

14. A method of producing a tree-structured template for directing an auto-editor to auto-edit media items into a production, the method comprising the steps of: selecting data elements including at least one of authored media data elements, input data elements and relational data elements, wherein the authored media data elements are established when the template is authored and the authored media data elements are not changed at run time; the input data elements are placeholders that are replaced by user selected media at run time; and 140206 640608 140206 640608 -42- N the relational data elements establish the relationship among media c items associated with child data elements of the relational data elements, as well as the relationship among the child data elements; and arranging the selected data elements in a hierarchical tree structure to thereby form the template.

A tree-structured template substantially as described herein with reference to the accompanying drawings.

16. A data structure for a tree-structured template substantially as described herein with reference to the accompanying drawings.

17. A method of auto-editing using a tree-structured template substantially as described herein with reference to the accompanying drawings.

18. A method of authoring a tree-structured template substantially as described herein with reference to the accompanying drawings. DATED this fourteenth Day of March, 2006 CANON KABUSHIKI KAISHA Patent Attorneys for the Applicant SPRUSON&FERGUSON 140206 640608 140206 640608