US20130124999A1 - Reference clips in a media-editing application - Google Patents

Reference clips in a media-editing application Download PDF

Info

Publication number
US20130124999A1
US20130124999A1 US13/361,932 US201213361932A US2013124999A1 US 20130124999 A1 US20130124999 A1 US 20130124999A1 US 201213361932 A US201213361932 A US 201213361932A US 2013124999 A1 US2013124999 A1 US 2013124999A1
Authority
US
United States
Prior art keywords
clip
media
clips
angle
camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US13/361,932
Other languages
English (en)
Inventor
Giovanni Agnoli
Shaun M. Poole
Kenneth M. Carson
Colleen Pendergast
Brian Meaney
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Apple Inc
Original Assignee
Apple Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Apple Inc filed Critical Apple Inc
Priority to US13/361,932 priority Critical patent/US20130124999A1/en
Assigned to APPLE INC. reassignment APPLE INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: AGNOLI, GIOVANNI, CARSON, KENNETH M., MEANEY, BRIAN, PENDERGAST, COLLEEN, POOLE, SHAUN M.
Priority to PCT/US2012/058138 priority patent/WO2013074207A1/en
Priority to EP12788327.0A priority patent/EP2781102A1/en
Priority to JP2014541061A priority patent/JP5856690B2/ja
Priority to CN201280055762.0A priority patent/CN103931199B/zh
Priority to EP15199415.9A priority patent/EP3012838B1/en
Publication of US20130124999A1 publication Critical patent/US20130124999A1/en
Priority to JP2015242604A priority patent/JP6214619B2/ja
Abandoned legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/19—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier
    • G11B27/28—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • G11B27/034—Electronic editing of digitised analogue information signals, e.g. audio or video signals on discs
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/34—Indicating arrangements 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/27—Server based end-user applications
    • H04N21/274—Storing end-user multimedia data in response to end-user request, e.g. network recorder
    • H04N21/2743—Video hosting of uploaded data from client
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47205—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/8549—Creating video summaries, e.g. movie trailer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/16—Analogue secrecy systems; Analogue subscription systems
    • H04N7/173—Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal
    • H04N7/17309—Transmission or handling of upstream communications
    • H04N7/17336—Handling of requests in head-ends

Definitions

  • Digital graphic design, image editing, audio editing, and video editing applications provide graphical designers, media artists, movie and television directors, and other users with the necessary tools to create a variety of media content.
  • Examples of such applications include Final Cut Pro® and iMovie®, both sold by Apple, Inc. These applications give users the ability to edit, combine, transition, overlay, and piece together different media content in a variety of manners to create a resulting media project.
  • the resulting media project specifies a particular sequenced composition of any number of text, audio, image, and/or video content elements that is used to create a media presentation.
  • Various media editing applications facilitate such composition through electronic means.
  • a computer or other electronic device with a processor and computer readable storage medium executes the media editing application.
  • the computer generates a graphical interface whereby designers digitally manipulate graphical representations of the media content to produce a desired result.
  • a director, editor, etc. may film a single scene from multiple different angles using multiple different cameras.
  • Manually editing numerous media clips into a timeline, trying to figure out how the clips should be aligned and trimmed, and where to switch between the different cameras can be a difficult and time-intensive process.
  • the editor has gone through this process in multiple locations in a timeline or in multiple different media projects, the editor will be hesitant to make any changes, as this will require making the same correction in multiple places.
  • Some embodiments of the invention provide a media-editing application that allows a user to create a multi-camera media clip that includes several media clips captured by multiple different media capture devices (e.g., several cameras recording a scene simultaneously from different angles).
  • the application uses metadata of the media clips to automatically assign the media clips in the set to different groups (e.g., one group for each different camera), order and space out the clips in each group (e.g., in time order with gaps between the clips), and synchronize the clips across the groups (e.g., align the capture times of the clips in a first group with the capture times of the clips in a second group).
  • the media-editing application When a user adds such a multi-camera clip to a composite presentation, the media-editing application creates a new clip instance in the presentation that refers to the already-created multi-camera clip such that modifications to the multi-camera clip will affect the clip instance in the composite presentation.
  • the media-editing application provides various user interface tools for editing the multi-camera clip as well as the instance of the clip in the composite presentation.
  • a user specifies several different media clips (e.g., video clips, audio clips, still images, etc.) from which to generate a multi-camera clip.
  • the media-editing application automatically sorts the specified media clips into groups (sometimes referred to herein as “angles” or “camera angles”). Some embodiments use metadata of the media clips, such as user-entered angle or camera names or automatically-generated camera information, to sort the media clips into the groups (e.g., creating a group for each camera used to capture the different media clips). With the clips sorted, the application automatically generates an ordered sequence of the clips for each of the different groups.
  • the application utilizes clip metadata to perform this ordering (e.g., timing data generated by the capture device used to capture the media clips), so long as the metadata fits particular criteria indicating that it matches a correct ordering.
  • the media-editing application spaces the clips within the sequence, inserting gaps for time when the capture device was not capturing media (e.g., the time when a camera was shut off in between shots).
  • the media-editing application After generating the sequences for each of the groups, the media-editing application synchronizes the sequences across the different groups. That is, the application aligns the sequences such that content captured by a first capture device at a particular time is aligned with content captured by a second capture device at the same time. Some embodiments use the same timing metadata used for sequencing one or more of the groups to synchronize the different groups. When information indicating a time at which a clip was captured by a first device has the same format as corresponding information from a second device (e.g., timecode data or time of day information), the application uses this information to align the clips so long as the timing information meets certain criteria indicating that the information should be compared between devices.
  • the media-editing application fine tunes the synchronization using audio data captured by multiple devices. That is, the application identifies similar audio captured by multiple devices (e.g., microphones of different cameras) and aligns this audio. Because the audio and video from a particular camera are tied to each other, this has the effect of also aligning the video from the different devices.
  • the application identifies similar audio captured by multiple devices (e.g., microphones of different cameras) and aligns this audio. Because the audio and video from a particular camera are tied to each other, this has the effect of also aligning the video from the different devices.
  • the media-editing application of some embodiments When creating a multi-camera media clip from several different media clips, the media-editing application of some embodiments generates a reference data structure for the multi-camera media clip.
  • This reference data structure stores information about each of the groups to which clips are assigned.
  • the application stores the reference data structure as a series of nested clip structures, with each group stored in a separate clip structure within the reference clip structure.
  • the clip structure for a particular group then stores an ordered set of clip structures for the clips assigned to the group, using gap structures to indicate the spacing between the clips in some embodiments.
  • the reference clip structure stores information to indicate the synchronization between the different groups.
  • the media-editing application provides a graphical user interface (GUI) with an editing display area for editing the reference multi-camera media clip.
  • this editing display area is a composite display area also used for creating and editing a composite media presentation (also called a “media project” or “composite media project”) as a sequence of media clips (e.g., as a sequence of clips arranged along a timeline).
  • the composite display area is a trackless display area for editing composite presentations, while providing the appearance of tracks for the different groups of a multi-camera clip.
  • the composite display area of some embodiments is used to create media projects as sequences of media clips arranged by a user.
  • the user of the media-editing application can add a multi-camera media clip to a media project.
  • the media-editing application then adds to its definition of the media project a clip instance of the multi-camera clip that refers to the reference clip.
  • the clip instance indicates one of the ordered groups of the multi-camera clip as an active group (in some embodiments, the instance indicates both an active audio group and an active video group).
  • the media-editing application retrieves source content that corresponds to the clips of the active group.
  • the clip instance inherits properties of the reference multi-camera clip, as well as properties of the active group for the clip instance.
  • the clip instance also can have properties of its own applied.
  • the multi-camera clip reference of some embodiments will have its own format and frame rate, which are inherited by the clip instance.
  • the active group will have its sequence in the clip reference, and a user may have applied effects to the clips in the active group. A user can then apply additional effects to the active group in the clip instance, which will not affect other instances (even those with the same active group).
  • the user may place retiming effects, markers, or keywords on the clip instance that stay with the clip instance regardless of which group is active for the instance.
  • reference clips are described for multi-camera media clips, some embodiments also include reference clips for individual media clips or compound media clips.
  • the application first applies properties (e.g., effects) of the reference clip and then applies properties of the clip instance.
  • a multi-camera media clip instance When added to a composite presentation, a multi-camera media clip instance can be edited much like other media clips in the presentation.
  • the media-editing application provides additional or different editing functionalities for such multi-camera media clips.
  • the media-editing application of some embodiments provides multiple different ways for a user to modify which group is active for a clip instance in a composite presentation. The user may select the clip instance to produce a drop-down menu of the different groups or a display area that allows the user to cycle through video or images from the different groups.
  • some embodiments provide a multi-camera clip viewer (or “angle viewer”) in the GUI.
  • the multi-camera clip viewer allows a user to simultaneously view content (e.g., video images) from multiple different groups of a single multi-camera media clip. The user can use the viewer to switch the active group of a clip instance or to break a clip instance into two separate instances with different active groups.
  • Some embodiments provide GUI items that allow the user to determine whether selecting a particular group in the viewer will switch (or split and switch) audio, video, or both audio and video for a clip instance to the particular selected group.
  • the GUI items have different colors (or other indicators, such as patterns, symbols, etc.) for different types of content (e.g., for audio, video, or both) and these colors are reflected in highlights surrounding the active group's content (e.g., video images being played back) in the viewer, as well as in banks of groups shown in the viewer.
  • the active group's content e.g., video images being played back
  • some embodiments display selectable banks of groups that a user can select in order to rotate through different sets of the groups.
  • FIG. 1 illustrates a graphical user interface (“GUI”) of a media-editing application with the ability to create and use multi-camera media clips.
  • GUI graphical user interface
  • FIG. 2 illustrates a user adding an instance of the multi-camera clip to the composite presentation being created in the composite display area of FIG. 1 .
  • FIG. 3 illustrates a GUI of a media-editing application of some embodiments.
  • FIG. 4 illustrates the creation of a new project in the GUI of a media-editing application of some embodiments.
  • FIG. 5 conceptually illustrates an asset data structure for a media asset, as well as an event data structure for an event that contains the media asset.
  • FIG. 6 conceptually illustrates a nested sequence of clip objects created by the media-editing application of some embodiments for an imported media file.
  • FIG. 7 illustrates a timeline with a project title “New Project” that includes four clips.
  • FIG. 8 conceptually illustrates a subset of the data structures for the project illustrated in FIG. 7 .
  • FIG. 9 conceptually illustrates the objects of FIG. 8 nested in a conceptual timeline.
  • FIG. 10 conceptually illustrates a process of some embodiments for generating a multi-camera media clip from a set of media clips.
  • FIG. 11 illustrates user input for creating a multi-camera media clip.
  • FIG. 12 conceptually illustrates a process for assigning clips to different angles of a multi-camera media clip.
  • FIG. 13 conceptually illustrates the assignment of a set of nine clips to three different angles for a multi-camera media clip.
  • FIG. 14 conceptually illustrates a process of some embodiments for generating a sequence of ordered clips for the various angles of a multi-camera media clip.
  • FIG. 15 illustrates the sequencing of the clips from FIG. 13 .
  • FIG. 16 conceptually illustrates a process of some embodiments for performing the initial synchronization of a multi-camera media clip utilizing timing metadata.
  • FIG. 17 illustrates the alignment of the sequences from FIG. 15 using the metadata for clips shown in FIG. 13 .
  • FIG. 18 conceptually illustrates a process of some embodiments for synchronizing media clips across a set of angles within a multi-camera media clip.
  • FIG. 19 illustrates the audio synchronization of the angles from FIG. 17 .
  • FIG. 20 conceptually illustrates a subset of the data structures of a multi-camera media clip created in an event of the media-editing application of some embodiments.
  • FIG. 21 illustrates the opening of an angle editor display area in a graphical user interface (“GUI”) of some embodiments.
  • GUI graphical user interface
  • FIG. 22 illustrates a GUI over three stages in which a multi-camera preview display area is opened and options for the display area are accessed.
  • FIG. 23 illustrates the manual alignment of a media clip in a first angle track with a clip in a second angle track by moving the clip along its track.
  • FIG. 24 illustrates a user changing the monitoring angle for a multi-camera media clip according to some embodiments.
  • FIG. 25 illustrates a user enabling and disabling the audio for various angles of the multi-camera reference clip.
  • FIG. 26 conceptually illustrates a process of some embodiments for playing back (or skimming) a multi-camera media clip open in the angle editor.
  • FIG. 27 illustrates the use of a function to automatically synchronize a selected clip to a clip in a different angle in an angle editor.
  • FIG. 28 illustrates the use of a frame matching technique for synchronizing clips in an angle editor.
  • FIG. 29 illustrates a user adding an angle to a multi-camera media clip in a GUI.
  • FIG. 30 illustrates a user renaming an angle of a multi-camera media clip in the GUI of FIG. 29 .
  • FIG. 31 illustrates a user deleting an angle of a multi-camera media clip in the GUI of FIG. 29 .
  • FIG. 32 illustrates a user reordering the angles of a multi-camera media clip in the GUI of FIG. 29 .
  • FIG. 33 illustrates applying a black and white effect from an effects display area to a clip in an angle editor.
  • FIG. 34 illustrates a user adding a multi-camera media clip to a media project.
  • FIG. 35 conceptually illustrates a subset of the data structures that define a media project that includes a multi-camera clip instance.
  • FIG. 36 illustrates an example of a user adding multi-camera clip to a media project.
  • FIG. 37 illustrates a user opening the angle editor for a multi-camera clip in a composite display area.
  • FIGS. 38-41 illustrate a selected clip as well as different tabs of the inspector display area with the clip selected.
  • FIG. 42 illustrates a user switching the active video angle for a selected clip instance in the inspector.
  • FIG. 43 illustrates an angle viewer of some embodiments, as displayed while a user is editing a media project in a composite display area.
  • FIG. 44A-D illustrates various examples using the angle viewer to edit a multi-camera clip in a media project.
  • FIG. 45 conceptually illustrates a process of some embodiments for modifying the data structures for a media project when a user blades a clip instance using the angle viewer.
  • FIG. 46 illustrates the effect of changing the size of an angle viewer when the viewer is displaying four angle preview displays.
  • FIG. 47 illustrates the use of a menu to switch the video angle of a selected multi-camera clip instance in a composite display area.
  • FIG. 48 illustrates a different manner of switching video angles in a composite display area of some embodiments.
  • FIG. 49 illustrates the use of a menu to switch the active audio angle of a selected multi-camera clip instance in a composite display area.
  • FIG. 50 illustrates the deletion of a through edit in a composite display area in order to combine two multi-camera clip instances into a single instance.
  • FIG. 51 illustrates a user modifying the volume level of two clip instances that are separated by a through edit.
  • FIG. 52 illustrates a user applying a video effect to a pair of clip instances and in a composite display area.
  • FIGS. 53 and 54 illustrate the difference in default trim edits applied at the edit point between two standard media clips and two multi-camera clip references in which the edit point is a through edit.
  • FIG. 55 conceptually illustrates a process of some embodiments for rendering images from a multi-camera clip instance in a media project.
  • FIG. 56 conceptually illustrates a timeline of a media clip that includes various media clips, including a multi-camera clip instance.
  • FIG. 57 conceptually illustrates a render graph for a particular image of the media project from FIG. 56 .
  • FIG. 58 conceptually illustrates the software architecture of a media-editing application of some embodiments.
  • FIG. 59 conceptually illustrates an electronic system with which some embodiments of the invention are implemented.
  • Some embodiments of the invention provide a media-editing application that allows a user to create a multi-camera media clip that includes several media clips captured by multiple different media capture devices (e.g., several cameras recording a scene simultaneously from different angles).
  • the application uses metadata of the media clips to automatically assign the media clips in the set to different groups (e.g., one group for each different camera), order and space out the clips in each group (e.g., in time order with gaps between the clips), and synchronize the clips across the groups (e.g., align the capture times of the clips in a first group with the capture times of the clips in a second group).
  • the media-editing application When a user adds such a multi-camera clip to a composite presentation, the media-editing application creates a new clip instance in the presentation that refers to the already-created multi-camera clip such that modifications to the multi-camera clip will affect the clip instance in the composite presentation.
  • the media-editing application provides various user interface tools for editing the multi-camera clip as well as the instance of the clip in the composite presentation.
  • a media clip in some embodiments, is a data structure representing a piece of source media (e.g., the video contained in a video file, the audio contained in an audio file or video file, a still image, etc.).
  • the media-editing application provides graphical representations of these clips to the user, who uses the clips to create a composite media presentation (also referred to as a media project) by editing the clips together in a composite display area.
  • a multi-camera media clip of some embodiments is a specific type of media clip that amalgamates media clips from multiple different capture devices and aligns these media clips within the clip structure.
  • FIG. 1 illustrates a graphical user interface (“GUI”) 100 of a media-editing application with the ability to create and use multi-camera media clips.
  • GUI graphical user interface
  • the GUI 100 includes a media library 105 , a project display area 115 , and a preview display area 125 .
  • the media library 105 displays representations of a set of media clips available to the user for creating a media project.
  • the media clips may be displayed different in different embodiments. For instance, the media clips 106 - 108 in the media library 105 display one thumbnail image from the video that they represent, and display both the clip name and camera metadata below the thumbnail. Other embodiments may display the clip names.
  • the media clips 106 and 108 (“Clip A” and “Clip C” respectively) are both from “Camera 1 ” while media clip 107 (“Clip B”) is from “Camera 2 ”.
  • This camera name metadata may be automatically generated from the source video (e.g., at time of import into the media-editing application) or may be user-entered.
  • the project display area 115 (also referred to as a composite display area) provides a visual representation of a media project being created by the user of the media-editing application. Specifically, it displays one or more media clip representations along a timeline that represent the media clips added to the media project. In some embodiments, as shown here, the representations for the video clips in the timeline display one or more images from each video clip. The user can select clips in the media library 105 and add the clips to the project being created in the project display area 115 .
  • the preview display area 125 displays preview images from media clips in the media library 105 or the project in the project display area 115 . Both video clips and the media project represent sequences of images, and the media-editing application plays these images in the preview display area according to user input.
  • the user views the media project by moving a playhead 135 over the project in the timeline. In stage 110 of FIG. 1 , the playhead 135 is at approximately the 2.5 second mark of the project, and the application displays an image corresponding to this time in the project in the preview display area 125 .
  • stage 110 the user has selected all three of the clips 106 - 108 in the media library 105 and has activated a menu 130 that includes a “Create Multi-Camera Clip” option. While this operation is shown as a selection operation being performed by a cursor, one of ordinary skill in the art will recognize that similar operations may be performed by using touchscreen operations, and furthermore that additional operations (e.g., dragging and dropping items, using a hotkey, etc.) may be performed in the GUI of some embodiments in order to achieve this same result. In fact, this application shows many such operations as being performed in a particular manner (e.g., with a cursor), and one of ordinary skill will recognize that these operations may be performed in a variety of different ways through a variety of different input devices in various different embodiments.
  • this initiates a multi-camera clip creation process using the selected clips.
  • the media-editing application automatically sorts the specified media clips into groups (sometimes referred to herein as “angles” or “camera angles”). Some embodiments use metadata of the clips, such as user-entered angle or camera names, or automatically-generated camera information, to sort the media clips into the different groups (e.g., creating a group for each camera used to capture the different media clips).
  • the three selected clips 96 - 108 include metadata indicating that they are from either “Camera 1 ” or “Camera 2 ”.
  • the media-editing application utilizes this metadata to sort the clips into two groups (one for each of the cameras).
  • the application automatically generates an ordered sequence of the clips for each of the different groups.
  • Some embodiments utilize clip metadata to perform this ordering (e.g., timing data generated by the capture device used to capture the media clips), so long as the metadata fits particular criteria indicating that it matches a correct ordering.
  • the media-editing application spaces the clips within the sequence, inserting gaps for times when the capture device was not capturing media (e.g., the time when a camera was shut off in between shots).
  • the application examines metadata of media clips 106 and 108 to determine the order in which the clips were captured and the time between the end of the first clip and the start of the second.
  • the media-editing application After generating the sequences for each of the groups, the media-editing application synchronizes the sequences across the different groups. That is, the application aligns the sequences such that content captured by a first capture device at a particular time is aligned with content captured by a second capture device at the same time. Some embodiments use the same timing metadata used for sequencing one or more of the groups to synchronize the different groups. When information indicating a time at which a clip was captured by a first device has the same format as corresponding information from a second device (e.g., timecode data or time of day information), and the timing information meets certain criteria that indicate that the information should be compared between devices, then the application uses this information to align the clips.
  • timecode data or time of day information e.g., timecode data or time of day information
  • the media-editing application verifies that the groups are synchronized properly and fine tunes the synchronization using audio data captured by multiple devices. That is, the application identifies similar audio captured by multiple devices (e.g., microphones of different cameras) and aligns this audio. Because the audio and video from a particular camera are tied to each other, this has the effect of also aligning the video from the different devices.
  • the media-editing application whose GUI is shown in FIG. 1 automatically synchronizes the timing of clip 107 (the clip from “Camera 2 ”) with that of clips 106 and 108 (the clips from “Camera 1 ”).
  • Stage 120 of FIG. 1 illustrates the GUI 100 after the media-editing application has created a multi-camera media clip represented by clip representation 109 .
  • the GUI 100 includes additional features of some embodiments for interacting with the multi-camera media clip. Specifically, the GUI 100 displays the multi-camera media clip for editing in the composite display area 115 , and displays a multi-camera clip viewer (or “angle viewer”) 140 .
  • the multi-camera media clip editing display in the composite display area 115 displays the two groups of media clips in multi-camera clip 109 . These two groups are labeled as “Camera 1 ” and “Camera 2 ” and show the clips of each group laid out along a single timeline. While in some embodiments the composite display area is a trackless display area for editing composite presentations, the application provides the appearance of separate tracks for each of the groups in the multi-camera media clip.
  • the first group includes “Clip A” and “Clip C”—as shown, “Clip A” is approximately six seconds long (running from a time of zero seconds to a time of six seconds), while “Clip C” is a bit shorter (running from approximately 8.5 seconds to 12.5 seconds).
  • “Clip B”, assigned to the “Camera 2 ” group, is slightly longer than either of the other two clips, and runs from approximately 1.5 seconds to nine seconds in the timeline.
  • the clips of the second group do not start at time zero because the media-editing application has automatically aligned the clips of the two groups with each other using timing metadata stored with the source video as well as the audio of the clips, as described above.
  • a user can name the groups of clips in the multi-camera media clip, add or delete groups, add or remove clips from the groups, move the individual media clips within a sequence (e.g., to perform additional alignment of the groups), change the order of the groups within the multi-camera clip, apply effects (e.g., transforms, pixel modifications, etc.) to individual clips or groups of clips in the multi-camera clip, etc.
  • effects e.g., transforms, pixel modifications, etc.
  • the multi-camera clip viewer 140 allows a user to simultaneously view content (e.g., video images) from multiple different groups of a single multi-camera media clip.
  • content e.g., video images
  • the viewer shows preview images for each of the groups corresponding to a location of the playhead 135 in some embodiments.
  • the preview display area 125 displays preview images for one of the groups.
  • a particular group is selected (e.g., by a user) for playback in the preview display area.
  • the application displays images from the particular clip in the preview display area rather than from the selected group at the same time.
  • “Camera 1 ” is the selected group, and thus the media-editing application displays the video image from this group corresponding to the time of the playhead 135 in the preview display area 125 .
  • the viewer provides functionalities when editing a multi-camera clip in a composite presentation, which will be described below by reference to FIG. 2 .
  • the media-editing application of some embodiments When creating a multi-camera media clip from several different media clips (e.g., the multi-camera media clip 109 ), the media-editing application of some embodiments generates a reference data structure for the multi-camera media clip.
  • This reference data structure stores information about each of the groups to which clips are assigned.
  • the application stores the reference data structure as a series of nested clip structures, with each group stored in a separate clip structure within the reference clip structure.
  • the clip structure for a particular group then stores an ordered set of clip structures for the clips assigned to the group, using gap structures to indicate the spacing between the clips in some embodiments.
  • the reference clip structure stores information to indicate the synchronization between the different groups.
  • a user can add a multi-camera media clip to a media project.
  • the media-editing application then adds to the media project a clip instance of the multi-camera clip that refers to the reference clip.
  • the clip instance indicates one of the ordered groups of the multi-camera clip as an active group (in some embodiments, the instance indicates both an active audio group and an active video group).
  • the media-editing application retrieves source content that corresponds to the clips of the active group.
  • the clip instance inherits properties of the reference multi-camera clip, as well as properties of the active group for the clip instance.
  • the clip instance also can have properties of its own applied.
  • the multi-camera clip reference of some embodiments will have its own format and frame rate, which are inherited by the clip instance.
  • the active group will have its sequence in the clip reference, and the clips in the active group may have effects applied. A user can then apply additional effects to the active group in the clip instance, which will not affect other instances (even those with the same active group).
  • the user may place retiming effects, markers, or keywords on the clip instance that stay with the clip instance regardless of which group is active for the instance.
  • reference clips are described for multi-camera media clips, some embodiments also include reference clips for individual media clips or compound media clips.
  • the application first applies properties (e.g., effects) of the reference clip and then applies properties of the clip instance.
  • FIG. 2 illustrates a user adding an instance of the multi-camera clip 109 to the composite presentation being created in the composite display area 115 over two stages 210 and 220 .
  • the user has selected the clip 109 and dragged this clip into the composite display area 115 .
  • the user edits an instance of the multi-camera clip 109 into the presentation.
  • the user adds a clip to a timeline through user interface buttons, hotkeys, selecting a menu option, etc., rather than through a drag-and-drop operation.
  • the second stage 220 illustrates that the composite presentation now includes an instance of the multi-camera clip.
  • the clip representation shown in the composite display area 115 includes information indicating the active video and audio groups for the clip instance.
  • the active video group is Camera 2
  • the active audio group is Camera 1 .
  • the clip instances can have different active groups for video and audio.
  • the application renders the presentation using clips from one group for the audio and clips from a different group for the video.
  • a multi-camera media clip instance When added to a composite presentation, a multi-camera media clip instance can be edited much like other media clips in the presentation.
  • the media-editing application provides additional or different editing functionalities for such multi-camera media clips.
  • the media-editing application of some embodiments provides multiple different ways for a user to switch which group is active for a clip instance in a composite presentation. The user may select the clip instance to produce a drop-down menu of the different groups or a display area that allows the user to cycle through video or images from the different groups.
  • the GUI 100 also includes the multi-camera clip viewer 140 , which displays images for multiple different groups in a multi-camera clip corresponding to a particular time in the clip.
  • the video images shown in the viewer 140 correspond to the images at that particular time of the multi-camera clip.
  • the multi-camera viewer 140 may be used for editing multi-camera clip instances in the composite display area.
  • the user can use the viewer to switch the active group of a clip instance or to break a clip instance into two separate instances with different active groups.
  • the GUI items 240 allow the user to determine whether selecting a particular group in the viewer will switch (or split and switch) the active audio and/or video group for a clip instance to the particular selected group.
  • the GUI items have different colors (or other indicators, such as patterns, symbols, etc.) for different types of content (e.g., for audio, video, or both) and these colors are reflected in highlights surrounding the active group's content (e.g., video images being played back) in the viewer, as well as in banks of groups shown in the viewer.
  • buttons indicate A, V, and A/V, and these symbols (“A” and “V”) are illustrated in the group previews shown in the viewer as well as in the bank 245 .
  • some embodiments display selectable banks of groups. A user can select one of the banks in order to rotate through different sets of the groups. In this case, as there are only two groups, there is no need for a user to switch between different sets.
  • FIGS. 1 and 2 illustrate examples of the creation and use of multi-camera clips in the media-editing application of some embodiments.
  • Section I describes the graphical user interface of a media-editing application of some embodiments.
  • Section II describes the data structures used by the media-editing application of some embodiments.
  • Section III describes the creation of multi-camera media clips.
  • Section IV then describes editing multi-camera reference clips, while Section V describes editing multi-camera clip instances in a media project.
  • Section VI describes how the media-editing application of some embodiments renders multi-camera media clips.
  • Section VII describes the software architecture of the media-editing application of some embodiments.
  • Section VIII describes an electronic system with which some embodiments of the invention are implemented.
  • FIG. 3 illustrates a GUI 300 of a media-editing application of some embodiments with additional features as compared to those described above.
  • GUI 300 is only one of many possible GUIs for such a media-editing application.
  • the GUI 300 includes several display areas which may be adjusted in size, opened or closed, replaced with other display areas (e.g., an angle viewer as shown in FIGS. 1 and 2 ), etc.
  • the GUI 300 includes a clip library 305 , a clip browser 310 , a timeline 315 , a preview display area 320 , an inspector display area 325 , an additional media display area 330 , and a toolbar 335 .
  • the clip library 305 includes a set of folders through which a user accesses media clips that have been imported into the media-editing application.
  • the user can import media files into the application, at which time media clips are created for the imported files.
  • These files may be imported from a camera, an external drive (e.g., an external hard drive, a flash memory drive, a network drive, etc.), or a drive internal to the device on which the media-editing application operates.
  • an external drive e.g., an external hard drive, a flash memory drive, a network drive, etc.
  • the creation of clip data structures upon import will be described in further detail below in Section II.
  • Some embodiments organize the media clips in the clip library 305 according to the device (e.g., physical storage device such as an internal or external hard drive, virtual storage device such as a hard drive partition, etc.) on which the imported media files represented by the clips are stored. Some embodiments also enable the user to organize the media clips based on the date the media represented by the clips was created (e.g., recorded by a camera). As shown, the clip library 305 includes media clips from 2009 that are stored on the hard disk (in this case, the boot disk of the device on which the media-editing application runs).
  • the device e.g., physical storage device such as an internal or external hard drive, virtual storage device such as a hard drive partition, etc.
  • the clip library 305 includes media clips from 2009 that are stored on the hard disk (in this case, the boot disk of the device on which the media-editing application runs).
  • users may group the media clips into “events”, or organized folders of media clips. For instance, a user might give the events descriptive names that indicate what media is stored in the event (e.g., the “Vacation” event shown in clip library 305 might include video footage from a vacation).
  • the media files corresponding to these clips are stored in a file storage structure that mirrors the folders shown in the clip library.
  • some embodiments enable a user to perform various clip management actions. These clip management actions may include moving clips between events, creating new events, merging two events together, duplicating events (which, in some embodiments, creates a duplicate copy of the media to which the clips in the event correspond), deleting events, etc.
  • some embodiments allow a user to create sub-folders of an event. These sub-folders may include media clips filtered based on tags (e.g., keyword tags). For instance, in the vacation event, all media clips showing statues are tagged by the user with a “statue” keyword, and underneath the “Vacation” folder is a keyword item for this tag.
  • the clip browser 310 allows the user to view clips from a selected folder (e.g., an event) of the clip library 305 .
  • a selected folder e.g., an event
  • the event folder “Vacation” is selected in the clip library 305 , and the clips belonging to that event are displayed in the clip browser 310 .
  • the user can create multi-camera clips in the clip browser 310 by selecting several clips in the browser and then selecting a multi-camera clip creation option. This creation process will be described in detail below in Section III.
  • Some embodiments display the clips as thumbnail filmstrips, as shown in this example.
  • a cursor or a finger on a touchscreen
  • the user can skim through the clip. That is, when the user places the cursor at a particular horizontal location within the thumbnail filmstrip, the media-editing application associates that horizontal location with a time in the associated media file, and displays the image from the media file for that time.
  • the user can command the application to play back the media file in the thumbnail filmstrip in some embodiments.
  • thumbnails for the clips in the browser display an audio waveform underneath the clip that represents the audio of the media file.
  • the audio plays as well.
  • the user can modify one or more of the displayed thumbnail size, the percentage of the thumbnail occupied by the audio waveform, whether audio plays back when the user skims through the media files, etc.
  • some embodiments enable the user to view the clips in the clip browser in a list view instead of the filmstrip view. In this view, the clips are presented as a list (e.g., with clip name, duration, etc.). Some embodiments also display a selected clip from the list in a filmstrip view in the browser (e.g., above the list) so that the user can skim through or playback the selected clip.
  • the timeline 315 (also called a composite display area or project display area) provides a visual representation of a media project being created by the user of the media-editing application. Specifically, it displays one or more geometric shapes that represent one or more media clips that are part of the composite presentation.
  • the timeline 315 of some embodiments includes a primary lane 350 (also called a “spine”, “primary compositing lane”, or “central compositing lane”) as well as one or more secondary lanes 355 - 365 (also called “anchor lanes”).
  • the spine 350 represents a primary sequence of media which, in some embodiments, does not have any gaps.
  • the clips in the anchor lanes are anchored to a particular position along the spine (or along a different anchor lane).
  • Anchor lanes may be used for compositing (e.g., combining pixels of one image with pixels of a different image, as done in green-screening), B-roll cuts (i.e., cutting away from the primary video to a different video whose clip is in the anchor lane), audio clips, or other composite presentation techniques.
  • compositing e.g., combining pixels of one image with pixels of a different image, as done in green-screening
  • B-roll cuts i.e., cutting away from the primary video to a different video whose clip is in the anchor lane
  • audio clips or other composite presentation techniques.
  • the user can add media clips from the clip browser 310 into the timeline 315 in order to add the clip to a project represented in the timeline.
  • the user can perform further edits to the media clips (e.g., move the clips around, split the clips, trim the clips, apply effects to the clips, etc.).
  • this clip in the timeline refers to the reference clip in the clip browser.
  • Some embodiments implement such reference clips for other clips in addition to the multi-camera clips (e.g., for compound clips, individual clips, etc.).
  • the length (i.e., horizontal expanse) of a clip in the timeline is a function of the length of media represented by the clip. As the timeline is broken into increments of time, a media clip occupies a particular length of time in the timeline. As shown, in some embodiments the clips within the timeline are shown as a series of images. The number of images displayed for a clip varies depending on the length of the clip in the timeline, as well as the size of the clips (as the aspect ratio of each image will stay constant).
  • the user can skim through the timeline or play back the timeline (either a portion of the timeline or the entire timeline).
  • the playback (or skimming) is not shown in the timeline clips, but rather in the preview display area 320 .
  • the preview display area 320 (also referred to as a “viewer”) displays images from media files that the user is skimming through, playing back, or editing. These images may be from a project in the timeline 315 or from a media clip in the clip browser 310 . In this example, the user has been skimming through the project in the timeline 315 , and therefore the application displays an image corresponding to the time of the playhead 340 in the preview display area 320 . As shown, some embodiments display the images as large as possible within the display area while maintaining the aspect ratio of the image.
  • the preview display area 320 also includes video editing and playback tools in some embodiments. Selectable items are included for rewinding, fast forwarding, playing/pausing, etc.
  • some embodiments include a set of tools 370 for editing the size and shape of images (e.g., video images) that have been added to a media project.
  • these tools 370 include a transform tool, a crop tool, and a distortion tool.
  • transform selectable item When the transform selectable item is selected, a user can modify the height and width of the source images within the output images.
  • the crop selectable item is selected, the user can select the portion of the source images that are displayed in the output images.
  • the distortion tool is selected, the user can distort the shape of the source images in the output images.
  • the inspector display area 325 displays detailed properties about a selected item and allows a user to modify some or all of those properties.
  • the selected item might be a clip, a composite presentation, an effect, etc.
  • the clip 345 in the clip browser 310 is selected, and thus the inspector displays information about this media clip.
  • This information includes the clip name, file format, resolution (1920 ⁇ 1080), frame rate (29.97 fps), date created, audio information, etc. for the selected clip.
  • different information is displayed depending on the type of item selected (e.g., video clip, audio clip, still image, multi-camera clip, media project, etc.).
  • Some embodiments may include different tabs in the inspector (e.g., video, audio, and info tabs).
  • the additional media display area 330 displays various types of additional media, such as video effects, transitions, still images, titles, audio effects, standard audio clips, etc.
  • the set of effects is represented by a set of selectable GUI items, with each selectable GUI item representing a particular effect.
  • each selectable GUI item also includes a thumbnail image with the particular effect applied.
  • the display area 330 is currently displaying video effects that a user can apply to a clip.
  • the toolbar 335 includes various selectable items for editing a project, modifying what the application displays in one or more display areas, etc.
  • the right side of the toolbar includes various selectable items for modifying the type of media displayed in the additional media display area 330 .
  • the illustrated toolbar 335 includes items for video effects (currently selected), visual transitions between media clips, photos, titles, generators and backgrounds, etc.
  • the toolbar 335 includes an inspector selectable item that toggles the display of the inspector display area 325 as well as items for applying retiming operations to a clip, adjusting color, and other functions.
  • the left side of the toolbar 335 includes selectable items for media management and editing. Selectable items are provided for adding clips from the clip browser 310 to the timeline 315 . In some embodiments, different selectable items may be used to add a clip to the end of the spine, add a clip at a selected point in the spine (e.g., the location of a playhead), add an anchored clip at the selected point, perform various trim operations on the media clips in the timeline, etc.
  • the media management tools of some embodiments allow a user to mark selected clips as favorites and add keyword tags to clips in the clip browser, among other options.
  • the timeline 315 can be toggled with other display areas.
  • a timeline specific to editing a selected multi-camera reference clip is displayed in this display area.
  • the timeline 315 is a trackless display area with a primary lane and several anchor lanes, some embodiments display the multi-camera reference clip as a set of tracks, with one track for each group (or angle) of the reference clip.
  • the GUI 300 also includes a project library toggle item 375 which, when selected, replaces the timeline 315 with a project library in the GUI.
  • the project library of some embodiments displays a list of current projects that a user can select and edit.
  • selecting one of the projects in the project library causes the timeline 315 to replace the project library with a representation of the selected project.
  • some embodiments allow users to create a new media project through the project library.
  • FIG. 4 illustrates the creation of a new project in the GUI of a media-editing application of some embodiments using the project library 400 in three stages 410 - 430 . Specifically, the stages illustrate the opening of a project creation dialog box 405 and the user of that dialog box to create the new project.
  • the first stage 410 illustrates the project library 400 , which as mentioned may occupy the same portion of the media-editing application GUI as the timeline 315 .
  • the GUI includes a project library/timeline toggle item 415 (similar to the toggle item 375 shown in FIG. 3 ) that allows a user to toggle between the timeline and the project library in this section of the GUI, as well as a new project GUI item 425 the selection of which begins the process for creating a new project.
  • Some embodiments provide these options as different types of selectable items, as menu selections, as hotkey selections, or a combination thereof.
  • the project library 400 displays a list of projects on its left side. In this case, there are two projects that have been previously created. With the name of the project, the library 400 also displays information about the project. In this case, the information includes the date the project was created, the length of the project, and the frame rate of the project. Other embodiments may display other information, such as the last time the project was edited, the output format, output resolution, etc. For each project, the library displays a filmstrip as a set of images for the project. In some embodiments, these images represent frames from the composite presentation that are evenly spaced throughout the project. As shown at stage 410 , the user has placed a cursor over the new project user interface item 425 and selected this item in order to create a new project.
  • the second stage 420 illustrates a dialog box 405 that appears when the user selects the new project item 425 .
  • This dialog box allows the user to enter information about a new project.
  • the user can enter a name for the project (in this case, “Proj 2”), select a default event for the project, and set video properties, render properties, and audio properties.
  • the default event in some embodiments, is the event to which the project automatically imports a media file when a user edits the media file into the project from a source other than an event. For instance, a user might drag and drop a video or audio file (e.g., from their desktop or other folder) into the timeline for a project. Doing so, in some embodiments, will cause the application to automatically import the file as an asset of the default event.
  • the user can also either select to use the video properties of the first clip added to the project as the video properties for the project, or select custom properties for the project.
  • the user has chosen to set custom properties.
  • the format field lets the user choose a format (e.g., 1080 p, 1080 i, 720 p, various standard definition formats, etc.) for the project.
  • the options presented in the resolution and rate fields are dependent on the selected format in some embodiments, so as to ensure settings that match up with a regularly-used video format.
  • the resolution options are 1920 ⁇ 1080, 1440 ⁇ 1080, and 1280 ⁇ 1080, while the frame rates are 23.98, 24, 25, 29.97, 30, 50, 59.94, and 60 frames per second.
  • Other formats are more restrictive: for instance, if the user selects NTSC SD, then only frame rates of 23.98 and 29.97 frames per second are available as options. The user may instead choose to set the properties for the media project automatically based on the first video clip added to the project.
  • the audio and render properties include a render format (in this case, the user has chosen Apple ProRes 422 , though other options are available).
  • the render format in some embodiments, is the encoding format used for cache files that are prepared to simplify playback (i.e., prepared ahead of time and used during playback).
  • the audio properties include an audio sample rate, and the choice between surround and stereo for audio channels.
  • the user can also select to use default settings (in this case, the options currently selected). Some embodiments provide an option to use settings from the most recently created or edited project.
  • the third stage 430 illustrates the result of the user selecting the OK item in the dialog box 405 in order to create the new project “Proj 2 ” with the properties shown in the dialog box.
  • the project library now shows a third project, the newly created “Proj 2 ”.
  • the user has not yet added any media clips to the project, so its filmstrip is blank.
  • the media-editing application of some embodiments will apply the spatial and temporal conform effects to these clips as necessary.
  • the above section describes various items in the user interface of the media-editing application of some embodiments, including events, media clips, and projects. Events contain media clips, which can be added to projects. In some embodiments, the media-editing application creates and stores various data structures to represent these different items.
  • the following section describes data structures for defining media clips and projects according to some embodiments; one of ordinary skill will recognize that various different data structures for media clips, media projects, etc. may be used in different embodiments of the media-editing application.
  • Some embodiments create an initial set of data structures for each media file upon import of the file into the media-editing application.
  • the media-editing application of some embodiments When a set of media files are imported together, the media-editing application of some embodiments generates media clips and prompts the user as to whether to add the clips to an existing event or create a new event for the clips.
  • the import process of some embodiments is described in further detail in the U.S. patent application Ser. No. 13/111,912, filed May 19, 2011 and entitled “Data Structures for a Media-Editing Application”, which is incorporated herein by reference.
  • the application For each imported media file, the application creates a series of data structures. In some embodiments, the application creates an asset data structure for each file in the event to which the files are added. The asset stores a reference to the file and any additional files created upon import (e.g., transcoded versions of the media file, analysis data about the media file, etc.). The application also creates a component clip data structure that stores a reference to the asset and the event containing the asset. When the imported file contains both audio and video data, some embodiments create component clips for the video and the audio, each of which refer to the same asset. When this is the case, the application creates another clip data structure that contains the component clips.
  • asset data structure for each file in the event to which the files are added.
  • the asset stores a reference to the file and any additional files created upon import (e.g., transcoded versions of the media file, analysis data about the media file, etc.).
  • the application also creates a component clip data structure that stores a reference to the asset and the event containing the
  • FIG. 5 conceptually illustrates an asset data structure for a media asset 500 , as well as an event data structure 505 for an event that contains the media asset.
  • the media file referenced by the asset is a video file with audio channels (e.g., a “.mov” file).
  • the event data structure 505 includes an event ID, a list of assets, and a list of clips.
  • the event data structure may include additional fields in some embodiments, such as the event name, event date (which may be derived from asset information), etc.
  • the event data structure 505 may be a CoreData (SQLite) database file that includes the assets and clips as objects defined with the file, an XML file that includes the assets and clips as objects defined with the file, etc.
  • SQLLite CoreData
  • the media asset 500 includes an asset ID 510 , source file metadata 515 , and references to various files. These files include an original imported media file 530 , any transcoded versions of the media file (e.g., a high resolution version 535 and a low resolution version 540 ), and any analysis data about the media file (e.g., a person detection data file 545 that identifies images in the source video with people in them, a color correction file 550 that stores automatic color balancing data, a shake correction file 555 that identifies and corrects camera shake in the video, audio correction files, etc.).
  • any transcoded versions of the media file e.g., a high resolution version 535 and a low resolution version 540
  • any analysis data about the media file e.g., a person detection data file 545 that identifies images in the source video with people in them, a color correction file 550 that stores automatic color balancing data, a shake correction file 555 that identifies and corrects camera shake in the video, audio
  • the metadata is information about the source file and its stored media in some embodiments.
  • the source file metadata 515 includes the file type (e.g., audio, video, movie, still image, etc.), the file format (e.g., “.mov”, “.avi”, etc.), the source device (i.e., the device that created the media, such as the particular type of camera on which a movie file was captured), the file creation date (e.g., the date a video was captured), a UUID (a unique identifier generated by a media creation device such as a camera), a set of video properties 520 , a set of audio properties 525 , timecode, and additional metadata.
  • Different types (or manufacturers) of cameras create different UUIDs differently.
  • a camera ID may be hashes of various data in some embodiments, such as a camera ID, record time (e.g., the time a user of the camera started recording), the duration of the recording, etc., so long as all UUIDs are unique.
  • record time e.g., the time a user of the camera started recording
  • duration of the recording etc.
  • different devices generate timecode differently. Some cameras use a running timecode in which each recorded video starts one frame after the previous video left off, while other cameras use a timecode that parallels actual time (i.e., leaves gaps between timecodes to reflect the time the camera was off). Still other cameras reset the timecode to zero for each recorded video.
  • the video properties 520 of some embodiments include such properties as a sample rate, a frame rate, the dimensions (or resolution) of the image (i.e., the number of pixels horizontally and number of rows of pixels), the pixel aspect ratio (i.e., the shape of the pixels, which may be square (HD video) or rectangular (e.g., NTSC 4:3 video has a ratio of 10:11)), a pixel transform (described in detail in U.S. patent application Ser. No. 13/111,912, incorporated by reference above), and the colorspace in which pixel values of the image are defined (e.g., ITU-R BT.709 for HD, ITU-R BT.601 for SD, etc.).
  • the dimensions (or resolution) of the image i.e., the number of pixels horizontally and number of rows of pixels
  • the pixel aspect ratio i.e., the shape of the pixels, which may be square (HD video) or rectangular (e.g., NTSC 4:3 video has
  • the audio properties 525 of some embodiments include a sample rate (i.e., the number of audio samples per second, often 48 kHz), the number of audio tracks stored in the underlying media file, and the number of audio channels stored in the underlying media file.
  • the asset may additionally store override data that modifies one or more of the video or audio properties. For instance, a user might enter that a media file actually has a frame rate of 24 frames per second, even though the file's metadata, stored in the asset, indicates that the video's frame rate is 23.98 frames per second. When presented to the user, or used within the application, the override will be used and the media file will be treated as having a frame rate of 24 fps.
  • FIG. 6 conceptually illustrates a nested sequence of clip objects created by the media-editing application of some embodiments for an imported media file.
  • each of the clip objects shown in this figure is a member of the same class, though the objects may be of different sub-classes.
  • the media file the clip structure of which is illustrated in FIG. 6 is a movie file that stores both audio and video information.
  • the figure illustrates a sequence 605 , a collection 610 , and two components 615 and 620 .
  • the sequence 605 , collection 610 and components 615 and 620 are sub-classes of clip objects (or anchored objects).
  • Some embodiments create a sequence (which may be another clip object or a different class of object) within the event object for each media file imported into the event.
  • the sequence 605 stores a sequence ID, a set of sequence attributes, and the collection object 610 .
  • the sequence ID is a unique identifier for the sequence object.
  • the sequence attributes include the resolution, frame rate, and other video and audio properties for the sequence. For event clips, these attributes are generally based on the underlying media (i.e., the information stored in the asset that is linked to through the collection and component clips).
  • the collection object 610 is an ordered array of clip objects. In the case of a clip imported into an event, as is the case with the object 610 , the collection stores one or more component clips in the array. Often, the collection stores the video component clip in the array, as shown here; any additional components (generally one or more audio components) are then anchored to that video component. In addition, the collection object 610 stores a collection ID, a total range, and a trimmed range. The collection ID is a unique identifier for the collection object. The total range of a clip object indicates the maximum range for the object based on the objects it contains, while the trimmed range indicates the actual range of the object in its parent object according to user edits.
  • the collection object stores a value for the total range because the total range is a variable assigned to all types of clip objects, but this range is not actually used by the application. Instead, the application dynamically calculates the total range based on the objects contained by the collection (as opposed to updating the range value every time a clip is added to the collection). Some embodiments, on the other hand, do update the total range in the collection object.
  • the array of the collection object 610 contains the media component 615 . Collection objects, in some embodiments, can contain both component clip objects and additional collections in their arrays.
  • a first object e.g., the collection object 610
  • a second object e.g., a media component 615 in the array of collection object 610
  • a first object contains a second object by storing a reference to the object (e.g., a pointer). This reference is stored as a strong pointer in some embodiments.
  • the media components shown include a video component 615 and an audio component 620 .
  • Each of the components stores a component ID, a source media range (i.e., total range), a trimmed range, an asset reference (that refers to an asset such as the asset 500 and the event containing the asset), and a roles setting.
  • the roles setting is a setting for a clip and is stored on the video and audio components (i.e., at the component level).
  • Some embodiments provide a set of roles options (e.g., music, dialogue, and effects for audio components; video and titles for video components) as well as allowing the user to input custom roles options (e.g., French, English, etc.). The user can then export a composite presentation that excludes clips with certain roles (e.g., removing all of the French audio).
  • the total range is the duration of the source media.
  • the trimmed range is the range selected by a user through various user edits (e.g., trim edits) and indicates both the start time within the total range and the duration, in some embodiments.
  • the trimmed range is generally a subset of the total range (i.e., does not include time outside the total range).
  • the application may use a range extender to extend the trimmed range past the total range (e.g., when needed for part of a transition, or if aligned audio runs slightly longer than video).
  • Some embodiments will generate video images using, e.g., the first and last images of the video file, to fill in the needed extra range.
  • the total range and trimmed range will generally be equal as the user will not have edited the clip at all yet.
  • the video component 615 includes an anchored item set that contains the audio component 620 .
  • clips can be anchored to other clips in some embodiments. When a particular clip is moved in the timeline, any clips anchored to the particular clip are moved along with it. Each clip can have numerous anchored clips, and can be anchored to a single clip. In the case of a media clip with audio and video components, some embodiments anchor the audio component to the video component.
  • the video component object 615 contains the audio component in its set of anchored items (and could include numerous such items if there were additional audio components), while the audio component object 620 includes a parent item reference to the video component to which it is anchored.
  • the child (anchored) object stores an anchor offset that indicates two values.
  • the first value is the position in the parent object to which the child object is anchored, and the second value is the offset of the anchor within the child object.
  • both anchor offset values are zero. However, this can be modified if, for example, there is a slight offset in the recordings of the two components and the user has to adjust the audio relative to the video.
  • FIGS. 5 and 6 illustrate the data structures for an object imported into an event in the media-editing application.
  • the data structures created by the media-editing application may be more complicated.
  • FIG. 7 illustrates a timeline 700 with a project title “New Project” that includes four clips 705 - 720 .
  • the clips 705 - 715 are in the primary compositing lane of the project's sequence, while the clip 720 is anchored to clip 710 at approximately 26 seconds into clip 710 .
  • the clip 710 is a compound clip that itself includes two clips.
  • FIG. 8 conceptually illustrates a subset of the data structures for the project illustrated in FIG. 7 .
  • the data structures of FIG. 8 are all contained within a project data structure that contains a single sequence 800 .
  • the project data structure for a project in the timeline is a sub-class of the same class that includes event data structures. While the event data structures can contain multiple sequences (i.e., one sequence for each clip in the event), the project data structure only contains a single sequence.
  • the sequence 800 includes a primary collection data structure 803 , which itself stores an array containing three collections 805 - 815 that correspond to the clips 705 - 715 in the timeline 700 .
  • the fourth clip 720 is contained within the collection 810 as an anchored item of the collection.
  • the component objects are not shown in this figure.
  • the sequence 800 includes a sequence ID, sequence attributes 835 , and a primary collection 803 .
  • These sequence attributes 835 include the video properties set for the project, such as the resolution, frame rate, and other additional properties (e.g., format, render properties, audio properties, etc.). As shown in the previous Section I, in some embodiments a user sets these properties when creating the project.
  • the primary collection 803 includes a collection ID, ranges (i.e., the total and trimmed ranges described above), an effects stack (which, in some embodiments, represents a video effects stack and an audio effects stack), and the array of media clips.
  • the array is ordered based on the location in the timeline and only includes media clips in the primary lane of the collection.
  • the media-editing application assumes that there is no gap between these items, and thus no timing data is needed between the items. As shown, each of these clips is represented as a collection.
  • an event e.g., the clip shown in FIG.
  • the effects stack stores a set of effects applied to the media clip. These effects may include automatically applied rate and spatial conform effects, user-initiated transform, crop, and distortion effects, user-applied pixel-manipulation effects (i.e., effects that modify pixel values of the images of the clip according to different algorithms, etc.).
  • the clips 805 , 815 , and 820 are individual clips that have been added to the timeline from the clip browser, and thus do not themselves include collections. Similar to the collection 610 , these objects include an ID, ranges, an effects stack, and an array of media components (e.g., a video component with one or more audio components anchored to the video component).
  • these objects include an ID, ranges, an effects stack, and an array of media components (e.g., a video component with one or more audio components anchored to the video component).
  • the clip 810 is a compound clip that includes multiple clips in its array, in addition to the collection ID, ranges, and effects stack. Specifically, the clip 810 includes two media clips 825 and 830 . Within the compound clip, the clips are both in the primary lane of the collection, and thus one follows the next. These clip objects are not shown in this figure, but each of the clips is similar to the clip 805 in that the clips each include an array of media components.
  • the clip object 810 includes a set of anchored items (in this case only the one item, clip 820 ). Some embodiments include a set of anchored items for each collection, which are empty for the other objects shown in FIG. 8 .
  • the anchor offset stored in clip 820 indicates that it is anchored 26 seconds into the clip 810 , and that the anchor is at the start of the clip 820 . These times refer to the trimmed ranges of the clips in some embodiments.
  • FIG. 9 conceptually illustrates the objects 800 - 830 nested in a conceptual timeline.
  • the collection objects 825 and 830 are nested inside the collection 810 , which is nested inside the primary collection object 803 along with the other collection objects 805 , 815 , and 820 .
  • the sequence's primary collection object 803 is itself nested inside the sequence object 800 .
  • the figure illustrates the anchoring relationships between the various clips as well as the durations (ranges) of the clips.
  • the lowest level collections 805 , 815 , 825 , and 830 each have an audio component anchored to a video component. While not shown, collection 820 could also have the same video/audio setup, or could be just a video component (or just an audio component). While each of the objects shown has a single audio component, one of ordinary skill will recognize that some embodiments will have multiple audio components (e.g., if a camera records several audio tracks as separate files and imports the files with a video file as part of a single clip).
  • the figure also illustrates the anchoring of clip 820 to clip 810 .
  • multiple clips will be anchored to the same primary lane clip, and the multiple anchored clips may overlap in time.
  • multiple secondary lanes may be used.
  • Some embodiments assign lane numbers to each clip object that indicates the clip object's lane within a collection.
  • All of the primary lane objects are assigned a lane number of zero in some embodiments, with lanes above the primary lane getting increasing numbers and lanes below the primary lane getting decreasing (negative) numbers.
  • a separate audio clip might be anchored to a clip in the primary lane and displayed below the primary lane.
  • the anchored clip 820 has a lane number of 1.
  • the lane numbers indicate compositing order for video in some embodiments. Whereas two audio files can be combined fairly easily (mixed), images from two video files cannot be displayed at the same time. Thus, some embodiments composite higher lane number clips on top of lower lane number clips.
  • compositing modes and effects may be used to combine the pictures (e.g., compositing modes such as subtract, darken, multiply, etc. that combine pixel information of two images, as well as effects such as scaling the top image to get a picture-in-picture, applying a color mask to the top image, etc.).
  • the items in a lower-level nested collection will also have lane numbers that refer to their lane order within that collection.
  • the collection object 810 has two clips 825 and 830 , that each has a lane number of zero. However, this collection object could have anchored clips in multiple lanes.
  • the items within the collection 810 would be composited initially according to the ordering within the collection, then the output of that would be composited within the primary collection according to the ordering of the primary collection.
  • the video components are all assigned to lane zero and the audio components are assigned to lane ⁇ 1.
  • FIG. 9 also illustrates the ranges (e.g., durations) of the various clip objects.
  • the total range is the full length of the media to which the components refer, whereas the trimmed range is a subset of this range based on user edits (e.g., trim edits).
  • user edits e.g., trim edits
  • the total range is the sum of the trimmed ranges of its primary lane clips, which in this case are collections 825 and 830 .
  • the variable Total Range 2 Trimmed Range A+Trimmed Range B.
  • the collection 810 is not separately trimmed, such that its trimmed range equals its total range. This means that, although there is more media within clip 830 , while editing the primary collection 803 the media-editing application will not allow a user to increase the duration of clip 810 beyond that of Total Range 2. However, a user could open up the clip 810 in the timeline and apply trim edits to either of clip 825 and 830 . Modifications to the trimmed range of these clips will affect the total range of the clip 810 .
  • a user can modify the trimmed range of clip 810 to shorten the clip. Trimming from the beginning would result in less of the media of collection 825 being used in the component presentation, while trimming from the end would result in less of the media of collection 830 being used in the composite presentation.
  • some embodiments store reference clips in the event data structures. Rather than copying these clips when a user adds the clip to a media project, the media-editing application creates a clip instance in the collection for the media project. This clip instance is similar to a collection clip, but does not contain clips of its own. Instead, the clip instance refers to the reference clip in the event. Any edits to the reference clip are then reflected in all instances of the clip in the media project (or other media projects).
  • the reference clips will be described below in detail for multi-camera clips, but some embodiments include reference clips for additional types of clips (e.g., individual video clips, compound clips, etc.).
  • some embodiments store generators (effects that create their own video images rather than modifying existing video images), transitions between clips, audition stacks (sets of clips that can be swapped out for each other), markers, and keywords as clip objects.
  • generators effects that create their own video images rather than modifying existing video images
  • transitions between clips transitions between clips
  • audition stacks sets of clips that can be swapped out for each other
  • markers markers
  • keywords keywords as clip objects.
  • the above-described data structures e.g., the clip objects, asset objects, event objects, project objects, etc.
  • are used by some embodiments of the media-editing application for displaying information in the GUI of the application and rendering a project, as well as for determining whether to apply a spatial or temporal conform effect to a clip.
  • Such a media-editing application may have the capability to generate and use multi-camera media clips, and may include features specific to such clips.
  • some embodiments generate multi-camera media clips that include several media clips captured by multiple different media capture devices.
  • the media-editing application automatically generates these multi-camera clips from the individual media clips using metadata stored with the media clips.
  • the generation process uses either a default multi-camera clip generation algorithm or adapts the algorithm based on preference settings input by a user.
  • FIG. 10 conceptually illustrates a process 1000 of some embodiments for generating a multi-camera media clip from a set of media clips.
  • the process 1000 is performed by the media-editing application of some embodiments in response to user input indicating a set of clips from which to generate the multi-camera clip.
  • This process will be described by reference to FIG. 11 , which illustrates user input for creating such a multi-camera media clip over five stages 1110 - 1150 .
  • the process 1000 begins by receiving (at 1005 ) a set of clips for a multi-camera clip.
  • these clips may be any clips stored in an event (e.g., video clips, still images, audio clips, etc.), including compound clips or previously-generated multi-camera clips.
  • Some embodiments require all of the clips to be stored in the same event, while other embodiments allow a user to combine clips from multiple events into a single multi-camera clip.
  • Some embodiments restrict the type of clips used to generate a multi-camera clip to prevent the use of compound clips or multi-camera clips, which may themselves have content captured by multiple different devices.
  • the first stage 1110 of FIG. 11 illustrates a clip browser 1105 showing several media clips.
  • the user has selected these clips and opened a drop-down (e.g., via a right-click, two-finger tap, or other such selection input).
  • the user has located a cursor over the menu option for “New MultiCam Clip”.
  • the second stage 1120 illustrates a dialog box 1115 that provides the user with several options for the multi-camera clip.
  • the user can title the clip, input a starting timecode, set video properties, and set audio and render properties.
  • the video properties includes the format (e.g., 720 p HD), the dimensions (e.g., 1280 ⁇ 720), and the frame rate (e.g., 30 p).
  • the user can either set these properties based on common clip properties (e.g., the properties most common among the selected clips, the properties of a particular clip, etc.) or provide custom settings (e.g., as shown in FIG. 4 for a new project).
  • the dialog box 1115 also allows the user to choose whether the application will use its default settings to create the multi-camera clip or whether the application should prefer certain metadata for assigning the selected clips to angles, ordering the clips within the angles, and synchronizing the angles.
  • the dialog box provides an option for determining whether the application should perform a second level of synchronization using audio data. As shown in stage 1120 , the user is selecting the custom option in order to view these various choices.
  • the process 1000 next assigns (at 1010 ) each of the clips in the set to a particular angle using clip metadata.
  • the angles are groups of clips the content of which was captured by the same device. For instance, a director might shoot a scene with four cameras simultaneously in order to capture four different angles of the scene (e.g., straight on, close up, overhead, etc.).
  • the metadata used includes the camera angle (e.g., user-provided metadata for the clip) and/or camera name (e.g., information automatically generated by the media-editing application for the clip upon import of the clip's content).
  • the application allows the user to choose to have each clip placed in its own angle.
  • the process generates (at 1015 ) a sequence of the assigned clips for each angle. That is, the process determines an order for the clips in the angle (e.g., using time of day or timecode metadata), then spaces these clips out over time according to this metadata. This operation will be described in detail below by reference to FIG. 14 .
  • the metadata used may include the timecode and the time of day (or creation date, as described above in Section II) of the clips.
  • Various different algorithms may be used by different embodiments to translate this metadata into a sequenced angle.
  • the process 1000 then synchronizes (at 1020 ) the sequences of the different angles in the multi-camera clip.
  • the synchronization operation aims to have content that was simultaneously recorded be aligned in time within the multi-angle clip.
  • an overhead shot and a close-up shot of the same action should ideally be lined up in the clip such that at a particular time within the clip the video images for each angle show the same time in the action.
  • the application can use time of day or timecode metadata for synchronization, as well as lining up the start of the first clip in each angle or the first marker in the clips of the angles.
  • various different algorithms may be used by different embodiments to translate the timing metadata into aligned angles.
  • some embodiments perform a second level of synchronization using audio data. These embodiments compare audio portions of clips located near to each other in time (after the initial metadata synchronization) from different angles to identify matching audio, then align the clips further using any matching audio. The synchronization operations will be described in detail below by reference to FIGS. 16 and 18 .
  • FIG. 12 conceptually illustrates a process 1200 for such assignment of clips to different angles of a multi-camera media clip.
  • the process 1200 will be described by reference to FIG. 13 , which conceptually illustrates the assignment of a set of nine clips 1305 - 1345 to three different angles for a multi-camera media clip.
  • the process 1200 begins by receiving (at 1205 ) a set of clips to assign to different angles for a multi-camera media clip.
  • a set of clips may be a set of clips selected by a user, and the content of the clips may have been captured by several different devices or a single device.
  • the media clips might be video clips, audio clips, still images, or other content.
  • a set of media clips could include video (with accompanying audio) taken by one or two different cameras, still images taken by yet another different camera, as well as audio-only clips recorded by a microphone separate from any of the cameras.
  • the clips will store metadata about their represented content. This metadata may include information about the device that captured the content and when the content was captured (in addition to other data, such as the video or audio properties).
  • FIG. 13 conceptually illustrates nine clips 1305 - 1345 with their metadata. These clips include five video clips 1305 - 1325 and four still images 1330 - 1345 .
  • the video clips include angle IDs (“Wide Shot” and “Close-up”), which are user-entered metadata.
  • a user can select a media clip (e.g., in the clip browser) and enter an angle ID (or camera ID) for the clip through the inspector display area.
  • the video clips include starting and ending timecode information and start time of day information.
  • the still images include a camera ID (“10101”) and Exif (Exchangeable image file format) date, which is data indicating the date and time an image was captured.
  • the process 1200 selects (at 1210 ) an unassigned clip from the set of received media clips. Some embodiments select the clips in the order they are stored in their event, in alphabetical order by clip name, in some other order, or in a random order. Initially, of course, all of the clips are unassigned.
  • the process then identifies (at 1215 ) an angle for the selected clip using clip metadata.
  • the media-editing application may use a variety of different metadata to identify the angle for a particular clip.
  • some embodiments prefer a user-entered angle ID, then a user-entered camera name (specified at time of import in some embodiments), then a camera ID automatically generated at time of import among data stored with the clip data structure, then look to other metadata stored with the source file if none of the previous data is available (e.g., a manufacturer ID or device name). If none of this metadata is available, then the application assigns the clip to its own angle. In the case of the clips shown in FIG.
  • the video clips 1305 - 1325 each have an angle ID
  • the still images 1330 - 1345 each have a camera ID.
  • the video clip angle IDs are user-entered information
  • the camera ID for the still images is automatically-generated data.
  • the process determines (at 1220 ) whether a set has already been created for the angle in the multi-camera clip.
  • the process checks each available piece of metadata for the clip to determine whether the clip could be assigned to an existing angle. For instance, if a first clip has a camera ID but no angle ID and a second clip has an angle ID as well as the same camera ID as the first clip, then the application will assign the second clip based on its camera ID rather than only examining the angle ID and assigning the second clip to a new angle because no angle yet exists for that angle ID.
  • some embodiments will always use the first piece of information for which the clip has a value. Thus, in the case just mentioned, such embodiments would assign the second clip to a new angle based on its angle ID. If the clip does not have any sort of angle identification metadata, then the clip will be assigned to its own angle.
  • the process adds (at 1225 ) the selected clip to the set of clips associated with the identified angle.
  • the process creates (at 1230 ) a set for the identified angle and adds the selected clip to the set.
  • the media-editing application temporarily stores these sets of clips in volatile memory (e.g., RAM) while performing additional operations of the multi-camera clip creation process (i.e., ordering and synchronization).
  • media-editing application data structures are not actually created for the sets of clips until the sets are ordered, at which point the application generates the angles of the multi-camera clip in a data structure for the multi-camera clip. The structure of such a multi-camera clip is described in detail below.
  • the process 1200 determines (at 1235 ) whether additional clips remain to be assigned to the different angles. When additional clips remain, the process returns to 1210 to select another unassigned clip for assignment to an angle. Once all of the clips have been assigned, the process 1200 ends.
  • FIG. 13 illustrates the assignment of the clips to three different angles. Assuming the clips 1305 - 1345 are assigned in order from “Clip 1” to “Clip 9”, the media-editing application first selects clip 1305 , creates an angle “Wide Shot” 1350 , and assigns the clip 1305 to this angle 1350 . Next, the application assigns the clip 1310 to a new angle “Close-up” 1355 based on this clip's angle ID. For the third clip 1315 , no new angle is needed, as this clip also has the angle ID “Wide Shot” and is thus assigned to the already-existing angle 1350 . The video clips are assigned to these two angles 1350 and 1355 as shown, and then the application gets to the still images.
  • the application identifies the camera ID as “10101” for the clip 1330 , determines that this is the only metadata for this clip useful for angle assignment and creates a new angle 1360 with this name.
  • camera IDs, camera names, or even clip names may be used to name angles, while other embodiments create generic angle names such as “Angle 3” when no angle ID metadata exists for the clips assigned to that angle.
  • the four still image clips 1330 - 1345 all have the same camera ID, and thus the application assigns these four clips to the angle 1360 .
  • these angles exist in volatile memory at this point in the clip creation process.
  • Other embodiments create a multi-camera clip structure (described below by reference to FIG. 20 ) during the angle assignment process and create angles within this clip structure as the angles are created by process 1200 (or a similar angle assignment process.
  • FIG. 14 conceptually illustrates a process 1400 of some embodiments for generating a sequence of ordered clips for the various angles of a multi-camera media clip.
  • the process 1400 will be described by reference to FIG. 15 , which illustrates the sequencing of the clips 1305 - 1345 from FIG. 13 .
  • the process 1400 begins by receiving (at 1405 ) sets of clips assigned to different angles for a multi-camera clip.
  • these clips have been assigned to the angles by the media-editing application using the process 1200 or a similar process.
  • These may be clips that have an extended duration (e.g., video and audio clips) or may only have an instantaneous time (e.g., still images, which consist of a single frame).
  • the media clips may store timing metadata about the source video to which they refer.
  • the video clips 1305 - 1325 include both timecode (“TC”) information and time of day (“ToD”) information.
  • TC timecode
  • ToD time of day
  • the metadata for Clip 1 1305 indicates that the timecode starts at 01:00:00:00 (given in hours:minutes:seconds:frames) and ends at 01:10:00:00, for a duration of exactly ten minutes.
  • the time of day information indicates that filming for this clip started at 5:35 PM on May 25, 2011.
  • cameras will record time of day down to the nearest second.
  • time of day information will generally not take into account frames, whereas timecode stores information at the frame level (i.e., fractions of a second) and is thus more precise.
  • this example shows only a starting time of day. Many cameras only record starting or ending times, but not both. To record the duration of the clip, the camera records starting and ending timecode information (duration could also be determined based on the frame rate and number of frames). Thus, the media-editing application would use the time of day to identify start times for the clips and timecode to derive the duration.
  • the process 1400 next selects (at 1410 ) a set of clips associated with a particular angle.
  • the information associating the clips with the different angles is stored in volatile memory at this point, in which case the application retrieves this information from the memory in order to evaluate the clips.
  • the process may evaluate the angles in a particular order such as the order in which they were created, or a random order. Some embodiments might also evaluate the angles in parallel, performing operations 1415 - 1425 at the same time for multiple angles.
  • the process identifies (at 1415 ) a useable set of timing metadata for the selected set of clips.
  • some embodiments first identify whether the clips in the selected angle can be sequenced using timecode.
  • different embodiments of the media-editing application apply different criteria to the timecode of the media clips. For instance, the first test the application applies in some embodiments is whether the timecode of any of the clips overlaps. In the case of the angle 1355 (“Close-up”) of FIG. 13 , these clips both start at a timecode of 00:00:00:00, so the timecode does not provide any useful ordering information.
  • the application determines whether the total span of the timecode of all of the clips is greater than a threshold duration (e.g., one hour, one day, three days, etc.). For example, if the clips are 100 hours apart in timecode, then they will probably not form a useful multi-camera clip sequence with this information.
  • a threshold duration e.g., one hour, one day, three days, etc.
  • some embodiments will not use timecode if all of the clips start one frame after the end of another clip in the set (or, in some embodiments, within a second of the end of the previous clip). This is generally indicative of a camera that uses continuous timecode but which does not roll through timecode when not recording, and thus will not correspond to actual time (common with digital tape-based recording).
  • the media-editing application of some embodiments will use the ordering (i.e., that a first clip should come before a second clip) from the timecode, as this will most likely be reliable. For spacing the clips when this is the case, the application will attempt to use time of day information.
  • timecode information cannot be used, the application will attempt to use the time of day information.
  • the time of day is not as precise as timecode because it does not generally record time at the frame level.
  • the timecode information starts at 0 for both clips, so the application would use the time of day information for ordering and spacing purposes (while using the timecode to determine duration).
  • the clip 1310 starts at 5:36:00 PM and has a duration of exactly seven minutes, while the clip 1320 starts at 5:55:00 PM and has a duration of eighteen minutes and nineteen frames, so no overlap exists in the time of day information.
  • Some embodiments apply a threshold duration test to the time of day as well, mandating that the duration from the start of the first clip to the end of the last clip not span more than a threshold duration (e.g., one hour, one day, three days, etc.). For instance, a clip might be marked as having been filmed in 1990 while another clip is from 2011, but it is unlikely the user would actually want a multi-camera clip with a length of 21 years.
  • a threshold duration test to the time of day as well, mandating that the duration from the start of the first clip to the end of the last clip not span more than a threshold duration (e.g., one hour, one day, three days, etc.). For instance, a clip might be marked as having been filmed in 1990 while another clip is from 2011, but it is unlikely the user would actually want a multi-camera clip with a length of 21 years.
  • the application will nevertheless attempt to at least use the information to put the clips in order, even if the information may not work for properly spacing the clips along a timeline.
  • the application would order the first clip before the second clip. If no time of day information is available at all, then the application places clips in order based on any other metadata that can be found, such as date of import of the source files represented by the clips.
  • some embodiments initially look at the Exif data (data generated by the camera according to a standardized format for recording image metadata) to find a date and time when the image was captured. When this information is not available, the application will look to the creation date (which might be the date of import of the image files) to order the still image clips.
  • the metadata for still images keeps time at the level of seconds, and thus multiple still images captured by a single camera one after another might have metadata indicating that they were taken at the same time.
  • the process orders (at 1420 ) the set of media clips according to the identified set of timing metadata.
  • the media clips may be ordered based on timecode, time of day information, or other information that the application can identify from the clips and utilize for ordering.
  • the media clips are stored in the array of an anchored collection within a multi-camera media clip structure, as described below by reference to FIG. 20 .
  • the process inserts (at 1425 ) gaps into the ordered set in between the media clips according to the identified set of timing metadata.
  • some embodiments do not insert any gap clips in the sequence and just leave the clips in order one after another.
  • the media-editing application spaces the clips out along a timeline and inserts gaps between the clips. If a first clip ends at the five minute mark and a second clip begins at the six minute mark, then the application inserts a one minute gap between the two clips. Due to the nature of the data structures used to store the multi-camera clips, in some embodiments these gaps are stored as separate clip structures within the array of the anchored collection that represents the angle.
  • FIG. 15 illustrates the ordered angles 1350 - 1360 .
  • the angles 1350 and 1355 contain video clips.
  • the media-editing application of some embodiments uses the timecode (as this information is not overlapping and is contained within a forty minute duration).
  • the angle 1350 starts at a timecode of 01:00:00:00 with clip 1305 . After ten minutes, this clip ends and a gap clip begins, until the 01:15:00:12 mark when the clip 1325 begins for just under ten minutes. Another gap clip begins at the twenty-five minute mark, until the clip 1315 begins at a timecode of 01:28:54:02. This clip lasts approximately eight minutes, for a total length of thirty-seven minutes.
  • the “Close-up” angle 1355 only contains two clips 1310 and 1320 . However, for these clips, timecode does not provide useful information because the camera that captured the clips starts each captured video at a timecode of 00:00:00:00. However, the time of day information does provide useful timing metadata. As shown, the first clip 1310 starts at 5:36:00 PM, and the timecode indicates that it has a duration of seven minutes. Next in the array for the angle is a large gap clip, which lasts until the second clip 1320 begins at 5:55:00 PM. The duration of the clip 1320 is eighteen minutes and nineteen frames (e.g., 19/24 of a second if the frame rate is 24 fps). Thus, the total length of the angle 1355 is thirty-seven minutes and nineteen frames.
  • the angles 1350 and 1350 only include video clips, which can have extended durations.
  • the media-editing application looks at the file names of the images, which are often automatically named in a number order by the camera (e.g., “IMG — 0035”, “IMG — 0036”, etc.).
  • the Exif dates for the still image clips are 5:40:00 PM for clip 1345 , 5:42:30 PM for clip 1330 , 5:50:00 PM for clip 1340 , and 5:55:00 PM for clip 1335 , all on May 25, 2011. These times are all within a threshold duration (so long as the threshold duration is longer than 15 minutes), and thus the Exif times pass the criteria for use in ordering the clips and identifying start points for the clips in the timeline.
  • FIG. 15 illustrates these four clips 1330 - 1345 along a timeline for the angle “10101” 1360 .
  • the timeline begins at 5:40 PM, the time of the first image clip 1345 .
  • This clip has a duration of two minutes and thirty seconds, until the next image clip 1330 begins at 5:42:30 PM.
  • the clip 1340 begins.
  • This clip ends at 5:55 PM, where the last clip 1335 begins.
  • this clip is left with an open-ended duration until after synchronization of the angles.
  • the still image clip, after synchronization will have a duration that extends until the end of the multi-camera clip, which will depend on the lengths and synchronization of the angles 1350 and 1355 .
  • some embodiments assign a fixed duration to the clips (e.g., one minute), which can be shortened automatically if another clip begins before that duration is up.
  • the process 1400 determines (at 1430 ) whether any additional angles remain that need to be sequenced. Once the process has ordered and sequenced all of the angles, the process 1400 ends. When additional angles remain, the process returns to 1410 to select the next angle and determine a sequence for the angle.
  • the media-editing application stores these angles, or ordered sequences of clips, as collection clip structures within a multi-camera media clip structure.
  • this multi-camera clip structure is a collection that includes a gap element to which each of the angle collection structures is anchored.
  • the media-editing application of some embodiments generates the sequence for each of the angles separately, then synchronizes the sequences once they are all generated.
  • FIG. 16 conceptually illustrates a process 1600 of some embodiments for performing the initial synchronization utilizing the timing metadata.
  • the process 1600 will be described by reference to FIG. 17 , which illustrates the alignment of the sequences 1350 - 1360 from FIG. 15 using the metadata for clips 1305 - 1345 shown in FIG. 13 .
  • the process 1600 begins by receiving (at 1605 ) a set of angles with the clips in each angle ordered in a sequence.
  • some embodiments store each of these angles as an anchored collection clip structure within a multi-camera clip structure (which is itself a collection structure).
  • the sequences have been generated by the process 1400 or a similar process.
  • the angles might all be sequenced according to the same type of metadata (e.g., timecode) or be sequenced using different types of metadata (e.g., as in FIG. 15 with one angle sequenced using timecode, another with time of day information, and a third with Exif dates).
  • at least one of the angles will have the clips in order, but not spread out (e.g., because the only available timing information for the clips was not deemed reliable enough to space out the clips).
  • the process 1600 selects (at 1610 ) a first one of the angles and places the selected angle in a set of synchronized angles.
  • the set of synchronized angles in some embodiments, is the set of angles that have been evaluated and had their start times in the multi-camera clip adjusted for alignment with the other angles.
  • the media-editing application in order to align a first angle with a second angle, inserts a gap clip at the start of the sequence for whichever of the first and second angles starts later. Because at this stage the process 1600 has not yet evaluated any of the angles, there are no angles with which to synchronize the first angle, and thus the process sets the first clip in this first selected angle as starting at time zero of the multi-camera media clip. That is, the first selected angle does not have a gap clip inserted at its start at this point.
  • the process selects (at 1615 ) one of the unevaluated angles as a current angle. If there is only one angle in the multi-camera clip, then no synchronization is necessary, of course.
  • the media-editing application selects these angles in the order in which they are stored in the multi-camera clip collection data structure in some embodiments. For instance, in the example shown in FIG. 15 , the media-editing application might first select the “Wide Shot” angle 1350 and set this as its initial synchronized angle, then subsequently select the “Close-up” angle 1355 to align this angle to the “Wide Shot” angle.
  • the different angles are each stored as an anchored collection in the multi-camera collection data structure, and each has a lane number to indicate order. Some embodiments select the angles starting from the highest or lowest lane number.
  • the process 1600 determines (at 1620 ) whether any timing metadata of the clips is useful to synchronize the current angle with at least one of the angles in the set of synchronized angles.
  • this timing metadata may be either the timecode or the time of day (creation date), as with the angle sequencing process.
  • Some embodiments first determine whether the timecode is useful for synchronization because timecode has a higher precision than time of day (as timecode uses increments of frames, whereas time of day only uses full second increments).
  • timecode uses increments of frames, whereas time of day only uses full second increments.
  • Professional setups may run multiple cameras attached (e.g., wirelessly, via a wired connection) to a single timecode synchronizer.
  • the timecode of a first camera is not related to the timecode of a second camera.
  • some embodiments first determine whether timecode was used to order both the angles. If timecode was not useful for ordering an angle (e.g., because the timecode of clips within the angle overlap), then the application will not use the timecode data for synchronizing the angle. For example, the timecode data for the “Close-up” angle 1355 was not useful for sequencing the clips of that angle, and thus will not be used for synchronizing angle 1355 with the “Wide Shot” angle 1350 . When both of the angles were sequenced using timecode data, then the application compares the timecodes for one of the angles with the timecodes for the other angle.
  • Some embodiments require that the timecodes of the first angle at least partially overlap with the timecodes of the second angle in order to use the timecode data, while other embodiments require that the total time spanned by the timecodes not be greater than a threshold duration (e.g., one hour, twenty-four hours, seventy-two hours, etc.). That is, if the timecodes of a first angle run from 01:00:00:00 to 03:00:00:00, while the timecodes of a second angle run from 22:00:00:00 to 26:00:00:00, this would be over a twenty-four hour threshold in some embodiments. Other embodiments compare the gap between the two sets of timecodes (nineteen hours in the above example) rather than the overall time spanned to the threshold duration.
  • a threshold duration e.g., one hour, twenty-four hours, seventy-two hours, etc.
  • Some embodiments compare the timecode data of the currently selected angle to each angle already synchronized to determine whether the currently selected angle can be synchronized with any of the other angles. For instance, the first two angles might be synchronized using time of day information, but the third angle could synchronize to one of the first two angles using timecode information.
  • the process uses this timecode data.
  • the media-editing application of some embodiments will attempt to use time of day information.
  • Some embodiments apply similar constraints on the use of time of day information as described above for timecode data, such as requiring that the span of the current angle at least partially overlaps or is within a threshold duration of another angle to which it is compared. If a user has set all of his different cameras to the correct time, then the time of day data will often be overlapping (at least in the case of simultaneous filming of a scene by the multiple cameras), and thus useful.
  • the timecode information of the example “Close-up” angle 1355 is not useful, the time of day information for its clips 1310 and 1320 (running from 5:36 PM to 6:13 PM) overlaps with the time of day information for the clips 1305 , 1325 , and 1315 of the “Wide Shot” angle 1350 (running from 5:35 PM to 6:12 PM). As in the previous section, the timecode information is still useful to determine the duration of the clips (and thus the duration of the angle).
  • Still images are ordered using their Exif data, or creation date information when the Exif data is unavailable.
  • the media-editing application compares the timestamps of these types of data with the time of day information of the video or audio clips in other angles in a similar manner to that described above (i.e., determining whether the timespan of the first angle overlaps or is close to that of another angle).
  • some embodiments may include still images and video/audio clips in the same angle (e.g., when a digital camera is used to both take still images and shoot video).
  • the Exif dates may be considered along with the video's time of day information as the same sort of metadata for both ordering and synchronization purposes.
  • the process synchronizes (at 1625 ) the current angle to the set of synchronized angles using the timing metadata.
  • the timing metadata either the timecode or the time of day information
  • the process synchronizes (at 1625 ) the current angle to the set of synchronized angles using the timing metadata.
  • the application will attempt to synchronize the current angle to the angle with overlapping time of day information using that angle's time of day information rather than its timecode.
  • a first angle has time of day information
  • a second angle has both time of day information and timecode information
  • a third angle only has timecode information
  • the application is enabled to synchronize these three angles together by synchronizing both the first and third angle to the second angle.
  • synchronizing an angle to a set of angles involves inserting (or adjusting) a gap clip at the start of one or more of the angles, unless the currently selected angle starts at exactly the same time as the earliest angle in the set.
  • the application inserts a gap at the start of the other angles with a length equal to the offset of the current angle. If one of these angles already has a gap clip, then the application extends this gap by the length of the offset.
  • the application inserts a gap at the start of the currently selected angle with a length equal to the offset.
  • FIG. 17 illustrates, when synchronizing the “Close-up” angle 1355 to the “Wide Shot” angle 1350 , the application inserts a one minute gap at the start of the “Close-up” angle 1355 because it starts at 5:36 (compared to 5:35 for the “Wide Shot” angle 1350 ).
  • the third angle “10101” 1360 starts at 5:40, so the application inserts a five minute gap at the start of this angle.
  • the application extends out the duration of the still image clip 1335 for eighteen minutes (from 5:50 to 6:13 PM).
  • Some embodiments do not extend a still image clip that is the last clip in an angle all the way to the end of the media clip, but instead place a maximum length on the still image (e.g., one minute, five minutes, etc.).
  • the process 1600 synchronizes (at 1630 ) the angle to the set of synchronized angles without using timing metadata.
  • the application starts the first clip of the angle at the same time as the first clip of the angle with the earliest start time. In some embodiments, this is accomplished by simply not inserting a gap clip at the start of the collection representing the angle. So long as the anchor offset of the angle collections is kept at zero, then any angles that do not start with a gap clip will have the earliest start time in the multi-camera media clip.
  • some embodiments allow a user to choose that the media-editing application synchronizes all of the angles at the start of the first clip of each angle.
  • the application orders the different angles using the timing metadata (or as specified by the user), then simply aligns the angles so that they each start at the same time.
  • this alignment is simply performed by not inserting gap clips at the start of any of the angles, because the angles are all anchored to a gap clip with the same anchor offset (0, 0), as described below by reference to FIG. 20 .
  • the application allows a user to choose to synchronize the angles via a first marker in each angle.
  • Users can set markers on clips, which in some embodiments are stored as clip data structures with a one-frame duration, anchored to the marked clip at a particular time in the clip. With this option selected, the application will identify the first marker in each ordered angle, then insert gap clips at the start of the angles as necessary so that these markers are aligned. This allows users to identify a particular frame in one clip from each of several cameras, and mark this frame for alignment with other such frames. This may be especially useful when a user films a scene from multiple different angles and can identify a specific frame as representing the exact same time in the different angles.
  • the process 1600 determines (at 1635 ) whether additional angles remain to be synchronized. When all of the angles have been synchronized, the process ends. Otherwise, the process returns to 1615 to select the next angle and synchronize this angle with the other previously synchronized angles.
  • the process 1600 is one conceptual example of a process of some embodiments for synchronizing angles using timing metadata (or other criteria if the timing metadata do not provide adequate information). Other embodiments use different variations of the synchronization process that may vary somewhat from process 1600 . For instance, some embodiments initially attempt to synchronize all of the angles at once using timecode information. The process determines whether all of the timecodes of the various angles overlap with each other to some extent (i.e., whether there are any gaps in the overall timecode, treating each angle as a continuous block) or whether all of the timecodes fit within a particular time range (e.g., a day), depending on the criteria used by the application (this criteria may be a default setting or a user-determined setting, in some embodiments).
  • the application aligns all of the data using these timecode. In some embodiments, when there are only one or two outlier angles, the application synchronizes the other angles using their timecode and then aligns the outlier angles with the start of the first clip of the earliest angle.
  • Some embodiments instead try to use time of day information for all of the angles when the timecode for at least one of the angles does not pass the criteria for synchronization.
  • the application then applies similar criteria to the time of day information for the angles (determining whether all of the angles overlap or whether all of the angles are within a threshold duration). If all of the angles can be aligned with the time of day information, then the media-editing application uses this time of day information to synchronize the angles. If the time of day information cannot be used for all of the angles, then different embodiments may apply different options.
  • some embodiments use whichever metadata passes the usefulness criteria for the greatest number of angles to synchronize those angles, then attempts to synchronize the other angles to any of the synchronized angles using time of day information. Any angles without useful timing metadata will start at the beginning of the multi-angle clip. As will be described in Section IV, some embodiments allow the user to re-align the angles manually after the application automatically creates the multi-camera clip.
  • some embodiments fine tune this alignment using audio comparisons between clips in different angles.
  • the media-editing application compares clips from different angles that overlap in time in order to identify clips that have the same (or similar) audio or same (or similar) sections of audio. The application can then determine an offset between the audio of the two clips and re-align one of the clips to compensate for the offset.
  • a process of some embodiments for analyzing audio in order to synchronize clips is described in detail in the U.S.
  • some embodiments assume that the initial synchronization will be accurate to within a particular threshold (e.g., five seconds, one minute, two minutes, etc.) and only compare audio of one clip to audio of other clips within that threshold in the timeline of the multi-camera media clip.
  • a particular threshold e.g., five seconds, one minute, two minutes, etc.
  • FIG. 18 conceptually illustrates a process 1800 of some embodiments for synchronizing media clips across a set of angles within a multi-camera media clip.
  • the process 1800 will be described by reference to FIG. 19 , which illustrates the fine-tuned audio synchronization of the angles 1350 - 1360 over four stages 1910 - 1940 .
  • the process 1800 begins by receiving (at 1805 ) a set of synchronized angles of a multi-camera media clip.
  • the angles (groups of media clips) have already been synchronized based on timing metadata of their media clips using the process 1600 or a similar alignment process.
  • some embodiments store these angles as collection clip structures within a multi-camera clip structure, which itself is a collection clip structure.
  • the clips might all be video clips with accompanying audio (e.g., clips with a video component to which an audio component is anchored, as described above in Section II), or include a number of audio clips with no video component, video clips with no audio component, and still image clips (which generally lack an audio component).
  • only the first two angles 1350 and 1355 contain clips with audio data, and thus only these clips will be adjusted by the audio synchronization.
  • the process selects (at 1810 ) the first unlocked clip in the overall timeline of the multi-camera media clip.
  • the process selects clips starting at the beginning of the timeline and moving towards the end of the timeline, locking the clips into place one at a time.
  • a clip is locked means that the process 1800 has evaluated the clip and will no longer move the clip along the overall timeline of the multi-angle media clip.
  • Once a clip is locked, other clips may be shifted along the timeline to align their audio with that of the locked clip.
  • the process determines (at 1815 ) whether the selected clip overlaps any locked clips in a different angle (i.e., an angle other than that to which the selected clip belongs). That is, the application identifies whether the span of the currently selected clip in the overall timeline of the multi-camera media clip overlaps the span of any clip that has been flagged as locked. In some embodiments, in order to save processing, the media-editing application only compares the audio of clips that overlap, rather than comparing each clip's audio to the audio of all other clips in different angles. When the selected clip does not overlap any locked clips, the process locks (at 1840 ) the selected clip. Because there are no locked clips when the first clip is selected, the application automatically locks this first clip into place.
  • a different angle i.e., an angle other than that to which the selected clip belongs
  • the first stage 1910 of FIG. 19 illustrates that initially the first clip 1305 is locked.
  • the locked clips are illustrated with solid lines while clips that have not yet been locked are drawn with dashed lines.
  • the other four video clips (with accompanying audio) are drawn with dashed lines at the first stage 1910 .
  • the clips of angle 1360 are shown as locked because they are not candidates for audio synchronization, as they represent still images without accompanying audio.
  • the process analyzes (at 1820 ) audio of the selected clip and all of the overlapping locked clips. In some cases, this will be many clips (when there are numerous angles), some of which will have already been aligned with each other by previous iterations of this process. Referring again to FIG. 19 , the next clip selected is the clip 1310 , which overlaps the locked clip 1305 . As shown, both of these clips include audio components, which can be compared with each other.
  • the media-editing application of some embodiments uses a process described in detail in U.S. patent application Ser. No. 13/019,986, incorporated by reference above, to compare the audio of two clips.
  • the algorithm of some embodiments compares two audio clips using cross correlation or phase correlation to produce a correlation function that indicates the likelihood of a match for all possible time offsets of the two clips. The peak value of this correlation function thus indicates the most likely alignment of the two audio signals.
  • some embodiments generate the function up to a maximum offset (e.g., two minutes).
  • a maximum offset e.g., two minutes.
  • some embodiments also compare previously-generated waveform data for the two clips.
  • the media-editing application automatically generates audio waveform data for each clip containing audio information.
  • the application uses this generated data to display audio waveforms for the clips (e.g., in the clip browser or the composite display area).
  • the application compares the waveform of the two clips for similarities. In many cases, this comparison can be performed significantly more quickly than comparing audio data.
  • some embodiments use a combination of the audio waveform data and the audio data to determine the correlation of the different time offsets. For instance, some embodiments initially use the audio waveform information to approximate a peak correlation offset for two clips, then further fine-tune the offset with the audio data.
  • some embodiments generate waveform data points for every 500 (or 100, 1000, etc.) audio samples.
  • the offset from the waveform data can be treated as precise to within 500 samples (if the audio sample rate is 48 kHz, then this is 1/96 seconds, or 1 ⁇ 4 frame for 24 fps video).
  • the application uses a small range of audio offsets (e.g., five seconds, ten seconds, etc.) around the identified offset to more precisely identify a peak offset for the two clips.
  • the process calculates (at 1825 ) the offset of the best matching locked clip and the selected clip.
  • each of the comparisons generates a peak offset correlation value that indicates (i) the offset of the two clips producing the best correlation and (ii) the likelihood that this peak correlation is indicative of a match.
  • Some embodiments identify the offset between the selected clip and any of the overlapping clips that produces the highest likelihood of a match (e.g., 20%, 75%, 90%, etc.).
  • the process determines (at 1830 ) whether this best matched clip has a high enough likelihood of correlation to be used for synchronization.
  • Some embodiments use a minimum threshold correlation value to prevent the media-editing application from shifting clips off of the initial synchronization when there is a low probability of the audio indicating a match. For instance, some embodiments require at least a 50% likelihood of correlation, or even a 70% likelihood, in order to shift the audio, though other embodiments do not require as high a likelihood of correlation.
  • some embodiments generate the correlation values up to a maximum offset.
  • Some embodiments initially use a first maximum offset (e.g., 10 seconds) and determine whether the peak offset has a high enough correlation. When the correlation at this offset is not above the threshold, the process expands the range and calculates correlation values up to a second, larger maximum offset (e.g., 2 minutes). If the peak correlation value is still not above the threshold likelihood, the process expands the range to the entirety of the two clips. At this point, if the peak offset does not have a high enough correlation, then the process assumes that there is no match.
  • a first maximum offset e.g. 10 seconds
  • a second, larger maximum offset e.g. 2 minutes
  • some embodiments identify multiple peaks (e.g., 10-20 peaks within the sampled range). The application then compares the highest peak to the other peaks to ensure that the maximum correlation found is more than a second threshold value higher than the other peaks. That is, the application determines whether the peak offset is significantly more likely of an alignment than other offsets in the range. If the difference is not large enough (i.e., does not exceed the second threshold value), then the process does not consider the offset to be a match.
  • peaks e.g. 10-20 peaks within the sampled range.
  • the application compares the highest peak to the other peaks to ensure that the maximum correlation found is more than a second threshold value higher than the other peaks. That is, the application determines whether the peak offset is significantly more likely of an alignment than other offsets in the range. If the difference is not large enough (i.e., does not exceed the second threshold value), then the process does not consider the offset to be a match.
  • the process locks (at 1840 ) the currently selected clip without re-aligning the clip.
  • the process shifts (at 1835 ) all unlocked clips in the angle of the currently selected clip by the peak offset time.
  • any previously locked clips in the angle (which may have already been synchronized to other locked clips) are not shifted, but the application of some embodiments implements a presumption that other unlocked clips should be shifted by the same offset (e.g., in case the clock for the device that captured the clips in that angle was off by a consistent duration).
  • some embodiments adjust the duration of the gap clip that precedes the selected clip (shortening the duration of the gap clip in order to move the selected clip earlier in the timeline, and lengthening the duration in order to move the selected clip later in the duration). Some embodiments may shift a locked clip in the specific instance in which a likely match is found for a selected clip with an offset that would move the selected clip so that it overlaps an earlier clip in the angle.
  • some embodiments will “release” the locked clip and move it earlier in time as needed to allow for the alignment of the selected clip (e.g., by removing the gap between the selected clip and the locked clip, then shortening the gap before the locked clip).
  • the media-editing application requires that the video be shifted in discrete amounts equal to the duration of a single frame (e.g., 1/24 or 1/30 of a second). However, audio recordings are typically sampled in much smaller increments (e.g., 1/48,000 of a second). In some embodiments, the application determines the offset at the level of the audio sample boundaries, then shifts the video to the nearest frame boundary. For the audio, some embodiments slightly offset the shifted audio by less than the increment of a video frame so that the audio of the shifted clip more closely aligns with the audio that it matches.
  • the first stage 1910 of FIG. 19 illustrates that the audio of the currently selected clip 1310 is slightly offset from the audio of the locked clip 1305 .
  • the media-editing application moves both the clip 1310 and the clip 1320 earlier in the timeline of the multi-camera media clip, the result of which is shown in stage 1920 .
  • the application modifies the duration of the gap clip that precedes the clip 1310 in the angle 1355 .
  • the process 1800 locks (at 1840 ) the selected clip. In some embodiments, as stated, this involves setting a flag (e.g., in volatile memory) that the clip is locked and should no longer be moved while performing the audio synchronization. The process then determines (at 1845 ) whether any additional unlocked clips remain to be evaluated. When additional clips remain, the process returns to 1810 to select the next unlocked clip in the multi-camera media clip. Once all of the clips have been aligned, the process 1800 ends.
  • a flag e.g., in volatile memory
  • the second stage 1920 of FIG. 19 illustrates that the media-editing application has also locked the clip 1325 without moving it.
  • This clip is the next clip selected after aligning clip 1310 , but does not overlap any locked clips in other angles. As such, the application locks this clip and moves on to evaluating the next clip 1320 .
  • the second stage 1920 illustrates that the audio of clip 1320 is slightly offset from that of the clip 1325 . As such, the application shifts the clip 1320 slightly later in the timeline (by increasing the duration of the gap clip between clip 1310 and clip 1320 , in some embodiments).
  • the third stage 1930 illustrates that the application has moved the clip 1320 slightly later in the multi-camera clip timeline and locked the clip 1320 .
  • the figure illustrates that the audio of the clip 1315 , in the first angle 1350 , is slightly offset from that of the now-locked clip 1320 .
  • the application moves the clip 1315 earlier in the timeline of the multi-camera media clip (by decreasing the duration of the gap clip between clip 1325 and clip 1315 ).
  • the fourth stage 1910 illustrates the timeline of the three angles of the multi-camera media clip after the audio synchronization process.
  • multi-camera media clips will have more than two tracks with audio clips and more than two or three clips per track.
  • the synchronization will propagate through the multi-camera clip (e.g., as shown in stages 1920 - 1940 for clips 1325 , 1320 , and 1315 ) as each individual clip is aligned within the overall multi-camera clip timeline.
  • FIG. 20 conceptually illustrates a subset of the data structures contained by a multi-camera media clip created in an event of the media-editing application.
  • the application stores a reference multi-camera clip in an event data structure of the application, then defines instance clips in one or more media projects that refer to the reference clip and inherit properties from the reference clip.
  • the reference multi-camera media clip 2000 illustrated in FIG. 20 is an example of such a reference clip. In some embodiments, reference clips are only used for multi-camera media clips.
  • reference clips and correspondingly, clip instances that refer to the reference clips
  • clip instances that refer to the reference clips
  • the nested sequence of clip objects that defines the multi-camera media clip is similar in structure to the objects illustrated in FIG. 6 .
  • the figure illustrates a sequence 2005 , an angle instance 2012 , a reference collection 2000 (the multi-camera media clip), a gap clip 2007 contained by the reference collection 2000 , one of the angle collections 2010 of the multi-camera media clip, and one of the clips 2015 of the angle collection 2010 .
  • each of the clip objects shown in this figure is a member of the same class, though the objects may be of different sub-classes.
  • the set of data structures shown in FIG. 20 might be the output of the multi-camera clip creation process described by reference to FIG. 10 .
  • some embodiments create a sequence within the event object for each media file in the event. This includes not only clips imported into the event, but also compound clips or multi-camera clips created in the event.
  • the sequence 2005 is an example of such a sequence for an event clip.
  • the sequence 2005 like sequence 605 of FIG. 6 , stores a sequence ID, a set of sequence attributes, and the angle instance object 2012 .
  • the sequence attributes include the resolution, frame rate, and other video and audio properties for the sequence.
  • some embodiments provide the user with the option of customizing these sequence attributes for a multi-camera clip or inheriting them from the media clips used to generate the multi-camera media clip.
  • the angle instance 2012 is an object that contains (or wraps) the reference collection 2000 .
  • the angle instance 2012 indicates an active video angle and active audio angle that are used when the multi-camera clip is added to a sequence in the timeline. These active angles can be selected by a user of the media-editing application in some embodiments (e.g., through the angle viewer or the inspector).
  • the angle instance data structure, as used in timeline sequences, will be described in further detail below.
  • the reference collection 2000 can be thought of as the multi-camera clip object.
  • the reference collection object 2000 is a collection object similar to collection objects 610 of FIG. 6 or 810 of FIG. 8 (a compound clip object), and similarly includes a collection ID.
  • the object includes a multi-camera flag. This indicates to the media-editing application to display a multi-camera indicator over the clip, as well as to provide the functionality associated with multi-camera clips (described in the sections below).
  • the reference collection 2000 includes a single gap clip 2007 in its array, with no content.
  • the gap clip 2007 includes a collection ID, a gap component with no content, and a total and trimmed range.
  • the gap clip is a component or a generator rather than a collection. Regardless of how the gap clip is implemented, it does not refer to any actual media, but instead produces a solid black output.
  • the duration of the gap clip 2007 varies based on the length of the longest angle in the clip.
  • the angles are stored as connected storylines, which are collections anchored to the gap clip in the reference collection.
  • the connected storylines are anchored to the start of the gap clip (i.e., with an offset of 0, 0).
  • the gap clip 2007 includes three anchored items (Angles 1, 2, and 3).
  • the multi-camera clip includes three angles.
  • the second of these angles is represented by the collection clip 2010 , and the other two would have a similar structure (with different clips in their respective arrays).
  • the collection clip 2010 includes an angle ID, an ordered array of two content clips separated by a gap clip, total and trimmed ranges, anchoring information to the parent gap clip (Gap Clip 1) with an offset of 0, 0 (as each angle is anchored to the start of the gap clip 2007 ), and a lane number that indicates the order of the anchored item in the multi-camera media clip (in this case, 2).
  • Some embodiments use non-zero anchor offsets to synchronize angles within a multi-camera clip rather than inserting gap clips at the start of the angles.
  • the angle ID is a unique ID that identifies the angle.
  • the angle collection 2010 also stores an angle name, which the user can change. Even when a user changes the angle name, the angle ID stays constant so that the media-editing application does not need to update any clip instances that refer to the angle.
  • the array includes all of the clips of the angle arranged in time order.
  • the angle has only two clips (Clip 2A and Clip 2B).
  • the application inserts a gap clip between clips in order to space out the clips in the multi-camera timeline.
  • some embodiments insert a gap clip as the first clip in the array.
  • Clip 2A starts at time zero of the overall timeline and thus this clip is the first clip in the array (i.e., there is no gap clip to start the array).
  • the clip 2015 (Clip 2A) is a collection clip similar to clip 610 , with a single media component in its array (e.g., a video component) and a second component (e.g., an audio component) anchored to the first component (not shown for simplicity).
  • these components are themselves clip structures that refer to media asset data structures.
  • the clips within the array of an angle collection could also be compound clips that contain additional collections.
  • Other embodiments only allow simple collection clips that contain components (such as clip 2015 ) within multi-camera clips.
  • the above section described the automatic creation of multi-camera media clips from several media clips (either using default settings of a media-editing application or preferences input from a user).
  • a user may edit the reference multi-camera clip as well as add instances of the multi-camera clip to a media project (both in the primary compositing lane of a project as well as in secondary anchor lanes).
  • These instance clips each have an active audio angle and active video angle, which determine the content of the multi-camera media clip that is actually part of the media project. Users can switch the active angle in a variety of different ways, or blade the clip instance into two clip instances at a particular frame boundary in order to enable seamless switching from one angle to the next.
  • the user can manually edit a multi-camera reference clip stored in an event.
  • the media-editing application provides a display area for manually adjusting the clips within the angles (e.g., aligning clips), adding effects to the clips in the reference clip, renaming, adding, and deleting angles, reordering the angles within the multi-camera clip, etc.
  • This multi-camera clip editing display area is a specific version of the composite display area for multi-camera clips in some embodiments, that does not display the gap clip to which the different angles are anchored, and displays each of the angles as a separate track.
  • FIG. 21 illustrates the opening of such a display area in a graphical user interface (“GUI”) 2100 of some embodiments over two stages 2110 and 2120 .
  • the first stage 2110 illustrates a clip browser 2105 that includes a multi-camera clip 2115 , along with various other clips in the same event.
  • Some embodiments display an indicator over the thumbnail (in this case, four squares) to indicate to the user that the clip is a multi-camera clip.
  • the user has selected the multi-camera clip 2115 and activated a drop-down menu for the clip (e.g., through a right-click, two-finger tap, or other user input). As shown, the user is in the process of selecting the option “Open in Angle Editor” for the clip.
  • the second stage 2120 illustrates the resultant GUI 2100 .
  • Many of the display areas in the GUI 2100 were described above in Section I by reference to FIG. 3 : the clip library, clip browser 2105 , preview display area (not currently displaying any preview images), and inspector display area (not currently displaying any information as no items are selected in the GUI).
  • the GUI 2100 includes the multi-camera editing display area 2125 .
  • the editing display area 2125 displays the location and name of the selected multi-camera media clip that is open for editing (in this case, the location is “New Event Oct. 28, 2011” and the name is “New MultiCam Clip”).
  • This editing display area displays separate tracks for each of the angles in the selected multi-camera media clip.
  • the angles include at least “Amir”, “Nathaniel”, “Viega”, and “Matt”, as these are the angles for which tracks are at least partially shown in the figure.
  • the media-editing application identifies each different angle of the selected multi-camera media clip and creates a track in the editing display area for the angle.
  • some embodiments store the angles as collection clips using the data model for the trackless system described in Section I, but convert the angles for display as tracks in the multi-camera clip editor because each angle can be treated as having a sequence of clips separate (though synchronized with) the clips in the other angles.
  • FIG. 22 illustrates a GUI 2200 over three stages 2210 - 2230 in which the multi-camera preview display area is opened and options for the display area are accessed.
  • the GUI 2200 resembles the GUI 2100 , including a preview display area 2205 .
  • the user has opened a drop-down menu 2215 and is selecting an option of “Show Angle Viewer” from the menu.
  • the second stage 2220 illustrates the result of this selection in the GUI 2200 .
  • the media-editing application decreases the size of the preview display area 2205 and displays the angle viewer 2225 in a portion of the area previously occupied by the main preview display area 2205 .
  • the clip library, clip browser, angle viewer, and preview display area can all be hidden and have their widths varied so as to provide more display room to one or the other of these areas.
  • different configurations of the GUI are also possible (e.g., with different display areas in the top or bottom, or the display areas in a different order across the GUI).
  • the angle viewer as described above by reference to FIGS. 1 and 2 , provides preview displays for multiple angles into which the application outputs simultaneous video images from different angles of a multi-camera media clip.
  • the third stage 2230 illustrates a close-up of the angle viewer 2225 as the user opens a settings menu for the angle viewer.
  • the angle viewer displays video images from the first four angles of the multi-angle clip currently open in the angle editor.
  • the four preview windows display an image from a clip in each of four different angles, as well as information about the angles.
  • these four images come from the same time (e.g., the location of a playhead in the angle editor).
  • the current information for the different angle previews shows the names of the angles displayed in each preview as well as the time of the currently shown image within the individual clip. In some embodiments, this time starts from zero at the beginning of the source video to which the individual clip refers.
  • the settings menu provides various options for the angle viewer 2225 .
  • the user can change the number of preview displays shown in the viewer (in this case, the options are 2, 4, 9, or 16 preview displays).
  • the user can also determine whether or not to display the timecodes, and whether to indicate the angle name, clip name, or not identify the angles at all.
  • One of ordinary skill in the art will recognize that in different embodiments, different combinations of information may be displayed in the angle viewer. With the angle viewer introduced, various functionalities of the angle editor will now be described.
  • FIG. 23 illustrates the manual alignment of a media clip 2305 in a first angle track 2315 with a clip in a second angle track 2325 by moving the clip 2305 along its track. This figure illustrates this movement over three stages 2310 , 2320 , and 2330 of a GUI 2300 .
  • the GUI 2300 includes, among other GUI items, an angle editor 2335 (which includes the tracks 2315 and 2325 , as well as additional tracks), an angle viewer 2340 , and a preview display area 2345 .
  • the user has selected the media clip 2305 , but has not yet started moving the clip.
  • the angle viewer 2340 currently displays four previews of the first four angles of the multi-camera media clip that is currently open in the angle editor 2335 (“Nathaniel”, “Amir”, “Lam”, “Viega”). As the playhead 2350 is at gaps in the “Nathaniel” and “Lam” angles, the angle viewer shows blank images for these angles.
  • the preview display area 2345 displays an image from the “Amir” angle that corresponds to a time in the multi-camera media clip over which a playhead 2350 is situated.
  • the track 2325 (for the “Amir” angle) is currently highlighted as a selected angle, or monitoring angle, and thus the preview display area 2345 displays images from the “Amir” angle.
  • the user can switch the monitoring angle (as described below) in order to determine which angle the media-editing application will display in the preview display area.
  • the media-editing application automatically swaps out the angle viewer 2340 for a second preview display area 2355 , which displays images of the selected clip at the location of the playhead 2350 .
  • the media-editing application maintains (in the first preview display area) the image from the monitoring angle that corresponds to the playhead location in the timeline.
  • the second preview display area skims through the images from the selected clip that correspond to the current time in the clip that is over the static playhead.
  • a user can select a particular image from a clip in the monitoring angle, then find the image from the selected clip that matches the particular image (i.e., was captured at the same time) and use the playhead to align these images in order to align the clips.
  • some embodiments display an indicator over the selected clip (the clip being moved) that shows how far the clip has moved from its previous location—in this case, the clip 2305 has been moved twenty seconds and six frames later in the timeline at stage 2320 .
  • the third stage 2330 illustrates the GUI 2300 after the user has finished moving the clip 2305 .
  • the media-editing application has replaced the second preview area 2355 with the angle viewer 2340 , and now displays an image from the monitoring angle in the preview display area 2345 .
  • the application does not ripple the clips in the angle of the moved clip. That is, when the user moves clip 2305 , the other clips in the track 2315 remain stationary. In this way, the application does not remove the alignment of other clips in the angle (which might already be aligned to clips in other angles).
  • the movement can be thought of as a slide edit of the selected clip, which changes the in and out points of the gap clips on either side of the selected clip.
  • the application will treat this as a roll edit rather than a ripple edit, and simply modify the length of the gap clip on the edited boundary of the media clip. If the user wishes to move all of the clips in an angle together, the user can select all of the clips and then move the clips in either direction along the timeline (which increases or decreases the length of the gap clip at the start of the angle.
  • the user can change the monitoring angle within the angle editor, as well as enable or disable audio for the different angles.
  • Some embodiments require only a single monitoring angle, but allow for multiple different angles to have enabled audio.
  • some embodiments provide GUI items for each track in the angle editor.
  • FIG. 24 illustrates a user changing the monitoring angle for a multi-camera media clip according to some embodiments.
  • This figure illustrates an angle editor 2400 over two stages 2410 and 2420 .
  • the track “Amir” 2405 is highlighted as the monitoring angle.
  • the media-editing application highlights a selectable item 2415 in the track. This selectable item 2415 both indicates that the track 2405 is the monitoring angle and would enable the user to select the track 2405 as the monitoring angle when a different track was selected.
  • the user has placed a cursor over the corresponding selectable item 2425 for the track 2430 , and selects this item with the cursor.
  • the second stage 2420 illustrates the result of this selection.
  • the track 2430 for the angle “Viega” is now the monitoring angle, and the application highlights the track as such.
  • the selectable item 2425 is now highlighted, and the application has removed the highlighting from the selectable item 2415 and the track 2405 .
  • the user determines which angle's images will be displayed in the preview display area (e.g., for aligning clips as shown in FIG. 23 ).
  • FIG. 25 illustrates a user enabling and disabling the audio for various angles of the multi-camera reference clip over three stages 2510 - 2530 showing the angle editor 2400 .
  • some embodiments output the audio of the clip. Whereas only one angle is displayed in the preview display area (the monitoring angle), the media-editing application of some embodiments can combine the audio of the multiple selected signals and output the combined audio.
  • each track in the angle editor 2400 has an audio selectable GUI item as well. These audio items enable a user to turn on or off the audio of the different tracks.
  • the monitoring angle “Viega” (track 2430 ) as well as the angle “Amir” (track 2405 ) have their audio enabled.
  • the selectable items 2525 and 2535 for these tracks are highlighted, indicating that the audio is enabled.
  • the application highlights the selectable items 2525 and 2535 by displaying sound waves coming from the microphone icons).
  • the user has placed a cursor over the selectable item 2540 for the angle “Nathaniel” (track 2545 ), and selects this item with the cursor.
  • the second stage 2520 illustrates the result of this selection, with the selectable item 2540 for track 2545 now highlighted, indicating that the application has now enabled the audio of these angles.
  • the user has placed the cursor over the selectable item 2525 , the audio item for the monitoring angle, and is selecting this item.
  • the item 2525 is no longer highlighted. As this illustrates, the application of some embodiments allows the user to disable the audio of the monitoring angle.
  • FIG. 26 conceptually illustrates a process 2600 of some embodiments for playing back a multi-camera media clip open in the angle editor.
  • the process begins by receiving (at 2605 ) a command to play back a multi-camera media clip.
  • the user can initiate playback through a keystroke (e.g., the space bar), selecting a menu option, selecting a user interface item, etc.
  • the playhead moves through the clip's timeline at a constant rate so that the video is played back at real-time speed.
  • the process identifies (at 2610 ) a video angle to display in the preview display area.
  • the application selects the video monitoring angle.
  • the process also identifies (at 2615 ) all enabled audio angles. As shown in FIG. 25 , multiple angles may be enabled for audio, and these may exclude the monitoring angle.
  • the process 2600 renders (at 2620 ) video images in the preview display area using the clips from the identified video angle (i.e., the video monitoring angle). As will be described below in Section VI, rendering these images involves reading the images from a storage, decoding the image (if required), applying effects to the image, and outputting the image. In addition to displaying the image of the identified video angle in the preview display area, some embodiments also display multiple video images for the angle viewer (if the angle viewer is currently displayed in the GUI). The process also renders (at 2625 ) audio as a composition of all of the enabled audio angles. This may involve using standard audio processing techniques to combine multiple audio signals from the various enabled audio angles. The process 2600 then ends.
  • FIG. 26 describes the selection of which angles to render when playing back a multi-camera media clip in the angle editor.
  • users can play back a particular angle other than the monitoring angle.
  • the media-editing application of some embodiments switches out the angle viewer for a second preview display area, similar to what is shown in stage 2320 of FIG. 23 .
  • the application then plays back the selected angle in the second preview display area while playing the monitoring angle in the primary preview display area.
  • some other embodiments play back the selected angle in the primary preview display while also playing back several angles in the angle viewer (thus swapping out the monitoring angle for the selected angle in the preview display area).
  • the skimming playhead in some embodiments, is separate from a primary editing playhead which remains in place while the user skims through the clip.
  • the skimming playhead shrinks down and moves through the track of the monitoring angle. In this case, the angle viewer continues to display the multiple displays for different angles while the preview display area displays images from the monitoring angle.
  • the skimming playhead is displayed over the entire timeline, while again the angle viewer remains to display simultaneous previews of multiple angles while the preview display area displays images from the monitoring angle.
  • the application of some embodiments swaps out the angle viewer for a second preview display area that displays the angle through which the user is skimming. In this way, the user can skim through the particular angle to identify a frame that matches that of the monitoring angle shown in the primary preview display area. Similar to the situation illustrated in FIG. 23 , the editing playhead remains fixed and thus so does the image from the monitoring angle displayed in the primary preview display area.
  • the application determines whether audio skimming is turned on. Some embodiments provide a GUI tool that enables users to determine whether to play audio when a user skims through a multi-camera media clip, media project, event clip, etc. When audio skimming is off, the process does not treat any audio as enabled. When either audio skimming is on, or the application is playing back the multi-camera media clip as shown in FIG. 26 , the process identifies which of the various angles of the multi-camera media clip have been designated as active.
  • Changing the video monitoring angle and enabling and disabling audio angles can be used to help with manually aligning media clips within a multi-camera clip.
  • some embodiments provide functionality for automatic synchronization using audio within the angle editor. For example, if a set of media clips from which the application generates a multi-camera clip does not provide any useful timing metadata, then the clips might be out of order and/or so far out of alignment that the automatic audio synchronization process (e.g., that described above in Section III.d) cannot find any audio matches to use for alignment. In such a case, a user might open the clip in the angle editor and manually align the clips. However, in the absence of specific images that the user can pinpoint as matching between two video angles, achieving perfect alignment may be difficult without audio synchronization.
  • FIG. 27 illustrates the use of a function to automatically synchronize a selected clip to a clip in a different angle in an angle editor 2700 over two stages 2710 and 2720 .
  • the user has selected a media clip 2705 (as indicated by the highlighting around the edges of the media clip), and activated a drop-down menu for the clip that includes the option “Sync to Monitor Angle via Audio”.
  • the user can select a monitoring angle and then synchronize clips in other angles to the monitoring angle using audio synchronization.
  • the second stage 2720 illustrates that the media-editing application has shifted the selected clip 2705 later in the timeline by approximately five seconds in order to align the audio of clip 2705 with the audio of clip 2715 . As shown, the peaks in the audio now line up in the timeline.
  • Some embodiments use similar audio synchronization techniques to those used when initially creating a multi-camera media clip. That is, the media-editing application compares the audio of a selected clip to the audio of a clip (or multiple other clips) in the monitoring angle to identify the likelihood of correlation at various offsets. In some embodiments, the application compares the selected clip to any overlapping clip or clips in the monitoring angle. Some embodiments compare the audio of the selected clip to various offsets within the monitoring angle (e.g., up to two minutes of offset, etc.), or to the entirety of the monitoring angle.
  • the application moves the selected clip in the multi-camera clip timeline by this offset value.
  • Some embodiments use a lower threshold for post-creation synchronization on the assumption that the user has more clearly expressed a preference to shift the selected clip in the timeline, and because the user will often select a clip that is very close to correctly aligned and will be using the audio synchronization for fine-tuning.
  • some embodiments provide additional options in the angle editor.
  • users can choose to perform a full audio synchronization of a multi-camera media clip, synchronize a first selected clip to a second selected clip, synchronize a selected clip to the best matching clip in any other angle, etc.
  • Some embodiments allow the user to specify the maximum offset range for the synchronization algorithm (i.e., placing a threshold on how far the media clip may be shifted, which can be used to decrease the processing time for synchronization).
  • FIG. 28 illustrates the use of such a frame matching technique over three stages 2810 - 2830 of a GUI 2800 .
  • the GUI 2800 includes a primary preview display area 2805 and an angle editor 2815 .
  • the user has selected a particular time (i.e., particular frame, indicated by the location of the playhead 2835 ) in a selected clip 2825 .
  • the user has initiated a menu that includes the option “Sync to Frame”.
  • the selection of the “Sync to Frame” menu option brings up a secondary preview display 2840 in the GUI 2800 .
  • the selected frame from the clip 2825 remains fixed in the primary preview display 2805 , while the user can skim through the clips in other angles (angles that do not include the selected clip 2825 ) in the secondary preview display 2840 .
  • the user selects this frame for synchronization (e.g., by clicking or tapping the location in the clip representation). In the figure, the user selects a particular frame in the clip 2845 at the second stage 2820 .
  • Stage 2830 illustrates the result of this selection.
  • the clip 2825 has been automatically moved approximately 20 seconds earlier in its track.
  • the application initially moves the selected clip so that the selected frames are exactly matched up, then analyzes audio data of the two clips within a predetermined range (e.g., 5 seconds, 10 seconds, etc.) around the two matched frames in order to more precisely align the selected clip.
  • a predetermined range e.g., 5 seconds, 10 seconds, etc.
  • Some embodiments include variations on the process shown in FIG. 28 .
  • a user might initially select a clip and the “Sync to Frame” option (or a variation thereof), then be allowed to skim through the selected clip in order to find the first frame for alignment.
  • the user would then be presented with the opportunity to skim through the other clips in other angles to identify a second frame (e.g., as shown in stage 2820 ).
  • the application enables the user to utilize the angle editor to modify clip properties such as the number of angles, the names of the angles, the order of the angles, etc. Some embodiments automatically reflect these properties in the angle viewer as soon as the user modifies the property in the angle editor.
  • FIG. 29 illustrates a user adding an angle to a multi-camera media clip in a GUI 2900 over two stages 2910 and 2920 .
  • the GUI 2900 includes both an angle editor 2905 and an angle viewer 2915 .
  • One of ordinary skill in the art will recognize that some embodiments will additionally include a preview display area, but this is not shown in this figure for simplicity.
  • the multi-camera media clip open in the angle editor 2905 includes five angles.
  • the angle viewer 2915 is set to display four angles at a time, but currently only has one angle in the top left corner.
  • the media-editing application divides the angles into sets (of up to four angles), and the user can choose between these sets by selecting the angle banks 2925 at the bottom of the angle viewer 2915 .
  • the user has selected the bank with a single display rather than the bank with four displays.
  • the user has initiated a drop-down menu 2930 for the multi-camera clip, and is in the process of selecting the “Add Angle” option from the menu.
  • the user can access this drop-down menu via a selectable item displayed in the header for each of the angles.
  • selectable item displayed in the header for each of the angles.
  • One of ordinary skill in the art will recognize that some or all of the options in the drop-down menu 2930 may be accessed through other user interface constructs, such as keystrokes, option-specific UI items, other menus, etc.
  • the second stage 2920 illustrates the result of this menu selection.
  • the application has added a new angle to the multi-camera media clip (directly after the angle from which the drop-down menu was initiated). In some embodiments, this involves adding an anchored collection to the multi-camera reference clip collection that represents the multi-camera media clip (i.e., another collection such as the collection 2010 of FIG. 20 ). In some embodiments, the application assigns the new anchored collection the lane number of the angle from which the menu was accessed, and increments the lane numbers of all of the higher-numbered angles.
  • the GUI 2900 now displays (i) a new track 2935 for the angle in the angle editor 2905 , (ii) a new display area 2940 for the angle in the angle viewer 2915 , and (iii) an additional square in the selected set of angles in the banks 2925 .
  • the new angle is automatically given a name (e.g., “Untitled Angle”, “Angle 6”, etc.).
  • the media-editing application uses processes described in Section III (or variations thereof) to order and align the newly added clips in the multi-camera clip. That is, the application first sequences the clips within the angle using timing metadata of the clips, then aligns the sequence with the other angles using timing metadata followed by audio synchronization.
  • FIG. 30 illustrates a user renaming an angle of a multi-camera media clip in the GUI 2900 over two stages 3010 and 3020 .
  • the user has selected and highlighted the name of the track 2935 in the angle editor 2905 , “Untitled Angle”.
  • selecting the name e.g., with a click, double-click, tap, double-tap, etc.
  • the angle editor causes the name to become a modifiable text box.
  • the user can type in a new name for the angle.
  • the second stage 3020 illustrates the GUI 2900 after the user has renamed the angle represented by the track 2935 with the name “Matt”.
  • the media-editing application modifies the angle name in the collection data structure for the angle, while keeping the unique angle ID constant.
  • data structures that refer to the angle do not need to be updated but instead continue to refer to the same unique angle ID.
  • items that refer to that angle ID will now retrieve different information for the angle name.
  • the preview display 2940 in the angle viewer 2915 now reads “Matt” rather than “Untitled Angle”.
  • FIG. 31 illustrates a user deleting an angle from a multi-camera media clip in the GUI 2900 over two stages 3110 and 3120 .
  • the user has initiated a drop-down menu 3105 (similar to the menu 2930 ) through the track header for the track 2935 (the track for the angle “Matt”). Within this menu, the user is in the process of selecting the “Delete Angle” option.
  • the second stage 3120 illustrates the result of this menu selection.
  • the application has removed the angle “Matt” from the multi-camera media clip.
  • this involves removing the anchored collection with the matching angle ID from the multi-camera reference clip collection that represents the multi-camera media clip (i.e., a collection such as the collection 2010 of FIG. 20 ).
  • the GUI 2900 now displays (i) one fewer tracks in the angle editor 2905 , (ii) a different set of preview displays in the angle viewer 2915 as the display 3115 for the angle “Amir” has moved into the first set of four displays, and (iii) one fewer square in the banks 2925 .
  • the application removes the anchored collection for that angle from the multi-camera media clip structure, and decrements the lane numbers of any higher-numbered angles in the multi-camera media clip.
  • FIG. 32 illustrates a user reordering the angles of a multi-camera media clip in the GUI 2900 over three stages 3210 - 3230 .
  • the first stage 3210 illustrates that a user has selected the track 3205 and begun moving the track upward.
  • the tracks in the angle editor 2905 have handles with which a user can select the track for reordering. In this case, the handles are located on the right side of the track header. At this point the selected track 3205 for the angle “Amir” is below the track 3215 for the angle “Jim”.
  • the second stage 3220 illustrates the GUI 2900 while the user continues to move the track 3205 upwards in the angle editor 2905 .
  • some embodiments automatically slide the other track into the spot previously occupied by the selected track.
  • the track 3215 is now below the selected track 3205 , although the user has not released the track 3205 yet.
  • some embodiments automatically update the multi-camera media clip data structure to reflect this reordering, by modifying the lane numbers of the two angles that have swapped places.
  • the application modifies the order of the preview displays in the angle viewer 2915 .
  • the viewer for the angle “Jim” is the third display (bottom left) while the viewer for the angle “Amir” is the fourth display (bottom right).
  • the viewer for these two angles have switched places, mirroring the change in the angle editor.
  • the third stage 3230 illustrates the GUI 2900 after the user has released the track 3205 in the second slot of the angle editor 2905 .
  • the preview display for the angle “Amir” in the angle viewer 2915 is now in the second (top right) position, and the application has again updated the multi-camera media clip data structure to reflect this change in ordering (e.g., by modifying the lane numbers of the angles).
  • FIG. 33 illustrates the application of a black and white effect from an effects display area 3315 to a clip 3305 in an angle editor 3310 . This effect applies a function to the color values of the affected clip.
  • users can apply transform, crop, distortion, etc. effects to the clips that affect the shape or size of the displayed images for the clip. In some embodiments, these effects are applied through on-screen controls in the preview display area or by modifying effect variables in the inspector.
  • Some embodiments allow a user to apply effects to entire angles instead of single clips. Users can select all of the clips of the angle and then select an effect from the effects display area 3315 , and the application will apply this effect to all of the clips. Similarly, in some such embodiments, defining a transform (or crop, or distort) effect through the on-screen controls of the preview display area will apply the transform to all clips in the selected angle. Other embodiments, however, only apply effects to a single clip at a time.
  • the previous section described the editing of a multi-camera reference clip in the angle editor.
  • the multi-camera clips can also be added to a media project, much like a standard media clip.
  • some embodiments create a clip instance in the data structure of the media project, which refers back to the reference clip stored in the event.
  • Each clip instance includes active video and audio angles that indicate which angle's video and audio should be used in the media project for the duration of the clip instance.
  • the data structures for the multi-camera clip instances differ from those of the timeline clips shown in FIG. 8 .
  • the user can perform various operations on the clip instances.
  • the user can swap out the active video and/or audio angles for a clip, split a clip into two clip instances with different angles for each of the instances, apply effects to specific angles of the clip instances, as well as other editing operations.
  • FIG. 34 illustrates the addition of a multi-camera media clip to a media project over three stages 3410 - 3430 of a GUI 3400 .
  • the GUI 3400 includes a clip browser 3405 and a composite display area 3415 , among other display areas.
  • the composite display area is currently displaying the timeline for a media project (“New Project”) with one video clip 3407 , which is a typical video clip of the sort described in Section II (i.e., a collection with a video component and anchored audio component that both refer to the same asset).
  • New Project media project
  • video clip 3407 which is a typical video clip of the sort described in Section II (i.e., a collection with a video component and anchored audio component that both refer to the same asset).
  • Within the clip browser 3405 are various clips in the event “New Event Oct. 28, 2011”, including a multi-camera reference clip 3425 .
  • the first stage 3410 illustrates that a user has selected the multi-camera reference clip 3425 .
  • the user drags this selected clip 3425 and drops the clip into the composite display area 3415 .
  • the third stage 3430 illustrates the result of this action, with a multi-camera clip instance 3435 now part of the media project in the composite display area 3415 .
  • the representation for the clip instance 3435 has an appearance similar to that of the video clip 3407 , but also includes a multi-camera clip indicator (four squares, similar to that displayed on the multi-camera reference clips in the clip browser) as well as an angle name rather than a clip name.
  • the active video and audio angle of the clip instance 3435 is “Amir”, which means that when the application plays back, renders, exports, etc. the media project, it sources the output for that portion from the assets referred to by the clips in the “Amir” angle.
  • Some embodiments copy the angle instance data structure from the event (e.g., angle instance 2012 ) and use the active video and audio angles set for this structure. These active video and audio angles may be modified by the user through the inspector display area, in some embodiments.
  • the clip 3435 in the composite display area 3415 is a clip instance, which refers to a multi-camera reference clip structure (e.g., that shown in FIG. 20 ).
  • FIG. 35 conceptually illustrates a subset of the data structures that define a media project that includes a multi-camera clip instance 3515 .
  • the data structures of FIG. 35 are all contained within a project data structure that contains a single sequence 3500 .
  • the sequence 3500 includes a primary collection data structure 3505 , which includes an array containing three clips 3510 - 3520 .
  • the first and third clips 3510 and 3520 are collection clips similar to clips 805 - 815 of FIG. 8 —either individual or compound clips that contain component clips which in turn refer to media assets.
  • the angle instance clip 3515 does not contain any nested clips.
  • the instance 3515 instead contains an instance ID, a clip reference, total and trimmed ranges, active audio and video angles, an effect stack dictionary, a marker, and speed effects.
  • the clip reference refers to a multi-camera reference clip stored in an event (e.g., reference clip 2000 of FIG. 20 ). In some embodiments, the clip reference contains both an event ID and the collection ID, while in other embodiments the reference just contains the collection ID.
  • the angle instance 3515 is the same type of data structure as angle instance 2012 , shown in FIG. 20 .
  • the angle instance 3515 in a timeline sequence only stores a reference (e.g., a pointer) to the reference collection (e.g., reference collection 2000 ), whereas the angle instance 2012 in an event sequence contains the reference collection.
  • a reference e.g., a pointer
  • the angle instance 2012 in an event sequence contains the reference collection.
  • the total range in some embodiments, is the total length of the multi-camera reference clip to which the instance refers, and is inherited based on the current state of the multi-camera reference clip.
  • the total length of the reference clip in some embodiments, is the duration of the longest angle in the reference clip, including any gap clips.
  • the primary gap clip in the reference clip receives this duration in some embodiments, and thus determines the duration of the reference clip.
  • the media-editing application updates the total range of the instance 3515 .
  • the application updates this information whenever a particular clip instance is accessed, in that the total range of a clip instance is a dynamic variable that the application populates when needed according to the present state of the reference clip.
  • the trimmed range like that of other clips, can be modified by a user within the composite display area by standard trim operations. When a user blades a clip instance into two such instances, this will also result in a change to the trimmed range.
  • some embodiments do not modify the trimmed range of the instances, although the rendered contents of those instance may change as a result of the edits.
  • the active video angle and active audio angle are variables stored by the angle instance. In some embodiments, when a user adds an angle instance to a timeline, these variables are set based on the active angles stored by the angle instance in the event. As will be described in detail in the next subsection, users can swap out the audio or video angles of a clip instance using a variety of operations.
  • the application allows the active video angle to be different from the active audio angle. For example, if a particular clip's video is not suitable and the user instead chooses to use a sequence of still images captured at the same time, the audio of the clip may still be desired for background of the still image sequence.
  • the media-editing application plays back, skims through, renders for export, etc. the composite presentation, the application uses the collection in the reference clip of the active angle to identify the source media to retrieve.
  • the effect stack dictionary contains a set of effect stacks: one audio effects stack and one video effects stack for each angle.
  • the media-editing application identifies the active video angle and uses the effects stack corresponding to this active angle.
  • some embodiments first apply effects contained within the reference clip (which will generally be contained in the effects stack for a particular clip within the reference clip), then subsequently apply the effects stack for the active angle stored in the clip instance.
  • the intermediate output image (after applying the effects stored in the reference clip) may be cached for use in rendering multiple clip instances.
  • the marker is a clip structure that is stored as anchored to an angle instance, and does not change when the user switches the active angles. Users can apply both markers and keywords to a clip in the composite display area, and these affect the clip in the same manner whether the clip is a standard collection clip or a multi-camera clip instance.
  • the speed effect is a retiming effect that has been applied to clip 3515 . These retiming effects are described in detail in U.S. patent application Ser. No. 13/109,989, filed May 17, 2011 and entitled “Retiming Media Presentations”, which is incorporated herein by reference. In some embodiments, retiming effects are not stored with the effects stack and are not stored as relating to a particular angle.
  • the retiming effects remain with the clip instance.
  • Retiming effects modify the length of a video clip (by either slowing down or speeding up the rate at which the video plays).
  • the media-editing application prevents a change to the active video angle from modifying the length of the clip.
  • some embodiments may also use reference clips for clips other than multi-camera media clips. For instance, a user could create a compound clip in an event with specific effects applied, then add multiple instances of the compound clip to a media project and apply different effects to the different instances. If the user then modified the clips within the compound clip reference, these changes would be inherited by the various instances of the compound clip.
  • FIG. 34 illustrates a user adding a multi-camera media clip to a composite display area, and the media-editing application creating a single instance of the multi-camera clip in the media project displayed in the composite display area.
  • the application automatically splits the multi-camera clip into several instances with different active angles.
  • users can mark clips or portions of the clips within the multi-camera reference clip as favorites, and the media-editing application uses this information to generate the clip instances with different angles.
  • FIG. 36 illustrates an example of a user adding multi-camera clip 3605 to a media project over two stages 3610 and 3620 .
  • the first stage 3610 illustrates the user adding a multi-camera clip 3605 to a composite media project and the second stage 3620 illustrates the resultant timeline for the composite media project.
  • the user has selected the multi-camera reference clip 3605 and dragged it into the previously empty composite display area 3615 .
  • the first stage 3610 illustrates the timeline for the multi-camera reference media clip 3605 .
  • the multi-camera clip 3605 includes three angles 3625 - 3635 . These angles each include a series of media clips, some of which have been marked by the user as favorites.
  • users can mark clips or segments of clips as favorites within the event browser.
  • some embodiments allow users to mark clips as favorites within the angle editor or the composite display area. Within the composite display area, users can identify a portion of a media clip that they will likely want to use in a presentation and mark the portion as such.
  • some embodiments allow users to mark portions as rejected, indicating segments of clips that the user wishes to avoid (e.g., because of a shaky camera, poor audio, etc.). Thus, if a single video clip has a good section as well as an unusable section, then the user may want to keep the clip but mark the unusable section so that it does not end up in the media project.
  • the user has marked a portion of media clip 3640 and the entirety of media clip 3645 as favorites (portions marked as favorites are shown with a gray bar in this example).
  • the user has marked the second half of clip 3650 as a favorite, which partially overlaps in time with clip 3645 from the first angle 3625 .
  • the user has marked clip 3655 as a favorite.
  • the second stage 3620 illustrates the composite display area 3615 after the user has added the multi-camera clip 3605 to the media project.
  • the media-editing application has automatically created four separate clip instances.
  • the first clip instance 3660 has the angle 3625 as its active angle, because that angle starts with a favorited clip. Some embodiments identify the first favorited clip portion in the timeline of the multi-camera clip and select that angle for the first clip instance.
  • the application extends the duration of a clip instance past the end of the favorited portion of an angle until identifying a favorited clip portion in a different angle.
  • the clip 3655 in angle 3635 is the next clip in the multi-camera clip marked as a favorite.
  • the favorited portion of the clip (which in this case is the start of the clip) begins at approximately the seven minute mark of the multi-camera reference clip timeline.
  • the multi-camera media clip is the first clip in the media project
  • times in the multi-camera clip timeline correspond to times in the media project timeline.
  • the second clip instance 3665 has the angle 3635 as its active angle, and starts at approximately the seven minute mark of the media project timeline.
  • the favorited portion of this clip ends shortly before the ten minute mark of the multi-camera reference clip timeline, but the next favorited clip segment does not begin until shortly after the ten minute mark.
  • the third clip instance 3670 begins just after the ten minute mark with an active angle of 3630 .
  • the multi-camera clip has both clip 3650 in angle 3630 marked as a favorite as well as clip 3645 in angle 3625 .
  • Some embodiments do not switch angles until the current angle is no longer marked as a favorite, and thus the switch to angle 3625 with a new clip instance 3675 does not come until after the fifteen minute mark, when the clip 3650 ends.
  • choosing to slice the clip instance according to marked favorites is a user option upon adding a multi-camera clip to a media project. That is, the user can either perform a normal add operation that chooses one active angle for the clip instance or choose to perform a different add operation that searches for favorites and divides the clip instance accordingly.
  • users can perform various editing operations with the multi-camera clip instances.
  • users can move the multi-camera clip instances, delete the clip instances, anchor the clip instances to other clips, have other clips anchored to the clip instances, trim the clip instances, split the clip instances in two, etc.
  • users can perform operations specific to multi-camera clips, such as switching the active angles (for both audio and video) and splitting a clip instance in two in order to change a portion of the clip to use a different active angle.
  • FIG. 37 illustrates a user opening the angle editor for a multi-camera clip over two stages 3710 and 3720 of a composite display area 3700 .
  • the composite display area displays a media project entitled “New Project”, which includes a multi-camera clip instance 3705 .
  • the user has selected the multi-camera clip instance 3705 and activated a menu 3715 for the clip (e.g., by right-clicking, two-finger tapping, etc.) with various selectable options related to the selected clip.
  • the figure shows the user selecting the option “Open in Angle Editor”.
  • the second stage 3720 illustrates the result of this selection.
  • the composite display area 3600 now displays the timeline of the reference clip to which the clip instance 3705 refers. As shown in the top left, rather than displaying “New Project”, the display area now displays “Untitled MultiCam Clip” from the event “New Event Oct. 28, 2011” to indicate that any edits the user makes will apply to the reference clip stored in the event rather than the clip instance in the media project.
  • some embodiments highlight the reference clip in the clip browser of the GUI. If the user then makes changes to the reference clip, these changes will also be reflected in the clip instance 3605 , as well as any other clip instances in the current or other media projects.
  • some embodiments display information about the selected clip in an inspector display area (e.g., the display area 325 of FIG. 3 ).
  • the inspector displays information about the active angles in the inspector.
  • different tabs of the inspector may display information about the multi-camera reference clip properties, the audio properties of the active audio angle, and the video properties of the active video angle.
  • FIGS. 38-41 illustrate a selected clip 3805 as well as different tabs of the inspector display area with the clip 3805 selected.
  • FIG. 38 specifically illustrates a composite display area displaying a media project that includes two multi-camera clip instances.
  • the first multi-camera clip instance 3805 which is selected, has an active video angle of “Viega” and an active audio angle of “Nathaniel”.
  • the second clip instance on the other hand, has “Viega” as both its active video and audio angle.
  • FIG. 39 illustrates the inspector display area 3900 as a result of the selection of the clip 3805 .
  • the video tab 3905 is selected in the inspector display area, which also has an audio tab 3910 and an info tab 3915 .
  • the top section of the inspector indicates that the selected clip is a multi-camera clip by use of a multi-camera indicator (similar to the indicator used for multi-camera reference clips in the clip browser or multi-camera clip instances in the composite display area), as well as indicating the active video and audio angles.
  • the inspector displays various video information pertaining to the active video angle of the multi-camera clip instance. Specifically, the inspector displays information about any effects applied to the active angle of the clip instance. In this case, the user has not yet applied any specific effects to the “Viega” video angle. However, if the user had applied a color correction, applied a pixel modification effect, transformed, cropped, or distorted the video, etc., this information would be displayed so long as “Viega” was the active angle. If the user were to switch the active video angle to “Nathaniel”, then information about any effects stored in the effect stack for that angle would be displayed in the inspector 3900 .
  • the video properties in the inspector 3900 of some embodiments do not show, however, any effects applied to the clips of the angle “Viega” within the multi-camera reference clip.
  • the media-editing application identifies the active video angle and accesses the video effect stack for that angle stored in the data structure for the instance. In order to view and modify the effects on clips in the multi-camera reference clip, the user can select the reference clip in the event.
  • FIG. 40 illustrates the inspector display area 3900 with the audio tab 3910 selected.
  • the top section of the inspector is the same as in FIG. 39 (indicating the active video and audio angles), but the section below that includes information about the audio effects rather than video effects.
  • the audio effects shown are those stored in the audio effect stack for the angle “Nathaniel”, the active audio angle for the selected clip instance.
  • the application identifies the active audio angle and accesses the audio effect stack for that angle stored in the instance data structure.
  • FIG. 41 illustrates the inspector display area 3900 with the info tab 3915 selected.
  • the info tab for a multi-camera clip instance displays video and audio properties of the multi-camera reference clip. These include video properties such as the format (720 p), dimensions (1280 ⁇ 720), and frame rate (30 fps), and audio properties such as the audio channel count and sample rate. Some of the other information is user-modifiable within the inspector, such as the clip name, roles, and camera name.
  • modifying this information within the inspector modifies this information for the multi-camera reference clip (in the case of information such as the camera name or clip name) or for the components of either the active angles or all of the clips in the reference clip (e.g., for the roles). In other embodiments, changes to this information only affects the clip instance.
  • FIG. 42 illustrates a user switching the active video angle for a selected clip instance in the inspector over two stages 4210 - 4220 of a GUI 4200 .
  • the GUI 4200 includes, among other display areas, a composite display area 4205 , an inspector display area 4215 , a preview display area 4225 , and an angle viewer 4227 .
  • the user has selected a multi-camera clip instance 4230 in the composite display area 4205 .
  • this clip instance has an active video angle of “Viega” and an active audio angle of “Nathaniel”.
  • the preview display area 4225 displays a video image from a media clip in the angle “Viega”.
  • the user has selected the video angle in order to activate a drop-down menu 4235 .
  • This drop-down menu includes selectable menu options for each of the five angles in the multi-camera clip reference.
  • the menu 4235 displays a check next to the currently selected option (“Viega”).
  • the user has placed a cursor over the option “Amir” in order to select this option for the video angle.
  • the second stage 4220 illustrates the result of the user's selection of the menu option for the “Amir” video angle.
  • the inspector display area 4215 now indicates that the active video angle is “Amir”, as does the clip 4230 in the composite display area 4205 .
  • the preview display area 4225 now displays a video image from a clip in this angle as well.
  • the application has highlighted the preview display 4240 for the “Amir” angle with a border and a video icon that indicates the active video angle.
  • users of the media-editing application may use the angle viewer to perform certain editing operations on a multi-camera clip instance.
  • the following examples will be described in the context of a viewer with four preview displays and which shows the angle name within the displays, but one of ordinary skill in the art will recognize that other options are available in some embodiments (e.g., different numbers of preview displays and with different information displayed about the angles, as shown in FIG. 22 ).
  • FIG. 43 illustrates an angle viewer 4300 of some embodiments, as displayed while a user edits a media project in a composite display area.
  • the angle viewer 4300 includes selectable items 4305 - 4315 , four preview displays 4320 - 4335 , and angle banks 4340 .
  • the selectable items 4305 - 4315 allow a user to switch the type of content for which the active angle of a clip instance will be modified by selection of the various preview displays.
  • the media-editing application either switches the active angle for a current multi-camera clip instance to the angle corresponding to the selected display or splits the current clip instance into two instances with one of the instances having the selected angle.
  • the selectable item 4305 is for switching both the audio and video angles to the selected angle
  • the selectable item 4310 is for switching only the video angle to the selected angle
  • the selectable item 4315 is for switching only the audio angle to the selected angle.
  • the selectable item 4305 is currently selected.
  • each of the items has a different particular appearance (e.g., color, pattern, shade of gray, etc.) when selected (e.g., yellow for both audio and video, blue for video only, and green for audio only).
  • This color coding (or other appearance differentiator) may be used in the angle viewer to identify the display associated with the active angles as well as identify the location of the active angle in the banks 4340 .
  • the four preview displays 4320 - 4335 each display images for one of the angles in the multi-camera media clip.
  • the application displays in the angle viewer the images from the different angles that correspond to the time of the playhead.
  • the preview displays also indicate when a particular angle is the active audio or active video angle.
  • the active audio angle is “Nathaniel”, shown in the top left display 4320 .
  • the display 4320 has a green highlighted border and shows the same audio icon used for the selectable item 4315 .
  • the active video angle is not one of the angles currently shown in any of the four displays.
  • the appearances (e.g., colors, pattern, etc.) of the highlighting for the preview displays match that of the selectable items 4305 - 4315 .
  • the angle banks 4340 illustrate the total number of angles in the multi-camera clip and how those angles are broken into viewable sets.
  • the banks 4340 include a first set of four angles and a second set of one angle. These sets are selectable within the banks 4340 , in that a user can select one of the banks in order to view the preview displays for the particular angles represented by the selected bank. For instance, the user could select the bank with a single square in order to view a preview display for the fifth angle of the multi-camera clip in the angle viewer 4300 .
  • the banks indicate the location in the sets of angles of the active video angle and the active audio angle. In this figure, the top left square in the first bank is shaded green to indicate the active audio angle (“Nathaniel”), while the single square in the second bank is shaded blue to indicate the active video angle.
  • some embodiments use a consistent color-coding between the selectable items 4305 - 4315 , the preview displays 4320 - 4335 , and the banks 4340 to indicate the different types of content for the active angles.
  • the video angle is blue
  • the audio content is green
  • that angle is yellow.
  • any different color or other indicator scheme may be used for the different active angles—e.g., using a different set of colors, using different shades of gray, using different patterns for different types of content, etc.—and that the selection of green, blue, and yellow is merely one specific choice.
  • FIG. 44A-D illustrates various examples of the use of the angle viewer to edit a multi-camera clip in a media project through seven stages 4410 - 4470 of a GUI 4400 .
  • the GUI 4400 includes an angle viewer 4425 , preview display area 4405 , and composite display area 4415 , among other display areas.
  • the angle viewer 4425 is similar to the viewer 4300 , with the angles in a different order and individual clip names shown in preview displays 4426 - 4429 rather than angle names.
  • the angles shown in the viewer at the first stage 4410 are “Nathaniel” in the display 4426 , “Lam” in the display 4427 , “Jim” in the display 4428 , and “Amir” in the display 4429 .
  • the fifth angle “Viega” is not shown in any of the four displays, but is accessible through the banks 4435 .
  • the viewer has three selectable items 4431 - 4433 , with the same functions as items 4305 - 4315 .
  • the composite display area 4415 displays a media project with several clips, including a multi-camera clip instance 4445 , the active angles of which at this stage are “Amir” for the video angle and “Viega” for the audio angle.
  • the user plays back the media project shown in the composite display area 4415 .
  • a playhead 4455 moves across the composite presentation over the course of these stages 4410 - 4470 .
  • the angle viewer 4425 shows, at stage 4410 , that the selectable item 4432 for modifying the video angle is currently selected.
  • the display 4429 is highlighted in blue with a video icon in order to indicate that “Amir” is the active video angle. Because this is the active angle, video from this angle also shows up in the primary preview display area 4405 .
  • the cursor turns into a razor blade icon in order to indicate that selecting the display will cause the application to blade the clip instance at the time currently indicated by the playhead 4455 , while switching the active angle of the newly created clip instance after the playhead.
  • the cursor appears as a blade and selection of the angle display causes the instance to split in two unless the user provides auxiliary input while making the selection, such as holding down a particular key, providing a different type of touch input, etc.
  • the second stage 4420 indicates the result of the user selecting the angle display 4426 in the previous stage.
  • the active video angle is now the “Nathaniel” angle, shown in the display 4426 and the preview display area 4405 .
  • the upper left square in the first bank is highlighted blue to indicate the new active video angle.
  • the media-editing application has split the clip instance 4445 into two instances by modifying the trimmed range of the clip 4445 and creating a new clip instance 4446 with a trimmed range starting at the time of the playhead in the previous stage and continuing to the end of the media clip (the previous ending time of the instance 4445 ).
  • the clip instance 4445 remains with the same active angles while the new clip instance 4446 has an active video angle of “Nathaniel” and an active audio angle of “Viega”.
  • the user selects the item 4431 in this stage.
  • the third stage 4430 illustrates that the selectable item 4431 is now highlighted (in yellow), as a result of the user selection at the previous stage. This item, as indicated above, enables the user to switch both the active audio and video angles to a selected angle. In addition, the user has placed the cursor over the display 4428 for the angle “Jim”, with the cursor again displayed as the blade icon.
  • the fourth stage 4440 illustrates the result of the selection of the angle display 4428 in the previous stage.
  • the active audio and video angle is now the “Jim” angle, shown in the display 4428 and the preview display area 4405 .
  • the banks 4435 no longer show either a blue or a green square, but instead display the bottom left square as yellow.
  • the media-editing application has split the clip instance 4446 into two instances by modifying the trimmed range of the clip 4446 and creating a new clip instance 4447 with a trimmed range starting where the clip instance 4446 's range leaves off.
  • the clip instance 4446 remains with the same active angles while the new clip instance 4447 has active video and audio angles of “Jim”.
  • the user selects item 4433 at this stage.
  • the fifth stage 4450 illustrates that the selectable item 4433 is now highlighted (in green), as a result of the user selection at the previous stage.
  • the user has selected the display 4429 to blade the clip instance 4447 and create a new clip instance 4448 with an active video angle of “Jim” and an active audio angle of “Amir”.
  • the previously yellow highlighted display 4428 is now highlighted blue to indicate the active video angle, while the display 4429 is highlighted green to indicate the active audio angle.
  • each of the banks is a separate selectable item, the selection of which causes the angle viewer to display the angle displays associated with the angles represented in the selected bank.
  • the sixth stage 4460 illustrates the result of the user selecting the bank with a single angle in the previous stage.
  • the angle viewer now displays images from the angle “Viega” in the display 4426 .
  • the user has placed the cursor over this display, and the application displays this cursor as a hand icon rather than a blade icon.
  • the user can hold down a particular key, or provide a different selection input in order to swap angles through the viewer rather than create a new clip instance with the blade tool.
  • the application provides different indicators in the GUI to let the user know whether a selection will split a clip instance or switch angles for the clip instance.
  • the seventh stage 4470 illustrates the result of the user selecting the display 4426 in the previous stage.
  • the media-editing application has modified the clip instance 4448 so that the active audio angle is “Viega” rather than “Amir”. To do this, the application need only modify the data structure for the clip instance to change the active audio angle variable. Additionally, both the display 4426 and the corresponding square in the banks 4435 are now highlighted in green.
  • FIG. 44 illustrates the case in which the angle instances are all located in the primary lane of the timeline.
  • a user might create a composite presentation with a first angle instance anchored to a second angle instance, such that both angle instances are located at certain times within the composite presentation, in different lanes.
  • Some embodiments display video from the angles of the angle instance located in the higher of the two lanes for those times, unless that instance is disabled.
  • the application displays the angles from the instance with the highest lane number in the angle viewer.
  • Some embodiments display a second angle viewer (either automatically or by user request) for such situations, enabling the user to view and edit either of the instances.
  • a compositing mode e.g., a blend
  • the primary preview display area of some embodiments displays the output image that is the result of the blend.
  • FIG. 45 conceptually illustrates a process 4500 of some embodiments for modifying the data structures for a media project when a user blades a clip instance using the angle viewer.
  • the process 4500 begins by receiving (at 4505 ) a selection through the angle viewer to blade and switch a multi-camera clip instance in a composite presentation.
  • FIG. 44 illustrates examples of such a selection; for instance, when the user selects the display 4426 in stage 4410 .
  • the process identifies (at 4510 ) the time of a new edit point in the multi-camera clip instance being trimmed.
  • this edit point is the location of a playhead over a multi-camera clip instance in the composite display area.
  • Some embodiments include both an editing playhead, which a user can lock in at a particular time or set to playback the composite presentation at a real time rate, as well as a skimming playhead that moves with a cursor when the cursor moves over clips in the composite display area.
  • the application uses the position of the editing playhead to determine the time in the multi-camera clip instance to set as the new edit point. While the example of FIG. 44 shows the user editing the clip instances as the playhead moves at a constant rate through the timeline, users may also select a time within the timeline to fix the playhead at that point, then select an angle in the angle viewer.
  • the process then trims (at 4515 ) the out point of the existing multi-camera clip instance in the composite presentation to match the identified edit point.
  • this involves modifying the trimmed range of the angle instance data structure such that the start time of the trimmed range does not change but the end time of the trimmed range matches the new edit point.
  • the process also generates (at 4520 ) a new clip instance in the presentation with an in point immediately subsequent to the new edit point and an out point matching the previous out point of the existing clip instance. That is, the trimmed range of the new clip instance starts one frame after the new out point of the already existing clip instance, and ends at the same time that the existing clip instance ended at prior to receiving the blade and switch input.
  • this new clip instance is created in the data structure for the composite presentation directly after the previously existing clip instance that was broken in two. If the instances are in the primary compositing lane of the composite display area (as in FIG. 44 ), then these clip instances will be defined as subsequent elements in the array of the primary collection for the composite presentation.
  • the process modifies (at 4525 ) the active angles of the new clip instance based on the type of content switched by the selection. That is, the new clip instance is a duplicate of the previously existing clip instance with the exception of the trimmed range and at least one of the active angles, as the new clip existence refers to the same multi-camera reference clip, has the same total range, etc.
  • the media-editing application changes the active audio angle of the new clip instance to the selected angle, with the active video angle staying the same as the previous clip instance.
  • the selection tool is only for modifying the video
  • the application changes the active video angle of the new clip instance to the selected angle, with the active audio angle staying the same as the previous clip instance.
  • the selection tool modifies both audio and video, the application changes both active angles to the selected angle.
  • edits to a multi-camera clip instance created by such a blading process are treated differently than other edits by the media-editing application.
  • these through edits essentially split a continuous multi-camera clip based on angle switches, some embodiments enable a user to quickly remove the edit and merge the two clip instances.
  • the application uses a different default trim edit when editing such a through edit as compared to other edit points.
  • FIG. 46 illustrates the effect of changing the size of an angle viewer 4600 when the viewer is displaying four angle preview displays, over three stages 4610 - 4630 .
  • the first stage 4610 illustrates the angle viewer 4600 , with four display areas, and a preview display area 4605 that borders the angle viewer.
  • the user has placed a cursor over the boundary, or seam, between the two display areas, which causes the GUI to display the cursor as a bar with arrows indicating the directions in which the seam can be moved.
  • the user moves the border to the left, thereby increasing the size of the preview display area 4605 and correspondingly decreasing the size of the angle viewer 4600 .
  • the user has continued to move the seam to the left.
  • This causes the angle viewer 4600 to no longer display the display areas in a 2 ⁇ 2 formation, but instead use a 1 ⁇ 4 display to more efficiently make use of the space available in the angle viewer display area 4600 .
  • the application correspondingly changes the formation of the banks in the angle viewer 4600 . While in the first and second stages 4610 and 4620 the bank 4625 of four angles is shown in a 2 ⁇ 2 square, the third stage 4630 illustrates this bank 4625 as a 1 ⁇ 4 set of rectangles.
  • the media-editing application switches the display of the angle viewer once the individual angle display areas will be larger in the new 1 ⁇ 4 formation than in the 2 ⁇ 2 formation.
  • Some embodiments also perform such formation changes for groups of less or fewer angle displays. For instance, a 2-up viewer (with two angle displays) may be displayed in 1 ⁇ 2 or 2 ⁇ 1 formation, and a 16-up viewer (with sixteen angle displays) may be displayed in 4 ⁇ 4 or 2 ⁇ 8 formation, among other options.
  • a through edit is when a first multi-camera clip instance has an out point that is one frame in the multi-camera clip reference prior to the in point of a second multi-camera clip instance that directly follows the first clip instance in the timeline.
  • FIG. 47 illustrates the use of a menu in a composite display area 4700 to switch the video angle of a selected multi-camera clip instance 4705 over three stages 4710 - 4730 .
  • the clip instance 4705 is selected and the user has activated a drop-down menu 4715 .
  • the angle “Amir” is the active video and audio angle for the clip instance 4705 .
  • the user has selected the option “Active Video Angle”, which causes the display of a sub-menu 4725 showing the five possible video angles for the clip instance. Because the currently selected video angle is “Amir”, this option is currently checked in the sub-menu.
  • the second stage 4720 illustrates the selection of the angle “Nathaniel” in the sub-menu 4725
  • the third stage 4730 illustrates the result of this action.
  • the multi-camera clip instance 4705 now has an active video angle of “Nathaniel” and an active audio angle of “Amir”.
  • the media-editing application of some embodiments modifies the active video angle of the angle instance data structure for the clip 4705 in order to effect this switch. Doing so also causes the application to swap in the video effect stack for the selected angle from the effect stack dictionary for the clip for any rendering operations to be performed with the clip.
  • FIG. 48 illustrates a different manner of switching video angles in a composite display area 4800 of some embodiments, over four stages 4810 - 4840 .
  • the media-editing application displays an angle view display area, or heads-up display (“HUD”), in response to user selection input.
  • HUD heads-up display
  • a multi-camera media clip instance 4805 with active video and audio angles “Amir” is selected.
  • the user has initiated the activation of a menu 4815 for this selected clip. Within the menu, the user selects the “View Angle HUD” option.
  • the second stage 4820 illustrates that as a result of this selection, the application now displays an angle viewing HUD 4825 in the GUI, above the selected clip instance.
  • the angle viewing HUD of some embodiments allows the user to rotate through the various video angles available for a multi-camera clip instance. The user can either playback or skim through the video and audio associated with the different angles.
  • the angle viewing HUD 4825 displays the video of the currently selected angle (“Amir”) in its center.
  • the other options are shown to the sides of the display area, in order for the user to rotate them into the center through interaction with the user interface (e.g., keys such as the right and left arrows, or selection of items within the HUD).
  • the multi-camera reference clip to which the clip instance 4805 refers has two angles, so only one other image is shown, to the left.
  • At the bottom of the display area are indicators that visually indicate to the user which of the angles is currently selected as the active video angle. As shown by the star indicator, the angle on the right (“Amir”) is currently selected.
  • the user can either select these indicators or select the image to the side of the display in order to rotate through the various angles. In this case, the user is shown selecting the other image on the left side of the HUD 4825 .
  • the third stage 4830 shows that after the user selects this image, the HUD now displays video images for the angle “Nathaniel” at its center.
  • the selection indicators still indicate that the “Amir” angle was selected when the user initiated the viewer 4825 , as the right of the two indicators remains a star.
  • the user selects the “Done” GUI item in order to select the “Nathaniel” video angle.
  • the fourth stage 4840 illustrates the result of this selection, as the multi-camera clip instance 4805 now indicates a video angle of “Nathaniel” and an audio angle of “Amir”. While this example shows the use of the angle viewing HUD 4825 to select the video angle, some embodiments allow the user to select the audio angle for a clip instance through the HUD as well. Some embodiments include selectable items similar to those shown in the angle viewer 4300 of FIG. 43 that allow the user to choose between audio, video, or both to determine for which type of content the active angle or angles will be modified by selection of a different angle in the viewing HUD.
  • FIG. 49 illustrates the use of a similar menu to switch the active audio angle of a selected multi-camera clip instance 4905 in a composite display area 4900 over two stages 4910 and 4920 .
  • the clip instance 4905 is selected and the user has activated a drop-down menu 4915 .
  • the active video angle for the clip instance 4905 is “Nathaniel” and the active audio angle is “Amir”.
  • the user has selected the option “Active Audio Angle” to cause the display of a sub-menu 4925 showing the five possible audio angles for this clip instance. Because the currently selected audio angle is “Amir”, this option is currently checked in the sub-menu.
  • the user selects the angle “Viega” in the sub-menu 4925 .
  • the second stage 4920 illustrates the result of this selection, as the multi-camera clip instance 4905 now has an active audio angle of “Viega”.
  • the media-editing application of some embodiments modifies the active audio angle of the angle instance data structure for the clip 4905 in order to effect this switch. Doing so also causes the application to swap in the audio effect stack for the selected angle from the effect stack dictionary for the clip for any rendering operations to be performed with the clip.
  • the media-editing application of some embodiments provides a variety of editing operations for affecting multi-camera clip instances.
  • many standard editing operations available for application to a standard media clip in a media project are also available for multi-camera clip instances: application of video or audio effects to an active angle (pixel modification effects, transforms, distortions, etc.), trim operations (ripple, roll, slip, slide), and compositing operations using multiple lanes (blending, picture in picture, etc.), among other operations.
  • some embodiments provide operations specific to multi-camera clip instances or vary how certain operations are applied to multi-camera clip instances.
  • a through edit occurs in some embodiments when a user blades a multi-camera clip instance and the resulting out point of the first clip is immediately prior to the resulting in point of the second clip (that is, time within the multi-camera reference clip timeline flows smoothly from the first clip to the second clip with no breaks).
  • playing back a media project across the two clip instances involves a continuous playback through the multi-camera clip, but with a switch in the active video or audio angle (or both).
  • the edit point between two clips in a media project is normally displayed in the GUI as a solid line
  • some embodiments display through edits as dashed lines to visually indicate the presence of the through edits to the user.
  • FIG. 50 illustrates the deletion of a through edit in a composite display area 5000 in order to combine two multi-camera clip instances into a single instance over two stages 5010 and 5020 .
  • the composite display area 5000 displays a representation of a media project that includes, among other clips, a first multi-camera clip instance 5005 and a second multi-camera clip instance 5015 . Both of these clips refer to the same multi-camera reference clip, but the first clip instance 5005 has active video and audio angles of “Amir” while the second clip instance 5015 has active angles of “Viega”.
  • the edit point between the two clip instances 5005 and 5015 is a through edit because, in addition to referring to the same reference clip, the clips refer to subsequent content in the multi-camera reference clip.
  • the user has selected this edit point and activated a menu 5025 , which includes the option “Delete Edit”.
  • this menu option is only available when the user selects a through edit, as opposed to a normal edit between two unrelated media clips, or even two clip instances that do not refer to subsequent reference clip material.
  • the second stage 5020 illustrates the result of this selection, as the media-editing application has deleted the second clip instance 5015 , and extended the duration of the clip instance 5005 .
  • the application removes the data structure for the second clip instance from the media project while extending the trimmed range of the first clip instance such that the out point in the reference clip timeline matches that of the now-deleted clip instance. While this example shows the second clip instance (the later clip in the timeline) being deleted, some embodiments automatically delete the earlier instance (and change the in point of the later instance), or allow the user to choose which of the two instances to delete. Furthermore, while this figure shows a menu selection, some embodiments allow the user to delete an edit point through other interactions, such as the delete key or a different set of keystrokes.
  • some embodiments will also automatically delete clip instances in specific editing scenarios. When two clip instances are separated by a through edit, and the user modifies one of the instances to have the same active angles as the other instance, some embodiments automatically recognize this and combine the two clip instances (by deleting one of the instances and modifying the in or out point of the other instance as needed).
  • some embodiments will treat the video or audio of the two clips as a single clip for certain actions when the two clip instances share one of the active angles. For instance, certain audio-related actions will be applied to both clip instances when the active audio angles are the same but with different active video angles.
  • FIG. 51 illustrates a modification to the volume level of two clip instances 5105 and 5115 that are separated by a through edit, over two stages 5110 and 5120 of a composite display area 5100 .
  • the first clip instance 5105 has a video angle of “Amir” and an audio angle of “Nathaniel”, while the clip instance 5115 has a video angle of “Viega” and an audio angle of “Nathaniel”.
  • the user has changed the clip display in the composite display area 5100 to better highlight the audio waveforms rather than the video thumbnails.
  • this audio volume line is a UI tool that enables the user to modify the volume of the audio for a media clip.
  • some embodiments provide a display indicating the volume of the particular clip—in this case, ⁇ 7 decibels. The user can move the UI tool up or down in order to modify the volume of the clip.
  • the second stage 5120 illustrates the clip instances 5105 and 5115 after the user has dragged the audio volume line upwards in order to change the volume of the clips to +9 decibels. As shown, the user drags the volume line within the first clip instance 5105 . However, because the edit point between the two clips is a through edit and the audio angle is the same for the two clips, the application automatically treats the audio as if it were one clip and applies the same effect to both. As such, not only is the volume of the audio for the clip instance 5105 raised to +9 dB, but the volume of the audio for the clip instance 5115 increases as well.
  • other audio effects panning, sound generation effects, etc.
  • other audio effects panning, sound generation effects, etc.
  • neighboring clip instances that have the same audio angle and are divided by through edits as well.
  • several subsequent clips that are all divided by through edits i.e., that collectively refer to a continuous timeline in a multi-camera reference clip
  • share the same audio angle can be treated collectively for audio effect purposes (such as shown in FIG. 51 ).
  • FIG. 52 illustrates a user applying a video effect to a pair of clip instances 5205 and 5215 in a composite display area 5200 over two stages 5210 and 5220 .
  • the first clip instance 5205 has an active video angle of “Viega” and an active audio angle of “Nathaniel”, while the second clip instance 5215 has an active video angle of “Viega” and an active audio angle of “Lam”.
  • a user can drag a video effect from an effects display area (e.g., the display area 330 of FIG. 3 ) over a clip in order to apply the effect to the clip.
  • an effect the Bokeh Random effect
  • the Bokeh Random effect the Bokeh Random effect
  • the clip instance 5205 has the same active video angle as the clip instance 5215 and is only separated from the other instance by a through edit.
  • the application applies the Bokeh Random effect to both of the clip instances 5205 and 5215 .
  • the second stage 5220 illustrates the result of the application of the effect in the inspector display area 5225 .
  • the user has selected the multi-camera clip instance 5205 , and the inspector display area 5225 displays the video information for this selected clip.
  • the video information includes the effect Bokeh Random, dropped over the clip instance 5215 in the previous stage. This indicates that the media-editing application has added this effect to the video effect stack for the “Viega” angle in the clip instance 5205 as well as that of clip instance 5215 .
  • this when the user modifies the effect parameters for one of the clips, this also affects the effect stack of the other neighboring clip instances.
  • users have the option whether to affect multiple instances separated by through edits or to only affect the clip to which the effects are directly applied.
  • the user may toggle a setting, hold down a particular key or set of keys, etc. in order to determine whether to affect one or multiple clips at the same time.
  • the media-editing application also uses different defaults in certain circumstances when applying user actions to multi-camera clip instances as compared to standard media clips. For instance, some embodiments apply different default trim edits to clips when the edit boundary being trimmed is a through edit between two clip instances. Trim edits, in some embodiments, are edits that modify the edit points between two clips. Examples of trim edits applied at the edit boundary include the ripple edit (which changes the duration of only one of the clips at the edit boundary) and the roll edit (which increases the duration of one of the clips while correspondingly decreasing the duration of the other clip).
  • trim edits include the slide edit (which moves a selected clip in the timeline such that the clip on one side of the selected clip decreases in duration while the clip on the other side of the selected clip increases in duration) and the slip edit (which changes the in and out points of a selected media clip without moving the clip in the timeline).
  • FIGS. 53 and 54 illustrate the difference in default trim edits applied at the edit point between two standard media clips and two multi-camera clip references in which the edit point is a through edit.
  • the media-editing application assumes that the user wishes to perform a trim edit at that boundary (e.g., a ripple edit).
  • the user can select a trim tool, which provides the ability to access multiple different trim edits (e.g., ripple, roll, slip, slide, as described above) by moving the cursor over different portions of a clip.
  • trim tool which provides the ability to access multiple different trim edits (e.g., ripple, roll, slip, slide, as described above) by moving the cursor over different portions of a clip.
  • the following figures indicate the difference in the default trim edit when the trim tool is not selected for some embodiments.
  • FIG. 53 illustrates a default trim edit as a ripple edit to standard media clips over three stages 5310 - 5330 of a composite display area 5300
  • FIG. 54 illustrates the default trim edit as a roll edit to multi-camera media clip instances over three stages 5410 - 5430 of a composite display area 5400 .
  • the user has selected an edit point between two clips 5305 and 5315 .
  • These clips are standard media clips with audio and video components (e.g., clip 805 of FIG. 8 ).
  • the cursor at the edit point displays a ripple edit indicator, as the user moves the cursor to the left.
  • the second stage 5320 displays the composite display area 5300 as the user has moved the edit point to the left by approximately two seconds
  • the third stage 5330 illustrates the composite display area 5300 after the user has released the edit point and finished the trim edit.
  • the media-editing application defaults to a ripple edit, this decreases the trimmed range of the clip 5305 .
  • the data structure for the clip 5315 is not affected, although the clip starts earlier in the timeline as a result of the shortened clip 5305 .
  • the user selects an edit point between two multi-camera media clip instances 5405 and 5415 .
  • These clip instances both refer to the same multi-camera reference clip and are separated by a through edit.
  • the media-editing application defaults to using a roll edit rather than a ripple edit.
  • the duration of the clip 5405 decreases while the duration of the clip 5415 increases.
  • the third stage 5430 illustrates the composite display area 5400 after the user has released the edit point and finished the trim edit.
  • the change decreases the trimmed range of the clip 5405 (moving the out point earlier in the reference clip) and increases the trimmed range of the clip 5415 (correspondingly moving the in point earlier in the reference clip). Because the range of the second clip 5415 still starts directly after the end of the range of the first clip 5405 , the edit point between the two clips remains a through edit. Were the user to apply a ripple edit at this boundary, the edit would no longer be a through edit as the clips would no longer be subsequent in the timeline of the reference clip.
  • the application In order to generate output images for a media project, the application renders an image for each frame in the presentation. The frequency of these frames depends on the frame rate specified for the media project (e.g., as shown in FIG. 4 ). To render an image for a particular frame in the media project, the application first identifies the media clip or clips to use for that time. When the time in the media project has clips in multiple lanes, or a compound clip with clips in multiple lanes, multiple source images may be needed.
  • the application To render an image from a standard video clip whose frame rate (typically taken from the source video file referred to by the video clip through an asset) matches the composite presentation, the application identifies a source image from the time in the clip that corresponds to the requested frame, reads and decodes the frame, and applies any effects in the clip's effects stack (e.g., transforms, pixel value modification, etc.).
  • any effects in the clip's effects stack e.g., transforms, pixel value modification, etc.
  • some embodiments use multiple source images from the video clip in order to perform a rate conform effect, as described in U.S. patent application Ser. No. 13/218,407, entitled “Rate Conform Operation for a Media-Editing Application” and filed Aug. 25, 2011, which is incorporated herein by reference.
  • some embodiments apply a spatial conform effect to the clips as described in application Ser. No. 13/218,407. These effects are stored in the effect stack of the media clip in some embodiments, and applied after the image or images are retrieved and decoded.
  • Clip instances do not directly contain component clips or references to assets, but instead refer to reference clips stored in an event (which themselves contain component clips and asset references).
  • the clip instances may store their own effect stacks, and in the case of multi-camera clip instances store different effect stacks for the different angles in the multi-camera reference clip.
  • the reference clips may have their own effects applied.
  • the clips contained within the reference clips may have their own effect stacks.
  • the source images may have a first set of video properties, the reference clip a second set of video properties, and the media project that contains the clip instance a third set of video properties.
  • the media-editing application of some embodiments may need to apply multiple layers of effect stacks, including conform effects, in order to render each output image.
  • FIG. 55 conceptually illustrates a process 5500 of some embodiments for rendering images from a multi-camera clip instance in a media project.
  • the process 5500 will be described by reference to FIGS. 56 and 57 .
  • FIG. 56 conceptually illustrates a timeline of a media project 5600 that includes various media clips, including a multi-camera clip instance 5605 .
  • FIG. 57 conceptually illustrates a render graph 5700 for a particular image of the media project 5600 .
  • the multi-camera clip instance 5605 refers to a multi-camera reference clip 5610 and has an active video angle of “Angle 2”.
  • the multi-camera reference clip 5610 includes, in the video angle “Angle 2”, various media clips including a particular clip 5615 .
  • the process 5500 begins by receiving (at 5505 ) a request to render an image for a particular time in a multi-camera clip instance.
  • the media-editing application of some embodiments requests rendered images for various reasons. When a user skims through a media project or compound clip in the composite display area or initiates playback of a media project, the application will render an image to output to the appropriate display area.
  • some embodiments render images in the background when the processing and memory resources are available, in order to have the images available for future use without having to render at that time.
  • the user might also have requested to render the media project for export (e.g., to a storage, for upload to a website, etc.).
  • FIG. 56 illustrates a playhead 5620 over the media project 5600 , which indicates a particular time within the multi-camera clip instance 5605 .
  • This clip instance 5605 is an instance of the multi-camera reference clip 5610 , which has a 720 p video format with a 30 fps frame rate and both a transform effect and a colorize effect applied.
  • the render graph 5700 indicates the operations applied to a source image in order to arrive at the desired output image for this particular time in the clip instance 5605 .
  • the process 5500 then identifies (at 5510 ) an active video angle for the clip instance.
  • the application stores the active video angle as a variable in the data structure for the clip instance.
  • the active video angle of the clip instance 5605 is “Angle 2”.
  • the process identifies (at 5515 ) the video clip and asset or assets for the particular time in the active angle of the multi-camera reference clip referred to by the clip instance.
  • the particular time requested in the media project corresponds to a particular time in the multi-camera reference clip.
  • the multi-camera reference clip of some embodiments has a set of angles that are each an ordered set of clips. As such, each angle has a clip that corresponds to the particular requested time (though this clip may be a gap clip).
  • This clip has a video component that refers to a media asset. If the clip is a compound clip, then it may refer to multiple assets by way of the different clips contained within the compound clip.
  • the active angle in the multi-camera reference clip 5610 includes clip 5615 at the requested time. As shown, this clip 5615 has a 1080 p video format with a 30 fps frame rate, and has a color correction effect applied.
  • the process 5500 retrieves and decodes (at 5520 ) an image from the media file corresponding to the requested time in the video clip.
  • the media asset identified at 5515 includes a reference to a media file that stores actual video images. These images are often stored in encoded form, and thus the media-editing application uses a decoder to decode the images and arrive at a set of pixels that represents the image.
  • the render graph 5700 begins with a decoded 1920 ⁇ 1080 bitmap image from the media asset referred to by the clip 5615 .
  • the process applies (at 5525 ) any video effects on the clip in the multi-camera reference clip to the decoded image.
  • the effects on the reference clip (or clips within the reference clip) are applied before any effects on the clip instance.
  • some embodiments apply the reference clip effects before conforming the retrieved images to the reference clip properties.
  • the clip 5615 has a color correction effect, and thus the application applies this color correction effect.
  • the node 5705 for the retrieved bitmap leads into a color correction node 5710 for the application of this effect on the clip 5615 .
  • This color correction node includes the parameters of the effect defined by a user (e.g., a mapping of input pixel values to output pixel values).
  • the process applies (at 5530 ) any spatial conform effect necessary to conform the image to the spatial properties of the multi-camera clip. If the multi-camera reference clip has the same spatial video properties as the retrieved image, then no spatial conform effect is necessary.
  • the media-editing application applies a spatial conform effect in order to generate an image of the requested size (in this case, that of the reference clip).
  • the application can either fit the entirety of the retrieved image into the output image (with either a pillarbox or letterbox), or fill the entirety of the output image (cutting off part of the retrieved image).
  • some embodiments do not temporally conform the contained video clip to the frame rate of the reference clip.
  • some embodiments use time-preserving rate conform effects such as frame sampling or blending, but avoid using rate conform effects that speed up or slow down the clip.
  • the clip 5615 has a 1080 p video format (1920 ⁇ 1080 images with square pixels), while the reference clip 5610 that contains the clip 5615 has a 720 p video format (1280 ⁇ 720 images with square pixels), so the application applies a spatial conform effect to scale the 1920 ⁇ 1080 images to 1280 ⁇ 720.
  • the next operation in the render graph 5700 is a transform node that scales the input image by a factor of 2 ⁇ 3 (0.67).
  • the output of this node 5715 will be a 1280 ⁇ 720 bitmap.
  • the process 5500 next applies (at 5535 ) to the image any video effects on the active angle in the multi-camera instance clip.
  • some embodiments store a video effects stack in the clip instance for each angle of the multi-camera reference clip.
  • the process identifies the effects stack for the active angle and applies any effects in this stack.
  • the active angle (“Angle 2”) of clip 5605 has a colorize effect and a transform effect applied.
  • the render graph 5700 thus has a node 5720 for the colorize effect and a node 5725 for the transform operation. Some embodiments switch the order of these two operations, and combine the transform 5715 with the transform operation 5725 .
  • transform effect 5725 enlarges the image, then combining these operations avoids a loss of resolution caused by downscaling the image and then subsequently upscaling the image.
  • some embodiments separate effects applied within the reference clip from effects applied to the active angle.
  • the process 5500 applies (at 5540 ) spatial and temporal effects to the image to conform the image to the spatial properties and frame rate of the media project that contains the multi-camera clip instance, then ends.
  • the spatial properties of the multi-camera reference clip are the same as those of the media project, then no spatial conform effect is needed. Otherwise, the spatial conform effect is applied as described above for operation 5530 .
  • the application applies a rate conform effect.
  • the rate conform effect actually requires multiple frames from the source video file, as the output image is a blend of two of the source images. Other rate conform effects only require one image. As indicated above, a detailed description of different rate conform techniques can be found in U.S. application Ser. No.
  • the reference clip properties (720 p, 30 fps) match those of the media project, and thus no conform effect is necessary.
  • the output of operation 5725 is the requested image for the media project.
  • the media-editing application does not retrieve an image, then determine which effects to apply from the reference clip, then apply these effects, then determine which effects to apply from the clip instance, and then apply these clip instance effects. Instead, the application builds a render graph such as graph 5700 , then begins retrieving and decoding images and applying the render graph operations to the decoded images. This allows the application to combine effects when possible, and avoid resampling pixels repeatedly. For instance, if a transform effect and a conform effect are both applied to an image, some embodiments combine these operations.
  • a more detailed discussion of the rendering operations of some embodiments can be found in U.S. application Ser. No. 13/111,895, incorporated by reference above.
  • FIGS. 55-57 described rendering for multi-camera clip instances that refer to multi-camera clips.
  • One of ordinary skill in the art will recognize that similar principles are applicable to other clips implemented as instances of reference clips as well, whether the reference clips are individual clips, compound clips, etc.
  • Some embodiments render such clips by first applying effects on the reference clip or on clips contained within the reference clip, then subsequently applying effects on the clip instance.
  • FIG. 58 conceptually illustrates the software architecture of a media-editing application 5800 of some embodiments.
  • the media-editing application is a stand-alone application or is integrated into another application, while in other embodiments the application might be implemented within an operating system.
  • the application is provided as part of a server-based solution.
  • the application is provided via a thin client. That is, the application runs on a server while a user interacts with the application via a separate machine remote from the server.
  • the application is provided via a thick client. That is, the application is distributed from the server to the client machine and runs on the client machine.
  • the media editing application 5800 includes a user interface (UI) interaction and generation module 5805 , a media ingest module 5810 , editing modules 5815 , a timeline generator 5820 , an angle viewer generator 5827 , a rendering engine 5825 , a playback module 5830 .
  • the figure also illustrates stored data associated with the media-editing application: source files 5835 , events data 5840 , project data 5845 , and render files 5850 .
  • the source files 5835 store media files (e.g., video files, audio files, combined video and audio files, etc.) imported into the application.
  • the events data 5840 stores the events information used by some embodiments to populate the clip library and clip browser, which includes multi-camera reference clips.
  • the events data may be a set of clip object data structures stored as one or more SQLite database files (or other format) in some embodiments.
  • the project data 5845 stores the project information used by some embodiments to specify a composite presentation in the timeline.
  • the project data may also be a set of clip object data structures stored as one or more SQLite database files (or other format) in some embodiments.
  • These clip object data structures include multi-camera clip instances that refer to the multi-camera reference clips stored in the event data 5840 , as well as other clip objects that refer to asset data structures stored in the event data 5840 .
  • the render files 5850 of some embodiments may include thumbnail-sized images for display in the clip browser or timeline, audio waveform displays for media clips, as well as rendered segments of a timeline sequence for use during playback.
  • the four sets of data 5835 - 5850 are stored in one physical storage (e.g., an internal hard drive, external hard drive, etc.).
  • the data may be split between multiple physical storages.
  • the source files might be stored on an external hard drive with the events data, project data, and render files on an internal drive.
  • Some embodiments store events data with their associated source files and render files in one set of folders, and the project data with associated render files in a separate set of folders.
  • FIG. 58 also illustrates an operating system 5855 that includes input device driver(s) 5860 , display module 5865 , and media import module 5870 .
  • the device drivers 5860 , display module 5865 , and media import module 5870 are part of the operating system even when the media editing application 5800 is an application separate from the operating system.
  • the input device drivers 5860 may include drivers for translating signals from a keyboard, mouse, touchpad, tablet, touchscreen, etc. A user interacts with one or more of these input devices, which send signals to their corresponding device driver. The device driver then translates the signals into user input data that is provided to the UI interaction and generation module 5805 .
  • the present application describes a graphical user interface that provides users with numerous ways to perform different sets of operations and functionalities. In some embodiments, these operations and functionalities are performed based on different commands that are received from users through different input devices (e.g., keyboard, trackpad, touchpad, mouse, etc.). For example, the present application illustrates the use of a cursor in the graphical user interface to control (e.g., select, move) objects in the graphical user interface. However, in some embodiments, objects in the graphical user interface can also be controlled or manipulated through other controls, such as touch control. In some embodiments, touch control is implemented through an input device that can detect the presence and location of touch on a display of the device. An example of such a device is a touch screen device.
  • a user can directly manipulate objects by interacting with the graphical user interface that is displayed on the display of the touch screen device. For instance, a user can select a particular object in the graphical user interface by simply touching that particular object on the display of the touch screen device.
  • touch control can be used to control the cursor in some embodiments.
  • the display module 5865 translates the output of a user interface for a display device. That is, the display module 5865 receives signals (e.g., from the UI interaction and generation module 5805 ) describing what should be displayed and translates these signals into pixel information that is sent to the display device.
  • the display device may be an LCD, plasma screen, CRT monitor, touchscreen, etc.
  • the media import module 5870 receives media files (e.g., audio files, video files, etc.) from storage devices (e.g., external drives, recording devices, etc.) through one or more ports (e.g., a USB port, Firewire port, etc.) of the device on which the media-editing application 5800 operates and translates this media data for the media-editing application or stores the data directly onto a storage of the device.
  • media files e.g., audio files, video files, etc.
  • storage devices e.g., external drives, recording devices, etc.
  • ports e.g., a USB port, Firewire port, etc.
  • the UI interaction and generation module 5805 of the media editing application 5800 interprets the user input data received from the input device drivers and passes it to various modules, including the media ingest module 5810 , the editing modules 5815 , the timeline generator 5820 , the angle viewer generator 5827 , the rendering engine 5825 , and the playback module 5830 .
  • the UI interaction module also manages the display of the media-editing application GUI, and outputs this display information to the display module 5865 . This UI display information may be based on information from the editing modules 5815 , the timeline generator 5820 , the angle viewer generator 5827 , the playback module 5830 , the data 5835 - 5850 , etc.
  • the module 5805 may generate portions of the UI based solely on user input—e.g., when a user moves an item in the UI that only affects the display rather than any of the other modules, such as moving a window from one side of the UI to the other or modifying the relative size of the clip browser and clip library.
  • the UI interaction and generation module 5805 generates a basic GUI and populates the GUI with information from the other modules and stored data.
  • the media ingest module 5810 manages the import of source media into the media-editing application 5800 . Some embodiments, as shown, receive source media from the media import module 5870 of the operating system 5855 . The media ingest module 5810 receives instructions through the UI module 5805 as to which files should be imported, then instructs the media import module 5870 to enable this import (e.g., from an external drive, from a camera, etc.). The media ingest module 5810 of some embodiments stores these source files 5835 in specific file folders associated with the application. In some embodiments, the media ingest module 5810 also manages the creation of event data structures upon import of source files and the creation of the clip and asset data structures contained in the events.
  • the editing modules 5815 include a variety of modules for editing media in the clip browser as well as in the timeline.
  • the editing modules 5815 handle the creation of projects, addition and subtraction of clips from projects, trimming or other editing processes within the timeline, application of effects and transitions, or other editing processes.
  • the editing modules 5815 create and modify project and clip data structures in both the event data 1980 and the project data 5845 .
  • the editing modules 5815 of some embodiments include modules for editing clips, media projects, etc.
  • the editing modules create media projects, trim clips within projects, apply effects to the clips, and other operations.
  • the editing modules modify the events data 5840 and project data 5845 .
  • the editing modules 5815 include a multi-camera creation module 5817 that performs automatic multi-camera clip creation (e.g., assigning clips to angles, ordering and sequencing the angles, and aligning the angles).
  • the editing modules 5815 can be used to manually edit multi-camera reference clips in the angle editor.
  • the timeline generator 5820 generates a timeline for display in the composite display area, based on a requested media project, compound clip, multi-camera reference clip, etc. For example, when the user requests to view a multi-camera reference clip in the angle editor, the timeline generator 5820 retrieves the reference clip information from the events data 5840 and generates the displays of the tracks for the different angles of the clip. The timeline generator 5820 then passes the generated timeline to the UI interaction and generation module 5805 for incorporation into the UI.
  • the angle viewer generator 5823 generates the angle viewer for display in GUI, based on a multi-camera reference clip.
  • the angle viewer generator requests the data for a reference clip being edited in the angle editor or to which an instance clip through which a user is skimming refers. Based on the reference clip the angle viewer generator 5823 generates the banks and preview displays, then requests the appropriate output images for the preview displays from the playback module 5830 .
  • the rendering engine 5825 handles the rendering of images for the media-editing application.
  • the rendering engine 5820 of some embodiments includes a render graph generator 5821 , a scheduler 5822 , and image processing operations 5823 .
  • the rendering engine manages the creation of images for the media-editing application.
  • the rendering engine outputs the requested image according to the project or event data.
  • the rendering engine retrieves the project data or event data that identifies how to create the requested image and the render graph generator 5821 generates a render graph that is a series of nodes indicating either images to retrieve from the source files 5835 or operations to perform on the retrieved images.
  • the scheduler 5822 schedules the retrieval of the necessary images through disk read operations and the decoding of those images.
  • the image processing operations 5823 are the various operations performed on the images to generate an output image. In some embodiments, these operations include blend operations, effects (e.g., blur or other pixel value modification operations, transforms, conform effects, etc.), color space conversions, etc.
  • the image processing operations 5823 in some embodiments are actually part of the operating system and are performed by a GPU or CPU of the device on which the application 5800 operates.
  • the output of the rendering engine (a rendered image) may be stored in the render files 5850 or sent to a destination for additional processing or output.
  • the playback module 5830 handles the playback of images (e.g., in a preview display area of the user interface, or one of the preview displays of the angle viewer). Some embodiments do not include a playback module and the rendering engine directly outputs its images to the UI interaction and generation module 5805 for integration into the GUI, or directly to the display module 5865 for display at a particular portion of the display device.
  • media-editing application 5800 While many of the features of media-editing application 5800 have been described as being performed by one module (e.g., the UI interaction and generation module 5805 , the media ingest module 5810 , etc.), one of ordinary skill in the art will recognize that the functions described herein might be split up into multiple modules. Similarly, functions described as being performed by multiple different modules might be performed by a single module in some embodiments (e.g., the playback module 5830 , angle viewer generator 5827 , and timeline generator 5820 might all be part of the UI interaction and generation module 5805 , and a single module might be used to perform the transforms as well as spatial conforms, even when the effects are stored separately).
  • Computer readable storage medium also referred to as computer readable medium.
  • these instructions are executed by one or more computational or processing unit(s) (e.g., one or more processors, cores of processors, or other processing units), they cause the processing unit(s) to perform the actions indicated in the instructions.
  • computational or processing unit(s) e.g., one or more processors, cores of processors, or other processing units
  • Examples of computer readable media include, but are not limited to, CD-ROMs, flash drives, random access memory (RAM) chips, hard drives, erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), etc.
  • the computer readable media does not include carrier waves and electronic signals passing wirelessly or over wired connections.
  • the term “software” is meant to include firmware residing in read-only memory or applications stored in magnetic storage which can be read into memory for processing by a processor.
  • multiple software inventions can be implemented as sub-parts of a larger program while remaining distinct software inventions.
  • multiple software inventions can also be implemented as separate programs.
  • any combination of separate programs that together implement a software invention described here is within the scope of the invention.
  • the software programs when installed to operate on one or more electronic systems, define one or more specific machine implementations that execute and perform the operations of the software programs.
  • FIG. 59 conceptually illustrates an electronic system 5900 with which some embodiments of the invention are implemented.
  • the electronic system 5900 may be a computer (e.g., a desktop computer, personal computer, tablet computer, etc.), phone, PDA, or any other sort of electronic or computing device.
  • Such an electronic system includes various types of computer readable media and interfaces for various other types of computer readable media.
  • Electronic system 5900 includes a bus 5905 , processing unit(s) 5910 , a graphics processing unit (GPU) 5915 , a system memory 5920 , a network 5925 , a read-only memory 5930 , a permanent storage device 5935 , input devices 5940 , and output devices 5945 .
  • GPU graphics processing unit
  • the bus 5905 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of the electronic system 5900 .
  • the bus 5905 communicatively connects the processing unit(s) 5910 with the read-only memory 5930 , the GPU 5915 , the system memory 5920 , and the permanent storage device 5935 .
  • the processing unit(s) 5910 retrieves instructions to execute and data to process in order to execute the processes of the invention.
  • the processing unit(s) may be a single processor or a multi-core processor in different embodiments. Some instructions are passed to and executed by the GPU 5915 .
  • the GPU 5915 can offload various computations or complement the image processing provided by the processing unit(s) 5910 . In some embodiments, such functionality can be provided using CoreImage's kernel shading language.
  • the read-only-memory (ROM) 5930 stores static data and instructions that are needed by the processing unit(s) 5910 and other modules of the electronic system.
  • the permanent storage device 5935 is a read-and-write memory device. This device is a non-volatile memory unit that stores instructions and data even when the electronic system 5900 is off. Some embodiments of the invention use a mass-storage device (such as a magnetic or optical disk and its corresponding disk drive) as the permanent storage device 5935 .
  • the system memory 5920 is a read-and-write memory device. However, unlike storage device 5935 , the system memory 5920 is a volatile read-and-write memory, such a random access memory.
  • the system memory 5920 stores some of the instructions and data that the processor needs at runtime.
  • the invention's processes are stored in the system memory 5920 , the permanent storage device 5935 , and/or the read-only memory 5930 .
  • the various memory units include instructions for processing multimedia clips in accordance with some embodiments. From these various memory units, the processing unit(s) 5910 retrieves instructions to execute and data to process in order to execute the processes of some embodiments.
  • the bus 5905 also connects to the input and output devices 5940 and 5945 .
  • the input devices 5940 enable the user to communicate information and select commands to the electronic system.
  • the input devices 5940 include alphanumeric keyboards and pointing devices (also called “cursor control devices”), cameras (e.g., webcams), microphones or similar devices for receiving voice commands, etc.
  • the output devices 5945 display images generated by the electronic system or otherwise output data.
  • the output devices 5945 include printers and display devices, such as cathode ray tubes (CRT) or liquid crystal displays (LCD), as well as speakers or similar audio output devices. Some embodiments include devices such as a touchscreen that function as both input and output devices.
  • CTR cathode ray tubes
  • LCD liquid crystal displays
  • bus 5905 also couples electronic system 5900 to a network 5925 through a network adapter (not shown).
  • the computer can be a part of a network of computers (such as a local area network (“LAN”), a wide area network (“WAN”), or an Intranet, or a network of networks, such as the Internet. Any or all components of electronic system 5900 may be used in conjunction with the invention.
  • Some embodiments include electronic components, such as microprocessors, storage and memory that store computer program instructions in a machine-readable or computer-readable medium (alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media).
  • computer-readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), a variety of recordable/rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.), magnetic and/or solid state hard drives, read-only and recordable Blu-Ray® discs, ultra density optical discs, any other optical or magnetic media, and floppy disks.
  • CD-ROM compact discs
  • CD-R recordable compact discs
  • the computer-readable media may store a computer program that is executable by at least one processing unit and includes sets of instructions for performing various operations.
  • Examples of computer programs or computer code include machine code, such as is produced by a compiler, and files including higher-level code that are executed by a computer, an electronic component, or a microprocessor using an interpreter.
  • ASICs application specific integrated circuits
  • FPGAs field programmable gate arrays
  • PLDs programmable logic devices
  • ROM read only memory
  • RAM random access memory
  • the terms “computer”, “server”, “processor”, and “memory” all refer to electronic or other technological devices. These terms exclude people or groups of people.
  • display or displaying means displaying on an electronic device.
  • the terms “computer readable medium,” “computer readable media,” and “machine readable medium” are entirely restricted to tangible, physical objects that store information in a form that is readable by a computer. These terms exclude any wireless signals, wired download signals, and any other ephemeral signals.
  • FIGS. 10 , 12 , 14 , 16 , 18 , 26 , 45 , and 55 conceptually illustrate processes.
  • the specific operations of these processes may not be performed in the exact order shown and described.
  • the specific operations may not be performed in one continuous series of operations, and different specific operations may be performed in different embodiments.
  • the process could be implemented using several sub-processes, or as part of a larger macro process.
  • the invention is not to be limited by the foregoing illustrative details, but rather is to be defined by the appended claims.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Computer Security & Cryptography (AREA)
  • Television Signal Processing For Recording (AREA)
  • Management Or Editing Of Information On Record Carriers (AREA)
  • Signal Processing For Digital Recording And Reproducing (AREA)
  • Indexing, Searching, Synchronizing, And The Amount Of Synchronization Travel Of Record Carriers (AREA)
  • Studio Devices (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
US13/361,932 2011-11-14 2012-01-30 Reference clips in a media-editing application Abandoned US20130124999A1 (en)

Priority Applications (7)

Application Number Priority Date Filing Date Title
US13/361,932 US20130124999A1 (en) 2011-11-14 2012-01-30 Reference clips in a media-editing application
PCT/US2012/058138 WO2013074207A1 (en) 2011-11-14 2012-09-28 Generation of multi -views media clips
EP12788327.0A EP2781102A1 (en) 2011-11-14 2012-09-28 Generation of multi -views media clips
JP2014541061A JP5856690B2 (ja) 2011-11-14 2012-09-28 マルチメディアクリップの生成
CN201280055762.0A CN103931199B (zh) 2011-11-14 2012-09-28 多媒体片段的生成
EP15199415.9A EP3012838B1 (en) 2011-11-14 2012-09-28 Preview of multi-views media clips
JP2015242604A JP6214619B2 (ja) 2011-11-14 2015-12-11 マルチメディアクリップの生成

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201161559710P 2011-11-14 2011-11-14
US13/361,932 US20130124999A1 (en) 2011-11-14 2012-01-30 Reference clips in a media-editing application

Publications (1)

Publication Number Publication Date
US20130124999A1 true US20130124999A1 (en) 2013-05-16

Family

ID=48280753

Family Applications (4)

Application Number Title Priority Date Filing Date
US13/361,931 Active 2034-02-03 US9437247B2 (en) 2011-11-14 2012-01-30 Preview display for multi-camera media clips
US13/361,930 Active 2033-05-27 US9111579B2 (en) 2011-11-14 2012-01-30 Media editing with multi-camera media clips
US13/361,934 Active 2032-07-17 US9792955B2 (en) 2011-11-14 2012-01-30 Automatic generation of multi-camera media clips
US13/361,932 Abandoned US20130124999A1 (en) 2011-11-14 2012-01-30 Reference clips in a media-editing application

Family Applications Before (3)

Application Number Title Priority Date Filing Date
US13/361,931 Active 2034-02-03 US9437247B2 (en) 2011-11-14 2012-01-30 Preview display for multi-camera media clips
US13/361,930 Active 2033-05-27 US9111579B2 (en) 2011-11-14 2012-01-30 Media editing with multi-camera media clips
US13/361,934 Active 2032-07-17 US9792955B2 (en) 2011-11-14 2012-01-30 Automatic generation of multi-camera media clips

Country Status (5)

Country Link
US (4) US9437247B2 (ja)
EP (2) EP3012838B1 (ja)
JP (2) JP5856690B2 (ja)
CN (1) CN103931199B (ja)
WO (1) WO2013074207A1 (ja)

Cited By (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110289413A1 (en) * 2006-12-22 2011-11-24 Apple Inc. Fast Creation of Video Segments
US8994828B2 (en) * 2013-02-28 2015-03-31 Apple Inc. Aligned video comparison tool
US9111579B2 (en) 2011-11-14 2015-08-18 Apple Inc. Media editing with multi-camera media clips
US9280262B2 (en) 2006-12-22 2016-03-08 Apple Inc. Select drag and drop operations on video thumbnails across clip boundaries
US9519420B2 (en) 2013-10-16 2016-12-13 Samsung Electronics Co., Ltd. Apparatus and method for editing synchronous media
US9578256B1 (en) * 2012-02-29 2017-02-21 Google Inc. Temporary intermediate video clips for video editing
US9830063B2 (en) 2006-12-22 2017-11-28 Apple Inc. Modified media presentation during scrubbing
US10057651B1 (en) * 2015-10-05 2018-08-21 Twitter, Inc. Video clip creation using social media
US10162506B2 (en) 2014-07-18 2018-12-25 Apple Inc. Systems and methods for selecting portions of media for a preview
US20190141388A1 (en) * 2017-10-10 2019-05-09 Shazam Investments Ltd. Systems and Methods for Performing Playout of Multiple Media Recordings Based on a Matching Segment Among the Recordings
US10504559B2 (en) 2016-03-09 2019-12-10 Yamaha Corporation Recorded data processing method and recorded data processing device
US10692536B1 (en) 2005-04-16 2020-06-23 Apple Inc. Generation and use of multiclips in video editing
US10728443B1 (en) 2019-03-27 2020-07-28 On Time Staffing Inc. Automatic camera angle switching to create combined audiovisual file
US10963841B2 (en) 2019-03-27 2021-03-30 On Time Staffing Inc. Employment candidate empathy scoring system
US11023735B1 (en) 2020-04-02 2021-06-01 On Time Staffing, Inc. Automatic versioning of video presentations
US11127232B2 (en) 2019-11-26 2021-09-21 On Time Staffing Inc. Multi-camera, multi-sensor panel data extraction system and method
US11144882B1 (en) 2020-09-18 2021-10-12 On Time Staffing Inc. Systems and methods for evaluating actions over a computer network and establishing live network connections
US11423071B1 (en) 2021-08-31 2022-08-23 On Time Staffing, Inc. Candidate data ranking method using previously selected candidate data
US20230125371A1 (en) * 2021-10-27 2023-04-27 Genetec Inc. Systems and methods for concurrently playing multiple remotely stored media files in a web browser
US11727040B2 (en) 2021-08-06 2023-08-15 On Time Staffing, Inc. Monitoring third-party forum contributions to improve searching through time-to-live data assignments
US11893668B2 (en) 2021-03-31 2024-02-06 Leica Camera Ag Imaging system and method for generating a final digital image via applying a profile to image information
US11907652B2 (en) 2022-06-02 2024-02-20 On Time Staffing, Inc. User interface and systems for document creation

Families Citing this family (222)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8520069B2 (en) 2005-09-16 2013-08-27 Digital Ally, Inc. Vehicle-mounted video system with distributed processing
US8503972B2 (en) 2008-10-30 2013-08-06 Digital Ally, Inc. Multi-functional remote monitoring system
US10078695B2 (en) * 2011-04-11 2018-09-18 Evertz Microsystems Ltd. Methods and systems for network based video clip generation and management
US9557885B2 (en) 2011-08-09 2017-01-31 Gopro, Inc. Digital media editing
US9232175B2 (en) * 2012-03-01 2016-01-05 Sony Corporation Generating multiple shots in storyboarding
US20130271472A1 (en) * 2012-04-12 2013-10-17 Motorola Mobility, Inc. Display of Value Changes in Between Keyframes in an Animation Using a Timeline
US20130283330A1 (en) * 2012-04-18 2013-10-24 Harris Corporation Architecture and system for group video distribution
KR101905638B1 (ko) * 2012-05-15 2018-12-05 삼성전자주식회사 동영상 재생 장치 및 방법
US9071628B2 (en) * 2012-05-18 2015-06-30 Andrew Milburn Method and apparatus for managing bandwidth by managing selected internet access by devices in a Wi-Fi linked audience
US9207834B2 (en) 2012-06-11 2015-12-08 Edupresent Llc Layered multimedia interactive assessment system
US8682144B1 (en) 2012-09-17 2014-03-25 Google Inc. Method for synchronizing multiple audio signals
US9019431B2 (en) 2012-09-28 2015-04-28 Digital Ally, Inc. Portable video and imaging system
US10272848B2 (en) 2012-09-28 2019-04-30 Digital Ally, Inc. Mobile video and imaging system
US20140101551A1 (en) * 2012-10-05 2014-04-10 Google Inc. Stitching videos into an aggregate video
US9871842B2 (en) 2012-12-08 2018-01-16 Evertz Microsystems Ltd. Methods and systems for network based video clip processing and management
US9129640B2 (en) * 2012-12-12 2015-09-08 Crowdflik, Inc. Collaborative digital video platform that enables synchronized capture, curation and editing of multiple user-generated videos
US20160155475A1 (en) * 2012-12-12 2016-06-02 Crowdflik, Inc. Method And System For Capturing Video From A Plurality Of Devices And Organizing Them For Editing, Viewing, And Dissemination Based On One Or More Criteria
US10521188B1 (en) 2012-12-31 2019-12-31 Apple Inc. Multi-user TV user interface
US20140199050A1 (en) * 2013-01-17 2014-07-17 Spherical, Inc. Systems and methods for compiling and storing video with static panoramic background
US9064480B2 (en) * 2013-01-25 2015-06-23 Inmusic Brands, Inc Methods and systems for an object-oriented arrangement of musical ideas
US9767845B2 (en) 2013-02-05 2017-09-19 Alc Holdings, Inc. Activating a video based on location in screen
US20140226955A1 (en) * 2013-02-12 2014-08-14 Takes Llc Generating a sequence of video clips based on meta data
EP2965182B1 (en) 2013-03-05 2019-06-19 Fasetto, Inc. System and method for cubic graphical user interfaces
FR3004565B1 (fr) * 2013-04-12 2016-11-11 Kolor Fusion de plusieurs flux video
US9646650B2 (en) * 2013-05-28 2017-05-09 Google Inc. Automatically syncing recordings between two or more content recording devices
JP2015038719A (ja) * 2013-07-16 2015-02-26 東芝テック株式会社 情報処理装置及びプログラム
WO2015009944A1 (en) * 2013-07-18 2015-01-22 Christmas Coy System and method for multi-angle videos
US9253452B2 (en) 2013-08-14 2016-02-02 Digital Ally, Inc. Computer program, method, and system for managing multiple data recording devices
US10390732B2 (en) 2013-08-14 2019-08-27 Digital Ally, Inc. Breath analyzer, system, and computer program for authenticating, preserving, and presenting breath analysis data
US10075681B2 (en) 2013-08-14 2018-09-11 Digital Ally, Inc. Dual lens camera unit
US9881645B2 (en) * 2013-08-20 2018-01-30 Google Llc Systems, methods, and media for editing video during playback via gestures
US9804760B2 (en) * 2013-08-22 2017-10-31 Apple Inc. Scrollable in-line camera for capturing and sharing content
KR101518453B1 (ko) 2013-08-29 2015-05-11 주식회사 픽스트리 컨텐츠 재생 장치 및 방법
KR20150027934A (ko) * 2013-09-04 2015-03-13 삼성전자주식회사 다각도에서 촬영된 영상을 수신하여 파일을 생성하는 전자 장치 및 방법
JP6187811B2 (ja) * 2013-09-09 2017-08-30 ソニー株式会社 画像処理装置、画像処理方法、及び、プログラム
US10095873B2 (en) 2013-09-30 2018-10-09 Fasetto, Inc. Paperless application
WO2015044947A1 (en) * 2013-09-30 2015-04-02 Yanai Danielle Image and video processing and optimization
WO2015054342A1 (en) * 2013-10-09 2015-04-16 Mindset Systems Method of and system for automatic compilation of crowdsourced digital media productions
WO2015060470A1 (ko) * 2013-10-23 2015-04-30 엘지전자 주식회사 단말기 및 그 제어방법
US9087405B2 (en) 2013-12-16 2015-07-21 Google Inc. Depth map generation using bokeh detection
US9584402B2 (en) 2014-01-27 2017-02-28 Fasetto, Llc Systems and methods for peer to peer communication
US11831692B2 (en) * 2014-02-06 2023-11-28 Bongo Learn, Inc. Asynchronous video communication integration system
US10191647B2 (en) 2014-02-06 2019-01-29 Edupresent Llc Collaborative group video production system
WO2015134537A1 (en) 2014-03-04 2015-09-11 Gopro, Inc. Generation of video based on spherical content
US9230602B2 (en) * 2014-03-17 2016-01-05 Clipcast Technologies, LLC Media clip creation and distribution systems, apparatus, and methods
US10108254B1 (en) * 2014-03-21 2018-10-23 Google Llc Apparatus and method for temporal synchronization of multiple signals
KR20150142348A (ko) * 2014-06-11 2015-12-22 삼성전자주식회사 사용자 단말 및 이의 제어 방법
US9832538B2 (en) * 2014-06-16 2017-11-28 Cisco Technology, Inc. Synchronizing broadcast timeline metadata
US20150363157A1 (en) * 2014-06-17 2015-12-17 Htc Corporation Electrical device and associated operating method for displaying user interface related to a sound track
CN106415475A (zh) 2014-06-24 2017-02-15 苹果公司 用于在用户界面中导航的列界面
CN111078110B (zh) 2014-06-24 2023-10-24 苹果公司 输入设备和用户界面交互
US9600723B1 (en) 2014-07-03 2017-03-21 Google Inc. Systems and methods for attention localization using a first-person point-of-view device
GB2528100A (en) 2014-07-10 2016-01-13 Nokia Technologies Oy Method, apparatus and computer program product for editing media content
CN106797337B (zh) 2014-07-10 2021-06-22 法斯埃托股份有限公司 用于消息编辑的系统和方法
US9685194B2 (en) 2014-07-23 2017-06-20 Gopro, Inc. Voice-based video tagging
US10074013B2 (en) 2014-07-23 2018-09-11 Gopro, Inc. Scene and activity identification in video summary generation
US9928878B2 (en) 2014-08-13 2018-03-27 Intel Corporation Techniques and apparatus for editing video
USD762702S1 (en) * 2014-08-28 2016-08-02 Microsoft Corporation Display screen with transitional graphical user interface
USD778285S1 (en) * 2014-08-28 2017-02-07 Microsoft Corporation Display screen with graphical user interface
USD763298S1 (en) * 2014-08-28 2016-08-09 Microsoft Corporation Display screen with graphical user interface
USD777769S1 (en) * 2014-08-28 2017-01-31 Microsoft Corporation Display screen with colored graphical user interface
CN105376651B (zh) * 2014-08-29 2018-10-19 北京金山安全软件有限公司 一种生成视频幻灯片的方法及装置
US10123153B2 (en) 2014-10-06 2018-11-06 Fasetto, Inc. Systems and methods for portable storage devices
US10437288B2 (en) 2014-10-06 2019-10-08 Fasetto, Inc. Portable storage device with modular power and housing system
US9710220B2 (en) * 2014-10-24 2017-07-18 Sony Corporation Context-sensitive media classification
US20160125916A1 (en) * 2014-10-31 2016-05-05 ShotClip Limited Collaborative Movie Creation
US9451307B2 (en) * 2014-12-08 2016-09-20 Microsoft Technology Licensing, Llc Generating recommendations based on processing content item metadata tags
US9734870B2 (en) 2015-01-05 2017-08-15 Gopro, Inc. Media identifier generation for camera-captured media
US9679605B2 (en) 2015-01-29 2017-06-13 Gopro, Inc. Variable playback speed template for video editing application
GB2536032A (en) * 2015-03-05 2016-09-07 Carnegie Tech Invest Ltd Method for processing video clip for live-broadcast replay
KR102308140B1 (ko) 2015-03-11 2021-10-05 파세토, 인크. 웹 api 통신을 위한 시스템 및 방법
US10529383B2 (en) * 2015-04-09 2020-01-07 Avid Technology, Inc. Methods and systems for processing synchronous data tracks in a media editing system
GB201507348D0 (en) * 2015-04-29 2015-06-10 Tomtom Int Bv Digital video camera
WO2016187235A1 (en) 2015-05-20 2016-11-24 Gopro, Inc. Virtual lens simulation for video and photo cropping
US9841259B2 (en) 2015-05-26 2017-12-12 Digital Ally, Inc. Wirelessly conducted electronic weapon
JP6602067B2 (ja) * 2015-06-18 2019-11-06 キヤノン株式会社 表示制御装置、表示制御方法、プログラム
US10013883B2 (en) 2015-06-22 2018-07-03 Digital Ally, Inc. Tracking and analysis of drivers within a fleet of vehicles
CN107810638A (zh) * 2015-06-24 2018-03-16 汤姆逊许可公司 通过跳过冗余片段优化顺序内容的传递
JP2017016275A (ja) * 2015-06-29 2017-01-19 ヤマハ株式会社 制御方法
CN105100839A (zh) * 2015-07-14 2015-11-25 网易传媒科技（北京）有限公司 一种视频文件下载方法、装置、服务器和终端设备
CN105933772B (zh) * 2015-08-18 2019-06-21 盯盯拍（深圳）技术股份有限公司 交互方法、交互装置以及交互系统
US10019415B1 (en) 2015-08-28 2018-07-10 Animoto Inc. System and method for consistent cross-platform text layout
US20170075652A1 (en) 2015-09-14 2017-03-16 Kabushiki Kaisha Toshiba Electronic device and method
CN105222718B (zh) * 2015-09-21 2017-05-17 天津大学 室内空间测量定位网络动态坐标测量多站数据同步方法
US10204273B2 (en) 2015-10-20 2019-02-12 Gopro, Inc. System and method of providing recommendations of moments of interest within video clips post capture
US9721611B2 (en) 2015-10-20 2017-08-01 Gopro, Inc. System and method of generating video from video clips based on moments of interest within the video clips
US9858965B2 (en) * 2015-10-23 2018-01-02 Microsoft Technology Licensing, Llc Video loop generation
US20170127150A1 (en) * 2015-11-04 2017-05-04 Ubitus Inc. Interactive applications implemented in video streams
US9858036B2 (en) 2015-11-10 2018-01-02 Google Llc Automatic audio level adjustment during media item presentation
US10929071B2 (en) 2015-12-03 2021-02-23 Fasetto, Inc. Systems and methods for memory card emulation
US10217489B2 (en) 2015-12-07 2019-02-26 Cyberlink Corp. Systems and methods for media track management in a media editing tool
CN105528427B (zh) * 2015-12-08 2019-05-10 腾讯科技（深圳）有限公司 媒体文件处理方法、社交应用中的分享方法和装置
US10109319B2 (en) 2016-01-08 2018-10-23 Gopro, Inc. Digital media editing
US9799134B2 (en) * 2016-01-12 2017-10-24 Indg Method and system for high-performance real-time adjustment of one or more elements in a playing video, interactive 360° content or image
US10083537B1 (en) 2016-02-04 2018-09-25 Gopro, Inc. Systems and methods for adding a moving visual element to a video
US10178341B2 (en) * 2016-03-01 2019-01-08 DISH Technologies L.L.C. Network-based event recording
US10678398B2 (en) 2016-03-31 2020-06-09 Intel Corporation Prioritization for presentation of media based on sensor data collected by wearable sensor devices
US9794632B1 (en) * 2016-04-07 2017-10-17 Gopro, Inc. Systems and methods for synchronization based on audio track changes in video editing
US9838731B1 (en) 2016-04-07 2017-12-05 Gopro, Inc. Systems and methods for audio track selection in video editing with audio mixing option
US9838730B1 (en) 2016-04-07 2017-12-05 Gopro, Inc. Systems and methods for audio track selection in video editing
US10372306B2 (en) * 2016-04-16 2019-08-06 Apple Inc. Organized timeline
USD791159S1 (en) * 2016-04-18 2017-07-04 Apple Inc. Display screen or portion thereof with graphical user interface
DK201670582A1 (en) 2016-06-12 2018-01-02 Apple Inc Identifying applications on which content is available
DK201670581A1 (en) 2016-06-12 2018-01-08 Apple Inc Device-level authorization for viewing content
WO2017218341A1 (en) * 2016-06-17 2017-12-21 Axon Enterprise, Inc. Systems and methods for aligning event data
US10157638B2 (en) 2016-06-24 2018-12-18 Google Llc Collage of interesting moments in a video
JP6819096B2 (ja) * 2016-06-30 2021-01-27 オムロン株式会社 画像処理装置、画像処理方法、および画像処理プログラム
US10185891B1 (en) 2016-07-08 2019-01-22 Gopro, Inc. Systems and methods for compact convolutional neural networks
KR101768163B1 (ko) * 2016-07-18 2017-08-16 리얼리티리플렉션 주식회사 3d 비디오 생성 장치
US10104292B2 (en) 2016-08-04 2018-10-16 Microsoft Technology Licensing, Llc Multishot tilt optical image stabilization for shallow depth of field
CN107799132A (zh) * 2016-09-05 2018-03-13 中兴通讯股份有限公司 一种音频剪辑方法和装置，及终端
US9836853B1 (en) 2016-09-06 2017-12-05 Gopro, Inc. Three-dimensional convolutional neural networks for video highlight detection
US10521675B2 (en) 2016-09-19 2019-12-31 Digital Ally, Inc. Systems and methods of legibly capturing vehicle markings
US11966560B2 (en) 2016-10-26 2024-04-23 Apple Inc. User interfaces for browsing content from multiple content applications on an electronic device
USD820303S1 (en) 2016-10-27 2018-06-12 Apple Inc. Display screen or portion thereof with graphical user interface
USD815137S1 (en) * 2016-10-27 2018-04-10 Apple Inc. Display screen or portion thereof with graphical user interface
US10284809B1 (en) 2016-11-07 2019-05-07 Gopro, Inc. Systems and methods for intelligently synchronizing events in visual content with musical features in audio content
US10262639B1 (en) 2016-11-08 2019-04-16 Gopro, Inc. Systems and methods for detecting musical features in audio content
DE102016221928A1 (de) 2016-11-09 2018-05-09 Siemens Aktiengesellschaft Verfahren zum Betreiben eines im Betrieb zyklisch belasteten Bauteils
CA3044665A1 (en) 2016-11-23 2018-05-31 Fasetto, Inc. Systems and methods for streaming media
JP6727106B2 (ja) 2016-11-30 2020-07-22 キヤノン株式会社 画像音響処理装置および画像音響処理方法、プログラム
JP7137173B2 (ja) * 2016-12-08 2022-09-14 チームラボ株式会社 コレクションシステム，端末用プログラム，及びコレクション方法
KR102028198B1 (ko) * 2017-01-26 2019-10-04 한국전자통신연구원 동영상 장면과 메타데이터 저작 방법
US10534966B1 (en) 2017-02-02 2020-01-14 Gopro, Inc. Systems and methods for identifying activities and/or events represented in a video
WO2018144833A1 (en) 2017-02-03 2018-08-09 Fasetto, Llc Systems and methods for data storage in keyed devices
US10171256B2 (en) 2017-02-07 2019-01-01 Microsoft Technology Licensing, Llc Interactive timeline for a teleconference session
US10193940B2 (en) 2017-02-07 2019-01-29 Microsoft Technology Licensing, Llc Adding recorded content to an interactive timeline of a teleconference session
CN108429931A (zh) * 2017-02-14 2018-08-21 优酷网络技术（北京）有限公司 一种多媒体文件播放方法及装置
US10070093B1 (en) 2017-02-24 2018-09-04 Microsoft Technology Licensing, Llc Concurrent viewing of live content and recorded content
US10127943B1 (en) 2017-03-02 2018-11-13 Gopro, Inc. Systems and methods for modifying videos based on music
US10185895B1 (en) 2017-03-23 2019-01-22 Gopro, Inc. Systems and methods for classifying activities captured within images
US11749312B2 (en) * 2017-03-24 2023-09-05 Adobe Inc. Timeline creation of electronic document creation states
US10083718B1 (en) 2017-03-24 2018-09-25 Gopro, Inc. Systems and methods for editing videos based on motion
US20180302454A1 (en) * 2017-04-05 2018-10-18 Interlock Concepts Inc. Audio visual integration device
US10187690B1 (en) 2017-04-24 2019-01-22 Gopro, Inc. Systems and methods to detect and correlate user responses to media content
US11166085B2 (en) * 2017-04-26 2021-11-02 Google Llc Generating media content playlists including interstitials combined with media content items
US10360942B1 (en) * 2017-07-13 2019-07-23 Gopro, Inc. Systems and methods for changing storage of videos
US10726872B1 (en) 2017-08-30 2020-07-28 Snap Inc. Advanced video editing techniques using sampling patterns
US10931988B2 (en) * 2017-09-13 2021-02-23 Amazon Technologies, Inc. Distributed multi-datacenter video packaging system
CN109561240B (zh) 2017-09-24 2023-02-17 福希特公司 用于生成媒体资产的系统和方法
US10763630B2 (en) 2017-10-19 2020-09-01 Fasetto, Inc. Portable electronic device connection systems
US10349097B2 (en) * 2017-10-27 2019-07-09 Mti Film, Llc Metadata editor for multimedia delivery
US10659685B2 (en) 2017-10-30 2020-05-19 Visual Supply Company Control of viewing angles for 360-degree video playback
JP6952584B2 (ja) * 2017-11-20 2021-10-20 Ｏｍデジタルソリューションズ株式会社 撮像装置、撮像方法及び撮像プログラム
CN107872620B (zh) * 2017-11-22 2020-06-02 北京小米移动软件有限公司 视频录制方法及装置、计算机可读存储介质
CN108024073B (zh) 2017-11-30 2020-09-04 广州市百果园信息技术有限公司 视频编辑方法、装置及智能移动终端
CA3083988A1 (en) 2017-12-01 2019-06-06 Fasetto, Inc. Systems and methods for improved data encryption
USD861724S1 (en) 2017-12-21 2019-10-01 Toontrack Music Ab Computer screen with a graphical user interface
JP2019165386A (ja) * 2018-03-20 2019-09-26 ヤマハ株式会社 信号解析方法および信号解析装置
US11064268B2 (en) * 2018-03-23 2021-07-13 Disney Enterprises, Inc. Media content metadata mapping
US11093839B2 (en) * 2018-04-13 2021-08-17 Fujifilm Business Innovation Corp. Media object grouping and classification for predictive enhancement
WO2019204524A1 (en) 2018-04-17 2019-10-24 Fasetto, Inc. Device presentation with real-time feedback
CN108682436B (zh) * 2018-05-11 2020-06-23 北京海天瑞声科技股份有限公司 语音对齐方法及装置
DK201870354A1 (en) 2018-06-03 2019-12-20 Apple Inc. SETUP PROCEDURES FOR AN ELECTRONIC DEVICE
JP7183597B2 (ja) * 2018-07-12 2022-12-06 京セラドキュメントソリューションズ株式会社 プレビュー画像表示装置及びプレビュー画像表示プログラム
CN108986184B (zh) * 2018-07-23 2023-04-18 Oppo广东移动通信有限公司 视频创建方法及相关设备
US11024137B2 (en) 2018-08-08 2021-06-01 Digital Ally, Inc. Remote video triggering and tagging
CN109151537B (zh) * 2018-08-29 2020-05-01 北京达佳互联信息技术有限公司 视频处理方法、装置、电子设备及存储介质
CN109275024A (zh) * 2018-09-14 2019-01-25 洪海光电集团有限公司 通过音频信号控制多媒体设备的装置及其方法
CN109120997A (zh) * 2018-09-30 2019-01-01 北京微播视界科技有限公司 视频处理方法、装置、终端和介质
CN109936762B (zh) * 2019-01-12 2021-06-25 河南图灵实验室信息技术有限公司 相似音频或视频文件同步播放的方法以及电子设备
KR102569032B1 (ko) * 2019-01-22 2023-08-23 삼성전자주식회사 전자 장치 및 그의 컨텐트 제공 방법
US11049525B2 (en) * 2019-02-21 2021-06-29 Adobe Inc. Transcript-based insertion of secondary video content into primary video content
US11545187B2 (en) 2019-02-28 2023-01-03 Vertigo Media, Inc. System and method for compiling user-generated videos
US11720621B2 (en) * 2019-03-18 2023-08-08 Apple Inc. Systems and methods for naming objects based on object content
US11683565B2 (en) 2019-03-24 2023-06-20 Apple Inc. User interfaces for interacting with channels that provide content that plays in a media browsing application
CN113711169A (zh) 2019-03-24 2021-11-26 苹果公司 包括内容项的可选表示的用户界面
EP3928526A1 (en) 2019-03-24 2021-12-29 Apple Inc. User interfaces for viewing and accessing content on an electronic device
US11962836B2 (en) * 2019-03-24 2024-04-16 Apple Inc. User interfaces for a media browsing application
JP2022528858A (ja) * 2019-04-01 2022-06-16 ブラックマジック デザイン ピーティーワイ リミテッド ビデオ編集システム用ユーザインターフェース
CN113811843B (zh) 2019-04-01 2024-05-03 黑魔法设计私人有限公司 媒体管理系统
CN111787354B (zh) * 2019-04-03 2022-06-07 浙江大学 一种视频生成方法及其装置
US20200374422A1 (en) * 2019-05-24 2020-11-26 Grass Valley Canada System and method of synchronizing video and audio clips with audio data
CN110198486B (zh) * 2019-05-28 2022-04-05 上海哔哩哔哩科技有限公司 一种预览视频素材的方法、计算机设备及可读存储介质
CN113906380A (zh) 2019-05-31 2022-01-07 苹果公司 用于播客浏览和回放应用程序的用户界面
US11863837B2 (en) 2019-05-31 2024-01-02 Apple Inc. Notification of augmented reality content on an electronic device
USD913315S1 (en) * 2019-05-31 2021-03-16 Apple Inc. Electronic device with graphical user interface
CN112083855B (zh) * 2019-06-13 2024-09-13 昆山研达电脑科技有限公司 多媒体档案管理方法及计算机可读取储存媒体
JP7386008B2 (ja) * 2019-07-19 2023-11-24 株式会社デンソーテン 表示制御装置、データ収集システム、表示制御方法および表示制御プログラム
WO2021022324A1 (en) * 2019-08-02 2021-02-11 Blackmagic Design Pty Ltd Video editing system, method and user interface
CN110597759B (zh) * 2019-08-28 2022-03-29 福建省天奕网络科技有限公司 LayaIDE中图集资源优化的方法、存储介质
US11997409B2 (en) 2019-10-30 2024-05-28 Beijing Bytedance Network Technology Co., Ltd. Video processing method and apparatus, and terminal and storage medium
CN110784674B (zh) * 2019-10-30 2022-03-15 北京字节跳动网络技术有限公司 视频处理的方法、装置、终端及存储介质
KR102227532B1 (ko) 2019-11-01 2021-03-12 키네마스터 주식회사 클립 이동 제어를 위한 동영상 편집 ui 제어방법 및 장치
KR20210065604A (ko) * 2019-11-27 2021-06-04 한국전자통신연구원 분산 네트워크 기반 멀티미디어 스트리밍 서비스에서 스트림을 선택하여 수신하는 방법 및 장치
BR102019027509A2 (pt) * 2019-12-20 2021-07-06 Globo Comunicação E Participações S.a. sistema e método de captação de vídeo e ordenação de sequência de cenas
US11843838B2 (en) 2020-03-24 2023-12-12 Apple Inc. User interfaces for accessing episodes of a content series
WO2021222319A1 (en) * 2020-04-28 2021-11-04 Editshare, Llc Heterogeneous media editing across storage platforms
WO2021251615A1 (ko) 2020-06-10 2021-12-16 삼성전자주식회사 전자 장치 및 전자 장치의 제어 방법
US11899895B2 (en) 2020-06-21 2024-02-13 Apple Inc. User interfaces for setting up an electronic device
CN111538489A (zh) * 2020-07-09 2020-08-14 北京美摄网络科技有限公司 一种数据处理方法和装置
CN111757013B (zh) 2020-07-23 2022-04-29 北京字节跳动网络技术有限公司 一种视频处理方法、装置、设备及存储介质
CN111901534B (zh) * 2020-07-23 2022-03-29 北京字节跳动网络技术有限公司 一种音视频分割的交互方法、装置、设备及存储介质
CN111866550A (zh) * 2020-07-24 2020-10-30 上海盛付通电子支付服务有限公司 视频片段的屏蔽方法和装置
CN111953921B (zh) * 2020-08-14 2022-03-11 杭州视洞科技有限公司 一种圆角泳道的展示及交互方法
USD1011359S1 (en) * 2020-09-22 2024-01-16 Igt Display screen or portion thereof with graphical user interface
US11721365B2 (en) 2020-11-09 2023-08-08 Blackmagic Design Pty Ltd Video editing or media management system
CN112380379B (zh) * 2020-11-18 2023-05-02 抖音视界有限公司 歌词特效展示方法、装置、电子设备及计算机可读介质
US11720229B2 (en) 2020-12-07 2023-08-08 Apple Inc. User interfaces for browsing and presenting content
US11934640B2 (en) 2021-01-29 2024-03-19 Apple Inc. User interfaces for record labels
CN115079908B (zh) * 2021-03-12 2024-07-23 北京字节跳动网络技术有限公司 媒体内容收藏方法、装置、电子设备及计算机可读存储介质
WO2022239281A1 (ja) * 2021-05-12 2022-11-17 ソニーグループ株式会社 画像処理装置、画像処理方法、及びプログラム
KR20230004028A (ko) * 2021-06-30 2023-01-06 삼성전자주식회사 제어 방법 및 그 방법을 이용하는 장치
CN113938759A (zh) * 2021-10-19 2022-01-14 维沃移动通信有限公司 文件分享方法和文件分享装置
CN114329063B (zh) * 2021-10-29 2024-06-11 腾讯科技（深圳）有限公司 视频片段检测方法、装置以及设备
EP4380170A1 (en) * 2021-11-09 2024-06-05 Samsung Electronics Co., Ltd. Electronic device and method for automatically generating edited video
CN114025229A (zh) * 2021-11-09 2022-02-08 上海爱奇艺新媒体科技有限公司 处理音视频文件的方法、装置、计算设备及存储介质
KR102553519B1 (ko) * 2021-12-07 2023-07-10 한국전자기술연구원 컴퓨터와 사람의 협업을 통한 동영상 반자동 편집을 위한 인공지능 기반 동영상 편집 추천 방법
US11495265B1 (en) * 2021-12-30 2022-11-08 Microsoft Technology Licensing, Llc Change-dependent pre-export for video editing projects
CN118511506A (zh) * 2022-01-01 2024-08-16 镭亚股份有限公司 多视图视频渲染、准备多视图高速缓存和实时多视图视频转换的方法和系统
IT202200002852A1 (it) * 2022-02-16 2023-08-16 Pyxie S R L Metodo per l'organizzazione di contenuti multimediali
CN114630146B (zh) * 2022-05-12 2023-01-17 海看网络科技(山东)股份有限公司 一种iptv中解决进度条预览卡顿的方法
US11950017B2 (en) 2022-05-17 2024-04-02 Digital Ally, Inc. Redundant mobile video recording
CN116112780B (zh) * 2022-05-25 2023-12-01 荣耀终端有限公司 录像方法和相关装置
CN117425057A (zh) 2022-07-07 2024-01-19 抖音视界(北京)有限公司 用于影像拍摄的方法、装置、设备和存储介质
CN115314757B (zh) * 2022-08-05 2023-10-24 聚好看科技股份有限公司 一种显示设备及引导窗口显示方法
CN115460448A (zh) * 2022-08-19 2022-12-09 北京达佳互联信息技术有限公司 一种媒体资源编辑方法、装置、电子设备以及存储介质
CN115334246A (zh) 2022-09-06 2022-11-11 抖音视界有限公司 用于影像拍摄的方法、装置、设备和存储介质
CN116033221A (zh) * 2022-12-22 2023-04-28 北京新唐思创教育科技有限公司 一种视频处理方法、装置、设备及介质
CN117499745B (zh) * 2023-04-12 2024-06-28 中科方寸知微(南京)科技有限公司 一种媒体编辑方法、装置、设备和存储介质
US12067755B1 (en) * 2023-05-19 2024-08-20 Verkada Inc. Methods and apparatus for detection-based object search using edge computing
US12033348B1 (en) 2023-08-15 2024-07-09 Verkada Inc. Methods and apparatus for generating images of objects detected in video camera data

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6539163B1 (en) * 1999-04-16 2003-03-25 Avid Technology, Inc. Non-linear editing system and method employing reference clips in edit sequences
US20100013842A1 (en) * 2008-07-16 2010-01-21 Google Inc. Web-based graphics rendering system
US7823056B1 (en) * 2006-03-15 2010-10-26 Adobe Systems Incorporated Multiple-camera video recording
US20120210230A1 (en) * 2010-07-15 2012-08-16 Ken Matsuda Media-Editing Application with Anchored Timeline

Family Cites Families (114)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5307456A (en) 1990-12-04 1994-04-26 Sony Electronics, Inc. Integrated multi-media production and authoring system
EP0526064B1 (en) 1991-08-02 1997-09-10 The Grass Valley Group, Inc. Video editing system operator interface for visualization and interactive control of video material
US5442744A (en) * 1992-04-03 1995-08-15 Sun Microsystems, Inc. Methods and apparatus for displaying and editing multimedia information
US5999173A (en) 1992-04-03 1999-12-07 Adobe Systems Incorporated Method and apparatus for video editing with video clip representations displayed along a time line
US5682326A (en) 1992-08-03 1997-10-28 Radius Inc. Desktop digital video processing system
US5659792A (en) 1993-01-15 1997-08-19 Canon Information Systems Research Australia Pty Ltd. Storyboard system for the simultaneous timing of multiple independent video animation clips
WO1994029868A2 (en) 1993-06-10 1994-12-22 Lightworks Editing Systems Ltd Video editing systems
US5664216A (en) 1994-03-22 1997-09-02 Blumenau; Trevor Iconic audiovisual data editing environment
US5659793A (en) 1994-12-22 1997-08-19 Bell Atlantic Video Services, Inc. Authoring tools for multimedia application development and network delivery
US5826102A (en) 1994-12-22 1998-10-20 Bell Atlantic Network Services, Inc. Network arrangement for development delivery and presentation of multimedia applications using timelines to integrate multimedia objects and program objects
CA2145812C (en) 1995-03-29 1998-11-17 Mario Bonja Digital video networks
JP3837746B2 (ja) 1995-04-08 2006-10-25 ソニー株式会社 編集システム
US5572261A (en) 1995-06-07 1996-11-05 Cooper; J. Carl Automatic audio to video timing measurement device and method
US5659539A (en) 1995-07-14 1997-08-19 Oracle Corporation Method and apparatus for frame accurate access of digital audio-visual information
US5732184A (en) 1995-10-20 1998-03-24 Digital Processing Systems, Inc. Video and audio cursor video editing system
US6188396B1 (en) 1996-03-29 2001-02-13 International Business Machines Corp. Synchronizing multimedia parts with reference to absolute time, relative time, and event time
US6161115A (en) 1996-04-12 2000-12-12 Avid Technology, Inc. Media editing system with improved effect management
US6154601A (en) 1996-04-12 2000-11-28 Hitachi Denshi Kabushiki Kaisha Method for editing image information with aid of computer and editing system
US5861889A (en) 1996-04-19 1999-01-19 3D-Eye, Inc. Three dimensional computer graphics tool facilitating movement of displayed object
EP0896772A4 (en) 1996-04-29 2001-12-05 Princeton Video Image Inc ELECTRONIC INSERTION OF VISUAL ELEMENTS ASSORTED WITH SOUND EFFECTS IN VIDEO SIGNALS
US6628303B1 (en) 1996-07-29 2003-09-30 Avid Technology, Inc. Graphical user interface for a motion video planning and editing system for a computer
US6154600A (en) 1996-08-06 2000-11-28 Applied Magic, Inc. Media editor for non-linear editing system
US5969716A (en) 1996-08-06 1999-10-19 Interval Research Corporation Time-based media processing system
US7055100B2 (en) 1996-09-20 2006-05-30 Sony Corporation Editing system, editing method, clip management apparatus, and clip management method
US6324335B1 (en) 1996-11-29 2001-11-27 Sony Corporation Editing system and editing method
US5893062A (en) 1996-12-05 1999-04-06 Interval Research Corporation Variable rate video playback with synchronized audio
US6262776B1 (en) 1996-12-13 2001-07-17 Microsoft Corporation System and method for maintaining synchronization between audio and video
US6054984A (en) 1997-05-27 2000-04-25 Hewlett-Packard Company System and method for graphically manipulating display elements in a signal measurement system
US6400378B1 (en) 1997-09-26 2002-06-04 Sony Corporation Home movie maker
JP2985847B2 (ja) 1997-10-17 1999-12-06 日本電気株式会社 入力装置
US5933141A (en) 1998-01-05 1999-08-03 Gateway 2000, Inc. Mutatably transparent displays
US6336093B2 (en) 1998-01-16 2002-01-01 Avid Technology, Inc. Apparatus and method using speech recognition and scripts to capture author and playback synchronized audio and video
JPH11203837A (ja) 1998-01-16 1999-07-30 Sony Corp 編集システムおよび編集方法
JP3944807B2 (ja) 1998-04-02 2007-07-18 ソニー株式会社 素材選択装置及び素材選択方法
JPH11289512A (ja) 1998-04-03 1999-10-19 Sony Corp 編集リスト作成装置
US20020023103A1 (en) 1998-04-21 2002-02-21 Rejean Gagne System and method for accessing and manipulating time-based data using meta-clip objects
US6144375A (en) 1998-08-14 2000-11-07 Praja Inc. Multi-perspective viewer for content-based interactivity
JP4207099B2 (ja) 1998-09-29 2009-01-14 ソニー株式会社 画像編集装置及びその方法
US6313854B1 (en) 1998-10-16 2001-11-06 International Business Machines Corporation Display mechanism for HTML frames
GB2344453B (en) 1998-12-01 2002-12-11 Eidos Technologies Ltd Multimedia editing and composition system having temporal display
US6453251B1 (en) 1999-10-07 2002-09-17 Receptec Llc Testing method for components with reception capabilities
US6429902B1 (en) 1999-12-07 2002-08-06 Lsi Logic Corporation Method and apparatus for audio and video end-to-end synchronization
AUPQ464099A0 (en) 1999-12-14 2000-01-13 Canon Kabushiki Kaisha Emotive editing system
US7725812B1 (en) 2000-03-31 2010-05-25 Avid Technology, Inc. Authoring system for combining temporal and nontemporal digital media
US20010036356A1 (en) 2000-04-07 2001-11-01 Autodesk, Inc. Non-linear video editing system
US7325199B1 (en) 2000-10-04 2008-01-29 Apple Inc. Integrated time line for editing
JP2002135709A (ja) * 2000-10-18 2002-05-10 Sony Corp 編集装置、記録再生装置、編集方法、及び記録媒体
US6907570B2 (en) 2001-03-29 2005-06-14 International Business Machines Corporation Video and multimedia browsing while switching between views
US7461002B2 (en) * 2001-04-13 2008-12-02 Dolby Laboratories Licensing Corporation Method for time aligning audio signals using characterizations based on auditory events
US7610205B2 (en) 2002-02-12 2009-10-27 Dolby Laboratories Licensing Corporation High quality time-scaling and pitch-scaling of audio signals
GB2374719A (en) 2001-04-20 2002-10-23 Discreet Logic Inc Image data processing apparatus with two timelines
GB2374748A (en) 2001-04-20 2002-10-23 Discreet Logic Inc Image data editing for transitions between sequences
US7540011B2 (en) 2001-06-11 2009-05-26 Arrowsight, Inc. Caching graphical interface for displaying video and ancillary data from a saved video
JP3617052B2 (ja) 2001-08-30 2005-02-02 日本電気株式会社 カメラ映像提供システム及びカメラ映像の時刻補正方法
US7432940B2 (en) * 2001-10-12 2008-10-07 Canon Kabushiki Kaisha Interactive animation of sprites in a video production
US7062712B2 (en) * 2002-04-09 2006-06-13 Fuji Xerox Co., Ltd. Binding interactive multichannel digital document system
US7073127B2 (en) 2002-07-01 2006-07-04 Arcsoft, Inc. Video editing GUI with layer view
US20050273321A1 (en) 2002-08-08 2005-12-08 Choi Won Y Audio signal time-scale modification method using variable length synthesis and reduced cross-correlation computations
US7863513B2 (en) * 2002-08-22 2011-01-04 Yamaha Corporation Synchronous playback system for reproducing music in good ensemble and recorder and player for the ensemble
US8009966B2 (en) 2002-11-01 2011-08-30 Synchro Arts Limited Methods and apparatus for use in sound replacement with automatic synchronization to images
US7627359B2 (en) * 2002-12-19 2009-12-01 University Of Washington Quantitative contrast enhanced black-blood imaging using quadruple-inversion recovery
EP1593272B1 (en) 2003-02-14 2017-08-30 Thomson Licensing DTV Automatic synchronization of audio and video based media services of media content
CN1526864A (zh) * 2003-03-07 2004-09-08 西安工程科技学院 针梳机上使用的机电式自调匀整测控系统
US7142250B1 (en) 2003-04-05 2006-11-28 Apple Computer, Inc. Method and apparatus for synchronizing audio and video streams
US8392834B2 (en) 2003-04-09 2013-03-05 Hewlett-Packard Development Company, L.P. Systems and methods of authoring a multimedia file
US7739715B2 (en) 2003-06-24 2010-06-15 Microsoft Corporation Variable play speed control for media streams
CN1291037C (zh) * 2004-01-19 2006-12-20 中国科学院上海微系统与信息技术研究所 一种芯片原位聚合酶链反应检测目标基因的方法
JP4955544B2 (ja) * 2004-06-03 2012-06-20 ヒルクレスト・ラボラトリーズ・インコーポレイテッド ズーム可能ユーザインターフェースのためのクライアント／サーバアーキテクチャおよび方法
US7903927B2 (en) 2004-07-08 2011-03-08 Sony Corporation Editing apparatus and control method thereof, and program and recording medium
US7411590B1 (en) 2004-08-09 2008-08-12 Apple Inc. Multimedia file format
GB2417176A (en) 2004-08-12 2006-02-15 Ibm Mouse cursor display
JP4229035B2 (ja) 2004-09-28 2009-02-25 ソニー株式会社 記録再生装置および方法、再生装置および方法、記録装置および方法、並びにプログラム
US7788592B2 (en) 2005-01-12 2010-08-31 Microsoft Corporation Architecture and engine for time line based visualization of data
US7843491B2 (en) 2005-04-05 2010-11-30 3Vr Security, Inc. Monitoring and presenting video surveillance data
US7669130B2 (en) 2005-04-15 2010-02-23 Apple Inc. Dynamic real-time playback
US7313755B2 (en) 2005-04-20 2007-12-25 Microsoft Corporation Media timeline sorting
US8156176B2 (en) 2005-04-20 2012-04-10 Say Media, Inc. Browser based multi-clip video editing
FR2887106B1 (fr) 2005-05-27 2009-07-31 Christophe Malassis Procede de generation d'un signal source, signal source, procede et dispositif de montage, moyens de stockage et programme d'ordinateur correspondants
US7644373B2 (en) * 2006-01-23 2010-01-05 Microsoft Corporation User interface for viewing clusters of images
CN101455081A (zh) 2006-03-27 2009-06-10 诺基亚西门子通信有限责任两合公司 生成数字数据流的方法
TW200809764A (en) 2006-04-24 2008-02-16 Sony Corp Image processing device, image processing method and program recording medium
US8682654B2 (en) 2006-04-25 2014-03-25 Cyberlink Corp. Systems and methods for classifying sports video
GB0608529D0 (en) * 2006-04-28 2006-06-07 Oxwest Media Ltd Generating a media program
US7623755B2 (en) * 2006-08-17 2009-11-24 Adobe Systems Incorporated Techniques for positioning audio and video clips
US20080072166A1 (en) 2006-09-14 2008-03-20 Reddy Venkateshwara N Graphical user interface for creating animation
US8035747B2 (en) * 2007-01-05 2011-10-11 Eastman Kodak Company Image digital processing based on edit status
US8249153B2 (en) 2007-06-12 2012-08-21 In Extenso Holdings Inc. Distributed synchronized video viewing and editing
US8037046B2 (en) 2007-06-29 2011-10-11 Microsoft Corporation Collecting and presenting temporal-based action information
JP2010541415A (ja) 2007-09-28 2010-12-24 グレースノート インコーポレイテッド マルチメディアイベントのプレゼンテーションの合成
US8199211B2 (en) 2008-03-12 2012-06-12 Sony Corporation Camera direct dailies
WO2009114134A2 (en) 2008-03-13 2009-09-17 United Video Properties, Inc. Systems and methods for synchronizing time-shifted media content and related communications
JP2011077564A (ja) 2008-04-13 2011-04-14 Thomson Canopus Co Ltd 映像および音声データの符号化装置とその符号化方法、及びビデオ編集システム
US20100077289A1 (en) 2008-09-08 2010-03-25 Eastman Kodak Company Method and Interface for Indexing Related Media From Multiple Sources
US9646648B2 (en) 2008-10-23 2017-05-09 Google Technology Holdings LLC Method and apparatus for creating short video clips of important events
US20100107126A1 (en) 2008-10-28 2010-04-29 Hulu Llc Method and apparatus for thumbnail selection and editing
WO2010068175A2 (en) 2008-12-10 2010-06-17 Muvee Technologies Pte Ltd Creating a new video production by intercutting between multiple video clips
JP2012054619A (ja) 2009-03-19 2012-03-15 Grass Valley Co Ltd 編集装置、編集方法、編集プログラム及びデータ構造
US8769421B2 (en) 2009-04-30 2014-07-01 Apple Inc. Graphical user interface for a media-editing application with a segmented timeline
US8522144B2 (en) 2009-04-30 2013-08-27 Apple Inc. Media editing application with candidate clip management
US9032299B2 (en) * 2009-04-30 2015-05-12 Apple Inc. Tool for grouping media clips for a media editing application
US8555169B2 (en) * 2009-04-30 2013-10-08 Apple Inc. Media clip auditioning used to evaluate uncommitted media content
US8359537B2 (en) 2009-04-30 2013-01-22 Apple Inc. Tool for navigating a composite presentation
US8612858B2 (en) 2009-05-01 2013-12-17 Apple Inc. Condensing graphical representations of media clips in a composite display area of a media-editing application
WO2010138777A1 (en) 2009-05-27 2010-12-02 Arsh Technologies, Llc Automatic resource retrieval and use
US20110072037A1 (en) 2009-09-18 2011-03-24 Carey Leigh Lotzer Intelligent media capture, organization, search and workflow
US8677240B2 (en) * 2009-10-05 2014-03-18 Harris Corporation Video processing system providing association between displayed video and media content and related methods
US9066674B2 (en) * 2010-03-17 2015-06-30 Siemens Medical Solutions Usa, Inc. Motion synchronized destruction for three-dimensional reperfusion mapping in medical diagnostic ultrasound imaging
KR20150095957A (ko) 2010-05-04 2015-08-21 샤잠 엔터테인먼트 리미티드 미디어 스트림의 샘플을 처리하는 방법 및 시스템
US20120030550A1 (en) * 2010-07-28 2012-02-02 Chin Ai Method for editing multimedia
US9099161B2 (en) 2011-01-28 2015-08-04 Apple Inc. Media-editing application with multiple resolution modes
US8621355B2 (en) 2011-02-02 2013-12-31 Apple Inc. Automatic synchronization of media clips
US8819185B2 (en) * 2011-08-05 2014-08-26 John Charles McLaughlin Streaming internet video editing and composite edit playback
US9437247B2 (en) 2011-11-14 2016-09-06 Apple Inc. Preview display for multi-camera media clips
US8682144B1 (en) 2012-09-17 2014-03-25 Google Inc. Method for synchronizing multiple audio signals

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6539163B1 (en) * 1999-04-16 2003-03-25 Avid Technology, Inc. Non-linear editing system and method employing reference clips in edit sequences
US7823056B1 (en) * 2006-03-15 2010-10-26 Adobe Systems Incorporated Multiple-camera video recording
US20100013842A1 (en) * 2008-07-16 2010-01-21 Google Inc. Web-based graphics rendering system
US20120210230A1 (en) * 2010-07-15 2012-08-16 Ken Matsuda Media-Editing Application with Anchored Timeline

Cited By (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10692536B1 (en) 2005-04-16 2020-06-23 Apple Inc. Generation and use of multiclips in video editing
US20110289413A1 (en) * 2006-12-22 2011-11-24 Apple Inc. Fast Creation of Video Segments
US9959907B2 (en) * 2006-12-22 2018-05-01 Apple Inc. Fast creation of video segments
US9830063B2 (en) 2006-12-22 2017-11-28 Apple Inc. Modified media presentation during scrubbing
US9280262B2 (en) 2006-12-22 2016-03-08 Apple Inc. Select drag and drop operations on video thumbnails across clip boundaries
US9335892B2 (en) 2006-12-22 2016-05-10 Apple Inc. Select drag and drop operations on video thumbnails across clip boundaries
US9792955B2 (en) 2011-11-14 2017-10-17 Apple Inc. Automatic generation of multi-camera media clips
US9437247B2 (en) 2011-11-14 2016-09-06 Apple Inc. Preview display for multi-camera media clips
US9111579B2 (en) 2011-11-14 2015-08-18 Apple Inc. Media editing with multi-camera media clips
US9578256B1 (en) * 2012-02-29 2017-02-21 Google Inc. Temporary intermediate video clips for video editing
US10593295B2 (en) * 2013-02-28 2020-03-17 Apple Inc. Aligned video comparison tool
US20150279330A1 (en) * 2013-02-28 2015-10-01 Apple Inc. Aligned video comparison tool
US8994828B2 (en) * 2013-02-28 2015-03-31 Apple Inc. Aligned video comparison tool
US9519420B2 (en) 2013-10-16 2016-12-13 Samsung Electronics Co., Ltd. Apparatus and method for editing synchronous media
US10162506B2 (en) 2014-07-18 2018-12-25 Apple Inc. Systems and methods for selecting portions of media for a preview
US10545654B2 (en) 2014-07-18 2020-01-28 Apple Inc. Systems and methods for selecting portions of media for a preview
US10057651B1 (en) * 2015-10-05 2018-08-21 Twitter, Inc. Video clip creation using social media
US10504559B2 (en) 2016-03-09 2019-12-10 Yamaha Corporation Recorded data processing method and recorded data processing device
US10757468B2 (en) * 2017-10-10 2020-08-25 Apple Inc. Systems and methods for performing playout of multiple media recordings based on a matching segment among the recordings
US20190141388A1 (en) * 2017-10-10 2019-05-09 Shazam Investments Ltd. Systems and Methods for Performing Playout of Multiple Media Recordings Based on a Matching Segment Among the Recordings
US11457140B2 (en) 2019-03-27 2022-09-27 On Time Staffing Inc. Automatic camera angle switching in response to low noise audio to create combined audiovisual file
US10728443B1 (en) 2019-03-27 2020-07-28 On Time Staffing Inc. Automatic camera angle switching to create combined audiovisual file
US10963841B2 (en) 2019-03-27 2021-03-30 On Time Staffing Inc. Employment candidate empathy scoring system
US11961044B2 (en) 2019-03-27 2024-04-16 On Time Staffing, Inc. Behavioral data analysis and scoring system
US11863858B2 (en) 2019-03-27 2024-01-02 On Time Staffing Inc. Automatic camera angle switching in response to low noise audio to create combined audiovisual file
US11127232B2 (en) 2019-11-26 2021-09-21 On Time Staffing Inc. Multi-camera, multi-sensor panel data extraction system and method
US11783645B2 (en) 2019-11-26 2023-10-10 On Time Staffing Inc. Multi-camera, multi-sensor panel data extraction system and method
US11861904B2 (en) 2020-04-02 2024-01-02 On Time Staffing, Inc. Automatic versioning of video presentations
US11184578B2 (en) 2020-04-02 2021-11-23 On Time Staffing, Inc. Audio and video recording and streaming in a three-computer booth
US11636678B2 (en) 2020-04-02 2023-04-25 On Time Staffing Inc. Audio and video recording and streaming in a three-computer booth
US11023735B1 (en) 2020-04-02 2021-06-01 On Time Staffing, Inc. Automatic versioning of video presentations
US11720859B2 (en) 2020-09-18 2023-08-08 On Time Staffing Inc. Systems and methods for evaluating actions over a computer network and establishing live network connections
US11144882B1 (en) 2020-09-18 2021-10-12 On Time Staffing Inc. Systems and methods for evaluating actions over a computer network and establishing live network connections
US11893668B2 (en) 2021-03-31 2024-02-06 Leica Camera Ag Imaging system and method for generating a final digital image via applying a profile to image information
US11727040B2 (en) 2021-08-06 2023-08-15 On Time Staffing, Inc. Monitoring third-party forum contributions to improve searching through time-to-live data assignments
US11966429B2 (en) 2021-08-06 2024-04-23 On Time Staffing Inc. Monitoring third-party forum contributions to improve searching through time-to-live data assignments
US11423071B1 (en) 2021-08-31 2022-08-23 On Time Staffing, Inc. Candidate data ranking method using previously selected candidate data
US20230125371A1 (en) * 2021-10-27 2023-04-27 Genetec Inc. Systems and methods for concurrently playing multiple remotely stored media files in a web browser
US11907652B2 (en) 2022-06-02 2024-02-20 On Time Staffing, Inc. User interface and systems for document creation

Also Published As

Publication number Publication date
CN103931199A (zh) 2014-07-16
EP3012838B1 (en) 2018-01-03
US9437247B2 (en) 2016-09-06
US20130124998A1 (en) 2013-05-16
US9792955B2 (en) 2017-10-17
US9111579B2 (en) 2015-08-18
JP2016105597A (ja) 2016-06-09
JP5856690B2 (ja) 2016-02-10
EP2781102A1 (en) 2014-09-24
WO2013074207A1 (en) 2013-05-23
CN103931199B (zh) 2017-07-21
US20130121668A1 (en) 2013-05-16
JP2015504629A (ja) 2015-02-12
EP3012838A1 (en) 2016-04-27
JP6214619B2 (ja) 2017-10-18
US20130125000A1 (en) 2013-05-16

Similar Documents

Publication Publication Date Title
US9437247B2 (en) Preview display for multi-camera media clips
US11157154B2 (en) Media-editing application with novel editing tools
US8966367B2 (en) Anchor override for a media-editing application with an anchored timeline
US8875025B2 (en) Media-editing application with media clips grouping capabilities
US9870802B2 (en) Media clip management
US11747972B2 (en) Media-editing application with novel editing tools
US8631047B2 (en) Editing 3D video
US20030146915A1 (en) Interactive animation of sprites in a video production
JP2022542451A (ja) ビデオ編集システム、方法、およびユーザインターフェース
AU2002301447B2 (en) Interactive Animation of Sprites in a Video Production

Legal Events

Date Code Title Description
AS Assignment

Owner name: APPLE INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:AGNOLI, GIOVANNI;POOLE, SHAUN M.;CARSON, KENNETH M.;AND OTHERS;REEL/FRAME:028074/0772

Effective date: 20120201

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION