JP2014506434A - Logging events in media files - Google Patents

Abstract

Logging the events in the media file, providing a logger tool for allowing the user to view the media in multiple ways and capture and review key events in the media file; bars and tones; Tracking and logging events in the media file by adding information to the media file including slate, content, logo, commercial black, quality control issues, subtitles and caption location.
[Selection] Figure 3A

Description

[Cross-reference with related applications]
This application is filed on Feb. 11, 2011, co-pending US Provisional Patent Application No. 61 / 429,720 entitled “Logger Technology” filed Jan. 4, 2011. US Patent Application No. 13 / 026,134 entitled “LOGGING EVENTS IN MEDIA FILES” and “Tech Logger” filed on September 13, 2011 Claims the benefit of the priority of US Provisional Patent Application No. 61 / 534,275. The disclosure of the aforementioned application is incorporated herein by reference.

  The present invention relates to event logging, and more particularly to displaying and logging events associated with media files.

  Manually creating an event list for a video file is cumbersome and error prone. If you review a tape or video file with one tool and manually enter the timecode with another tool, mistakes and inconsistencies may occur. This type of problem can make it more difficult to reliably process video files in the library.

  Embodiments of the present invention display audio / video from a data file and attach metadata to the file.

  In one implementation, a method for logging events in a media file is disclosed. The method includes providing a logger tool to allow a user to view media in multiple ways and capture and confirm key events in a media file; bars and tones, slate, Tracking and logging events in the media file by adding information to the media file, including content, logos, commercial black, quality control issues, subtitles and caption locations.

  In another implementation, a logger tool for logging events in a video is disclosed. The logger tool includes an adjustable film strip of thumbnails representing at least a portion of the video, at least one audio waveform of the video, video timing information, a plurality of events associated with the video and within the video of the event Reuse from video, position, at least one interface for displaying and playing video and at least one audio waveform, at least one interface for creating, editing and deleting events for video It includes at least one interface for creating possible clips and at least one interface for editing, importing, and copying events or groups of events within a file or across multiple files.

  In yet another implementation, a non-transitory tangible storage medium that stores a computer program for logging events in a media file is disclosed. This computer program allows users to view media in multiple ways and capture and review key events in media files, as well as bars and tones, slate, content, logos, commercial black, quality control issues, Includes executable instructions that cause the computer to track and log events in the media file by adding information to the media file, including subtitles and caption locations.

  Other features and advantages of the present invention will become readily apparent to those skilled in the art after reviewing the following detailed description and accompanying drawings.

4 is a screenshot of a logger queue page, according to one implementation of the invention. A snapshot of a logger's video page that is reached by clicking on the title containing the media file name. FIG. 4 is a snapshot of a stack view in a logger's video page, according to one implementation of the invention. It is a screen shot of a list of filters displayed when the filter tab is selected. It is a figure which shows the video information displayed when the video information tab is selected in a tab area | region. It is a figure which shows the logo information displayed when a logo is selected in a tab area | region. It is a figure which shows a computer system and a user's expression. It is a functional block diagram which shows the computer system which hosts a logger. FIG. 6 is a flow diagram illustrating a method for logging events in a media file according to one implementation of the invention. FIG. 6 shows an implementation of a logger user interface for presenting, selecting, adapting, matching, logging, etc. of audio / video elements (frames, tracks, segments, clips, waveforms, filmstrips, events, etc.). FIG. 6 shows an implementation of a logger user interface for presenting, selecting, adapting, matching, logging, etc. of audio / video elements (frames, tracks, segments, clips, waveforms, filmstrips, events, etc.). FIG. 6 shows an implementation of a logger user interface for presenting, selecting, adapting, matching, logging, etc. of audio / video elements (frames, tracks, segments, clips, waveforms, filmstrips, events, etc.). FIG. 6 shows an implementation of a logger user interface for presenting, selecting, adapting, matching, logging, etc. of audio / video elements (frames, tracks, segments, clips, waveforms, filmstrips, events, etc.). FIG. 6 shows an implementation of a logger user interface for presenting, selecting, adapting, matching, logging, etc. of audio / video elements (frames, tracks, segments, clips, waveforms, filmstrips, events, etc.). FIG. 6 shows an implementation of a logger user interface for presenting, selecting, adapting, matching, logging, etc. of audio / video elements (frames, tracks, segments, clips, waveforms, filmstrips, events, etc.). FIG. 6 shows an implementation of a logger user interface for presenting, selecting, adapting, matching, logging, etc. of audio / video elements (frames, tracks, segments, clips, waveforms, filmstrips, events, etc.). FIG. 6 shows an implementation of a logger user interface for presenting, selecting, adapting, matching, logging, etc. of audio / video elements (frames, tracks, segments, clips, waveforms, filmstrips, events, etc.). FIG. 6 shows an implementation of a logger user interface for presenting, selecting, adapting, matching, logging, etc. of audio / video elements (frames, tracks, segments, clips, waveforms, filmstrips, events, etc.). FIG. 6 shows an implementation of a logger user interface for presenting, selecting, adapting, matching, logging, etc. of audio / video elements (frames, tracks, segments, clips, waveforms, filmstrips, events, etc.). FIG. 6 shows an implementation of a logger user interface for presenting, selecting, adapting, matching, logging, etc. of audio / video elements (frames, tracks, segments, clips, waveforms, filmstrips, events, etc.). FIG. 6 shows an implementation of a logger user interface for presenting, selecting, adapting, matching, logging, etc. of audio / video elements (frames, tracks, segments, clips, waveforms, filmstrips, events, etc.).

  Some implementations disclosed herein display audio / video from a data file and attach metadata to the file. After reading this description, it will become apparent how to implement the invention in various other implementations and applications. However, although various implementations of the invention are described herein, it should be understood that these implementations are presented by way of example only and not limitation. Accordingly, the following detailed description of various other implementations should not be construed as limiting the scope or breadth of the present invention.

  In one implementation, a software tool called a logger is used to log events in media files such as moving images. This logger tool allows users to watch videos in multiple ways and add information to the file including the location of bars and tones, slate, content, logo, commercial black, quality control issues, subtitles and captions, Provides a user interface that allows events to be tracked and logged. The logger tool allows the user to capture and confirm key events in the media file that are required to enable downstream automated post-production processing and workflow.

  In one implementation, the user interface provides access to the media file and also provides an interface for creating, tracking, and editing events for the media file. This user interface can automatically present events and associate them with the correct location of the media file, thereby improving data throughput and quality. Events can be generated manually by the user in the logger tool or by importing an externally created list or table of events. These events can then be associated with media files within the logger tool. For example, a user can import a quality control report into a logger tool and use the logger tool to create an event for a file that matches a quality control entry. In another implementation, the logger tool may present information and views regarding frame matching and / or differentiation based on imported matching and / or differentiated data.

  FIG. 1 is a screenshot of a logger queue page 100 according to one implementation of the present invention. The queue shown in the queue page 100 is designed to track the progress of the media file through each state of the logging process.

  In the implementation shown in FIG. 1, the logger queue page 100 includes a status bar 110, an item counter 112, a “sort by drop-down” 114, a search field 116, “expand / collapse all” 118, a title 120, and an identifier 130. , Expansion 122, thumbnail 124, folding 126, file description 128, “add movie” field 132, and items / fields such as logout. Clicking on the status bar 110 can display all files in the selected state including loading, logging preparation, user work, review preparation, completion and rejection. The item counter 112 displays the number of files indicating this selected state. When the “sort by drop-down” item 114 is clicked, an identifier (title, status, task ID, date added, feature, user assignment, kit ID, etc.) for arranging and viewing the file can be selected. The search field 116 displays files that satisfy the entered keyword criteria. By clicking on the “Expand / Collapse All” item 118, additional file information (ie, file details) of all current files can be expanded or collapsed. The title 120 contains the file name, which when clicked takes you to the logger's video page. The identifier field 130 indicates file-specific identification information. When the expansion icon 122 is clicked, additional file information can be displayed. Thumbnail 124 shows a single frame selected to visually represent the file. When the fold icon 126 is clicked, the additional file information can be hidden. File description 128 shows additional technical file information. The “add movie” field 132 is used to load selected files that are not currently in the logger tool.

  FIG. 2 is a snapshot of a logger's video page 200 that is reached by clicking on a title that includes a media file name (such as 120 in FIG. 1). In one implementation, the logger's video page 200 includes parts, controls, and instructions used to view, confirm, and obtain events. For example, a logger's video page 200 may include an adjustable thumbnail film strip representing all or part of a video file, video audio waveform, video including timing information (timecode, tape timecode, frame number, etc.) Events related to video and the position of these events in the file (by timecode, etc.), interface for displaying and playing video and audio waveforms, for creating, editing and deleting video file events Interface, interface for creating reusable clips (such as creating a new logo) from a video file, for editing, importing and copying events or groups of events within one file or across multiple files Inn of Over the face, to provide / display the interface with the user via a web browser.

  In the implementation shown in FIG. 2, the video page 200 includes a page selector 210, an event overview 212, a master strip 214, a looking glass 216, an event strip 218, an event indicator 220, an anchor 222, an audio waveform 224, an audio magnification 226, and a standard time. It includes parts such as code 228, tape time code 230, frame number 232, player control 234, magnification slider 236, volume slider 238, player pane 242 and stack view 240, controls and instructions. The page selector 210 is used to select a page to watch (such as a queue, video or audio). The event overview 212 represents the portion of the file that contains the event. In one example, known events and unknown events are represented in different colors.

  Master strip 214 represents the entire file timeline from start to end. The looking glass 216 is located in the master strip 214 and enlarges the enclosed portion of the file in the event strip 218. The default position of the looking glass 216 when a new file is opened includes the entire file. The event strip 218 is an enlarged portion located in the looking glass 216 on the master strip 214 that can divide the file into events. The event indicator 220 is a stroke indicating the outline of each individual event. For example, the first thumbnail in event indicator 220 is the first frame of the event, and the last thumbnail in event indicator 220 is the last frame of the event. Anchor 222 is represented by a vertical line across the event strip 218 and the audio waveform, and represents a position in the file. This file position is displayed in the player pane 242. The player control 234 is a button for controlling basic playback tasks such as playback, pause, fast forward, and rewind. The magnification slider 236 adjusts the size of the looking glass 216 that can increase or decrease the amount of the master strip 214 displayed in the event strip 218. The player pane 242 displays a frame located on the right side of the anchor 222. Stack view portion 240 is the action center of logger's video page 200.

  In one implementation, the logger video page 200 can be navigated using the parts, controls and instructions described above. For example, the master strip 214 can be navigated to display different portions of the file in the event strip 218 by clicking and dragging the looking glass 216 left and right. The size of the looking glass 216 is adjusted by moving the magnification slider 236 toward minutes to increase the size of the looking glass 216 and moving toward the frames (frame) to reduce the size of the looking glass 216. be able to. In another example, event strip 218 can be navigated by clicking on anchor 222 and dragging left or right along event strip 218. The event strip 218 can be dragged left and right while the anchor 222 remains in the same position. When the event strip 218 is dragged, the looking glass 216 in the master strip 214 is also moved. Clicking on the desired event on the event strip 218 moves the event strip 218 so that the anchor 222 is positioned before the first frame of the selected event. Either pressing the Enter key or clicking on an event on the event strip 218 can expand the event at the center of the strip 218. You can also use the up and down arrow keys to move to the next or previous event. In yet another example, selecting an event in the stack view 240 moves the event strip 218 so that the anchor 222 is positioned before the first frame of the selected event, expanding the event at the center of the event strip 218. Is done.

  FIG. 3A is a snapshot of a stack view 300 of a logger video page 200 according to one implementation of the present invention. The stack view 300 shows completed tasks, as well as filter tools and other information. In the implementation shown in FIG. 3A, the stack view window 300 includes track information 310 (including a track drop-down button 312 and an add track button 314), a filter 332 (see FIG. 3B), and video information 334 (see FIG. 3C). And a tab 330 indicating a logo 336 (see FIG. 3D) and an event row 320. As described above, known events and unknown events can be represented by different colors 322. The stack view pane 300 further includes an “all annotation expansion function” 316 and an “annotation expansion function” 318. The track information 310 portion provides options for importing quality control reports, captions, subtitles or script alignments, copying from selected titles, or creating unknown default events that represent the entire file.

  FIG. 3B is a screen shot of a list of filters 332 displayed when the filter tab is selected. By selecting one or more filters from the list of filters, events included in individual tracks can be viewed by category. Therefore, by selecting a filter, only events of the selected filter category can be displayed in the track. By pressing a plurality of filter buttons, a plurality of filters can be turned on at the same time so that events in the selected filter category can be viewed.

  FIG. 3C shows video information 334 displayed when the video information tab is selected in the tab area 330. Video information 334 provides information such as frame rate, language, and other related video information.

  FIG. 3D shows logo information 336 displayed when a logo is selected in the tab area 330. To see the logo in the stack view 300 logo window, click the logo button below the track name. To search for a logo, place the cursor in the search field and click with the logo window open. To create a new logo, create an event that represents the logo from start to finish, click on the "Edit Mode" icon in the stack view of the event that contains this logo, Select the corresponding logo type (such as logo, production company logo, distribution logo, or work logo), place the anchor on the frame that most accurately represents the logo in the event strip, and click the “OK” button, or Double-click the correct event type in the event category menu, and when the logo window appears on the stack view, enter the desired logo name in the search field and click the "Create New" button to add a new logo in the stack view. Appears, click the “Submit” button to assign the newly created logo to the event. Step to run.

Returning to FIG. 3A, each event row 320 displays the assigned event type, event description, duration, and start and end. Based on the highlighted measurement field, the duration measurement and the start and end information are displayed. Each event type is represented by a different color 322 in the “Event Type” column in the stack view 300. Available event types are defined in Table 1 shown below.

  Each track contains at least one event that represents the entire file from start to end, or many imported or copied events in combination include the entire file. Each new event is part of an existing event. Therefore, to create a new event, place the anchor on or just before the first frame of the event to be created in the event strip. As a result, the first frame of the event is displayed in the player pane. Next, choose to split the current event into two events. The frame on the right side of the anchor now represents the first frame of the new event, and the frame on the left side of the anchor represents the last frame of the previous event. This event is automatically classified as unknown.

  FIG. 4A shows a representation of computer system 400 and user 402. User 402 uses computer system 400 to log events in media files such as moving images. The computer system 400 stores and executes the logger 490.

  FIG. 4B is a functional block diagram illustrating a computer system 400 that hosts a logger 490. The controller 410 is a programmable processor and controls the operation of the computer system 400 and its components. Controller 410 reads instructions (in the form of a computer program, etc.) from memory 420 or built-in controller memory (not shown) and executes these instructions to control the system. Controller 410 provides logger 490 as a software system that, when executed, enables logging of events in media files. Alternatively, this service may be implemented as a separate hardware component within controller 410 or computer system 400.

  Memory 420 temporarily stores data used by other components of computer system 400. In one implementation, the memory 420 is implemented as a RAM. In one implementation, the memory 420 also includes long-term or permanent memory such as flash memory and / or ROM.

  The storage device 430 temporarily or long-term stores data used by other components of the computer system 400, such as data used by the logger 490. In one implementation, the storage device 430 is a hard disk drive.

  Media device 440 accepts removable media to read data and / or write data to the inserted media. In one implementation, for example, the media device 440 is an optical disk drive.

  User interface 450 includes components for accepting user input from a user of computer system 400 and presenting information to the user. In one implementation, the user interface 450 includes a keyboard, a mouse, audio speakers, and a display. The controller 410 uses the input from the user to adjust the operation of the computer system 400.

  The I / O interface 460 includes one or more I / O ports for connecting with corresponding I / O devices such as external storage devices (such as printers or PDAs) or auxiliary devices. In one implementation, the ports of the I / O interface 460 include ports such as USB ports, PCMCIA ports, serial ports, and / or parallel ports. In another implementation, the I / O interface 460 includes a wireless interface for communicating wirelessly with an external device.

  The network interface 470 includes wired and / or wireless network connections, such as RJ-45 or “WiFi” interfaces that support Ethernet connections (including but not limited to 802.11).

  Computer system 400 includes additional hardware and software typical of computer systems (such as power systems, cooling systems, operating systems, etc.), but these components are illustrated in FIG. 4B for simplicity of illustration. Not shown. In other implementations, different configurations of computer systems (such as different bus or storage configurations or multiprocessor configurations) may be used.

  FIG. 5 is a flow diagram illustrating a method 500 for logging events in a media file according to one implementation of the invention. In the illustrated implementation, the method includes configuring the logger tool in box 510 so that the user can view the media in multiple ways (box 512). The user also captures and confirms key events in the media file in box 514. In box 522, track events in the media file in box 520 by adding information to the media file including the location of bars and tones, slate, content, logo, commercial black, quality control issues, subtitles and captions. Logging.

Various implementations can include, but are not limited to, one or more of the following items.
Provide an adjustable thumbnail film strip that represents all or part of the video fileDisplay video audio waveformVideo with timing information (timecode, tape timecode, frame number, etc.) • Displaying events related to video and their location in the file (by timecode, etc.) • Providing a UI (User Interface) to control the display and playback of video and audio waveforms Provide a UI for creating, editing, and deleting events for video files. Provide a UI for creating reusable clips from video files (such as creating a new logo). Doing · Within a file or across multiple files Provide a UI for editing, importing and copying events or events • Automatically creating selected events (such as commercial black or slate) by analyzing files Providing UI and operations for frame matching that allow frames within or across files to be matched-Providing UI and operations for audio adaptation that find similarities and differences in audio waveforms・ Provide UI and operations for creating audio components ・ Export frame match data to reference video of Advanced AAF (Advanced Authoring Format) / EDL (Edit Decision List) or Quicktime (Quicktime) Provide UI and action for AVID export function Provide UI and action for on-screen annotation (ie hand-draw on frame) in video UI UI and action for QC report generation Provide UI and operation for auto text in image detection Provide UI and operation for speech-text processing and result display with editing capability UI for manual transcription tool Provide an interface to the user via a web browser, instead of or in addition to using file downloads and local copies, and audio through a logger using streaming from the server Providing video

  In one implementation, the logger includes components to support a set of functions that allow a user to match frames from the same or different video files. The basic algorithm of this frame matching function is based on the basic concept known as the law of absolute difference, and compares the positive and negative frames to determine the relevance of the fit, and the prescribed threshold value Returns results based on. This feature provides the user with the ability to create a textless master and a foreign language text master by matching the insert to the original program. In one example, the first file contains the original video, and the second file contains an insert, ie a corresponding group of frames in the original (such as a frame containing text localized for a particular language), Contains a group of frames to be replaced. The user may use frame matching to identify the original frame that matches the insert frame and indicate (manually and / or automatically) which frame in the original video file should be replaced with the frame from the insert file. it can. The logger can then output a new version using the original frame with the selected frame replaced by the selected insert frame. Alternatively, the logger can create a file (such as a reference table) that guides playback between the original file and the inserted file. The user can then create another file for a different language using a different insert file.

  The UI provides the user with the ability to see the resulting playback side-by-side in the player pane, and the user can repair and adjust the default storyboard mode and if there is an inconsistent mismatch between the inserted and original frames Includes a view of possible "Precision" mode. Also, the user can preview these versions in “real time” and switch languages. This allows these “virtual edits” to be previewed before the foreign language master is rendered into an actual file. During playback in the preview area, the insertion is added to the moving image on the fly by keying off the EDL created during the matching process. The user can also select audio and text elements and render them as part of the preview.

  In one implementation, the logger includes components to support a series of functions for audio adaptation and audio analysis that allow users to compare waveforms to each other to find similarities and / or differences. . This technology is part of the logger's feature set and part of the audio UI. In another implementation, audio analysis and / or audio UI can be implemented in a separate program or component. In the subsequent figure, an aspect of the audio UI is shown. The user can select a “gold” reference channel that is a channel to which all other channels fit (eg, offset or shifted to synchronize). When the result is returned, this audio channel is fixed in place and the offset value is recorded. Thereafter, the user confirms that the fitting result is accurate and fixes the component. When a new component is taken into this title, this automatic fitting process is executed and only the unfixed component is analyzed.

  In another implementation, the logger includes components to support a series of functions for creating audio components. For example, a user can combine multi-part audio components into one part to render 6 reels of audio in the form of a long playing file. In one implementation, the creation of this audio component is also part of the logger's audio UI. The audio UI provides users with the ability to capture multipart audio components, adapt them, and render new components. The resulting components can then be used in distribution systems or other post-production workflows. One implementation for creating audio components also provides functions such as sample rate conversion, sync pop removal, basic envelope and real-time preview.

  The above description of the disclosed implementations is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these implementations will be readily apparent to those skilled in the art, and the general principles described herein may be applied to other implementations without departing from the spirit or scope of the invention. You can also Therefore, additional embodiments and modifications are also included in the scope of the present invention. For example, these examples focus on displaying and logging videos, but on other videos, such as television shows, Internet videos, or user-generated content, or radio or podcasts, Alternatively, the logger can be specialized for audio, such as games or other content such as text, or a combination thereof (video, audio and text matching and matching for script matching and tracking, etc.). In a particular logger implementation, not all functions of each example are necessarily required. Further, it is to be understood that the description and drawings presented herein represent subject matter that is broadly contemplated by the present invention. Furthermore, it is to be understood that the scope of the present invention fully encompasses other implementations that will be apparent to those skilled in the art, and thus is not limited by anything other than the appended claims.

Claims (16)

A method of logging events in a media file,
Providing a logger tool for allowing a user to view media in multiple ways and capture and confirm key events in the media file;
Tracking and logging events in the media file by adding information to the media file including bar and tone, slate, content, logo, commercial black, quality control issues, subtitles and caption location;
A method comprising the steps of: The media is a video,
The method according to claim 1. The key event is necessary to enable downstream automatic post-production processing and workflow,
The method according to claim 1. The logger tool automatically presents the event and associates the event with the correct location of the media file.
The method according to claim 1. Tracking and logging events in the media file includes
Importing a quality control report having quality control entries into the logger tool;
Generating an event for the media file that matches the quality control entry;
The method of claim 1, comprising: The event is manually generated by the user in the logger tool,
The method according to claim 1. A logger tool for logging events in a video,
An adjustable film strip of thumbnails representing at least a portion of the video;
At least one audio waveform of the video;
Timing information of the video;
A plurality of events associated with the video and the location of the event within the video;
At least one interface for displaying and playing back the video and the at least one audio waveform;
At least one interface for creating, editing and deleting events for said video;
At least one interface for creating a reusable clip from the video;
At least one interface for editing, importing and copying events or groups of events within a file or across multiple files;
A logger tool characterized by comprising: The timing information includes at least one of a standard time code, a tape time code, and a frame number.
The logger tool according to claim 7. The at least one interface for creating a reusable clip includes at least one interface for creating a new logo;
The logger tool according to claim 7. The at least one interface for displaying and playing the video and the at least one audio waveform includes at least one user interface via a web browser;
The logger tool according to claim 7. The at least one interface for displaying and playing the video and the at least one audio waveform includes a master strip, a looking glass, an event strip, an event indicator, an anchor, an audio magnification, a player control, a magnification slider, a volume slider, Including at least one of a player pane and a stack view,
The logger tool according to claim 7. A non-transitory tangible storage medium storing a computer program for logging events in a media file, the computer program comprising:
Allowing a user to view media in multiple ways and capture and confirm key events in the media file;
Tracking and logging events in the media file by adding information to the media file including bars and tones, slate, content, logo, commercial black, quality control issues, subtitles and caption location;
Including executable instructions that cause the computer to
A non-transitory tangible storage medium characterized by the above. The media is a video,
The non-transitory tangible storage medium according to claim 12. The key event is necessary to enable downstream automatic post-production processing and workflow,
The non-transitory tangible storage medium according to claim 12. Executable instructions that cause a computer to allow a user to view media in multiple ways and capture and confirm key events in the media file automatically present the events and Including executable instructions that cause a computer to associate with the correct location of the media file;
The non-transitory tangible storage medium according to claim 12. Executable instructions that cause the computer to track and log events in the media file are:
Importing a quality control report with quality control entries;
Executable instructions that cause a computer to generate an event for the media file that matches the quality control entry;
The non-transitory tangible storage medium according to claim 12.