FR2768843A1 - USER INTERFACE SYSTEM AND USER INTERFACE METHOD - Google Patents

Abstract

The present invention describes a system and method for providing a configurable user interface or GUI (705) in relation to audio / video (A / V) data. In one embodiment, a data descriptor engine (710) is coupled to the A / V file system (715) which contains the A / V data. A type of user interfacing on the GUI is identified. A display generator (720) accesses the A / V file system and generates a display of the A / V data based on the type of user. A menu builder (725) generates user selectable options / features for display in the GUI, the options / features displayed depending on the type of user. A manipulation engine (730) receives user input and determines operations to be performed, the determination being based on the type of user.

Description

FIELD OF THE INVENTION

  The present invention relates to the field of audio / video editing. More specifically, the present invention relates to a method and apparatus for editing digitized audio / video data over a network.

DESCRIPTION OF THE PRIOR ART

  Currently, audio / video editing is most frequently used in accordance with the system shown in Figure 1. An editing workstation nowadays includes local discs on which audio / video information is stored. Editing can be done via a display screen, but it is limited to locally stored audio / video data. This type of system ensures insufficient functionality to edit a large amount of

  audio / video data stored on multiple workstations.

  More specifically, this system is insufficient because current editing workstations do not perform border editing of workstations on networks. Audio / video data on each workstation must be edited separately and the output of the edit can then be entered into a mixing or mixing console to create a final audio / video output. If there is any problem in connection with the final audio / video output, then the tedious process of editing the various audio / video data tracks residing on each separate workstation must be repeated. This process continues until the

final output is satisfactory.

SUMMARY OF THE INVENTION

  The present invention describes a system and method for constituting a configurable user interface in relation to audio / video (AN) data. According to one embodiment, a data descriptor engine is coupled to the AN file system which contains the AN / V data. A type of user interface on the graphical user interface (GUI) is identified. A display generator accesses the AN file system and generates a display of AN data according to the type of user. A menu builder generates user selectable options / features to be displayed in the GUI, the options / features displayed depend on the type of user. A manipulation engine receives user input and determines operations to be performed, the determination being

based on user type.

BRIEF DESCRIPTION OF THE DRAWINGS

  The present invention is illustrated by way of an example and not by way of limitation in the figures of the appended drawings in which identical reference indices relate to similar elements, and among these drawings: FIG. 1 represents a method of audio editing of the prior art; FIG. 2 represents an overall view of an embodiment of the present invention; Figure 3 shows in more detail the software modules according to an embodiment of the present invention; Figure 4 shows a data structure which defines a track according to an embodiment of the present invention; FIG. 5A to 5E represent various editing operations carried out on an audio data track; FIG. 6 represents an example of the graphical user interface which can be used to carry out an edition; Figure 7 shows an embodiment of a system which generates user configurable interfaces according to the teachings of the present invention Figure 8 shows an example of a graphical user interface for a different type of user; and Figure 9 shows another example of an interface

  graphical user for a particular user.

  DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

  The present invention is a method and apparatus for generating a user configurable interface. In the

  detailed description which follows, many specific details are put

  in order to ensure a complete understanding of the present invention. It will be apparent to those skilled in the art that these specific details need not be used for the practical implementation of the present invention. In other cases, well-known structures, interfaces and processes have not been presented in detail so as not to

  not unnecessarily obscure the present invention.

  FIG. 2 represents an overall view of an embodiment of the present invention. Although the following sections describe an audio editing process, the same technique can be successfully applied to a video editing process. As shown, the system according to the present invention comprises at least one editing station coupled to a plurality of remote stations (remote stations 1 to N). Remote stations 1 to N contain audio / video data stored in digitized form on one or more storage disks. According to this embodiment of the present invention, a sound engineer can edit the digitized data which reside on any of the remote stations 1 to N from a single location, namely an editing station 200. sound engineer can also edit the digitized data on remote stations 1 to N for example by means of a copy of audio information from a track on a station and by means of pasting the audio information cut on a track on another station. Editing operations on remote stations are described in a way

more detailed below.

  The editing station 200 can be a personal computer (PC) or any other type of system making it possible to display a graphical application, to accept an input via the graphical application and to communicate the input up to a remote location. The remote stations 1 to N may also include a PC or a private audio / video information storage / editing station such as a player-recorder system from Sony Corporation (registered trademark). Remote stations 1 to N can include slots for multiple removable disks such as lomega Jaz disks, which increases the storage available for the remote station and makes it easy to move data stored from a

  remote location to another.

  FIG. 3 represents the software modules according to an embodiment of the present invention. The editing station 200 includes an editing application 302, dynamic link libraries (DLL) 304 and a remote procedure call (RPC) mechanism 306. The editing application 302 allows the engineer sound to perform editing functions via a graphical user interface (GUI). The mechanism of RPC 306 constitutes a communication link between

  the editing station and the remote stations 1 to N (as seen in FIG. 2).

  According to an embodiment of the present invention, the RPC mechanism 306 is an RPC mechanism based on Microsoft Windows NT (registered trademark). Other RPC mechanisms or other network mechanisms for remote communication may

also be used.

  FIG. 3 also represents an example of the software modules on a remote station presented by way of example according to an embodiment of the present invention. The remote station shown as an example includes an RPC mechanism 308 (which corresponds to the RPC mechanism 306 on the editing station 300), DLLs 310 and a control mechanism 312 which interacts with a processing system. digital signal (DSP) 314. A sound engineer interacts with the remote stations 1 to N from the editing station 200 via a GUI, the functions specified via the GUI are communicated between the editing station 200 and the stations at distance 1 to N via the RPC mechanisms 306 and 308, and the DLL 310 and the DSP system carry out all the processing necessary for the implementation

performs specified functions.

  FIG. 4 represents the data structure of an audio track on a remote station presented by way of example according to an embodiment of the present invention. An audio track is a static representation of audio information that specifies the location of the audio data on a disc, the organizational data needed to properly play the audio data, and a variety of other parameters associated with the audio data. According to an embodiment of the present invention, the various parameters associated with audio data are

each specified in a list.

  The various lists shown in Figure 4 therefore correspond to a single audio track. This track is represented by a data structure which links all the lists together in a logical way. By using a single data structure to link all lists together, an embodiment of the present invention allows all data associated with a particular track of audio information to be edited collectively through a common editing interface. Consequently, for example, if a sound engineer decides to edit part of an audio track, all the lists associated with this part of the audio track can be edited simultaneously unlike the systems of interior art in which various aspects of the track may have to be edited

separately.

  The mechanism which performs edits on the data structure in the form of a track in FIG. 4 involves several components. According to one embodiment, the components are implemented as C ++ classes. The primary components are: tracks, event lists and events. Each of these components is shown in Figure 4 and is described in a way

  more detailed below. Although the following description assumes

  a C ++ implementation, the same functionality, or a similar functionality may be implemented in other programming environments. A "track" is a public object that is used by audio applications to represent static trains of audio information as well as other data on a time basis. A track is implemented as a named object (that is, it has a chain name) which maintains a collection of event lists (as described below). A track exposes an editing interface which is applied to each list of events. According to one embodiment, a track maintains at least one list

  of events, namely the list of sounds (see Figure 4).

  "Event lists" are linked data structures

  doubly which maintain two basic types of events.

  The default event is called a "silence" event and the other event is called a "non-silence" event. There is a separate event list for each type of silence / non-silence event combination. For example, as shown in FIG. 4, solid circles along the time axis of the audio information represent the occurrence of a sound while non-solid circles represent silence. Other lists are also being considered. For example, as shown in Figure 4, the gain list represents the magnitude of the gain that is applied to particular parts of an audio track. For example, the solid circle indicates that a certain scalar gain value should be applied while the non-solid circle represents that a unit gain should be applied. Figure 4 also shows "other lists", which, as will now be apparent to those skilled in the art, may include a variety of parameters including parameters related to video information or other operations of editing. Each type of event maintains at least a length measured in terms of samples and in terms of other data necessary to define the behavior of the event. According to one embodiment, virtually the entire behavior of editing event lists is implemented

according to a C ++ class.

  Each type of event maintains two important virtual functions: SlideStart (...) and SlideEnd (...). When new non-silence events are created, each event must implement these functions again. This is due to the fact that when a start time and / or an end time of an editing operation does not correspond to the start of an event, a "connection" must be created. This is accomplished by copying the appropriate event by "dragging" the end of the source event and then "dragging" the start of the copied event. This sliding mechanism is

  shown in more detail below (see Figure 5C below)

  after). Event lists are time-based data structures that contain linked events, each event

  maintaining at least a length measured in terms of samples.

  The sum of the lengths of all events plus the start offset (as explained in more detail below) is equal to the length of the event list. By default, the first event begins at absolute zero sample time unless a start offset is present, in which case the first event begins at the value of the start offset. The absolute time (measured in samples) at the start of an event is measured by adding the lengths of all the events preceding it, including the

start offset.

  In the context of this discussion, tracks and event lists are used interchangeably. The tracks represent multiple lists of events. A work event list is a list that contains the events that are built up to model a specific type of audio behavior. The work event lists are initially created with a very important silence event. Once the list of work events has been created, editing operations can be applied to it. Unique events can be added to the list by calling an appropriate Add (...) function. In addition, according to an embodiment of the present invention, six primitive editing operations can be applied. These are: Clear (, either Reset 0 Cut (, or Cut (Copy 0, or Copy O Paste (, or Paste 0 Insert (), or Insert ()

  Insert 0 (space), i.e. Insert O (space).

  Tracks, event lists, and any other objects that shape editing use the terms mentioned above when defining an editing interface. The editing operations, when they apply to lists of events, are the subject of a discussion below and are represented in FIGS. 5A to 5E. According to another embodiment of the present invention, other

  editing operations can also be performed.

  "Clear" replaces an event section with silence without affecting the instants of the events after the end time. Clear is actually an overwrite operation. The "Clear" editing operation is shown in FIG. 5A. Paste operations paste a source list into a destination list. The start and end times of the collage are determined by the source list. The paste operations overwrite, and it follows that the instants of the events after the end time remain unchanged, as shown in the

Figure 5B.

  The "Insert" operations insert a source list into a destination list. The insertion point is determined by the start time of the source list. The instants of all events at and after the insertion point are offset by the length of the source list. This is described below in relation to Figure 5C. "Insert space" simply inserts a blank event list of specified length in place of a source list in a destination. "Cut" removes an event section from an event list. All events after the end time are shifted earlier in time by the length of the cut element, as shown in Figure 5D. Finally, the "Copy" operations copy a section of an event list while the source list remains unaffected, as shown in the

Figure 5E.

  FIG. 6 represents an example of a GUI screen which must be generated by an editing application 302 at an editing station 200. This GUI screen allows the sound engineer to specify

  a range for editing functions to be performed on the audio data.

  For example, the screen in Figure 6 shows a display of seven separate tracks (FXL, FXC, FXR, FXS, FoleyL, FoleyC and FoleyR). These seven tracks can reside on a single station or on separate stations on a network. This ability of a sound engineer to access multiple stations on a network for simultaneous editing from a single remote location represents a significant advance on the processes of the interior art of information publishing. audio / video. According to an embodiment of the present invention, the sound engineer can carry out multiple editing operations on as many remote stations as he wishes, which gives the sound engineer the flexibility to edit a quantity important audio information in a very simple way. Therefore, for example, a sound engineer can perform any of the operations

  above described in the backlit part of the screen.

  The backlit portion corresponds to a portion of audio data on multiple tracks of audio data, each track comprising of

multiple lists.

  The visual representation of audio information gives the audio engineer great flexibility in specifying editing functions. For example, the sound engineer can highlight a predetermined part of the screen and can specify editing functions to perform on the whole of the highlighted area. This backlit area, as shown in Figure 6, can include both shaded and unshaded parts of the screen. The shaded parts indicate the active audio parts (of non-silence) of each track while the blank spaces indicate a silence. When the sound engineer specifies an editing function, the function is applied to the tracks,

  which allows the user to edit multiple tracks simultaneously.

  More significantly, these tracks may reside on separate machines on the network as opposed to interior art in which an editing station can only edit data that

reside at a local station.

  This ability to perform digitized audio / video editing from a single editing station on a network provides significant practical benefits that go beyond simply making the process more efficient. In a common film editing installation, for example, various editing studios may each contain multiple editing stations in order to edit different parts of audio / video data, the output of all editing stations being entered in a mixing or mixing console. In the event that a machine is reassigned to a different studio, the entire editing station must be physically moved to the new location. In contrast, according to an embodiment of the present invention, the remote physical stations do not have to be moved. Instead, the remote stations can reside at any location, and each studio can simply be assigned a predetermined set of remote stations. In this way, the number of remote stations assigned to a studio can vary on a daily basis or even

  within the hour without significant problem.

  In another embodiment, the data structure can be used to support a variety of GUIs that are customized for different types of users. For example, in the film industry, different individuals work on the production of

  sound, music composition, music editing and sound editing.

  The work carried out by the different individuals involves access to the

  entire data structure or only part of it

  Ci, an example of which is shown in Figure 4. However, these

  different people are used to different user interfaces.

  For example, people producing sound or composing work with staves, measures and beats to identify the particular parts to which an operation relates. The sound editors including mixers or mixers work with an SMPTE code. Similarly, if audio information for non-video applications has been processed in this system, the different parts of the audio information can be identified by a MIDI code. According to this other embodiment, the

  system further includes a data description engine, such as

  shown in Figure 7. With reference to Figure 7, the GUI 705 which

  contains the A / V data is created by the description engine of

  710 data by accessing the A / V database 715. The

  data description 710 can be implemented as

  software intended to be run on existing processors or on other processors. According to a variant, it can be implemented in a dedicated system constituted by hardware components or by software. The GUI generated depends on the type of user. Therefore, for example, the user may initially see himself

  ask the data description engine to identify the type

  GUI desired, for example sound editing, sound production, etc ...

  Alternatively, the system may implement a default type of GUI based on login or other user identification.

  The data description engine accesses the database

  A / V data 715 and the display generator displays the data of interest according to a representation configured for the type

  of user. This is shown graphically in Figures 6 and 8.

  As noted above, Figure 6 is a representation of a GUI configured for a sound editor. Figure 8 is a representation of a GUI for a duplicating device. It should be noted that the data from the different tracks are represented graphically in a different way. For example, with reference to FIG. 6, the sound editor is constituted as a graphic representation of the audio data (for example the area 655) in addition to the production of the identification of the particular audio information currently accessed and represented in each of the tracks represented. The display of the duplicating device also provides for the identification 805 of the audio data accessed; however, when the editing operations are not performed at a duplication station, the graphic representation of the audio data is not ensured. Similarly, duplication frequently uses offsets to offset different tracks from one another (for example for mixing or mixing); however, offsets are typically not used during the editing process. Consequently, offset zones are provided (for example in 810) in the GUI of FIG. 8

  and not in the GUI in Figure 6.

  The representations can be pre-generated and stored for example in the reference data file 735 or in any other medium in the form of memory or storage. The representation can be stored as descriptive data or as executable code (for example in Java or similar). He must

  be noted that there may be some community of representation.

  For example, time codes representing entry points, exit points etc ... (for example 666 to 690 and 830 to 836)

  can be the same, depending on the type of user.

  In addition, the menu constructor 725 (Figure 7) generates user selectable features / options / operations for the particular type of GUI. As in the case of data representations, the menus can be pre-generated and stored for example in the reference data file 735 or in any other medium in the form of memory or storage. The representation can be stored as descriptive data or as executable code (for example in Java or similar). Examples of the different features / options / operations selectable by user can be seen by referring again to FIGS. 6 and 8. In the GUI of the duplicating device, several options such as security "s" 812, record "R" 814, entry "1" 816, solo "S" 818 and silence "M" 820 can be envisaged. These options are implemented as buttons so that, when the user makes a selection, for example by placing the cursor on the button area and by "clicking" the mouse / marker, the function is validated. For example, the security button 812 implements a write protection mechanism over the entire track. The 814 record button initiates

  immediately recording the entry on the selected track.

  The 816 input button allows the user to hear what's at

  entry level (as opposed to the exit level).

  The mute button 820 mutes the selected track and the solo button 818 mutes all tracks except

of the selected track.

  In the editing GUI of Figure 6, similar functions are provided but there are some that are implemented differently in order to adapt to the way different users operate. As shown, buttons are provided for the solo 678, mute 680, input 682, and record 684 functions. However, unlike a duplication application, an editing application implements a tape which is based on a multi-track recording model. Therefore, if a user selects the recording function, the button will flash to indicate a wait to initiate a recording and will change to a solid color to indicate when a recording is initiated, as controlled by the user who selects a master record button (eg button 691) on the display. Once the master recording is selected, additional track recording buttons can be selected and immediately enabled. Also, if the master recording is selected again to turn off the master recording function, the track recording buttons will flash again to indicate a

  waiting for a recording state.

  Identical functions are also provided such as the tape control functions, for example, fast forward, reverse advance, play, stop etc ... (690 and 840) and that scrolling according to a time code (692 and 842 ). In addition, certain functions performed are only found in particular types of GUIs. For example, the programmable track selection buttons 844 and the selection buttons for all tracks 845 or no tracks 846 are very useful in the duplication application in which the user is manipulating many different tracks. at a time. Furthermore, graphically, the functions can be represented differently from what is defined by the menu constructor. For example, the gain value is represented as a slide icon in the edit GUI and as a numeric value in the duplicate GUI. Selecting the gain column or icon in the appropriate GUI will display the gain progress window as shown in Figure 9, allowing the user to adjust the gain using a slider graphic. It should be noted that Figure 9 fully shows another embodiment of a duplication GUI that can be configured by the user. In this illustration, tracks that separately identify an input and an output are used to simplify the information.

  displayed. Other embodiments are also envisaged.

  Once the user selectable data and functions are displayed, the user can apply input to the system to perform certain tasks. The user input is preferably processed through the manipulation engine (730, FIG. 7) which interprets the input and which generates instructions which are understood by the system. The advantage is that the user can use the particular nomenclature or the particular "language" typically used and that the system will understand what has to be done and how to perform the task. Therefore, some functions are unique for a particular type of user and some are common although referenced differently. For example, when producing sound, an operation is called "switching", when editing music, the same operation is called "verse" and when

  of an edition of the sound, the same operation is called "sound recording".

  The three terms refer to the operation of temporal timing from a determined point in order to record multiple versions of the same sound. Therefore, the data recording operation will be the same even if it is called differently by different types of users. The manipulation engine can be implemented in a variety of ways. According to one embodiment, a dictionary is constructed and stored for example on a storage device 735 intended to contain possible user functions and corresponding internal functions. The manipulation engine can then search the dictionary for the particular function in order to

  determine the internal function to be performed.

  As can be seen, the present illustration can be extended beyond the audio domain to examples of the video domain or to other types of audio or video users. Therefore, it is envisioned that the same system could support a wide variety of users without having to provide user hardware components.

  specialized for a particular type of user.

  Therefore, a method and apparatus for generating a user configurable graphical user interface are described. These specific arrangements and methods described herein are merely illustrative of the principles of the present invention. Many modifications in terms of form and details can be made by those skilled in the art without departing from the scope of the present invention. Although the present invention has been presented in connection with a particular preferred embodiment, this is not considered to be so limited. The present invention is

  limited only by the scope of the appended claims.

Claims (1)

CLAIM   1. System characterized in that it comprises: a graphical user interface (GUI) (705) an audio / video (A / V) file system (715) comprising A / V data; and   a data description engine (710) coupled between the GUI   and the AVN file system, said data description engine   comprising: a display generator (720) configured to apply A / V data in a form for display in the GUI for a particular type of user; a menu constructor (725) configured to generate user selectable options / features for display in the GUI, the user selectable options / features generated dependent on the particular type of user; and a manipulation engine (730) configured to receive user input and to determine an operation to be performed, the determination of the operation to be performed dependent on the particular type of user.