WO2010072747A2

WO2010072747A2 - Method, device, and system for editing rich media

Info

Publication number: WO2010072747A2
Application number: PCT/EP2009/067726
Authority: WO
Inventors: Jean-Christophe Mielnik
Original assignee: Thales
Priority date: 2008-12-23
Filing date: 2009-12-22
Publication date: 2010-07-01
Also published as: EP2368245A2; FR2940481B1; WO2010072747A3; FR2940481A1

Abstract

A method, a device and a system for editing rich media are provided. The production of rich media according to the invention includes the following steps: collecting native media and native synchronisation information and generating alternate formats referred to as proxy media; simultaneously editing the synchronised proxy media and the synchronisation information, during which an operator will place editing events on the synchronised proxy media which will enable the automatic generation, in a single iteration, of the edited version of the native media, and placing, deleting and/or moving the synchronisation information; and automatically, uniquely generating final rich media based on native media.

Description

Method, device and system for rich media editing

The invention relates to the production of enriched media content and more particularly enriched media for managing the intellectual and educational heritage of businesses.

Rich media content (or richmedia) is particularly suitable for the presentation of a face-to-face event organized in front of an audience. These are most often conference-type events, technical convention, educational presentation. Where appropriate, events can be organized specifically for the production of an enriched media, such as a training session or a commercial presentation.

Figure 1 shows an example of rich media. An enriched medium 10 generally comprises several frames 1 1, 12, 13 in which are placed media, which are synchronized either from the actions of a user, or from the temporal sequencing of the presentation. There is generally at least one first frame 1 1 hosting an audio and / or video stream type media, also called main media thereafter, and at least one second frame 12 hosting a medium such as transparencies, for example, called media secondary thereafter. In rich media the media is synchronized. The user can generally directly access a given portion of the enriched media through an index 13. In an enriched media media are heterogeneous, that is to say of a different nature. An enriched media comprises audio and / or video stream media and at least one other media of a different nature, for example a textual media transparencies. In summary, an enriched media is set of heterogeneous media synchronized with each other and can be viewed interactively.

Rich media production covers a very wide range of activities, from source creation, editing, indexing and scripting (defining the layout of the media) to dissemination and management (archiving, distribution, protection) of the enriched media heritage thus constituted.

The segmentation of these activities still corresponds today to a segmentation of trades and tools used, the upstream part of the production cycle mainly in the audiovisual field, the part swallows mainly in the field of computing and telecommunications.

As soon as the productions exceed a threshold of minimum complexity, requiring the post-production edition of one or more media, many round trips are necessary between these currently non-integrated activities. This results in significant development costs that do not allow companies to fully benefit from the potential of enriched media, including their unparalleled effectiveness as a vehicle for knowledge sharing and intellectual property management. Hence for example the few richmedia presentations on the Internet compared to the growing amount of videos.

In the current context of digitization of all stages of production, the persistence of the current segmentation of production activities can be explained, in part, by the use at each extreme of the production chain of two very different types of production. representation and manipulation of digital information on the other hand:

• Upstream, the most audiovisual-rich information possible, extremely resource-intensive, whose formats are specialized by type of media, in order to optimize the quality of signal processing.

• Downstream, due to the need to manipulate and distribute a large number of media simultaneously, more integrated and highly compressed information, supplemented with purely digital media (vector graphics and midi sequences for example), with a priority given to saving of resources and not the quality of the signal.

A first example of richmedia content gives priority to the quality of the media, not to their scripting. This is typically institutional communication, with limited interactivity, such as that found in most DVD titles (with a few exceptions, a small number of chapters, no indexation, detailed navigation, resulting in a considerable loss of the qualities of knowledge vector richmedia contents). For example, for richmedia production of a one-day conference, consisting of 10 richmedia presentations of half an hour each, it will be possible to select the presentation to view, but not access to a finer indexing within each presentation.

A second example of richmedia content gives priority to interactivity, to the detriment of the overall quality of the media. These are typically courses broadcast live from a university with a webcam, rebroadcast off-line but without editing.

FIG. 2 represents a diagram of the steps of an enriched media editing process according to the prior art. This process is generally composed of three main steps: the collection of native media 10,210, the edition of media 220 and the synchronization of these media 230.

At the Native Media Collection Stage 210, media are collected independently of each other with the sole exception of the audio and video of the filmed portions. The collection of native media is divided into two main categories:

15 - the creation 21 1 for presentation or animation type media, for example,

Capture 212, that is recording with at least one camera and a microphone for film-type media.

In either case, very precise writing work is required, to allow the subsequent synchronization of the media.

The capture is necessarily accompanied by an acquisition phase 213 which can be simultaneous (direct storage on hard disk in case of digital recording directly on disk) or delayed

(conversion and storage in case of digital recording on tape or

Digitization, conversion and storage, in case of analog recording).

During the step of editing the media 220, the different native media are edited separately. The edition depends on the type of each media. There are secondary media publishing 221 and audiovisual publishing 222.

30 The secondary media are mainly of the presentation type

(typically PowerPoint) or animation (typically flash). Depending on the degree of precision of the edition required, the edition of these media may require the availability of their authoring tool (for example for a media in a non-editable pdf type format, it will be necessary to have the editable source non-pdf 35 if there are errors to correct). After capture and acquisition, audiovisual-type media are edited using a virtual editing tool, such as Avid Media Composer or Final Cut Pro, based on the metaphor of a timeline and synchronized audio and video tracks on this timeline. Audiovisual media publishing usually includes the following tasks:

• A first task is to re-align the tracks in case the recording has been done in multi-camera without live mixing. For each registration, it is necessary to locate a registration point (most commonly: sound clap, accented consonant or photo flash, if not precise movement).

• A second task is to visualize in detail the recording of each conference, to identify the cuts to be made, then to make these cuts. It is on this occasion that the indexes that will be used in the synchronization step 230 are identified to create or modify hypermedia navigation links.

• A third task is to choose which image to show at each moment (choice of the plan) in case the recording has been done in multi-camera. • A fourth task is to add the necessary effects, such as fades at the beginning and end of each conference.

• A fifth task is to adjust the sound levels.

A sixth task consists in exporting the final result of this step 220 in a format adapted to the synchronization step 230. The synchronization step 230 of the media comprises their spatial integration and their time synchronization. The spatial integration makes it possible to position frames in which will be placed the media, which will be synchronized either from the actions of the user, or from the temporal sequencing. Frames hosting static secondary media allow synchronization by interactive (random) event. The user typically asks to go from one slide to the next slide by selecting the corresponding button. In this case, the audiovisual media and secondary media of other executives are notified of this event and synchronize accordingly. The frames hosting the audiovisual media allow synchronization 232 by prior temporal indexing 231 (sequential). Each index typically corresponds to a moment in the film where the presenter has moved to the next slide. In this case, the static computer media of other executives are notified of this sequence and synchronize accordingly.

It is therefore necessary firstly to perform time indexing 231 audiovisual media. This is done by asking an index for each transition. This gives a list of indexed (list of time codes). These indexes and time codes are previously recorded in the editing step, but without the possibility of checking the final rendering, which is only possible in the last step. Conversely, if it is necessary to correct this data from the controllable rendering in this last step, these corrections will have to be taken again during a return to the editing step.

First of all, it is necessary for the spatial integration 233 to define the different scenes (templates) in which the synchronized media will be placed. There is, for example, one main stage for the global conference, and as many secondary scenes as sessions inside the workshop.

It is then necessary to import the media (audiovisual and computer static) in each of the frames reserved for them. In the case of audiovisual media, they must first have been indexed. The enriched media editing process described above has several disadvantages:

It most often requires numerous iterations between the editing and synchronization steps 220 to obtain the desired medium. Indeed no visualization of the final rendering is available during the editing step 220. The rendering problems appear only after the synchronization step.

• Each iteration between these two steps consumes time and significant IT resources mainly because of the work done separately and sequentially on the native media on the one hand and on the compressed media (s) on the other hand. • The 220 edition stage and the 230 synchronization stage require the intervention of different types of qualified personnel (video editor, audio editor, graphic designer, computer scientist, presenter, specialist in the field, etc.). Document WO 2004/062258 discloses a method for producing a multimedia edition from oral services on the basis of a previously recorded audio / video stream and complementary documents. This method notably comprises a step of collecting and processing heterogeneous data possibly associated with time codes. The collected data makes it possible to generate selective access tables allowing targeted access to sequences of the audio / video stream. This method is a variant of the method described above, and therefore has the particular disadvantage of requiring manual input of heterogeneous data and the round trip between expensive steps. Document US2002 / 0133520 discloses a method for preparing a multimedia recording of a live presentation. This method allows automatic synchronization between an audio / video recording and the transparencies commented during the presentation. In this method, the possibility of editing media in post-production is not taken into account. Hence the possible use of this approach in academic circles, but unfit for the constraints of control of communication in the industrial environment.

The invention aims to overcome the problems mentioned above by proposing a method, a device and a system for the integrated production of enriched media. The object of the invention is to facilitate the process of creating enriched media by facilitating scripting, maintaining good media quality and sparing media processing resources.

For this purpose, the subject of the invention is a method for editing an enriched medium from an oral performance, said enriched medium comprising a first media of the audio and / or video stream type and at least one second medium each of said media being synchronized, the first and second media being heterogeneous, said method comprising a step of collecting and acquiring (301) in digital form media and synchronization information, said acquired media being called native media, said characterized in that it further comprises the following steps:

the creation of a proxy medium for each of the native media, a proxy media being a version, adapted to the edition, of the native media with which it is associated,

simultaneous editing of the synchronized proxy media and synchronization information, including modifying at least one proxy media, updating the synchronization information of each of said proxy media, generating editing events for each modified proxy media, an editing event including a proxy media change, and a time code,

- the creation of the enriched media by posting the editing events on each of the corresponding native media. According to one characteristic of the invention, the simultaneous editing step comprises an automatic indexing step from the synchronization information, the synchronization information defining synchronization points, an index being created for each of said synchronization points. According to a characteristic of the invention, the enriched media being intended to be read on a given rich media player, the step of creating the enriched media further comprises the creation, from the native media, compatible media with the reader and layout of media and indexes according to the organization of the reader in a given format. The invention has the advantage of being able to be implemented without qualified personnel to edit the enriched media. The generation of enriched media can thus be carried out by an end user or by a specialist in the field of oral presentation. In addition, the invention provides a significant time and resource saving due to the simultaneous edition of the synchronized media, allowing the deletion of the iterations between the steps of the editing method according to the prior art. Finally, it also brings a significant time saving due to the use of a single step of final generation of the support.

The invention is based on the integration of the editing step 220 and the media synchronization step 230, made possible by the replacing the native media collection step 210 with a multimedia production step, in which all the media are acquired in a synchronized manner, during the event organized in front of an audience, so as to have the initial synchronization of the media as soon as the capture. The invention therefore has the advantage of grouping all the engineering work within a single editing step, this step using the editing device according to the invention.

The editing device of the invention allows, on the one hand, at any time, to edit the media, without having to change environment, so without having to perform expensive format conversions (these changes are in fact performed on proxy media, from which it will be possible to automatically postpone them to native media).

The invention also makes it possible at any time to modify the synchronization on this basis, without having to change the environment, without having to perform expensive format conversions.

The invention has the advantage of giving the opportunity for the neophyte to perform this work, without having to have knowledge or audiovisual or computer. The invention will be better understood and other advantages will appear on reading the detailed description given by way of non-limiting example and with the aid of the figures among which:

Figure 1, already presented, represents an example of enriched media. FIG. 2, already presented, represents a diagram of the steps of an enriched media editing method according to the prior art.

FIG. 3 represents a diagram of the steps of the enriched media editing method according to the invention.

FIG. 4 represents an exemplary embodiment of the editing device according to the invention.

FIG. 5 represents an exemplary implementation of the device according to the invention.

FIG. 6 represents an exemplary embodiment of the system according to the invention. FIGS. 7a and 7b respectively represent a diagram of the steps of an enriched media editing method according to the prior art and a diagram of the steps of the enriched media editing method according to the invention. The production of an enriched medium according to the invention comprises the following steps:

- the collection of native media and native synchronization information and the generation of adapted media called proxy media,

simultaneous editing of the synchronized proxy media and synchronization information, during which an operator will place editing events on the synchronized proxy media which will automatically generate, in a single iteration, the edited version of the native media; one part (before possibly an ultimate compression step), and place, delete and / or move the synchronization information,

- the automatic and unique generation of a final enriched media from native media.

FIG. 3 represents a diagram of the steps of the method for editing enriched media from an oral performance according to the invention. An example is an oral performance such as a lecture at which the speaker comments on a digital presentation. For example, it is desired to obtain an enriched medium comprising a first media of the audio / video stream type and a second medium corresponding to transparencies, these two media being synchronized. The method comprises a step of collecting and acquiring 301 in digital form of media and synchronization information. The media thus acquired are called native media. This step includes a capture phase to obtain a native media type audio / video stream. This step also includes a phase of collecting synchronization information for the other media. For example, in the case of transparencies, a device allows recording throughout the oral performance, when the speaker goes to the next transparent. The synchronization information includes, for example, a series of time codes associated with a reference to the media, in the case present, a transparency number and optionally other data (eg index label).

The method of the invention also includes a step of creating a proxy media 302 for each of the native media. A proxy media of a native media is a version adapted to the edition, for example a compressed version of said native media, allowing the creation of editing events in a light environment, which events will be used for the postponement of these editions on the native media. In the case of a native media of the audio and / or video stream type, the proxy media is, for example, a compressed audio and / or video stream and having a lower resolution than the native media. For transparencies, an example of proxy media may be that of transparencies cleared of their graphic information, which allows in a lighter environment the editing of textual parts, before their transfer to native transparencies. The method according to the invention also comprises a step of simultaneous editing of the synchronized proxy media 303 and synchronization information. During editing, the user is required to apply different editing functions to different proxy media. These editing functions result in the modification of at least one of these proxy media and its impact on other media. One of the peculiarities of the invention is that these editing functions maintain the synchronization of the proxy media by updating the synchronization information. Moreover, the application of the editing functions generates editing events for each of the modified proxy media, an editing event indicating, for example, a modification on a proxy medium and one or more time codes.

In the case of an editing function, such as a section in the video, that can edit multiple media, the cut function generates a cut event for other media, such as transparencies, to maintain synchronization. The time codes of each media are recalculated to account for the editing event.

However, an edit function can only impact a single medium, for example, adjusting the audio / video stream's audio volume. The synchronization of the proxy media is not affected. The editing tasks for the audio / video stream are of the same nature as those of the prior art but they are applied simultaneously to all the proxy media.

The method according to the invention also makes it possible to edit the synchronization information. By way of example, in the case of a transparent secondary medium, the synchronization information defines synchronization points corresponding to the transition from one transparency to another. One or more synchronization points are often generated by mistake during the sensed presentation (untimely passage to the next transparency, followed by a return to the initial transparency). It is then possible to delete all or part of these synchronization points. Likewise, the initial position of the sync points may require moving. In addition, synchronization points initially created with portions of video deleted during editing are also automatically deleted. It is also possible to create new synchronization points.

The method of the invention also allows the creation, modification or deletion of indexed and their hierarchization. It is possible to create, modify or delete an index on all or part of synchronization points, and to attach a label. The hierarchy allows in the final rendering to have more or less detailed views of the indexes according to their level of depth in the hierarchy.

In particular, it is possible for transparent secondary media to automatically create the indexes during initial synchronization point registration, and to use this native indexing as a starting point for editing indexes.

The method according to the invention also comprises a step of creating the enriched media 304 by reporting the editing events on each of the corresponding native media. For example, an edit event representing a cut applied to a proxy media during the editing step, defined by two time codes indicating the beginning and end of the cut, will be applied to the corresponding native media so that get the same effect.

FIG. 4 represents an exemplary embodiment of the editing device according to the invention. The device according to the invention 400 comprises: means for receiving proxy media 401, a proxy media being an adapted version of a native media with which it is associated,

means for receiving synchronization information 402 of the proxy media,

a module for the simultaneous editing of the proxy media 403 and synchronization information, said editing module comprising means for displaying the proxy media and synchronization information.

The module for editing further comprises means 404 for editing and displaying indexed and means for automatically creating the indexes from the synchronization information.

The editing device according to the invention is a computer medium that can take the form of, for example, a CD ROM allowing the execution

15 on a desktop computer processing modules described above. As the edition is based on the proxy media embedded in the editing device, the size of the medium and the computational load are much smaller than for direct editing of native media.

Figure 5 shows an example of implementation of the means

The display device 500 of the enriched media editing device according to the invention.

The display means comprise a first frame 501 allowing a preview of the enriched medium and a second frame 502 representing an enriched time line.

Rich media includes a 503 video, 504 transparencies

25 and index 505 to access different parts of the enriched media. The second frame 502 includes a time line 509 and then different lines representing the sequencing of the media along that timeline 509. A first line 506 represents the sequencing of the transparencies and indicates when the transparency will be displayed. A second line

30 507 represents the positioning of the various indexes (chapters) of the presentation. Each new transparency potentially corresponds to an index of hierarchical level 1, which effectively becomes an index only if it is attached a label with possibly changing the hierarchical level. The editing device comprises means 51 1 for modifying

The hierarchical level of an index, an arrow to the left to increase the hierarchical level of an index, an arrow to the right to decrease it. A third line 508 indicates when the different sections are made in the presentation or any other audiovisual editing operation. A cursor 510 placed on the timeline 509 indicates when the pre-display of the enriched media presented in the first frame 501 is located. The movement of the cursor along the timeline allows the user to access the different parts. enriched media. The second frame makes it easy to view and edit how media is synchronized.

The display means further includes a read button for viewing a preview of the final result of the enriched media in the first frame 501.

In the example corresponding to FIG. 5, the enriched media comprises five chapters (Introduction, Richmedia, Demo, Application and Conclusion). The first line 506 comprises seven transparencies. When viewing the rich media, transparencies 2 and 3 are displayed in the introduction chapter, transparencies 4, 5 and 6 in the Richmedia chapter and transparency 7 in the Demo chapter. The invention also relates to a system for producing enriched media from an oral performance. FIG. 6 represents an exemplary embodiment of the system according to the invention. Such a system comprises computer implementation means comprising processing and storage modules and display means. Such a system for editing an enriched media from an oral performance includes:

means for capturing and acquiring native media from said oral performance, for example, microphones and cameras, the native media being able to be stored in a dedicated storage space 607,

means for the acquisition 603 of synchronization information, for example a device recording a time code during a change of transparency of the speaker,

means for creating 602 proxy media from native media, means for editing the proxy media and synchronization information, said editing means generating editing events, the means for editing comprising the device for the edition 400 of an enriched media according to the invention, means for the generation of enriched media (605) from the editing events and the corresponding native media, these means being, for example, a module 605 for the generation of enriched media creation orders 406 and setting up a rich media player 607 from the editing events, timing information, and corresponding native media.

The means for the acquisition 603 of synchronization information, for the creation 602 of proxy media and for the edition of the proxy media as well as the module for generating creation orders are implemented by the computer processing means of the system.

Rich media creation orders are orders to apply proxy media editing events to native media. These creation orders can come from standards.

For audiovisual media, this is the format called EDL (Decision List Edition) An EDL includes a list of editing events associated with a time code. For other media, creative orders are defined in a more specific way. For example, in the case of transparencies whose proxy is a text file, the creation orders are made in the form of a macro command modifying the transparencies from the corrections applied to the text file.

The editing device according to the invention is independent of the enriched media players. Nevertheless, a parameterization phase is necessary so that the enriched media generated is compatible with a given reader. This setting consists in generating, from the native media, media compatible with the reader and arranging media and indexes according to the organization of the reader.

FIG. 7a represents a diagram of the steps of an enriched media editing method according to the prior art. The method consists of separately editing each of the media (Media-i, Media ₂ , ... and Media _n ) and each of the respective indexes (h, i ₂ , ..., and i _n ) for these media. A modification of a index must be reported manually on all other indexes. The edition is represented by a circular arrow on each of the media and each of the indexes. The method results in the creation of an enriched media including a container 701 grouping the different media (Media-i, Media ₂ , ... and Media _π ). An index i provides synchronized access to different parts of the media.

FIG. 7b represents a diagram of the steps of the enriched media editing method according to the invention. The method consists in uniquely editing proxy media (Media'i, Media ' ₂ , ... and Media' _π ) grouped together in a lightweight container 702, which is a version adapted to the simplified edition of the container. final 701 of the enriched media. A unique index i 'is edited. This feature simplifies the editing work and in particular removes the problem of postponing changes from one index to another during editing.

Claims

A method for editing an enriched media from an oral performance, said enriched media comprising a first media type audio and / or video stream and at least one second media, each of said media being synchronized, the first and the second medium being heterogeneous, said method comprising a step of collecting and acquiring (301) in digital form the media and synchronization information, said acquired media being called native media, said method being characterized in that further comprises the following steps:

the creation (302) of a proxy medium for each of the native media, a proxy medium being a version, adapted to the edition, of the native media with which it is associated,

simultaneous editing (303) of the synchronized proxy media and synchronization information, including modifying at least one proxy media, updating the synchronization information of each of said proxy media, generating the event events; editing for each of the modified proxy media, an editing event including a change to a proxy media and a time code,

the creation of the enriched media (304) by posting the editing events on each of the corresponding native media.

A method for editing an enriched media according to claim 1, characterized in that the simultaneous editing step (303) comprises an automatic indexing step from the synchronization information, the synchronization information defining synchronization points, an index being created for each of said synchronization points.

3. A method for editing an enriched media according to one of the preceding claims, characterized in that, the enriched media being intended to be read on a given rich media player, the step of creating the enriched media comprises in addition, the creation, from native media, media compatible with the reader and the layout of the media and indexes according to the organization of the reader in a given format.

4. Device for editing an enriched media (400) from native media and synchronization information acquired during an oral performance, said enriched media comprising a first media type audio stream and / or video and at least one second medium, each of said media being synchronized, said device being characterized in that it comprises:

means for receiving proxy media (401), a proxy medium being an adapted version of a native medium to which it is associated, means for receiving synchronization information (402) from the proxy media,

a module for simultaneous editing of the proxy media (403) and synchronization information, said editing module comprising means for displaying the proxy media and synchronization information.

5. Device for editing an enriched media according to claim 4, characterized in that the module for editing further comprises means for editing and displaying indexed from the information of synchronization.

6. System for producing an enriched media from an oral performance comprising means for capturing and acquiring (601) native media from said spoken performance, and means for acquiring (603 ) synchronization information, said system being characterized in that it further comprises:

means for creating (602) proxy media from native media,

means for editing (400) the proxy media and synchronization information, said editing means generating editing events, said editing means comprising the device for editing an enriched media according to the one of claims 4 or 5.

means for the generation of enriched media (605) from the publishing events and the corresponding native media.