FR2814255A1 - Recording and real-time interactive navigation within a panoramic digital film, where a field of vision is displayed on a computer monitor with a navigation interface beneath it allowing the field of vision and zoom to be changed - Google Patents

Abstract

Method for navigating within a panoramic digital film. The navigation uses two parts. The first visualization window (21) displays a panoramic film of a given duration, having a field of vision that can be altered. The film also has active zones allowing actions, such as zooming, to be triggered so that a new window of a particular object in the film is displayed. The second part is a navigation window (22) with various graphical symbols for representing the field of vision. The navigation window (22) has a time bar (23) and a circular representation (24) of the filmed panoramic space. The circle indicates, in real-time, the field of vision selected by a user and displacements (29) of persons (33) present in the film. The field of vision is represented by a section of the circle, the section being moved to change the field of vision displayed on the visualization window. The invention also relates to a system for recording a panoramic image with a number of objectives arranged in a circle around the area being filmed with a terminal for controlling real time image acquisition. An Independent claim is made for a navigation system for moving around a panoramic digital film image having a time bar and circular graphic image for controlling the field of vision.

Description

<Desc / Clms Page number 1>

"Method and system for interactive navigation within a digitized panoramic film, and device for acquiring a panoramic film" DESCRIPTION
The present invention relates to an interactive navigation method within a panoramic film digitized and generally viewed on a microcomputer screen. The invention also relates to a video image acquisition device for producing such a panoramic film. By panoramic film is meant a film acquired at an angle of 360 degrees.

 Document WO 9918725 (IPIX) discloses a device for taking 360-degree images using a specific lens in the form of a half-sphere mounted on a support. To obtain a panoramic image, four series of images are taken in the direction of the four cardinal points. In each series, two images are taken, a right image and a left image for which the camera is offset by a given distance relative to the taking of the right image. This document does not appear to concern the capture of video images.

 Document WO 9750252 (BEHERE) is also known, in which a camera system is described comprising a lens capable of directly taking a 360-degree image as well as a device for microphones in order to create sound effects. This document also describes a visualization interface comprising a rectangular visualization window in which the complete image taken at 360 degrees is displayed.

 This image acquisition system requires the implementation of a specific lens and this document discloses an inert display of the images obtained.

<Desc / Clms Page number 2>

 The object of the invention is to remedy these drawbacks by proposing an interactive navigation method when viewing a panoramic film.

 Another object of the invention is to provide a fast, efficient and inexpensive video image acquisition device in order to produce such a panoramic film.

 The above objectives are achieved with an interactive navigation process within a digital panoramic film, in which: a viewing window is displayed on a part of a screen in which the panoramic film is viewed according to a current field of vision, the dimensions of this window being variable; a time bar representing the time progress of the panoramic film is displayed under the viewing window; and - a graphic representation of the panoramic space filmed is displayed in the form of a circle in which the current field of vision of the panoramic film is represented by means of a quarter, this field of vision can be modified by moving the quarter within the circle.

 The center of this navigation circle represents the place where the camera or cameras that were used to acquire the panoramic film were placed.

 The 360 degrees of the panoramic film are not spread in the viewing window as in the document WO9750252, only the video sequence corresponding to a given viewing angle is displayed. To explore the 360 degrees of the film, we modify the field of vision by directly manipulating the panoramic film. To do this, we click (with the mouse pressed) on the panoramic film, and we perform a lateral translation movement. The unfolding of the film

<Desc / Clms Page number 3>

 can be carried out in looping or in linear by possibly leading to another panoramic film.

 With such a method, the temporal and spatial dimensions of the panoramic film are clearly distinguished by means of the time bar and the navigation circle. The field of vision is represented by a luminous quarter inscribed in the navigation circle. To change the field of vision, you can move the light district by activating one of the active elements arranged around the navigation circle. These active elements can be buttons which can be clicked using a mouse of a microcomputer.

 According to an advantageous characteristic of the invention, the panoramic film comprises active areas such that when one clicks on one of these active areas, a predetermined event is triggered.

By way of example, these predetermined events can be the passage to another panoramic film, the display of an additional video window comprising a subjective plane such as a zoom on a particular object, the display of an area of text, the triggering of a sound, the opening of a browser on a given Internal site, or any other event planned by the author of the interactive scenario.

 Advantageously, the timeline can include graphic markers placed at predetermined times corresponding to the presence of active areas on the panoramic film.

 According to the invention, the navigation circle may include markers representing particular elements present in the panoramic film. Preferably, these elements are mobile and represent the displacement of the characters present in the panoramic film. By observing the navigation circle, we can thus see the movement of a given character. In addition, by moving the quarter so that the marker representing the character remains constantly in the quarter, we move the field of vision at the same time as this

<Desc / Clms Page number 4>

 character which therefore remains constantly visible in the viewing window.

 According to another aspect of the invention, an interactive navigation system is proposed within a digital panoramic film. The system includes acquisition means for acquiring video images, processing means for developing said panoramic film and creating a graphical navigation interface. This interface includes elements described above, namely, a viewing window, a time bar and a graphic representation of the panoramic space filmed.

 The invention provides a device for acquiring images in order to produce the panoramic film. According to the invention, this device comprises: - a plurality of objectives, at least six for example, arranged in a crown, each objective being capable of filming a predetermined field of vision; - a mirror placed in the center of the crown and composed of at least as many facets as there are objectives; each facet being able to receive a light beam emitted by a lens facing it; - sensors arranged in a ring on a plane smaller than that of the objectives and capable of receiving light beams returned by the facets of said mirror, these sensors preferably being components with high resolution charge coupling such as tri-CCDs; a terminal provided with at least one real-time acquisition card for receiving and digitally processing the images coming from the objectives, said objectives operating simultaneously for a predetermined duration.

 In order to achieve sound effects, each lens can be surmounted by a directional microphone pointing in the axis of the lens

<Desc / Clms Page number 5>

 so as to constitute as many sound tracks as fields of vision filmed. We will thus be able to constitute several sound tracks in order to reconstruct the real sound environment.

 Advantageously, the adjoining fields of vision may include an overlap area of around 30%. This ensures perfect continuity when mounting the panoramic film.

 Each facet is inclined by 45 relative to the vertical axis of the crown so as to return the light beams received with a 90-degree angle downwards.

 In addition, the device includes a control transfer system allowing a user to control the objectives remotely.

- la figure 2 est un schéma simplifié d'une interface de navigation interactive affichée sur un écran d'ordinateur pour la visualisation du film panoramique. Other advantages and characteristics of the invention will appear on examining the detailed description of a mode of implementation in no way limiting, and the appended drawings in which: - Figure 1 is a simplified diagram of a device d 'acquisition of images for the production of a panoramic film; and

- Figure 2 is a simplified diagram of an interactive navigation interface displayed on a computer screen for viewing the panoramic film.

 In FIG. 1, a diagrammatic arrangement of objectives used for capturing images is illustrated. For reasons of simplification, only two objectives 1 and 2 are represented instead of the six normally present. The six objectives are arranged in a crown 12 so that their focal points converge and each objective sees opposite a diametrically opposite objective. The field of vision covered by each lens has a common part, or 30% overlap area, with the fields of vision covered by the adjoining objectives, this guarantees coverage of 360 degrees.

<Desc / Clms Page number 6>

At the center of the crown 12, there is a mirror 13 composed of as many facets 3, 4 as there are objectives. These facets are inclined at 45 degrees relative to the vertical axis of the crown 12. Each facet intercepts the light beam emitted by the objective which faces it. It returns this beam 90 degrees downwards on a ring 14 made up of high-resolution 5.6-tri-CCD sensors which transform the light signal into an electronic signal as for a conventional electronic camera. The tri-CCD sensors have a high resolution in number of pixels in order to compensate for the absence of optical zoom.

 This device constitutes a camera which is equipped with a command transfer system allowing a shooting operator to simultaneously trigger recording on all of the lenses.

This remote control also makes it possible to independently control each objective, this makes it possible to carry out the recording with a determined number of fields of vision. This flexibility of use is perfectly suited to all types of scriptwriting and post-production processing. We can decide to have only a few fields of vision shot on video and to reconstruct the rest only in post production with 3D environments for example. The same camera can also be used for complementary shooting of the subjective plane type, for example by zooming in on a particular object in the film in a field of vision.

 The control delay also allows for each iris, focus and shutter speed settings.

 On the crown 12, each objective is surmounted by a directional microphone 15, 16 pointing in the axis of the objective and allowing a preferential recording of the filmed sector. The sound recording is synchronous with the lens recording mode.

 The operator should not appear in the field of vision, the camera is placed on a fully movable camera stand

<Desc / Clms Page number 7>

 remote control. The command postponement allows the shooting operator to perform camera movements of the "traveling" type without the risk of intrusion into the field of vision.

 The electronic stream 9.10 from each tri-CCD sensor and from each microphone is sent to a real-time acquisition card placed in a microcomputer 11. This acquisition card manages the digitization of a video stream of 25 images with a resolution of up to 1440x1152 non-square pixels per second, or a bandwidth of 36 MB / s for a compression ratio of 2: 1. We must add to this bandwidth 1 MB / s for the sound sampled at 48000 Hz.

 The digital stream generated by the acquisition card is stored on hard disks formatted in "RAID" (Redundant Array of Inexpensive Disks), a formatting allowing the cooperation of several hard disks.

 Optimally, each tri-CCD sensor and the corresponding microphone are connected to an acquisition card present in the computer 11.

 The acquisition and the editing of the panoramic film are managed by software of the virtual editing type which allows to preview the connection, or "stitching" in English, of the different plans constituting a panorama of 360 degrees. To do this, we use the Quick Time 'software. A sound track is then assigned to the connected panoramic film. In fact, this track is composed of as many sound sub-tracks as there are filmed sectors. With this software, you can edit several panoramic films with their sound track and establish transitions between these different films. The transition between a first film and a second film can, for example, be effected by actuating an active area when viewing the first film.

 When the editing is finalized, the film is compiled. During this compilation, we decide on the compression-decompression format (codes) and the target bandwidth. The software used

<Desc / Clms Page number 8>

 allows complete editing with possibility of overlay on the main plane of text, still or moving images, sounds etc.

 The looped playback of the panoramic film is obtained by means of a specific "plug" implemented in a Quick Time 'format.

 The viewing of the panoramic film allows interactive navigation, the graphical interface of which is represented in FIG. 2. There are two parts in perfect correlation. The first part is a viewing window 21 on which the panoramic film is viewed according to a given field of vision. The second part 22 is a navigation interface comprising temporal and spatial elements.

d'une souris) directement la fenêtre en la faisant défiler latéralement. The viewing window 21 therefore displays the panoramic film which in this case represents a character 33 turning around a fountain 34 in a garden comprising trees 31 and 32. Only a field of vision is displayed (current field of vision), we do not view the 360 degrees spread over the viewing window 21. So if we stay in a field of vision, we see the character 33 enter from the right, cross the window and then disappear from the left. But it is possible to keep the field of vision on the character and go around the fountain 34 at the same time as this character. For this, we manipulate (by means

window) directly by scrolling the window sideways.

 You can also use the navigation interface to move the field of vision.

 This navigation interface 22 includes a time bar 23 which indicates in real time and graphically the time elapsed 36 in the playback of the panoramic film. The time bar includes graphic markers 35 placed at predetermined times and indicating to the user the presence of an active area somewhere in the space of the panoramic film. The graphic marker 35 corresponds to a period during which the character 33 is between the trees 32 and 31. The active area can be all of the foliage of the tree 32. An action of

<Desc / Clms Page number 9>

 the user, such as a mouse click, on this active area while the duration 36 crosses the marker 35, triggers an event which is the opening of a new window on the screen. This new window is a video image showing for example a couple of birds perched in the tree 32. This event can be another panoramic film replacing the first or any other event. The markers also make it possible to replay the film from these predetermined moments. The explored markers change color which allows the user to have a visual memory of his navigation. These markers can be re-initialized at any time.

 The navigation interface 22 also includes a graphic representation of the panoramic space filmed in the form of a viewing circle 24. The graphic representation indicates in real time which field of vision is chosen by the user. This field of vision is represented by a quarter 28 inscribed in the navigation circle 24. The center 30 of the navigation circle corresponds to the position of the image acquisition camera. The character 33 is represented in the panoramic space by the mobile element 29. It is thus possible to visualize the displacement of the character 33. On the edge of the navigation circle 24, active elements 25, 26, 27 allow the user to continue to an action (click for example) to instantly move to a specific zone in the filmed space by moving the quarter 28 therefore the current field of vision of the viewing window 21.

The navigation interface 22 also includes a text area 37 on which information on the filmed sector is written.

The sound is also panoramic. The volume and the balance of the sound (east - west / north-south) adjust according to the movements of the user in the panorama and become a strong indicator of what is happening outside the field of vision.

<Desc / Clms Page number 10>

Of course, the invention is not limited to the examples which have just been described and numerous modifications can be made to these examples without departing from the scope of the invention, in particular one can envisage a navigation interface in three-dimensional representation not not of a single panoramic film but of a universe of panoramic films linked together. This would increase the possibilities of navigation no longer within a single sequence but within the universe that composes an interactive scenario as a whole. A video database can be represented as a plurality of concentric video crowns forming a torus. Each video crown can be a panoramic film respecting space-time continuity or a juxtaposition of several independent panoramic films, and each panoramic film can belong to a given class of films. This arrangement would make it possible to optimize navigation between different panoramic films classified for example by themes, by periods or by issues.

The graphic interface thus described can relate to the following applications: - reporting, music group video clip, filmed concerts; - documentary, shooting of a film as "bonus Track" of a DVD video; - interactive fiction, scripted games, quest, police investigation, "sitcoms" ...

- artistic experimental achievements; - virtual tours, discovery of the Louvre in multilingual, or - conferences and symposia.

Claims (18)

 1. A method of interactive navigation within a digital panoramic film, characterized in that: - a display window (21) is displayed on a part of a screen in which said panoramic film is displayed according to a current field of vision ; - A time bar (23) representing the time progression (36) of the panoramic film is displayed under the viewing window; - a graphic representation (24) of the filmed panoramic space is displayed comprising a graphic means (28) for representing the current field of vision of the panoramic film, said field of vision being able to be modified by displacement of said graphic means (28). 2. Method according to claim 1, characterized in that said graphic means (28) is moved by actuating one of the active elements (25, 26, 27) arranged around said graphic representation (24).

3. Method according to one of the preceding claims, characterized in that the panoramic film (21) is directly manipulated to modify the current field of vision. 4. Method according to any one of the preceding claims, characterized in that the panoramic film comprises active areas such that when one clicks on one of these active areas, a predetermined event is triggered. 5. Method according to any one of the preceding claims, characterized in that the time bar comprises graphic markers (35) placed at predetermined times corresponding in the presence of active areas on the panoramic film. <Desc / Clms Page number 12> 6. Method according to any one of the preceding claims, characterized in that the graphic representation (24) of the filmed panoramic space comprises a circular shape on which are arranged markers (29) representing particular elements (33) present in the panoramic film. 7. Method according to any one of the preceding claims, characterized in that the panoramic looping film is viewed. 8. Method according to any one of claims 1 to 6, characterized in that the panoramic film is viewed linearly. 9. Method according to any one of the preceding claims, characterized in that the dimensions of the viewing window are variable. 10. Interactive navigation system within a digital panoramic film, comprising acquisition means for acquiring video images, processing means for developing said panoramic film and creating a graphical navigation interface, characterized in that said interface navigation graph includes: - a viewing window (21) for viewing said panoramic film according to a current field of vision; - a time bar (23) to indicate the time progress (36) of the panoramic film; - a graphic representation (22) of the filmed panoramic space comprising graphic means (28) for representing the current field of vision of the panoramic film, said field of vision being able to be modified by displacement of said graphic means. 11. System according to claim 10, characterized in that the graphic representation (24) comprises a circular shape and in that <Desc / Clms Page number 13>  said graphic means (28) representing the current field of vision comprises a triangular shape inscribed in said circular shape. 12. System according to one of claims 10 and 11, characterized in that the panoramic film comprises active areas for accessing predetermined events. 13. System according to any one of claims 10 to 12, characterized in that the acquisition means comprise: - a plurality of objectives (1,2) arranged in a ring (12), each objective being capable of filming a predetermined field of vision; - A mirror (13) arranged in the center of the crown and composed of at least as many facets (3,4) as there are objectives; each facet being able to receive a light beam emitted by a lens facing it; - sensors (5,6) arranged in a ring (14) on a plane smaller than that of the objectives and capable of receiving light beams (7, 8) returned by the facets of said mirror; - a terminal (11) provided with at least one real-time acquisition card for receiving and digitally processing the images originating from the objectives, said objectives operating simultaneously for a predetermined duration.

14. System according to claim 13, characterized in that each objective is surmounted by a directional micro (15, 16) pointing in the axis of the objective so as to constitute as many sound tracks as filmed fields of vision. 15. System according to one of claims 13 and 14, characterized in that the adjoining fields of vision include an overlap zone. <Desc / Clms Page number 14>  16. System according to any one of claims 13 to 15, characterized in that each facet (3, 4) is inclined by 450 relative to the vertical axis of the crown (12). 17. System according to any one of claims 13 to 16, characterized in that it further comprises a command transfer system allowing a user to control the objectives remotely. 18. System according to any one of claims 13 to 17, characterized in that the sensors (5,6) comprise components with high resolution charge coupling.