AU6453300A

AU6453300A - Device for interactive and smooth direction control in a compressed digital film and software for managing same

Info

Publication number: AU6453300A
Application number: AU64533/00A
Authority: AU
Inventors: Valentin Lefevre; Bruno Uzzan
Original assignee: Total Immersion
Current assignee: Total Immersion
Priority date: 1999-07-16
Filing date: 2000-07-13
Publication date: 2001-02-05
Also published as: JP2003505775A; CA2379606A1; FR2796513A1; WO2001006514A1; FR2796513B1; EP1200961A1

Description

VERIFIED TRANSLATION OP M WO 01/06514 PCT/FROO/02029 A device for smooth interactive directional control of a compressed digital film, and software for managing the device The invention relates to a device for smooth interactive direGtional control of a compressed digital film, and software for managing the device. Increasingly frequent use is nowadays made of films for presenting e.g. a flat or tourist travel or demonstrating equipment. This is usually done with a video display unit such as a television set or a computer monitor, but a spectator or user cannot vary the sequence of images. More and more frequently, however, it appears that the user wishes to be able to adjust the running of the film by varying the speed so as to accelerate or slow down the succession of images, i.e. to run the film at the desired speed with smooth changes, or to choose the direction taken by the film. When viewing a flat, for example, the user should be enabled to look at a particular room before the dining room or to return to the room before going to the kitchen. In the more general case of a visit to a site, the user should be able to choose the direction he takes at certain crossings, e.g. to go straight ahead, to left or to right. In a gymnasium- it may be appropriate to show a user a picture of a forest or country track, for example. so as to encourage him when using a cardio-training apparatus such as a bicycle, mat, rowing machine, treadmill or the like.

2 One aim of the invention accordingly is to provide a device for interactive control of a film, enabling the user at choice to modify the speed at which the pictures succeed on another in a film. Another aim of the invention is to provide a said device enabling a single film to be viewed a number of times without the succession of pictures always being identical. These aims, together with others which will appear hereinafter, are addressed by a device for fluid. interactive control of a compressed digital film, comprising a computer having a sound output. a video output both connected to a display system, and an input connected to a direction-controlling means in series with a means for controlling the running speed of the film, the device according to the invention being characterised in that the direction-controlling means comprises a number of buttons each corresponding to a direction in space, and each enabling the computer to process the direction chosen by a user. Preferably, the speed-control means is a lever which is operated by the user in order to unwind the film at a variable speed. Advantageously, the speed-control means is a speed pick-up directly connected to a cardio-training apparatus. in a first variant embodiment, the speed pick-up comprises a roller which is in contact with the moving part of the cardio-training apparatus and transmits the speed of rotation thereof to a toothed wheel situated between an infrared transmitter and an infrared receiver. In a second variant embodiment, the speed pick-up comprises a magnet which moves at regular intervals in front of an induction coil providing information to the computer, which uses it to deduce the speed.

3 In a third variant embodiment, the speed pick-up comprises an optical pick-up co operating with the moving part of the cardio-training apparatus. The invention also relates to a program or software characterized in that it comprises a step of acquiring the speed and direction desired by a user, a first series of steps for determining the running speed of the images and a simultaneous second series of steps for determining the orientation from the desired speed and direction, and a step for displaying the thus-determined series of pictures. Preferably use is made of a configuration file containing the following information: - name of digital file to be opened in accordance with the filmed route, - information regarding the nature of the route and branches thereof and regarding the possible choices of direction, - speed at which the film has been shot,,and - recovery of the information relating to the type of speed pick-up used. Advantageously. the instantaneous speed of a user is calculated as the average of the n preceding speeds Vi. Preferably, the known information regarding the speed of motion of the camera when shooting the film and the number of pictures taken by the camera per second when running the film is used, in dependence on the speed to be simulated, to determine the jump S corresponding to the number of pictures to be skipped before each displayed picture and the time T corresponding to the time interval between two displayed pictures. Advantageously, arrows indicating the directions to be chosen are overprinted on the film a few seconds before each crossing of the filmed route.

4 The following description, which is in no way limitative, should be read with reference to the accompanying drawings in which: Fig. 1 is a block diagram of a device for interactive direction control of a film in accordance with the invention; Fig. 2 is a flow chart of the software for working the device in Fig. 1, and Fig. 3 is an example of a directional diagram for a route or orientation device. A device for interactive directional control of a compressed digital film in accordance with the invention comprises, as shown in Fig. 1, a computer I having a sound output 2 and a video output 3 both connected to a display system 4, and an input 5 connected to a direction-controlling means 6 in series with a means for controlling the speed 7 at which a displayed film is run. The display system 4 is e.g. a television screen, a computer monitor, a plasma or liquid-crystal screen, or more generally any picture-restoring system for displaying the pictures in a film, e.g. shot on location, facilitating visual immersion of a user. The computer 1 has a coding and decoding system called CODEC for decompression of digital pictures. The CODEC is chosen inter alia from among known systems such as MJPEG (Motion Joint Picture Expert Group) or MPEG (Motion Picture Expert Group). These various CODECs can be used with or without material acceleration. The picture-decompressing device used in practice is generally either a very powerful general-purpose processor or a specific card dedicated to picture decompression.

5 The computer 1 should also be capable of storing the digital film. More particularly, use may be made of the following storage media: hard discs, optical compact discs (CD-ROMs) or digital video discs (DVDs). The direction-controlling means 6 can comprise a number of buttons or levers each corresponding to a direction In space. The minimum number of buttons is three, i.e. a left button, a right button and a straight-ahead button. Pressure on a button will cause the computer to store the direction chosen by the user. For example, a few seconds after each crossing of the filmed route, the computer prints arrows over the film informing the user that he must choose a direction in order to move further forward. The number of arrows displayed is equal to the number of possible directions, and the arrows extend in the directions which they indicate. The speed-controlling means 7 is a lever which, when operated by a user, can run a film at a variable speed which the user determines directly or indirectly. The device 7 may also be a speed pick-up directly connected to a cardio-training apparatus. In a first variant embodiment, the speed pick-up comprises a roller which is in contact with the moving part of the cardio-training apparatus and transmits the speed of rotation thereof to a toothed wheel situated between an infrared transmitter and an infrared receiver. A pick-up of this kind can be called a roller pick-up. In a second variant embodiment, the speed pick-up comprises a magnet which moves at regular intervals in front of an induction coil providing information to the computer 1. which uses it to deduce the speed. A pick-up of this kind can be called a magnetic pick-up.

6 in a third embodiment, the speed pick-up comprises an optical pick-up which co operates with the moving part of the cardio-training apparatus; for example the moving part can comprise a mat on which lines are disposed at regular intervals and read by the optical pick-up. The computer 1 comprises software which loads the film selected by a user, receives information transmitted by the speed pick-up 7, and manages the dialogue with the user by displaying the film pictures at the right speed and in the direction chosen by the user. The software must have real-time characteristics so as not to produce jerky sequences of pictures which are unpleasant to watch. The simulated rate of advance should therefore be as smooth as possible. The software, the flow chart of which is shown in Fig. 2, uses a configuration file containing the following information: - name of digital file to be opened in accordance with the filmed route, - information regarding the nature of the route and branches thereof and regarding the possible choices of direction, - speed at which the film has been shot, and - recovery of the information relating to the type of speed pick-up used, i.e.: - characteristics of the lever (where a lever is used), - radius of the roller of the speed pick-up (where a roller pick up is used) and - characteristics of the magnetic pick-up (in the case of a magnetic pick-up). The computer 1 must record an instantaneous speed, using one of the said three pick-ups. In a preferred embodiment of the invention, the pick-up comprises a roller having an axis of rotation coupled to the electronics originating from a commercial mouse, Rotation of the roller will cause the pointer to rotate along 7 the X axis. In this preferred embodiment, the instantaneous speed Vi is calculated by the method described hereinafter. Let - R be the radius of the roller, - nb pixels be the number of pixels travelled by the mouse cursor since the previous cycle, - nb_pixels_perrevolution be the number of pixels travelled during a complete revolution of the roller. - T be the cycle time and - nbradians be the number of radians travelled by the roller during the time T. Under these conditions, the instantaneous speed is determined by using the following formula: Vi = (nb _radians x R) / T, since the value of the curved abscissa is (nb radians x R) and with nb radians = (nbpixels x 2 7r) / (nbpixels per revolution). The instantaneous speed of the user is therefore: Vi = (nb_pixels x 2 xR) / (nbpixels per revolution x T) The instantaneous speed Vi, which can alternatively be obtained by one of the other methods of speed pick-up described hereinbefore, is stored and an average speed is determined. using the n previous instantaneous speeds. The result is low-pass filtering of the instantaneous speed (in practice n is equal to 5, giving a very satisfactory compromise between filtering and response time). As already mentioned, the computer 1 comprises a video decompression CODEC. This technology has the following three advantages in particular: - average time for decompressing a picture is practically constant, 8

-

indexed access to any Picture in the film,

-

pictures can be displayed at a rate faster than the television standard. In practice, up to 50 pictures per second can be shown. A variable running or unwinding speed Is obtained by varying the following two parameters: - T. time between consecutive display of two consecutive pictures of the route, and - S: number of pictures jumped between each picture displayed. To calculate the time T, It will be assumed that: - dt-film is the time interval between two pictures when shooting the film, - v.film is the speed at which the film was shot, and - v is the speed to be simulated. In a first step, $ is taken to be 1, i.e. a speed is to be simulated without skipping pictures in the film, We therefore have the following series of equations: v = dx/T in which dx is an elementary displacement, v_film = (dxfilm/dt film), but dx film is equal to dx since the elementary displacement is smallest if no picture is skipped (S = 1). Consequently dx = v film x dt film and v = (v-film x dt film) / T, so that T is: T = (v_film x dtfilm) / V 9 In practice, using a CCIR (international Radio and Television Consultative Committee) camera, the value of dtfilm is 40 milliseconds (ms). However this formula for T is not very satisfactory, since if v increases greatly with respect to v film, T will decrease greatly, as when a speed of 80 km/h is to be simulated whereas the film was shot at 5 km/h, There is therefore a special TTHRESHOLD below which T cannot be reduced. In practice, TTHRESHOLD is about 10 ms. In order nevertheless to simulate very high speeds, pictures in the original film are skipped. This is done by using a recursive algorithm (presented hereinafter in the DESCARTES form) to calculate the jump S and the time T: INITIALISATION- S = 1, TINITIAL = T (calculated by the formula hereinbefore). AS LONG AS (T < T_THRESHOLD) OPERATION S=S+ 1 T T_INITIAL x S END OF OPERATION END AS LONG AS The subscript of the next picture for display is calculated, beginning with an initialisation function. The software stores the route logic and the order in which the "route ends" are stored in the file. In addition, for each "route end", the software must store the number of pictures therein and the subscript of the starting picture in the complete file. Accordingly each picture in a digital film is given a subscript corresponding to the position of the picture in the film. PrQcessing of the choice 10 of direction by the software can be understood most clearly by taking the following simple example shown in Fig. 3. In this example, the filmed route comprises five "route ends", the logic of which is stored by the software during the initialisation phase, i.e.: P1 to P4: left P2 to P5: straight ahead P2 to P3: right P3 to P2: right P3 to P1: straight ahead P4 to P2: right P4 to P1: left PS to P4: right Also during the initialisation phase, the software stores the order in which the route ends" are stored in the file (avi). P4S Example of a route file The software stores the number of pictures making up each "route end' and the starting picture in the complete file. The algorithm for calculating the next picture for displaying is as follows, it being agreed that - P is the present route, - P-G is the route corresponding to a left turn after the present route, P_TTD is the route corresponding to travel straight ahead after the present route and 11 -PD is the route corresponding to a right turn after the present route (note: P_G, PTD and PD need not be defined). OPERATION Picturesubscript = subscript.preceding picture + S END OF OPERATION If (picture-subscript > number-of pictures-in present-route) If userchoice = LEFT picturesubscript - starting subscript (PG) END if If userchoice = STRAIGHT AHEAD picture_subscript = starting subscript (PTD) END IF If userchoice = RIGHT picturesubscript = starting subscript (PD) END if END if When running the film, the following requirements must be taken into account: the film must be as smooth as possible with stable movement of cameras, i.e. without jerks, visible changes of trajectory, or vibration. These conditions must be fulfilled in their entirety, since the films will subsequently be replayed at faster speeds. As the skilled man will understand, as a result of the invention the film will run in dependence on the effort made by the user, who will also choose his own itinerary, avoiding any fatigue and any passivity.

Claims

1. A device for fluid, interactive control of a Compressed digital film, comprising a computer (1) having a sound output (2). a video output (3), both Connected to a display system (4), and an input (5) connected to a direction controlling means (6) in series with a means (7) for controlling the running speed of the film, characterised in that the direction-controlling means (6) comprises a number of buttons each corresponding to a direction in space and each enabling the computer (1) to process the direction chosen by a user.

2. A device according to claim 1, characterised in that the speed-controj means (7) is a lever which is operated by the user In order to unwind the film at a variable speed.

3-n A device according to claim 2n characterized in that the speed-control means (7) is a speed pick-up directly connected to a cardio-training apparatus.

4. A device according to claim 3, characterized in that the speed pick-up comprises a roller which is in contact with the moving part of the cardio-training apparatus and transmits the speed of rotation thereof to a toothed wheel situated between an infrared transmitter and an infrared receiver

5. A device according to claim 3, characterised in that the speed pick-up (7) comprises a magnet which moves at regular intervals in front of an induction coil providing information to the computer (1), which uses it to deduce the speed

6. A device according to claim 3, characterised in that the speed pick-up comprises an optical pick-up co-operating with the moving part of the cardio training apparatus.

7. A method of operating a device according to any of claims I to 6, characterized in that it comprises a step of acquiring the speed and direction 13 desired by a user, a first series of steps for determining the running speed of the pictures and a simultaneous second series of steps for determining the orientation from the desired speed and direction, and a step for displaying the thus-determined series of pictures.

8. A method according to claim 7, characterised by use of a configuration file containing the following information: - name of digital file to be opened in accordance with the filmed route, - information regarding the nature of the route and branches thereof and regarding the possible choices of direction. - speed at which the film has been shot, and - recovery of the information relating to the type of speed pick-up used.

9- A method according to claim *7, characterised in that the instantaneous speed of a user is calculated as the average of the n preceding speeds Vi.

10. A method according to claim 7, characterised in that the known information regarding the speed of motion of the camera when shooting the film and the number of images taken by the camera per second when running the film is used, in dependence on the speed to be simulated, to determine the jump S corresponding to the number of pictures to be skipped before each displayed picture and the time T corresponding to the time interval between two displayed pictures.

11. A method according to claim 7, characterised in that arrows indicating the. directions to be chosen are overprinted on the film a few seconds before each crossing of the filmed route.