WO1999025120A1

WO1999025120A1 - Composite movie generating camera assembly and method

Info

Publication number: WO1999025120A1
Application number: PCT/IL1998/000544
Authority: WO
Inventors: Shmuel Banitt; Daniel Shor
Original assignee: Shmuel Banitt; Daniel Shor
Priority date: 1997-11-12
Filing date: 1998-11-09
Publication date: 1999-05-20
Also published as: IL122183A0

Abstract

A method and apparatus for generating a composite, multi-field of view movie, by filming a sequence in a main, central field of view (5) and simultaneously filming at least two lateral sequences in fields of view (6, 7) lateral to the main, central field of view (5). The main, central sequence and the two lateral sequences are filmed at different times. The field of view (6, 7) of each of the two lateral sequences is wider than the field of view (5) of the main, central sequence. The main, central sequence and the lateral sequences are combined and a composite, three-dimensional like movie sequence is generated by selecting the overlap (8, 9) between each of the lateral sequences and the corresponding portion of the main, central sequence, which provides a suitable match between adjacent and overlapping fields of view.

Description

COMPOSITE MOVIE GENERATING CAMERA ASSEMBLY AND

METHOD

Field of The Invention

The present invention relates to apparatus and methods for generating

composite movies.

BACKGROUND OF THE INVENTION

There are three several known methods to perceive depth. Examples are a

stereoscopic perception of depth, a psychological perception of depth using

visual cues and the use of a changing focus to perceive depth. A viewer

viewing a scene in which the focal point changes will perceive that the scene has depth.

Stereoscopic depth perception requires that the viewer view a scene from two

different locations at the same time. In real life, this means that he sees the

same scene using both his left eye and right eye to see different images.

When two dimensional visual images are substituted for real life, it requires

the use of two images taken from different right eye and left eye locations

respectively.

On the other hand, a viewer can perceive depth by virtue of a number of

psychological depth perception cues which are present in both real life and in

two dimensional visual images. Common cues include a real perspective effects where the change of coloring or softening of the shapes gives a feehng

of distance to far objects, distribution of light and shade on an object,

overlapping of contours as when a near object hides part of a more distant

object, geometrical perspective for example in the case of the converging of the

parallel rails of a railroad track as they extend in the distance, movement of

an object in a visual image either towards or away from the viewer or

audience, and movement for stimulating the Pulfrich effect. This is true of all

the various devices in which two dimensional visual images are projected onto

screens. Such devices include home entertainment systems including

televisions, motion picture theaters, training simulators, multi-media systems

and so on having applications in the areas of entertainment, work, education,

exhibitions, training, etc.

A number of devices have been proposed to permit a viewer or an audience to

view three dimensional visual images or at least the illusion of three

dimensional visual images when viewing two dimensional visual images.

These fall into three groups: those which provide stereoscopic viewing, those

which create the psychological illusion of depth and those which provide three

dimensional visual images. Display images on a three-dimensional display

system, such as a surrounding screen.

The devices providing stereoscopic viewing generally require the projection of

two pictures of the same view, taken simultaneously by more than one

camera from slightly different angles, to emulate the binocular vision of normal eyes. Several examples are shown in U.S. Patent Nos. 4,420,230 to

McElveen, 4,714,319 to Zeevi, and 5,225,861 to Marks. The use of a vibrating

mirror or screen is shown in U.S. Patent Nos. 4,130,832 to Sher and 3,814,513

to Nims et al. Other systems require special spectacles to be worn by the

viewer or other devices which direct the proper view to the proper eye. All

these systems are cumbersome and costly to use. Also, they tend to cause an

uneasy feeling in the observer often resulting in headaches that may become

severe. Thus these systems have failed to become popular.

The second approach is shown in U.S. Patents 4,651,219 to Rickert, 3,582,961

to Shindo and 4,000,367 to Field wherein depth perception is provided by

isolating the image to be viewed. In this way, the stereoscopic cues of the

space surrounding the image are suppressed so the inherent psychological

depth indicators in the picture can be sensed by the observer who perceives a

two dimensional visual image as a three dimensional scene. Other devices

which produce a similar result are described in U.S. Patent No. 3,820,873 to

Redington et al., U.S. Patent No. 4,154,514 to Harvey which utilize a curved

screen, and U.S. Patent No. 4,941,041 to Kenyon which utilizes the Pulfrich

illusion.

U.S Patent No. 2,468,046 to de los Monteros discloses an apparatus for

projecting and viewing images with a depth effect through the use of at least

one mirror to reflect the image from a projector onto a screen such that a degree of displacement between the images produces the illusion of

stereoscopy.

U.S. Patent 3,514,871 to Tucker discloses a wide angle visual display for a

training simulator which eliminates distracting discontinuities found in many

wide angle displays. It does so by providing three virtual image lenses

between the viewing point and the two dimensional displays and by providing

substantial duplication of the projected image on the marginal portions of the

displays. This creates binocular vision at the corners of the displays which

gives the illusion of three dimensionality .

U.S. Patent 5,274,405 to Webster describes a device into which the head is

placed wherein images are projected both to the area of visual attention and

the area of peripheral vision of each eye. The images are focused on the

screens to provide sharp stereoscopic viewing throughout the field of view.

Still another method which has gained some popularity is the so-called Imax

system., of the IMAX Corporation, (Toronto, Ontario, Canada). In this system,

the images are taken simultaneously by several motion picture cameras

rigidly mounted together, or a single camera with large field of view with a 70

mm film. The images thus acquired are then displayed in a properly fitted

cinema theater with similarly configured projectors or a single projector for 70

mm film with special optical correction. The resulting three-dimensional

motion picture gives the viewer a very realistic feehng that the action is actually taking place around him. Further, this system does not suffer from

the drawbacks of stereoscopic viewing systems - the need for special glasses

and the uncomfortable feeling.

Other methods for producing three-dimensional moving images are based on

computer generated images. An example is U.S. Patent 5,184,956 to Langlais

et al., where computer generated data is used to produce a three-dimensional

display for the purposes of driving training. It is easy to appreciate by any one

skilled in this field that it is not difficult to adapt computer generated

imaging techniques to many types of display systems such as stereoscopic

display, wide angle display, and also three-dimensional display. This is so

because entire information required of each part of the image is known to the

computer's program - including all the three-dimensional structure of each

object, its surface color, texture, and so on. This seeming advantage is also the

major disadvantage of such systems: the preparation of a realistic image

requires painful and elaborate design of each and every detail. Thus the

display is either non-realistic or else extremely expensive and laborious.

It is known in the art to prepare 3D-like movies (the so-called "Cineramic"

movies) by filming each scene using three connected cameras which operate

simultaneously. This arrangement is schematically shown in Fig. 1.

According to this arrangement, three distinct cameras, 1, 2, and 3 are

connected to a common platform 4, and are directed in different directions. The angular position of each camera is selected such that their fields of view,

1', 2' and 3', at the desired distance, are complementary, as seen in Fig. 1.

However, the arrangement of Fig. 1 is impractical in many situations,

particularly when filming scenes involving actors or effects, since in order to

film the central (main) field the lateral fields must be used by personnel and

equipment. For this reason, panoramic movies made by the technique

illustrated in Fig. 1 are almost entirely limited to outdoor scenes, such as

panoramic views or flight scenes.

Recently, a method has been developed by the same apphcants hereof, and

disclosed in published PCT Patent Application WO 95/33340

(PCT/US95/06700), the description of which is incorporated herein by

reference, which provides for the displaying of three dimensional visual

images and for the stimulation of the feehng of viewer participation with the

displayed image. This includes a method for displaying three dimensional

visual images, comprising the steps of:

(a) displaying a primary two dimensional visual image of a first

scene; and

(b) displaying at least two secondary visual images on at least two

peripheral screens inclined with respect to said main screen

thereby to transform said primary two dimensional visual image

into a three dimensional visual image, wherein said at least two secondary visual images are processed from

images of at least second and third scenes which differ in time or

significantly in space from said first scene; and

wherein each of said at least two secondary visual images matches said

primary visual image so as to create the appearance of displaying a single

scene.

Briefly, said PCT application, and a copending U.S. Patent Application Ser.

No. 08/760514, filed December 5, 1997, by the same apphcant herein, and the

contents of which are also entirely incorporated herein by reference, describes,

inter alia, a method of generating a composite, three-dimensional-like image

sequence by the assembly of two or more separate image sequences,

comprising the steps of:

a) providing a main sequence of images;

b) providing at least one secondary sequence of images the contents of

which are at least roughly related to the contents of said main sequence;

c) selecting the positioned relationship of each of said at least one

secondary sequence of images and of said main sequence of images;

d) processing each of said at least one secondary sequence of images so

as to optimize its matching with said main sequence in the selected positioned

relationship with it; and

e) displaying said main sequence of images and said secondary

sequences of images at the same time and in substantially the same

positioned relationship as selected in step (c) above. Therefore, there is a need which the prior art has failed to fill, for apparatus

and methods which can be used for making composite movies in an efficient

manner, taking into account all the aforesaid problems and limitations.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide apparatus which can be used in manufacturing multiple-screen movies.

It is another object of the invention to provide a method for manufacturing multiple-screen movies, which comprises the simultaneous filming of more than one field of view.

It is yet another object of the invention to provide a method for combining scenes filmed by the said method with other scenes, into a multiple-screen, 3D-like movie.

Other objects and advantages of the invention will become apparent as the description proceeds.

The method for generating a composite, multi-field of view movie, according

to the invention, comprises the steps of:

a) filming a sequence in a main, central field of view;

b) simultaneously filming at least two lateral sequences in fields of

view lateral to said main, central field of view, wherein:

1) said main, central sequence and said at least two lateral

sequences are filmed at different times; and 2) the field of view of each of said at least two lateral sequences

overlaps with the field of view of said main, central sequence;

c) combining said main, central sequence and said at least two lateral

sequences, to generate a composite, three-dimensional like movie sequence,

by selecting the overlap between each of said at least two lateral sequences

and the corresponding portion of the main, central sequence, that provides a

suitable match between adjacent, overlapping fields of view.

Step c) above can be performed in different manners. For instance, the

combination of the central and the lateral sequences can be effected by

cropping part of the lateral sequences which are complementary to the

central sequence. The overlap between the adjacent fields of view is useful

in aiding the process of adjusting the movements of the camera, and allows

for slight mismatchings in the movement of the camera between the central

main sequence and the lateral sequences.

As used herein, the term "lateral sequences" is intended to include both

sequences belonging to the right or left side of the central main field of view,

as well as those which belong to fields of view located above or below the

central main field of view. Furthermore, the term "central field of view" or

"main sequence" are meant to indicate the sequence around which the

lateral sequences are combined, but it is not meant to limit the type of shot

or its content in any way. For instance, a lateral view can be used as a "main sequence", and the central and the other lateral view can be prepared

by other means, as further described hereinafter.

According to a preferred embodiment of the invention each lateral portion is

matched to the main, central portion independently of the second lateral

portion. According to another preferred embodiment of the invention both

lateral portions are matched to the main, central portion simultaneously,

using the fact that both lateral sequences were filmed from the same rigid

platform.

While it is possible to employ more than two lateral sequences, typically two

lateral sequences are filmed. According to a preferred embodiment of the

invention the two lateral sequences are filmed in a synchronized manner.

In one particular embodiment of the invention the number of frames of the

central field and of the lateral fields is the same. In another preferred

embodiment of the invention, the number of frames of the central field

differs from that of the lateral fields.

When the same type of film is used for all fields of view, it is convenient that

the cameras used for each field of view are essentially the same. However, it

is possible - and this is an advantage of the present invention - to operate so

that the lateral fields are filmed with a different quality. The invention is also directed to apparatus for generating a composite,

multi-field of view movie, comprising:

a) means for filming a sequence in a main, central field of view;

b) means for simultaneously filming at least two lateral sequences in

fields of view lateral to said main, central field of view, wherein:

1) said main, central sequence and said at least two lateral

sequences are filmed at different times; and

2) the field of view of each of said at least two lateral sequences

is wider than the field of view of said main, central sequence;

c) means for combining said main, central sequence and said at least

two lateral sequences, to generate a composite, three-dimensional like

movie sequence, by selecting the overlap between each of said at least two

lateral sequences and the corresponding portion of the main, central

sequence, that provides a suitable match between adjacent, overlapping

fields of view.

As will be apparent to a skilled person, it is desirable to provide in the

apparatus means for synchronizing the filming of the two lateral sequences.

According to a specific preferred embodiment of the invention the apparatus

for filming two lateral sequences, comprises two cameras connected to a

common platform, said cameras being connected at an angle such that the

axes of their respective lenses cross. In an alternative embodiment of the invention a two field camera is built

according to the known art of stereo cameras, including means of

synchronization and recording two simultaneous image sequences.

However, as opposed to a stereo camera, the lenses of the camera of the

invention point outwards.

The movie generated according to the invention can be displayed in a

variety of ways, e.g., it can be displayed on a flat surface, or on a concave

surface, or on a plurality of juxtaposed surfaces, depending on the effect that

it is desired to obtain, and on technical requirements and limitations of the

location where the movie is displayed.

All the above and other characteristics and advantages of the invention will

be better understood through the following illustrative and non-limitative

description of preferred embodiments thereof.

Brief Description of the Drawings

In the drawings:

Fig. 1 is a schematic representation of a three-camera arrangement

according to the prior art;

Fig. 2A illustrates the side-by-side connection of fields of view,

according to the prior art; Fig. 2B illustrates the generation of a composite view, according to a

preferred embodiment of the invention; and

Fig. 3 schematically illustrates an arrangement according to a

preferred embodiment of the invention, which is suitable for carrying out

the method.

Detailed Description of Preferred Embodiments

Turning now to Fig. 2, the method of the invention is illustrated in

comparison with the prior art method. In Fig. 2A a three-screen movie is

generated by placing side-by-side three simultaneously taken scenes, e.g., as

illustrated in Fig. 1, so that fields of view 1', 2' and 3', placed side by side,

provide a panoramic picture.

Fig. 2B, on the other hand, illustrates a method according to a preferred

embodiment of the invention. According to this method, a central, main field

of view 5 is filmed separately. Then, the two lateral fields of view, 6 and 7,

are simultaneously filmed. The fields of view 6 and 7 are filmed using a

wider lens than the main field 5. Fields 6 and 7 can be filmed before or after

the main field, i.e., after the equipment and personnel needed for filming

the main, central field 5, have been removed.

Because of the fact that a wider lens is used for the lateral fields, there is

overlap between the central and the lateral fields in the areas indicated by

arrows 8 and 9. Thus, when manufacturing the composite movie it is possible to adjust each frame, or sequence of frames, by moving (or

otherwise manipulating) areas 6 and 7 relatively to central field 5, so as to

obtain the best fit between the three fields and to generate a whole

composite picture providing the best feeling of continuity between the fields.

The resulting lateral sequences are indicated by numerals 10 and 11. The

feeling of continuity is of importance in order to provide the feeling of 3-

dimensionality of the movie. Of course, since the angle at which the lateral

fields have been filmed is known and both cameras are rigidly mounted on

the same platform, the process of fitting the lateral and the central fields is

greatly simplified.

One arrangement suitable for carrying out the invention is schematically

shown in Fig. 3, which shows two cameras, 12 and 13, connected to a

common support 15. These two cameras have crossing fields of view, as

shown in the figure, and are synchronized. As will be apparent to the skilled

person, this arrangement permits to avoid parallax problems, since the two

cameras can be located with their lenses near to each other.

Numeral 14 indicates the position of the main camera, which is shown in

broken lines since it is not present when cameras 12 and 13 are in use, but

at a different time. Again, the spatial positioned relationship of camera 14

with cameras 12 and 13 is such that all fields can be filmed substantially

from the same position, as they would be seen from a spectator, thus

enhancing the feeling of reality. The method of the invention substantially reduces problems associated in

the prior art with parallax effects. This is illustrated by Fig. 3, in which it

can be seen that it is possible to position lateral cameras 12 and 13 at a

distance which is roughly half their minimal distance when central camera

14 is also physically present. Roughly speaking, if the prior art system

shown in Fig. 1 permits to film bodies as near as 20 meters, without

meeting with strong parallax effects, in the arrangement of Fig. 3 this

distance can be reduced to about 10 meters.

Of course, as stated above, more than one consecutive lateral fields can be

provided on each side of the central main sequence. Providing additional

adjacent sequences is carried out similarly to what has been described above

for the first adjacent field of view, and the same techniques can be applied,

mutatis mutandis, also to multiple lateral additions.

Furthermore, the angle between the cameras may need to vary when zoom

is applied to a given shot, in order to keep the lateral fields in match with

the central field. For instance, the angle needs to be reduced when a zoom in is applied to a given shot.

The above description has been provided for the purpose of illustration and

is not intended to limit the invention in any way. Many modifications can be

made in the method and apparatus of the invention. For instance, different frame matching techniques can be used, and different cameras can be

employed for filming scenes involving the lateral fields; or more than two

lateral scenes can be taken, to provide larger field movies, all without

exceeding the scope of the invention.

Claims

1. A method for generating a composite, multi-field of view movie,

comprising the steps of:

a) filming a sequence in a main, central field of view;

b) simultaneously filming at least two lateral sequences in fields of

view lateral to said main, central field of view, wherein:

1) said main, central sequence and said at least two lateral

sequences are filmed at different times; and

2) the field of view of each of said at least two lateral sequences

is wider than the field of view of said main, central sequence;

c) combining said main, central sequence and said at least two lateral

sequences, to generate a composite, three-dimensional like movie sequence,

by selecting the overlap between each of said at least two lateral sequences

and the corresponding portion of the main, central sequence, that provides a

suitable match between adjacent, overlapping fields of view.

2. A method according to claim 1, wherein each lateral portion is matched to

the main, central portion independently of the second central portion.

3. A method according to claim 1, wherein both lateral portions are matched

to the main, central portion simultaneously. 4. A method according to any one of claims 1 to 3, wherein two lateral

sequences are filmed.

5. A method according to any one of claims 1 to 3, wherein more than two

lateral sequences are filmed.

6. A method according to any one of claims 1 to 5, wherein the lateral

sequences comprise fields of view located above and/or below the central

field of view.

7. A method according to any one of claims 4 to 6, wherein the lateral

sequences are filmed in a synchronized manner.

8. A method according to any one of claims 1 to 7, wherein the number of

frames of the central field and of the lateral fields is the same.

9. A method according to any one of claims 1 to 7, wherein the number of

frames of the central field differs from that of the lateral fields.

10. A method according to any one of claims 1 to 9, wherein the cameras

used for each field of view are essentially the same.

11. A method according to any one of claims 1 to 9, wherein the lateral fields

are filmed with a lower quality. 12. Apparatus for generating a composite, multi-field of view movie,

comprising:

a) means for filming a sequence in a main, central field of view;

b) means for simultaneously filming at least two lateral sequences in

fields of view lateral to said main, central field of view, wherein:

1) said main, central sequence and said at least two lateral

sequences are filmed at different times; and

2) the field of view of each of said at least two lateral sequences

overlaps with the field of view of said main, central sequence;

c) means for combining said main, central sequence and said at least

two lateral sequences, to generate a composite, three-dimensional like

movie sequence, by selecting the overlap between each of said at least two

lateral sequences and the corresponding portion of the main, central

sequence, that provides a suitable match between adjacent, overlapping

fields of view.

13. Apparatus according to claim 12, comprising means for synchronizing

the filming of the lateral sequences.

14. Apparatus according to claim 12 or 13, wherein the cameras used for

each field of view are essentially the same. 15. Apparatus according to claim 12 or 13, comprising means for filming the

lateral fields with a lower quality than that of the main, central field of

view.

16. Apparatus for filming two lateral sequences, comprising two cameras

connected to a common platform, said cameras being connected at an angle

such that the axes of their respective lenses cross.

17. A method for generating a composite, multi-field of view movie,

substantially as described.

18. Apparatus for generating a composite, multi-field of view movie,

substantially as described and illustrated.