WO2001037028A1

WO2001037028A1 - Method of improved image depth perception

Info

Publication number: WO2001037028A1
Application number: PCT/NZ2000/000227
Authority: WO
Inventors: Gavin Paul Davey
Original assignee: Equal Reverse Thoughts Limited
Priority date: 1999-11-15
Filing date: 2000-11-14
Publication date: 2001-05-25
Also published as: AU1901701A; WO2001037028A8

Abstract

In a first aspect, curvature distortions of an image recorded with a wide angle or fisheye lens are corrected by electronically or optically reversing the curvature, to create a substantially flat image. In a second aspect, an image is created by mosaicing several images (3A, 3B, 3C) of a view, having different respective orientations (1A, 1B, 1C). The resultant image generates a perception of depth, due to the peripheral images (3B, 3B) having a more sharply angled perspective than the center image (3A).

Description

METHOD OF IMPROVED IMAGE DEPTH PERCEPTION

TECHNICAL FIELD

This invention relates to a method of improving depth perception.

BACKGROUND ART

Two-dimensional representations of images are used in a wide variety of applications, ranging from television to photography to film, printing and so forth.

The visual impact of an image can be influenced by a variety of factors.

One limitation with two-dimensional images is that they often lack a perception of depth. This is a function of the two dimensional representation of an image, missing the third dimension of depth the humans perceive with their eyes.

Attempts have been made to give depth to a two dimensional image.

One example is holographic images. These are produced with laser technology, and are therefore expensive and complicated to produce. These images change appearance depending on the viewing position.

Some other methods include filing multiple shots of a scene simultaneously, around a scene (for example with ring cameras), and combining the images so that the views may be viewed from any angle. However, although this provides greater scene coverage, does not improve depth perception.

In films. IMAX technology has been used to give audiences a three-dimensional feel. The IMAX technology demands the use of an extremely large curved projector screen, and the use of multiple projection units. This understandably is quite expensive. Virtual imaging which allows interaction by a viewer with a scene is known. But this is expensive to generate and requires considerable processing power and equipment.

Other methods have the use of two-tone glasses in combination with the projected image to give a three dimensional effect. These have varying degrees of success, and require the viewing spectacles and a special type of film technology.

What would be of great advantage is a way of improving the depth perception of a two-dimensional image cheaply and with minimal technology.

Another problem with the recording and playing of images is that there can be a significant amount of visual information happening in a scene which cannot be readily recorded and relayed to a viewer in such a manner that the viewer can assimilate or even be exposed to all of that information.

It is an object of the present invention to address the foregoing problems or at least to provide the public with a useful choice.

Further aspects and advantages of the present invention will become apparent from the ensuing description, which is given by way of example only.

DISCLOSURE OF INVENTION

According to one aspect of the present invention there is provided a method of creating an image characterised by the steps of

(a) recording a first image or images using a greater angle of view than is normally available to the human eye, and

(b) manipulating the recorded image or images to form a second image by altering the angle of view to that approximating the normal human eye. An alternate way of considering the present invention is that the initial recorded image effectively looks curved, and the second step of the invention is to convert the image to effectively look substantially flat.

In preferred embodiments of the present invention the image is directly derived from a 'real' environment (as opposed to virtual).

In some embodiments however, the recorded image may be graphically produced. The present invention can be used for animated sequences as well.

The normal angle of view of a human is often represented in photography terms b) the use of a 50mm lens recording directly onto a 35mm film. A lens less than 50mm recording onto 35mm film tends to give a panoramic or wide angle view which has a greater angle of view than normally perceived by a human. An extreme example of a greater angle of view is that given by a fish-eye lens which is in the order of 5mm. This gives a distorted view similar to that looking at a reflection on a curved convex surface^' such as the back of a spoon.

Thus, these recorded images contain far greater information than is normally assimilated by a person looking at a view. However, the distortion perceived by the individual makes it difficult to assimilate or appreciate the images.

In a basic embodiment of the present invention described later on, the greater angle of view can be achieved by recording multiple images of a subject at different angles from a given viewpoint. For example, a camera taking still photographs which is mounted on a pivot can take those photographs at different angles.

Another variation which is described in greater detail later on is the taking of a single image, but with a wide angle of a view such as that afforded by a suitable lens, for example a fisheye lens.

The applicant has found that by manipulating these images, so they can be presented to a viewer in a less distorted form (e.g. substantially flat) gives considerable depth of field or perception of depth than a standard recorded image of the same view which uses a angle of view compatible with the average human eye.

According to one aspect of the present invention there is provided method of creating an image using an image recorder capable of creating a two dimensional image of a view, characterised by the steps of:

(i) creating an image ofthe view in a first orientation,

(ii) creating at least one further image of the view of at least a further orientation, having a common reference point to the first orientation,

(iii) mosaicing the images so that corresponding points on the images substantialh' align.

According to another aspect of the present invention there is provided an apparatus for creating an image, the apparatus including,

a first image recorder capable of creating a two dimensional image of a view, in a first orientation, and

at least a second image recorder capable of creating a two dimensional image, in at least a further orientation having a common reference point to the first orientation, and

a control means configured to process the images created by the first and second image recorders,

characterised in that the apparatus mosaics the images created so that corresponding points on the images substantially align.

According to a further aspect of the present invention there is provided an apparatus for the creation of a two dimensional image, which includes

at least, one image recorder capable of creating a two dimensional image,

control means to control the movement of the image recorder from the first orientation to a further orientation, having a common reference point to the first orientation,

characterised in that the apparatus is configured to create a first image in the first orientation, and

the control means moves the position of the image recorder to the further orientation of the view, where at least a second image is created, the images having corresponding points of reference.

In some embodiments the processing and control means of the present invention may be configured to mosaic the images taken, so that corresponding points on the images substantially align.

Reference to corresponding points on images means the objects or points recorded in separate images are aligned, so as to reduce or remove any overlapping lines, to create substantially one image.

There may be software provided that automatically calculates the optimum orientations and corresponding like points on the images produced in accordance with the present invention, to create the optimal depth perceptive effect.

Reference to orientations having a common reference point, may refer to a common curvature or rotation point for the orientations.

The orientations may be uniformly positioned about this reference point.

The orientations are intended to refer to views of an image at different angles. According to another aspect of the present invention there is provided an image produced by a method and apparatus substantially as described above.

Reference to an image recorder may be made with reference to a device that creates a two dimensional image from a three-dimensional view. For example this may include a photographic still camera, digital camera, video recorder, projection film camera and so forth.

In preferred embodiments however, the image recorder may be made with reference to a moving camera and includes the film required. Reference hereafter will therefore be made with reference to moving and still cameras, although this should not be seen to be limiting.

The number of views taken by the recording device will depend on the field of view that is to be represented on the two dimensional image.

The applicants have found that two images will produce the desired depth effect. However, three or greater mosaiced images created according to the present invention have been found to give an improved depth perception.

Reference to the view may be made with reference to a scene or shot from which an image is to be taken.

Reference to a view may also be made with reference to the image from a single frame of film.

Reference to moving the recording device may be made with reference to adjusting the camera orientation without, as much as is practically possible, moving the centre orientation of the camera. This includes moving the camera arc-wise.

Inevitably some displacement of the recording device from the point of rotation, or arc-wise movement may occur, however the effect on the image will be negligible if this movement is kept to a minimum.

The applicants have further found that first taking an image of substantially the centre orientation of a view, followed by the surrounding orientations produces an image with a perception of depth.

This is because when a person is viewing a real scene, images in the periphery appear curved or angled, having a more angled perspective.

The present method attempts to replicate this effect by photographing the periphery of the view so as to create greater perspective from the area surrounding the centre of the image.

This is because perspectives in human views are more greatly angled at the periphery of human vision. By providing images that have a more angled perspective, this effect is reproduced in a two-dimensional image.

The present invention allows the creation of depth perception that does not require complex technology, and is inexpensive.

According to one aspect of the present invention there is provided a method of creating an image characterised by the steps of:

(a) recording an image with a certain curvature of view, and

(b) manipulating the image with a reverse curvature of view

The present invention also lends itself to sophisticated techniques using a higher degree of technologies.

For example, a single image may be recorded with a wide angle or fisheye lens. Please note the term wide angle or fisheye lens should not be seen to be limited to necessarily photographic lenses, but refers to the recording of an image with the greater angle of view than by the normal human eye.

This image may be recorded on a variety of media. For example, the image may be recorded onto moving film, photographic film, or slide film. Such images may then be scanned or otherwise converted into an electronic image. In preferred embodiments, the images are recorded directly onto electronic media (such as a digital camera or charge coupled device (CCD) such as a television camera). Once the recorded image is in electronic form, then it can be readily manipulated in accordance with the present invention.

According to one aspect of the present invention there is provided software for manipulating an image recorded with a wide angle of view including the steps of mapping the image onto a corresponding co-ordinate system which effectively inverts the wide angle of view.

The invention includes any media onto which the software is recorded.

For example, the recorded image may effectively have a curvature conforming to the lens through which the image was recorded. The software program can then map that image onto a pixel net and then reverse the effect of curvature of the image creating a flat picture as an end result. This reversal of curvature creates a new image in accordance with the present invention which gives greater depth of perception than previously available.

It should be appreciated at the recording of the image into electronic media means that pixel recognition can be achieved making it relatively easy to join and

images. This could be a problem if normal physical media is used instead.

It should be appreciated that in some embodiments the reverse curvature may be achieved optically, for example with a traditional projector.

Alternatively, there may be provided cameras or other recording devices in accordance with the present inventions which has lenses that reverse the curve of the lens through which the original image is received.

In other embodiments, the image may be manipulated by providing the viewer with individual curved lenses which reverse the curvature of the recorded image.

In some embodiments, the film itself may actually be curved to reverse the curvature of the image caused by the lens. For example, curved guide tracks may be provided to wind the film along.

However, it is envisaged that the present invention be particularly useful when utilised in the electronic media.

For example, once the additional information is recorded in electronic media, it is possible for there to be provided the ability of a virtual viewer to look at specific areas of interest in the manipulated image.

As an example, the manipulated image may be a single scene. Because the scene was originally recorded at a wider angle than usual, the viewer can look at different aspects to that scene than would otherwise be unavailable with a still, flat photographic image.

In one embodiment of the present invention there may be provided the additional step of recording an image using two recording devices or media so as to simulate a stereoscopic view, akin to that normally experienced by a human viewer using both left and right eyes. In some embodiments of the present invention the images will not be fed separately to the eyes and instead overlaid to each other giving an effect of bifocal viewing which is common to most viewers.

It should be appreciated that this additional step in accordance with the present invention can give additional depth of perception. For example, if the left and right images are viewed independently to the person's left and right eyes (say through the use of special glasses) the depth perception is enhanced.

Reference has been made so far to the use of the present invention in relation to moving film. However, it should be appreciated that the present invention has application also to still film.

In some embodiments the angle of view may surround totally the viewer for example. 360 degrees. This may be achieved by using a number of cameras to film a certain view. Using software in accordance with the present invention to join the images together allows the viewer to experience what is happening around, behind, above and below them by moving their visual point.

For example, the present invention may be used with moving images such as that recorded for film or television.

A simple example of how the present invention may be used is the recording of a standard one camera interview. Normally, a standard camera is used to film one person responding to questions, then separately to record the other person asking the questions. The result is normally very stilted interview with little spontaneity or responsiveness shown between the interviewer and interviewee.

With the present invention, a single camera can be used (with the appropriate savings in cost and labour) to record both parties at the same time. This can be achieved by having a wide angle lens which can take in both of the parties unlike a conventional camera. The software programmes in accordance with the present invention can allow the editor of the recorded interview to show one or either or both of the parties being interview by merely selecting which parts of the information recorded is shown once manipulated to give a normal angle of view. This also has the added bonus of greater depth of perception.

The present invention can also be used in action filming. For example, a sports event may be filmed with the camera using the fisheye lens. That recorded image may then be manipulated by the present invention to provide a normal angle of view (and possibly a greater depth of perception). Once in an electronic media, it is possible for others to select individual parts ofthe action and focus on these.

As an example, a single camera may cover the whole of a rugby field with the fisheye lens. During the action a try may be scored. This may only occur in a small percentage of the whole of the action being recorded. However, the use of electronic media in accordance with the present invention enables a viewer to effectively select and zoom in on that action without distortion of the image.

In some embodiments of the present invention this selection of action may be utilised in a television broadcast. For example, there may be in-built a small delay between recording the image, manipulating it and then transmitted the selected action to the television watchers.

The present invention can also be used to edit a sequence to pull out specific elements. For example, several actions may be happening in a single scene. Each part of that scene may be pulled out and displayed in different sequence or be manipulated independently of each other to produce the final film version.

In one embodiment of the present invention the depth of perception may be enhanced by placing a black border around the screen on which the film sequence is being viewed. An object on it moving to the foreground of the screen may be shown as overlapping part of the black border giving the illusion of this part of the image as emerging from out ofthe screen, thus enhancing the three dimensional effect.

Once the present invention has a sequence recorded on electronic media it is not only possible to choose an event in that sequence, but also possible to effectively follow an event. For example, the event may be kicking a ball. The view can be displayed to follow the movement of that ball within the scene. Advanced variations of the present invention may even use pattern recognition to track certain players or object within the scene.

In other embodiments of the present invention, the image manipulation process in accordance with the present invention can be conducted on an image produced using a normal view lens, such as a 50mm lens, in order to give the image a perception of depth. It should be noted however that unlike the preferred embodiments of the present invention there will be no increasing of the density of information on the image.

It should now be seen that the present invention offers a number of advantages over the prior art.

One advantage is that a reduced number of cameras is required to record information from a scene or sequence. This is a result of the first step in the present invention to use wide angled or fisheye lenses to take in the additional information.

The subsequent manipulation of that wider angle of view to a more natural perspective gives greater depth of perception as well as enabling the viewer to take in additional information.

The use of electronic media to record the image onto means that the additional information can be readily selected according to the needs of the viewer.

There is minimal equipment and processing power required in the operation of the present invention.

BRIEF DESCRIPTION OF DRAWINGS

Further aspects of the present invention will become apparent from the following description which is given by way of example only and with reference to the accompanying drawings in which: Figure 1 illustrates various vertical views taken according to the present invention, and

Figure 2 illustrates, from a plan perspective, other views taken according to the present invention, and

Figure 3 illustrates conceptually a software mapping technique in accordance with the present invention.

BEST MODES FOR CARRYING OUT THE INVENTIONS

Figure 1 shows a side on representation of one aspect ofthe present invention.

A camera 1 is positioned in front of a view which includes objects 2 and 2A.

Camera 1 is shown in three orientations, 1A, IB, 1C. The orientations are intended to show three vertical views ofthe scene of object 2 and 2A.

Orientation 1A relates to view 3A shown by the dotted lines. Orientation 1 C relates to view 3C and orientation IB relates to view 3B.

The views are converted to a two dimensional image such as a photograph.

It can be seen that the views 3A, 3B and 3C have intersecting or common image points as the image fields overlap.

When the photographs for views 3A, 3B and 3C are mosaiced so that the common points align, the images in 3B and 3C will have a more sharply angled perspective aspect, when compared to the centre view 3A.

By staring at the centre of the image created by the mosaiced images collectiveh . and more realistic, a greater feeling of depth is appreciated by the viewer.

This is because perspectives in human views are more greatly angled at the periphen of human vision. By providing images that have a more angled perspective, this effect is reproduced under two-dimensional image.

Figure 2 shows a plan view of the view in figure 1. However, views 5A, 5B. and 5C correspond to camera 1 horizontal orientations 4A, 4B and 4C respectively. For example, referring to figure 1 again, in orientation 1C, view 3C may consist of three horizontal views at that pitch, which would correspond to 5A, 5B, and 5C in figure 2.

Camera 1 may be an apparatus that includes a plurality of image recorders, which may lenses, and films, digital image recorders and so forth. There may be one image recorder for each ofthe orientations 3A, 3B and 3C. dedicated for those views.

The camera 1 may therefore achieve the mosaiced effect by recording one shot of the view, which will include all ofthe desired perspective orientations.

There may be processing circuitry provided to automatically combine the images taken by the image recorders, intersecting the common image points, producing the image with the greater feeling of depth.

Alternatively, camera 1 may include a control means that may automatically shifts the orientation of the camera 1 to correspond to the views 3 A, 3B and 3C and/or horizontal orientations 4A, 4B and 4C.

The processing circuitry may be configured to automatically calculate and assess the optimum orientation angles to achieve the greatest depth perception effect possible. and may also establish what common image points will result in the image with the greatest depth perceptive effect.

The greater the number of perspective orientations photographed, will determine the complexity and effect of the feeling of depth.

For example a mosaiced image of nine views may have a greater feeling of depth that using only three mosaiced images.

A great deal more perspective information is now in the periphery of the mosaiced image.

In this way, images created according to the present invention are more in accordance with how the brain perceives images at the periphery of a view, and therefore gives a greater, depth perception.

Figure 3 illustrates conceptually software mapping in accordance with the present invention.

The wide angle of view (or curvature) with which the image was recorded is input into the program. The program then effectively places the pixels as if they were on that curvature. Then, the program effectively retracts the pixels off a focal point onto a surface of equivalent curvature. This effectively flattens the image.

Aspects of the present invention have been described by way of example only and it should be appreciated that modifications and additions may be made thereto without departing from the scope ofthe appended claims.

Claims

CLAIMS:

1. A method of creating an image characterised by the steps of

a) recording a first image or images using a greater angle or view than is normally available to the human eye, and

b) manipulating the recorded image or images to form a second image by altering the angle of view to that approximating the normal human eye.

2. A method as claimed in claim 1 wherein the initial recorded image effectively looks curved, and the recorded image is manipulated to convert the image to effectively look substantially flat.

3. A method as claimed in either claim 1 or claim 2 wherein multiple images are recorded of a subject at different angles from a given view point.

4. A method as claimed in any one of claims 1 to 3 wherein a single image is recorded with a wide angle of view.

5. A method of creating an image using an image recorder capable of creating a two-dimensional image of a view, characterised by the steps of:

i) creating an image ofthe view in a first orientation,

ii) creating at least one further image of the view of at least a further orientation having a common reference point to the first orientation, and

iii) mosaicing the images so that corresponding points on the images substantially align.

6. An apparatus for creating an image, the apparatus including,

a first image recorder capable of creating a two-dimensional image of a view, in a first orientation, and

at least a second image recorder capable of creating a two-dimensional image, and at least a further orientation having a common reference point to the first orientation, and

a control means configured to process images created by the first and second image recorders,

7. An apparatus for the creation of a two-dimensional image, which includes

at least one image recorder capable of creating a two-dimensional image,

a control means to control the movement of the image recorder from the first orientation to a further orientation, having a common reference point to the first orientation,

8. A method as claimed in claim 7 wherein the processing control means is .configured to mosaic the images taken, so that corresponding points on the images substantially align.

9. Software which automatically calculates the optimum orientations and corresponding light points on the images produced in accordance with the present invention, in order to create the optimal depth perceptive effect.

10. An image produced by the methods and/or an apparatus substantially as claimed previously.

11. A method of creating an image characterised by the steps of:

a) recording an image with a certain curvature of view, and

b) manipulating the image with a reverse curvature of view.

12. Software for manipulating an image recorded with a wide angle of view which includes the steps of mapping the image onto a corresponding co-ordinate system which effectively inverts the wide angle of view.

13. Software as claimed in claim 12 which maps an image onto a pixel net and reverses the effect of curvature of the image to create a substantially flat image.

14. A method of manipulating an image characterised by the steps of using curved lenses which reverse the curvature ofthe recorded image.

15. A method of broadcasting characterised by the steps of recording an image, manipulating the image in accordance with the present invention and then broadcasting the manipulated image.

16. A method of creating an image substantially as herein described with reference to and as illustrated by the accompanying drawings.

17. An apparatus for creating an image substantially as herein described with reference to and as illustrated by the accompanying drawings.

18. An image produced by the methods substantially as herein described with reference to and as illustrated by the accompanying drawings.

19. Software for manipulating an image substantially as herein described with reference to and as illustrated by the accompanying drawings.

20. Recording devices substantially as herein described with reference to and as illustrated by the accompanying drawings.