GB2297877A

GB2297877A - Visual simulation using 'pin-cushion' and 'barrel' distortion

Info

Publication number: GB2297877A
Application number: GB9502656A
Authority: GB
Inventors: Ian Robert Lawrence
Original assignee: Loral Europe Ltd
Current assignee: Lockheed Martin UK Ltd
Priority date: 1995-02-10
Filing date: 1995-02-10
Publication date: 1996-08-14
Also published as: GB9502656D0

Abstract

Apparatus for producing an image for visual simulation, for example, for virtual reality systems, comprises a display 10 having substantially uniform resolution across the field of view, and means for displaying an intermediate image of a source on the display so that a first portion of the source occupies a disproportionately larger fraction of the intermediate image than a second portion 'barrel distortion' 20. An optical system 14 for imaging the display restores the original proportionality between the first and second portions displayed in the intermediate images by 'pin-cushion distortion' 22 so that the first portion is imaged with a higher resolution than the second portion. Preferably, the higher resolution region has a resolution greater than or equal to the limit of resolution of an observer's eye.

Description

VISUAL SIMULATION IMAGE PRODUCTION The present invention relates to a method and apparatus for producing an image for visual simulation, for example for virtual reality systems.

With the current upsurge in virtual reality systems, for information, educational and entertainment purposes, the parallel need has arisen for display systems which present the user with images of sufficient realism and over a sufficiently wide angle to give the sensation of reality.

The human eye is capable of resolving detail to approximately one minute of arc and has a field of view which in places exceed loo0. To provide the sensation of realism in a display system, particularly a helmet mounted display system where the real world view is excluded, it is important to provide a field of view in excess of 600. This leads to the widely held view that the illusion of reality can only be provided by display systems which have a resolution approaching 4000 lines (60 minutes over 600 = 3600 minutes of arc).

Generally a compromise is adopted, ranging from comparatively low resolution TV systems operating over a narrow field of view to highly expensive wrap-around display systems employing a plurality of contiguously integrated image projectors to provide wide field and high resolution. Although the wrap-around display systems can be very realistic, they are highly complex which is an obvious disadvantage.

According to the present invention, there is provided apparatus for producing an image for visual simulation, which has a higher resolution in one region than another, comprising: a display having substantially uniform resolution across the field of view; means for displaying an intermediate image of a source on the display in which a first portion of the source occupies a disproportionately larger fraction of the intermediate image than a second portion; and a transforming optical system for imaging the display which restores the original proportionality between the first and second portions displayed in the intermediate image, whereby the first portion is imaged with a higher resolution than the second portion.

Therefore, an image produced in accordance with the present invention has a higher resolution in one region (high resolution region) than an in another region (low resolution region). Preferably the high resolution region is surrounded by the low resolution region. In an especially preferred form, the high resolution region has a resolution greater than or equal to the limit of resolution of an observer's eye.

The present applicants have appreciated that a high resolution image need only be provided across a fraction of the total field of view. The human eye is only capable of resolving to its maximum degree, one minute of arc, over a very narrow field of view, approximately 20. Beyond this region of optimum sensitivity, known as the foveal region of the retina, the resolving power of the eye falls off rapidly. For example at a field angle of 100 the resolving power has fallen to 2.5 minutes of arc and at approximately 45 this has fallen further to 10 minutes of arc. The natural variation in the resolution acuity of the human eye can be used to beneficial effect in providing a wide field of view display.If the high resolution portion of the image fills the foveal region of an observer's eye, the image would be perceived by the observer as having high resolution throughout the entire field of view, even when the resolution in the image outside the foveal region is much lower.

Preferably, there is a gradual variation in the resolution in the image between the regions of high and low resolution. In the preferred embodiment, the intermediate image is a barrel distorted image of the source and the optical system produces a pin-cushion distortion of the intermediate image.

The degree of distortion in the intermediate image, which is corrected by the optical system, can be selected to give an image with a resolution which varies with angle in much the same way as the limit of resolution of the eye deteriorates with angle.

In a preferred arrangement, there is provided an eye monitoring system which is linked to the image source. The eye monitoring system determines any shift in the viewing axis of the eye and alters the image source accordingly. Preferably, the apparatus includes viewing optics, linked to the eye monitor, which can be varied so that the high resolution region of the image is always directed to the foveal region of the eye.

A system without the eye monitoring system will only provide a convincing high resolution image across the field of view if the foveal region of the eye is axially aligned with, or at least filled by, the high resolution region of the image.

This condition is readily attained if the observer is looking through gunsights or the like. However, in other applications such as flight simulators where it is necessary for the eye to move to search off-centre areas of the image, the fall off in resolution would readily become apparent as soon as the eye moved away from the high definition region.

The image source maybe generated by computer, and the intermediate image formed by a mathematical mapping operation performed on the computerised image source.

An embodiment of the invention will now be described in detail, by way of example, with reference to the accompanying drawing in which Figure 1 shows schematically apparatus in accordance with the invention.

An intermediate image of a source (not shown) is displayed on a display 10. The intermediate image is viewed by an observer represented by 12 through a pin-cushion optical system 14. The intermediate image is a barrel distorted image of the source; the barrel distortion being corrected by the pincushion optical system 14. The apparatus includes viewing optics which consists of two mirrors 16,17 which are positioned in a simple "periscope" type arrangement.

The source is a high definition picture stored on a computer. The barrel distortion can be effected by transforming the computerised picture using a mapping operation about the centre of the picture. The distortion essentially enlarges those portions of the picture closest to the centre and reduces those portions furthest away without introducing discontinuities. The effect of the mapping operation on a uniform grid pattern is shown at 20 in Figure 1 by way of illustration The display 10 is a conventional TV screen on which images are represented by a uniform array of pixels produced by a raster scan. A pixel is the smallest element with controllable colour and brightness in the display, and the spacing between adjacent pixels is the limit of resolution of any intermediate image on the display.

The pin-cushion optical system 14 is a lens arrangement which has the reverse distortion characteristic to the barrel distortion, so that the combined effect of the two is a distortion-free image of the computer picture. In practice, the pin-cushion optical system 14 is chosen first, and the barrel distortion calculated to be complementary to it to ensure distortion-free imaging. A uniform grid pattern imaged through the pin-cushion optical system is shown at 22 in Figure 1 by way of illustration. When the display is viewed through the pincushion optical system 14, central portions of the intermediate image are diminished whilst peripheral portions are enlarged.

In achieving this, the pin-cushion optical system changes the resolution in the image. In the diminished portions, the pixel spacing is apparently reduced, whilst in the enlarged portions, the pixel spacing is apparently increased.

When an observer looks at the centre of the display, through the pin-cushion optical system 14, the high resolution region of the image occupies the foveal region of his eye, while the lower resolution regions of the image fall on peripheral, less resolution-sensitive parts of the eye.

The resolution in the lower resolution regions decreases with increasing distance from the centre of the image in a manner which approaches the natural variation in resolution with angle in the eye. This gives the impression to the observer of an image of high resolution over the entire field of view, so long as the eye does not move off centre. If the visual simulation is actually used in, for example, a periscope, the eye will always be trained on the central portion of the display. The observer looks at a different part of the image by moving the periscope not his eye. Any movement in the periscope can be detected and used to bring about a corresponding movement in the image source so that the impression of high resolution over the entire field of view can be maintained.

In other applications, for example where a headset is used, the observer may move his eye not the headset to look at a different part of the image. As soon as the foveal region of the eye is no longer focused on the high resolution region of the image, the drop in resolution would rapidly become apparent.

Therefore, to maintain the impression of a wide angle, high resolution image, any movement in the eye must be detected and measured, and the information used to bring about a corresponding change in the image source. The intermediate image must move in sympathy with any movement in the eye to maintain the perception of reality. The eye movement can be monitored with a small charge coupled TV camera 24 which sees the eye through mirror 16 which is half silvered. The camera 24 is linked to a simple image processing system which locates the position of the observer's eye pupil relative to a nominal forward viewing axis. Any angular movement in the eye can be used to alter the image source by an analogous amount. The half silvered mirror 16 in the viewing optics is deflected by an amount determined by the image processing system to ensure that the high resolution region of the image is always aligned with the foveal region of the eye. This enables the pin-cushion optical system 14 and the intermediate image to remain aligned with the centre of the display 10.

Extension from a monocular system to a binocular system, for which limited stereoscopic viewing effects could be simulated, could be achieved by a doubling of the viewing channels with the sharing of a common image database.

Claims

1. Apparatus for producing an image for visual simulation which has a higher resolution in one region than another, comprising: a display having substantially uniform resolution across the field of view; means for displaying an intermediate image of a source on the display in which a first portion of the source occupies a disproportionately larger fraction of the intermediate image than a second portion; and an optical system for imaging the display which restores the original proportionality between the first and second portions displayed in the intermediate image, whereby the first portion is imaged with a higher resolution than the second portion.

2. Apparatus according to claim 1 in which the second portion surrounds the first portion.

3. Apparatus according to claim 1 or 2 in which there is a gradual variation in the resolution in the image between the first and second portions.

4. Apparatus according to claim 3 in which the intermediate image is a barrel distorted image of the source.

5. Apparatus according to claim 4 in which the optical system produces a pin-cushion distortion of the intermediate image.

6. Apparatus according to any preceding claim further comprising means for sensing the position of the pupil of an observer's eye when viewing the image, the sensing means being in communication with the image source such that the intermediate image displayed on the display moves in sympathy with any movement of the eye.

7. Apparatus according to claim 6 further comprising means for aligning with the centre of the display with the eye, the aligning means being in communication with the sensing means and operating in dependence on the position of the pupil of the observer's eye.

8. Apparatus according to any preceding claim further comprising a computer for generating the source.

9. A method of producing an image which has a higher resolution in one region than another, comprising: displaying an intermediate image of a source on a display in which a first portion of the source occupies a disproportionately larger fraction of the intermediate image than a second portion, the display having substantially uniform resolution; and imaging the display through an optical system which restores the original proportionality between the first and second portions displayed in the intermediate image, whereby the first portion is imaged with a higher resolution than the second portion.

10. A method according to claim 9 in which the second portion surrounds the first portion.

11. A method according to claim 9 or 10 in which the resolution in the image varies gradually between the first and second portions.

12. A method according to claim 11 in which the intermediate image is a barrel distorted image of the source.

13. A method according to claim 12 in which the optical system produces a pin-cushion distortion of the intermediate image.

14. A method according to any of claim 9 to 13 further comprising: sensing the position of the pupil of an observer's eye when viewing the image; and changing the intermediate image displayed on the display in dependence on any change in the position of the pupil.

15. A method according to claim 14 further comprising aligning the centre of the display with the direction of observation.

16. A method according to any of claims 9 to 15 further comprising generating the source by computer.

17. Apparatus for producing an image for visual simulation substantially as herein before described with reference to Figure 1.

18. A method of producing a image which has a higher resolution in one region than another substantially as herein before described with reference to Figure 1.