WO2023089181A1

WO2023089181A1 - Display system

Info

Publication number: WO2023089181A1
Application number: PCT/EP2022/082646
Authority: WO
Inventors: Andrzej KACZOROWSKI; Alfred James NEWMAN; Darius Martin Sullivan
Original assignee: Vividq Limited
Priority date: 2021-11-22
Filing date: 2022-11-21
Publication date: 2023-05-25
Also published as: GB202116782D0

Abstract

A display system comprising an image generating unit, an image replicating combined and an optical system. The image generating unit is configured to generate an image with depth information. The image replicating combiner comprises an input surface and an output surface, and is configured to generate a plurality of replications of an image incident on the input surface at output surface. The optical system is positioned in an optical path between the image generating unit and the input surface of image replicating combiner, and is arranged to transform a set of rays from the image generating unit into a plurality of sets of rays with respective different focal points, such that the plurality of sets of rays, when incident on the input surface, combine on extraction at the output surface to give each replication substantially the same focal point at a viewing position.

Description

DISPLAY SYSTEM

Technical Field

The present invention relates to a display system.

Background

Image replicating combiners, also known as waveguide combiners, are optical components used to expand an “eyebox” provided by an image source by generating multiple spatially-separated replications of the images. As used herein, an “eyebox” defines pupil positions in which an image can be viewed; the volume in which a viewer’s pupil can be positioned to view an image from an image source.

A replication may be defined as a real or virtual image of the image-source exit pupil with a given position, orientation and magnification as seen by the viewer’s pupil. In the case of planar image-replicating waveguide combiners, the magnification is always magnitude one

An image replicating combiner comprises an input surface to receive an input image and an output surface where the multiple replications are extracted. When light rays corresponding to the input image are coincident on the output surface, a portion of the light is refracted through, and extracted from the image replicating combiner. The portion of light not extracted out undergoes an internal reflection and remains in the image replicating combiner. This process of partial refraction and partial internal reflection is used to generate multiple replications as viewed through the output surface.

One application of image replicating combiners is as part of a head-mounted display (HMD) to expand the range of positions in which a viewer’s pupil can be positioned to see the display. Image replicating combiners work well when the image they display is a far-field image, such as one positioned at an infinite focus point, because this corresponds to generally collimated input and output rays forming the image. For a far-field image, the collimation is maintained for each replication. However, for near-field images, image-replicating combiners suffer from various image quality problems.

One such problem is “focus spread”. Focus spread occurs because each replication has a different path length. For near-field images the rays representing the image are diverging, not collimated. The combination of different path lengths and divergent rays creates a different apparent focal point for each replication. Not only does this mean that the depth of the image varies by replication, but a viewer’s field of view may include several replications, so that the viewer perceives the image at different depths simultaneously. This creates a poor viewing experience.

One solution to focus spread is to ensure the image is collimated at the input surface, i.e. focussed in the far-field, such as at infinity. However, this cannot be used in applications with depth information in the image, such as Computer-Generated Holography (CGH).

It would be desirable to reduce focus spread in image replicating combiners while preserving depth information in source image.

Summary

According to a first aspect of the present invention, there is provided a display system. The display system comprises: an image generating unit, an image replicating combiner and an optical system. The image generating unit is configured to generate an image with depth information. An image with depth information refers to an image with a focal point, or focal points, that reflect the perceived distance from the viewer. Examples of images with depth information include a CGH image or a 4D light field image. (This is unlike, for example, a 2D display that has the same focal point irrespective of the perceived distance from the viewer). The depth information could be due to a physical property of the rays that form the image, such as is utilised in phase-only computer-generated holography. Alternatively, the image with depth information could be one of a series of images that are displayed to a user with a time- sequential varying focus, wherein the depth of the scene changes in each image to give the perception of depth to the viewer.

The image replicating combiner comprises an input surface and an output surface, it is configured to generate a plurality of replications of an image incident on the input surface as viewed through the output surface. As used herein, the term “image replicating combiner” includes any optical element that generates multiple spatially separated replications of an image, therefore some embodiments may use an image replicator as the image replicating combiner. The input surface and the output surface may be distinct surfaces or regions of a larger surface. In some examples the input surface and the output surface may be different regions of the same surface (such as spaced apart from each other on the same surface).

The optical system is positioned in an optical path between the image generating unit and the input surface of the image replicating combiner and is arranged to transform a set of rays from the image generating unit having a first focal point into a plurality of sets of rays with respective different focal points, such that the plurality of sets of rays, when incident on the input surface of the image replicating combiner, combine at the output surface of the image replicating combiner to give each replication substantially the same focal point as seen from an exit pupil of the image replicating combiner. The first focal point may be located on the image generating unit exit pupil.

Such a display system, as a whole, produces a plurality of exit pupils, wherein each of the plurality of exit pupils comprises a sum of multiple images of the exit pupil of the image generating unit. The optical system may be arranged to generate multiple copies of an exit pupil of the image generating unit. The image replicating combiner may be arranged to produce a plurality of images of each of these multiple copies of the exit pupil of the image generating unit, as generated by the optical system. The optical system may further be arranged to produce positions for the plurality of copies of an exit pupil of the image generating unit, such that a set of rays diverging from the first focal point upon exiting the image generating unit also diverge from a substantially common point as viewed from any one of the plurality of exit pupils. This may enable sets of rays from the multiple copies of the exit pupil of the image generating unit to combine at the exit pupil of the image replicating combiner, giving each replication substantially the same display system exit pupil. In this way, each of the plurality of replications appears at generally the same depth.

It has been found that an optical system with these properties can be designed by considering the path of a single bundle of rays through the display system and designing the optical system such that the rays converge on a point at both the input and the output. This can be achieved with analytical or numerical methods using software such as Zemax OpticStudio, commercially available from Zemax Europe Limited, Stansted, UK.

In one design method, rays incident on the output surface, diverging from a point, are traced back from the output to the input and an optical system designed which creates corresponding virtual images at the input. (This method considers a reverse optical path of diverging rays from output to input.) In another design method, rays incident on the input surface, having diverged from a point, are traced from input to output and an optical system designed to converge on a single point at the output. (This method considers an optical path of diverging rays from input to output.) Either method could be analytical or numerical. When numerical methods are used, they may be iterative. Regardless of how the optical element is designed, for a particular replication, it has been found that designing for rays converging to a single point in the eyebox results in approximate convergence for all points in the eyebox. Furthermore, other replications also exhibit increased convergence so image quality is improved across all replications even though the optical system can be designed by considering a single bundle of rays between a single point at the input and a single point at the output.

The optical system may have any suitable form. In one example, the optical system comprises a replicating optical element to generate multiple copies of a ray, and a lens. The lens may be a separate component. In other examples, it may be integral with another component, for example the lens may be an additional diffractive phase profile on a grating forming the input surface, potentially providing the function of the lens with a lower component count. The lens may be positioned in an optical path between the replicating optical element and the input surface of the image replicating combiner.

The replicating optical element may be a diffractive optical element or a further image replicating combiner. Both of these can take a single input image and output a plurality of images with different respective focal positions.

The optical system may be configured to generate multiple copies of a ray from the image generating unit, and wherein the optical system is arranged so that one copy from the multiple copies is incident on the input surface of the image replicating combiner. This can reduce the effect of cross-talk in the output image.

The display system may further comprise an adjustment system for adjusting the location of output replications. It is possible that a viewer’s pupil does not align exactly with one of the output replications. The adjustment system can align a replication with the location of a viewer’s pupil. Various adjustment systems can be used. In one example, the adjustment system comprises a manually operable actuator configured to adjust the position of the display system as whole relative to the viewer’s pupil. Example manually operable actuators include screw threads, slide controls, adjustable headbands, and electronic actuator systems such as worm drive or servo system.

Alternatively, or additionally, the display system may further comprise an eye tracking system configured to control the adjustment system based on a location of the viewer's pupil. The eye tracking system may automatically control the adjustment system so that a replication is steered towards the location of the viewer’s pupil. Some rays entering the image replicating combiner may be incident on the input surface at angles larger than those applicable to first and / or second order paraxial approximation, introducing aberrations to the image. This can reduce image quality. In some examples, higher order terms (such as higher orders than tip/tilt and focus) are used in the design of the optical system using numerical analysis methods. For example, the optical system may be arranged to have an optical power in the x axis which is different from an optical power in the y axis. This can reduce aberrations, compensating for any astigmatism introduced by the image replicating combiner.

The display system may be configured to display a computer-generated hologram image or a 4D light field.

According to a second aspect of the present invention, there is provided a head-mounted display comprising the display system of the first aspect.

According to a third aspect of the present invention, there is provided a head-up display comprising the display system of the first aspect.

Further features and advantages of the invention will become apparent from the following description of preferred embodiments of the invention, given by way of example only, which is made with reference to the accompanying drawings.

Brief Description of the Drawings

Figure 1 is a schematic diagram of an example display system;

Figure 2 is a schematic diagram of a further example display system;

Figure 3 shows a method of designing an example display system;

Figure 4 is a schematic diagram of part of a display system according to a first example;

Figure 5 is a further schematic diagram of the part of the display system according to the first example;

Figure 6 is a schematic diagram of a further part of the display system according to the first example;

Figure 7 is a schematic diagram of a still further part of the display system according to the first example;

Figure 8 is a schematic diagram of the still further part of the display system according to the first example; Figure 9 is a schematic diagram of a second display system according to a further example;

Figure 10 is a schematic example of another display system;

Figure 11 is a desired eyebox according to an example;

Figure 12 shows the field of view of an image replicating combiner in a display system according to a worked example;

Figure 13 A shows various perspectives of a single ray traced through the display system according to the worked example;

Figure 13B shows various perspectives of a plurality of rays traced through the display system according to the worked example;

Figure 14 shows various perspectives of a plurality of replicated rays produced by the display system according to the worked example;

Figure 15 A shows various perspectives of an example simulation wherein a linear phase profile of each phase surface on a diffractive optical element has been set so that the rays from different replications intersect the nominal eye pupil at the same point;

Figure 15B shows various perspectives of the example simulation shown in Figure 15A with a plurality of rays spanning the entire field of view of the image replicating combiner;

Figure 16 shows an exit pupil filled with collimated rays from a virtual point source at infinity using the display system according to the worked example;

Figure 17 shows the (X, Y) angles of an example set of fields chosen to evaluate imaging performance of the display system according to the worked example;

Figure 18 is a grid of spots traced from in front of an eye through an optimal replication of the display system according to the worked example;

Figure 19 is simulated retinal image from the optimal replication produced by the display system according to the worked example;

Figure 20A shows a simulation of an entire pupil replicated across the eyebox in the display system according to the worked example;

Figure 20B is a simulated eyebox produced by the display system according to the worked example;

Figure 21 is a grid of spots traced from in front of the eye through the left-centre replication of the display system according to the worked example; Figure 22 is simulated retinal image from the centre-left replication produced by the display system according to the worked example; and

Figure 23 is a simulated eyebox in the absence of an optical system according to the present disclosure.

Detailed Description

Image replicating combiners expand the size of the viewable area of the image for a viewer. Image replicating combiners comprise an input surface, also known as an in-coupler or an entrance pupil, to receive light rays corresponding to an input image. The notion of an entrance pupil corresponds to the limiting aperture in an input of the image replicating combiner. The input surface is a coupling feature of the image replicating combiner that couples light waves propagating externally from the image replicating combiner to the inside of the image replicating combiner. The coupling feature may be, for example, a mirror, a prism, a diffraction grating or a hologram. An image replicating combiner further comprises an output surface, also called an out-coupler, to output light corresponding to the input image. The output surface is a further coupling feature of the image replicating combiner, which may use the same technology as the input surface. Image replicating combiners may be manufactured from materials with high-refractive indices that support total internal reflection over a wide range of internal incidence angles. Lanthanum dense flint glass, for example N-LASF46 manufactured by Schott™, has a critical angle of ()_c = 31° at wavelength 2 = 530nm. An image replicating combiner will propagate waves by total internal reflection at all incident angles above the critical angle.

In an example, an image replicating combiner takes the form of a substantially planar sheet. The planar sheet may be constructed from a transparent material, such as glass. In this case, one arrangement for the input and output surfaces is to position them on the same side of the planar sheet, such that light enters and exits at the same side of the planar sheet. In another arrangement, the input and output surfaces may be positioned on opposite sides of the planar sheet. The particular arrangement may be selected based on a function of the image replicating combiner.

Figure 1 is a schematic diagram of an example display system 100 without an optical system according to the present disclosure, so that the challenges of designing image replicating combiners for use with near field images can be better understood. The display system comprises an image generating unit 102 configured to generate a near-field image 110 and an image replicating combiner 104 comprising an input surface 106 and an output surface 108. Multiple replications of the image 110 are generated at the output surface 108. The image generating unit is arranged such that the exit pupil of the image generating unit 102 coincides with the input surface 106. In Figure 1, three replications 112, 114, 116 are shown, each corresponding to light that has undergone different numbers of internal reflections within the image replicating combiner 104.

As can be observed from Figure 1, light corresponding to the image 110 converges on the exit pupil of the image generating unit 102 (by definition). As the light travels beyond the exit pupil, it diverges. This results in each of the replications 112, 114, 116 diverging on extraction at the output surface 108 of the image replicating combiner 104. Consequently, a pupil of a viewer’s eye 124, aligned with replication 114 will also receive light rays from adjacent replications 112 and 116. As can be further observed from Figure 1, each of the replications 112, 114, 116 are located at different horizontal positions along the image replicating combiner 104 resulting in the viewer observing multiple copies of the image 110. Each of the replications 112, 114, 116 represent the same image horizontally translated, with respect to the figure. This produces a noticeable effect in the resulting replications due to parallax, wherein the viewer would perceive three (in this example) copies of the image 110 at different perceivable positions.

This contrasts with a far-field image, wherein all rays entering the image replicating combiner 104 appear to emanate from at least one point source at infinity. Thus, the horizontal distances between successive replications have little effect on the perceivable position of the images. Due to parallax of the image 110, the difference in position of the image in each replication 112, 114, 116 becomes more noticeable as the image is moved closer to the image replicating combiner 104. That is, the perceivable effect is a function of the distance of objects displayed within the image 110 from the image generating unit exit pupil.

Further, because each replication 112, 114, 116 results from light undergoing different numbers of internal reflections before extraction, and therefore a different optical path length, each replication is perceived as having a different focal depth. In other words, replications 112, 114, 116 appear at different depths and therefore have different apparent sizes. The viewer therefore sees three copies of the image 110 at slightly different depths/sizes resulting in an unsatisfactory viewing experience. This is known as “focus spread”. To include depth information in an image, for example a CGH image, objects within the image may need to be displayed close to the viewer. In some examples, regions of the image may need to be displayed as close as 100mm from the viewer. As described above, the display system 100 is not sufficient to generate a satisfactory image. It will be appreciated that unsatisfactory image quality (such as from replicated images of different apparent sizes and perceived position) applies at all depths other than infinity, but has a greater effect the closer the apparent distance of the image region from the viewer. As such, image quality at display depths in the range of 50mm to 300mm and further, such as Im or 2m can still suffer from unsatisfactory image quality.

Figure 2 is a schematic diagram of an example display system 200 including an optical system according to the present disclosure. The display system 200 comprises an image generating unit 202, also referred to as a picture generating unit (PGU), an image replicating combiner 206, and an optical system 204 positioned in an optical path between the image generating unit 202 and the image replicating combiner 206. As will be explained in more detail below, the optical system 204 is configured to generate a plurality of input images to the image replicating combiner, to counteract the problems described above, such as focus spread.

The image generating unit 202 is configured to generate an image 212, wherein the image 212 comprises depth information. The image generating unit may include an at least partially coherent light source, in this case a laser diode which is configured to illuminate a light modulation element in the form of a spatial light modulator. The laser diode is, for example, a Sumitomo Electric (RTM) SLM-RGB-T20-F-2 laser diode, but other laser diodes may be used. An RGB diode can rapidly switch between emitting different colours of laser light, sequentially emitting red, green, and blue light. By modulating the laser light at different times when the different colours are emitted, the appearance of a colour holographic image may be created for a viewer through persistence of vision. It will be appreciated that other examples may be monochrome or use simultaneous red, green and blue light sources and that the present disclosure is not tied to a particular light source.

An example spatial light modulator is a Compound Photonics (RTM) DP1080p26 micro-display and configured to adjust the phase of light. By controlling the phase of light, it is possible to use interference to create a holographic replay image. The present disclosure is not tied to a particular spatial light modulator technology or component. The holographic replay image is output from the image generating unit at a terminal optical element. In other examples, the image generating unit 202 may be configured to generate a 4D light field. In this case, the image generating unit 202 can be any suitable 4D light field generator. One such suitable 4D light field generator is described in “Near-eye light field displays”, Lanman et al., available from https://research.nvidia.com/sites/default/files/pubs/2013-l l_Near-Eye-Light-Field/NVIDIA- NELD.pdf, published 1 November 2013 and first archived by web.archive.org on 14 November 2020. The Lanman 4D light field generator consists of an OLED microdisplay and an intermediate microlens array between the microdisplay and the viewer’s eye. A 4D light field specifies the intensity of light as a function of x, y, 0, (p, wherein x, y are spatial coordinates at the location of the centre of a viewer’s pupil, and 0, (p are angular coordinates of light at that location.

The image replicating combiner 206 is comprised of material(s) with optical properties such that light rays undergo internal reflection within the image replicating combiner. The image replicating combiner 206 comprises an input surface 208, also known as an in-coupler, to receive light corresponding to an image. The image replicating combiner 206 further comprises an output surface 210, also known as an out-coupler, to generate a plurality of replications of the image incident on the input surface 208.

The optical system 204 is arranged to transform sets of rays 212 corresponding to the image into a plurality of sets of rays, such that rays emanating from a common point on the image generating unit exit pupil 213 converge on multiple focal points thereby forming copies 214 of the image generating unit exit pupil, and such that at least one of the exit pupils 216 is formed from a replication of each of the plurality of sets of rays, converging at a viewing position. Figure 2 shows the concept of the disclosure in diagrammatic fashion, the set of rays is transformed by the optical system into three sets of rays forming multiple copies 214 of the image generating unit exit pupil 213. The image replicating combiner 206 then generates three replications of each set of rays, shown by the differing dashed lines. For example, an exit pupil 216 comprises a plurality of replications corresponding to sets of rays which have travelled substantially different paths through the combiner due to different numbers of internal reflections before extraction at the output surface 210.

Such a display system 200 is capable of preserving depth information of the image in the plurality of replications because the differing input replications are designed so that the optical path length between the image generating unit 202 and the viewing position, through the image replicating combiner 206 is substantially the same, modulo wavelength. The result of this is that rays at the viewing position are in phase. Thus, when the replication is viewed, the different replications visible combine to an image with a consistent depth.

An example process of generating the plurality of replications so that they converge on a viewing position will now be discussed in more detail with respect to Figures 3-7.

Figure 3 shows an example method 300 of designing an optical system for use in the display system of Figure 2. The display system comprises an image generating unit, an image replicating combiner, and an optical system positioned in an optical path between the image generating unit and the image replicating combiner. In the present method 300, the optical system is designed by considering the reverse path from the output to the input.

At block 302, the method 300 involves selecting a point source to act as an optimal pupil location in an output of an image replicating combiner. The optimal pupil location is an area which will provide the optimal viewing experience for a viewer pupil. For example, it may correspond to a replication generally in a centre of the image replicating combiner.

Referring now to Figure 4, which shows a 2-dimensional representation of an example image replicating combiner 400. Figure 4 depicts an example point 402 on a nominal pupil location. The combiner 400 comprises an input surface 406 and an output surface 404. A position of a single point 402 in the output of the image replicating combiner 400 is selected. The nominal pupil location may be selected to represent the position of the centre of an eyebox positioned at the output.

Figure 4 further shows three sets of light rays 408, 410, 412 emanating from the single point 402 in the output, towards the output surface 404. Each set of light rays 408, 410, 412 comprises three light rays. Each light ray in a first set of light rays 408 is represented by a chain line arrow. Each light ray in a second set of light rays 410 is represented by a solid arrow. Each light ray in a third set of light rays 412 is represented by a dash line arrow. These rays generally extend over the field of view of the pupil, with set of rays 408 corresponding to a first replication within the field of view, set of rays 410 corresponding to a second replication within the field of view and set of rays 412 corresponding to a third replication within the field of view. For a near field image, an image comprising depth information, rays from all replications can be seen from point 402.

At block 304, the method 300 involves tracing sets of rays entering an output surface of the image replicating combiner (such as those depicted in Figure 4), through the image replicating combiner to the input surface. Each set of rays undergoes a different number of internal reflections from others of the sets of rays. The ray tracing may be performed analytically and / or numerically using appropriate ray tracing software. Tracing the sets of light rays results in a formation of a plurality of virtual images of the nominal pupil location, at the input. Locations of the virtual images will depend on the design of the image replicating combiner 400. This will be explained further with reference to Figure 5.

Figure 5 shows the image replicating combiner 400, the sets of light rays 408, 410, 412 have been traced through the combiner 400, and out through the input surface 406. The three sets of rays were diverging from the point 402 and therefore continue to diverge when traced through to the input surface as shown in Figure 5. Tracing these sets of light rays diverging from the input surface 406 back to their respective focal planes, shows that three virtual images at points 414, 416, 418 are produced, respectively corresponding to the three sets of light rays 412, 410, 408 emanating from the single point 402.

Virtual image 416 results from the second set of light rays 410 having undergone n total internal reflections within the image replicating combiner 400 before being extracted, wherein n is a positive integer. Virtual image 414 results from the third set of light rays 412 having undergone n+2 total internal reflections within the combiner 400 before being extracted. Virtual image 418 results from the first set of light rays 408 having undergone n~2 total internal reflections before being extracted. In other words, each further extraction corresponds to a difference of two total internal reflections. Positions of each of the virtual images 414, 416, 418 depend on properties of the combiner 400. A horizontal distance between successive virtual images, for example a horizontal distance between a first virtual image 414 and a second virtual image 416, is approximately equal to a spacing of successive replications, formed at the output surface 404, when a single image enters the input surface 406. The spacing of successive replications is known as the combiner extraction spacing and is dependent on the design of the combiner and the material properties. Similarly, a vertical distance between successive virtual images, for example a vertical distance between the first virtual image 414 and the second virtual image 416, depends on properties of the combiner 400, such as a thickness, a refractive index, and the combiner extraction spacing.

At block 306, the method 300 involves determining an optical system to form a real image corresponding to the sets of rays traced through the input surface. With the determined optical system, each ray emitted by a focal point on the input side of the image replicating combiner is transformed into a plurality of rays that together converge on multiple points, such that the plurality of rays, when incident on the input surface, converge on the single point that was selected at block 302 of the method 300, when extracted from the output surface.

One way of determining such an optical system at block 306 will now be described with reference to Figures 6 and 7. First, a lens is positioned to converge the diverging beams at the input. Figure 6 shows the image replicating combiner 400 with a lens 426 positioned in front of the input surface 406 of the image replicating combiner 400. The lens 426 is configured to form real images 420, 422, 424 corresponding to the respective virtual images 414, 416, 418. That is, the lens 426 converges the light rays diverging from the virtual images 414, 416, 418 to the points 420, 422, 424 in the input of the image replicating combiner 400, via refraction or diffraction. An example lens for this purpose has an aperture size similar to the maximum size of a viewer’s pupil. Atypical aperture size is therefore around 5mm. The suitable lens 426 may therefore have a focal length of between 5-50mm. For example, if the lens 426 has an f-number of f/2 then the focal length is 10mm.

Note that due to the varied distances of the virtual images 414, 416, 418 from the input surface 406, the corresponding real images 420, 422, 424 are also formed at various distances from the input surface 406. Further, because the horizontal and vertical distances between successive virtual images 414, 416, 418 is deterministic, locations of each of the real images 420, 422, 424 can be readily calculated based on properties of the lens 426. The real images 420, 422, 424 can then be focussed to a single point by addition of a further optical component, which is a diffractive optical element (DOE) in this example. Other examples may use refractive optical components such as a Fresnel lens or a microlens array.

Figure 7 shows the image replicating combiner 400 and lens 426 shown in Figure 6, wherein a DOE 428 is arranged to converge the light rays emanating from the lens 426, corresponding to the virtual images 414, 416, 418 to a point 430 in the input of the image replicating combiner 400. The DOE 428 may be any diffractive element capable of producing an image at the point 430 from the virtual images 414, 416, 418. Example DOEs include beamsplitters, pattern generators, and diffraction gratings.

The optical arrangement shown in Figures 4 to 7 is constructed in reverse with respect to the direction that light will traverse the arrangement in operation, as has been discussed above. This allows each component to be selected based on a ray tracing analysis. The ray tracing analysis provides a deterministic framework for designing display systems such that components can be selected based on required properties (a focal length of the lens 426 required to form the images 420, 422, 424 etc.).

One can appreciate that reversing the direction of the light rays in Figure 7, so that an image generated at the point 430 and entering the image replicating combiner 400 through the input surface 406, results in an image at the single point 402 in the output of the image replicating combiner 400. By construction, the single point 402 comprises converging sets of light rays that have undergone different numbers of internal reflections before being extracted. The result is that the first, second and third sets of rays 408, 410, 412 have the same optical path length, modulo wavelength, between the point 430 at which the image is generated and the point 402. Thus, the image formed at the single point on the nominal pupil location 402 has a single perspective and so images comprising depth information (such as CGH) can be supported.

The method 300 of designing a display system involves mapping a point in the input of an image replicating combiner to a point in the output. Mappings between further pairs of points will now be described with regards to Figure 8.

Figure 8 shows the image replicating combiner 400, lens 426 and DOE 428 shown in Figure 7, wherein a further point 432 representing a further point on the nominal pupil location has been selected in the output of the combiner 400. As in block 304 of the method 300, sets of light rays 434, 436, 438 emanating from the further point 432 are shown traced through the image replicating combiner 400 and extracted at the input surface 406. This can be achieved as described above with reference to Figure 5 using standard ray tracing techniques. The result of the ray tracing is the appearance of virtual images 440, 442, 444, wherein each virtual image 440, 442, 444 results from the sets of rays 434, 436, 438 having undergone different numbers of internal reflections before being extracted. As before, virtual image 442 results from a fifth set of light rays 436 having undergone n total internal reflections within the image replicating combiner 400 before being extracted, wherein n is a positive integer. Virtual image 440 results from a sixth set of light rays 438 having undergone n+2 internal reflections before being extracted. Virtual image 444 results from a fourth set of light rays 434 having undergone n~2 internal reflections before being extracted. It can then be determined that the virtual images 440, 442, 444 are transformed by the lens 426 and DOE 428 to form the single point 446 in the input. Thus, light emanating from the single point 446 will converge to a single point 432 in the output. As an image is generated by a finite exit pupil of an image generating unit, one can observe that rays emanating from different points on the image generating unit exit pupil, result in rays that converge on different points on the exit pupil of the image replicating combiner 400.

Figure 9 shows a display system 900 according to another example. The display system comprises an image generating unit 920, lens 926 and a first image replicating combiner 902. The display system 900 may be designed in a similar manner to the system shown in Figures 4 to 8. However, in this example, rather than implementing a DOE, the display system 900 includes a second image replicating combiner 940.

The display system 900 further comprises an image generating unit 920 configured to generate light corresponding to a focal point on the image generating unit exit pupil 930. Light rays emanating from the point 930 on the image generating unit exit pupil enter the second image replicating combiner 940 though a second input surface 942. As discussed above, light rays will undergo different numbers of internal reflections within the second image replicating combiner 940 before being extracted at a second output surface 944 of the second image replicating combiner 940. This results in a plurality of replications being extracted. In Figure 9, replications 910, 912, 914 are shown as images formed behind the first image replicating combiner 902. Light rays corresponding to the images 910, 912, 914 enter the first image replicating combiner 902 at a first input surface 908 and are extracted, after having undergone respective numbers of internal reflections, at an output surface 906. By design, the extracted rays converge on a point 904 in the output of the image replicating combiner 902, representing a point on the area of an optimal pupil location.

Figure 10 shows a further example display system 1000. The display system 1000 is the same as the display system 200 shown in Figure 2. The optical system 1004 is arranged to generate multiple copies 1008, 1010, 1012 of an image generating unit exit pupil. As discussed above, the optical system 1004 can be selected such that sets of rays from the copies 1008, 1010, 1012 combine at the output surface of the image replicating combiner 1006 to give each replication substantially the same exit pupil 1014, 1016, 1018, 1020, 1022. It can be seen from Figure 10 that the plurality of sets of rays from the copies 1008, 1010, 1012 undergo different numbers of internal reflections before being extracted. For example, a first exit pupil 1014 comprises: rays from the copy 1008 that have undergone n internal reflections before extraction. A second exit pupil 1016 comprises: rays from the copy 1010 that have undergone n internal reflections, and rays from the copy 1008 that have undergone n+2 internal reflections before extraction. A third exit pupil 1018 comprises: rays from the copy 1012 that have undergone n internal reflection, rays from the copy 1010 that have undergone n+2 internal reflections, and rays from the copy 1008 that have undergone /z+4 internal reflections before extraction. A fourth exit pupil 1020 comprises rays from the copy 1012 that have undergone n+2 internal reflections, and rays from the copy 1010 that have undergone n+ internal reflections before extraction. An exit pupil 1022 comprises rays from the copy 1012 that have undergone /z+4 internal reflections before extraction. Each successive extraction corresponds to a difference of two total internal reflections. Thus, even though the image generated by the image generating unit is formed in the near-field, the fact that there are rays being received from multiple copies 1008, 1010, 1012 of the image generating unit exit pupil ensures that the exit pupils of the image replicating combiner are spatially-separated and comprise a single perspective, ensuring depth information is preserved in each replication.

The optical system 1004 generates multiple copies 1024, 1026, 1028 of a single ray 1030 entering the optical system 1004. As shown in Figure 10, each copy 1024, 1026, 1028 is spaced from the other copies. It can be seen that for any given ray 1030 output by the image generating unit 1002, only one of the copies 1024 enters through an entrance pupil of the replicating combiner, and that other rays 1026, 1028 are not transmitted through the image replicating combiner input surface. Essentially, the other rays 1026, 1028 are blocked by arranging the optical system 1004 so that only the copy 1024 is incident on the input surface of the image replicating combiner and so only copy 1024 enters the image replicating combiner 1006. This ensures that additional replications resulting from multiple copies of the same ray do not make it to any of the exit pupils of the image replicating combiner 1014, 1016, 1018, 1020, 1022.

It was shown above with reference to Figure 8 how the optical system can be designed by considering a single point but also works to converge other points within the same exit pupil. A further advantageous property is that other exit pupils also have improved convergence and therefore an improved viewing experience.

Display systems, such as the display systems shown in Figures 7 and 9 may be used to provide a viewer with an improved viewing experience by preserving depth information of an input image in the replications. While the viewing area is increased, the viewing experience can be further improved if a viewer’ s pupil is aligned with a single replication rather than being positioned between replications. Some examples comprise an adjustment system to align an exit pupil with a viewer’s pupil. The adjustment system may be manual or automatic, and automatic systems may include some form of eye-tracking. The position of the exit pupil itself may be adjusted by physically moving elements of the display, or even the entire display, relative to the viewer. Other adjustment systems may operate by adjusting the position of the input image, with a resulting change on the output image.

Where adjustment systems are provided, the range of movement may be relatively small, perhaps at most the spacing between replications, such as 8mm or smaller. This allows a nearest replication to be centred to the pupil.

The method of designing an optical system shown in figures 4 to 9 illustrate examples of block 304 of the method 300 using the paraxial approximation. In practice, block 304 of the method 300 may be conducted using a non-paraxial ray tracing scheme. Then the optical system determined at block 306 will additionally include compensation for any geometric aberrations resulting from the image replicating combiner, which will primarily be expressible as an astigmatic term. For example, the optical system may be determined to have different optical powers in an x direction than in a y direction.

The display system described herein may form part of a head-mounted display (HMD). In another example, the display system described herein may for part of a head-up display (HUD). Such a display system allows the reproduction of images with depth information on HMDs and HUDs with an expanded viewing area.

Worked Example of Designing an Optical System

In the method 300, the optical system is designed by considering the reverse path from the output to the input. However, the optical system may also be designed by considering a path from the input to the output. The required components can be determined from a ray tracing scheme, as discussed above. In some cases, though, it may be simpler to determine an arrangement and physical properties of the optical system components using numerical modelling via software such as Zemax OpticStudio, commercially available from Zemax Europe Limited, Stansted, UK. Numerical analysis allows parameters of the display system to be determined a priori in order to determine other parameters of the display system. Examples of parameters that are to be predetermined are the size of the optimal exit pupil and spacing between neighbouring exit pupils. Ideally, these should be included as independent parameters that dictate the designing of the display system and can inform design parameters choices for the optical system.

For example, the diameter of the exit pupil may be determined by the range of diameter of a human pupil. The diameter of the pupil changes based on the amount of light entering the pupil, so a range of diameters can be considered. With the exit pupil diameter fixed, the area of the pupil determines the maximum spacing between neighbouring exit pupils so that the pupil mainly receives light from one exit pupil at any given time, in other words so that a single exit pupil can be positioned inside the area of the pupil. For example, an exit pupil of diameter 2mm may be a good compromise for resolution and field of view of a 2k x 2k pixel spatial light modulator of the image generating unit. A 5mm replication pitch (the spacing between nearest neighbouring exit pupils) ensures that the eye pupil will not be covered by two or more replicated pupils, providing that the entrance pupil position can be changed dynamically, following the position of the eye pupil. It is in principle possible to support an 8mm diameter eye pupil with a 5mm replication pitch of 2mm pupils. This eye pupil size is typical of scotopic vision (i.e. viewing in near darkness). An example target eyebox size is 10-15mm horizontally and 8-12mm vertically. This is supported by 3 replications in both horizontal and vertical directions.

Figure 11 shows an example eyebox. In this example, exit pupils 1102 are shown with a diameter of 2mm. The spacing 1104 between the centres of horizontally (and vertically) neighbouring exit pupils is 5mm. In this case, the maximum human pupil diameter 1106 that is supported by the eyebox, such that only a single exit pupil overlaps the pupil, is 8mm, indicated by circle 1110. A pupil of diameter 8mm would likely require steering of the exit pupil overlapping the pupil so that the exit pupil remains fixed to the centre of the pupil as the pupil moves with respect to the eyebox. The maximum pupil diameter 1108 that contains no more than one whole exit pupil, when the exit pupil is as far from the centre of the pupil as possible, is 5mm, indicated by circle 1112.

The specifications of the image replicating combiner are determined by the physical requirements of the combiner. Specifically, the image replicating combiner should be able to support total internal reflection over a wide range of internal incidence angles. Lanthanum dense flint glass, for example N-LASF46 manufactured by Schott™, has a critical angle of ()_c = 31° at wavelength 2 = 530nm. An image replicating combiner will propagate waves by total internal reflection at all incident angles above the critical angle. In an example in which the optical system comprises a lens and a diffractive optical element, such as the optical system of Figure 7, it can be determined how many copies of the image are required to produce an eyebox with the desired exit pupil spacing and field of view. Figure 12 illustrates an example image replicating combiner 1200 producing a plurality of replications 1202. The distance between the image replicating combiner 1200 and the exit pupil 1204 is 20mm, the replication pitch is 5mm, and the field of view 1206 is 30 degrees. It can be seen that, at the image replicating combiner 1200, the ray envelope covers 3 replications. This determines the number of discrete pupil images to be generated at the diffractive optical element (i.e. 3x3).

The image replicating combiner 1200 comprises input and output couplers. A surface relief grating may act as an input coupler. This serves as the input interface to the combiner 1200. Free space waves are coupled into the combiner 1200 at the input coupler by diffraction. A surface relief grating with a groove frequency of 2580 lines/mm can serve as an input or output coupler between free space and total internal reflection within N-LASF46. More complex surface relief gratings may be manufactured that exhibit optical power in addition to simple diffraction from parallel equally-spaced linear grooves. In this worked example the input coupler to the combiner 1200 is a diffractive surface acting as a combination of a convex lens with focal length 15mm and linear diffraction grating with 2580 grooves per millimetre.

The diffractive optical element (DOE) generates nine distinct images of the entrance pupil by a supposition of 9 phase profiles. This is known as phase multiplexing. The combiner input coupler is a phase surface with linear phase components (“tilt”), and quadratic phase components (“power”). The optical power ensures that the entrance pupil images are reimaged at the combiner’s exit pupil.

The DOE is positioned in the focal plane of the input coupler. This ensures that rays passing through a point on the DOE are collimated ray bundles after passing through the combiner and the exit pupil.

With the optical elements selected with the parameters described above, the starting element positions and sizes (pupils, gratings etc) are defined. Figure 13 A shows various views of the result of a numerical simulation wherein a chief ray from the entrance pupil centre to the eye pupil is traced through the image replicating combiner 1200 via 2 bounces vertically and 3 bounces horizontally. The entrance pupil of the simulated model is the preferred location for an image generating unit exit pupil. In the simulation, the image replicating combiner 1200 is positioned in the z=0 plane. A first view 1302 shows a face-on view of the ray tracing simulation. A second view 1304 shows a side on view. A third view 1306 shows another side- on view. The third view 1306 shows the horizontal reflections of the ray and the second view 1302 shows the vertical reflections. A fourth view 1308 shows a 3D isometric view corresponding to the first, second and third views 1302, 1304 and 1306.

The DOE has a flat phase surface (no power or aberration correction). Figure 13B shows the same views as shown in Figure 13 A for a simulation that traces a plurality of rays from the centre of the entrance pupil over the extent of the combiner entrance aperture.

The simulation can be extended to trace rays for all replications differing by one bounce in the vertical and horizontal directions. Figure 14 shows the results of such a simulation. It can be seen that this forms a 3x3 grid of rays emitted from the image replicating combiner 1200, but that only the central replication strikes the nominal eye pupil.

For each bundle of rays exiting the image replicating combiner 1200 after a given number of vertical and horizontal bounces, a phase surface on the DOE can be defined. The linear phase profile of each phase surface can be set so that the rays from different replications intersect the nominal eye pupil at the same point. Figure 15A shows the result of such a selection. It can be seen that rays emitted from a single point on the entrance pupil, at different angles, now intersect the exit pupil at a single point. It is always possible to fulfil this condition with replications of a single ray.

The simulation shown in Figure 15A can be extended to trace fans of rays that span the entire image replicating combiner entrance aperture, the result of which is shown in Figure 15B. Rays are shaded by the number of vertical and horizontal reflections inside the image replicating combiner 1200. Only three of the nine replications are shown for clarity. It can be seen that, due to the choice of element positions and sizes, rays of a given angle have only one available path through the image replicating combiner to the exit pupil. This ensures that multiple copies of the same ray do not make it to the exit pupil of the image replicating combiner. In later steps it will be confirmed that this condition holds for rays emitted from other pupil positions.

Nine different rays have been taken from a single point on the entrance pupil, and then propagated via different phase profiles on the DOE, and internal reflections within the image replicating combiner 1200, to a single point on the exit pupil. However, to consistently reimage the entrance pupil onto the exit pupil, corrections for focus and aberrations (e.g. astigmatism) onto the various phase surfaces on the DOE may be added. Viewed from the entrance aperture of the combiner 1200, the location of the virtual image of the exit pupil (nominal eye pupil) varies depending on the configuration of reflections within the combiner 1200.

Optical power corrections may be used for the different paths, because the exit pupil virtual images appear at different depths as well as lateral positions. The variation in lateral position is accounted for by linear phase corrections across the DOE surface (“tilt” corrections). The variation in depth is accounted for by different optical powers in the DOE phase profiles.

Next, the tilt, power and astigmatism Zemike terms are adjusted at the DOE to bring all rays from the entrance pupil vertex to focus at the nominal eye pupil. This can be achieved with the optimization functionality of optical design software such as Zemax™.

Each phase profile at the DOE generates a separate outgoing ray. Incident rays are split into 3x3 outgoing rays, corresponding to a different number of ray bounces within the image replicating combiner 1200. We identify the profiles by a pair of integers (z, j) where z is the number of ray bounces in the first replication direction, and j is the number of bounces in the second replication direction. Table 1 below lists the Zernike polynomials denoted (Z2, Z3, Z4, Z6) and table 2 below lists the Zernike coefficients of each profile, denoted (A2, A3, A4, A6). These correspond to phase corrections for: tilt (in two axes), defocus and one axis of astigmatism, respectively.

The values of the well-known Zemike polynomials are defined in terms of radial coordinates (p,0). The radial term p is normalised by an aperture radius, ro, such that (p = r / ro). In this worked example, this value is n = 4 mm.

Table 1 The phase shift in radians, at wavelength 520nm, of each DOE profile can be obtained by multiplying these Zemike polynomials by the coefficients defined in Table 2, as shown in Eq. 1.

Phase shift = 2T A2 Z2 + A3 Z3 + A4 Z4 + A6 Z6)

Eq. 1

Table 2

It can be seen, from Table 2, that the central profile (3,3) has broadly similar magnitudes of tilt, power and astigmatism correction. In contrast, the other, peripheral profiles are dominated by strong tilt components. All the profiles apply more than 10 wavelengths of both power and astigmatism correction.

The fundamentals of the display system according to the worked example have now been described. The design can now be validated at different pupil positions and over the required eyebox. The optical model has so far considered the reimaging of the entrance pupil to the exit pupil. Now, the imaging performance of the model can be validated directly. A set of virtual object field positions are chosen and the images formed of these fields are checked. Since a depth preserving image replicating combiner is being designed, one may validate object fields at different depths.

On converting the model to behave as an imaging system through the design pupils, it can be seen that the rays from certain object field positions travel with different numbers of bounces through the image replicating combiner 1200, depending on the pupil position. Figure 16 shows an exit pupil filled with collimated rays from a virtual point source at infinity. At this field position, the rays take paths with different numbers of bounces through the image replicating combiner 1200 depending on the ray position at the pupil. Note that in this example the rays from the entrance pupil are not collimated but converging. In this case, the converging light beam brings forward the plane of focus of the zero-order light from the light-modulation element allowing the zero-order light to be removed preventing a bright light being passed on to the viewer. Further discussion of this can be found in PCT application publication no WO 2021/151816 published on 5 August 2021 and incorporated herein by reference for all purposes. Other examples may use collimated (parallel) rays.

A set of field positions must be chosen in which one can evaluate the imaging performance. Figure 17 shows the (X, Y) angles of an example chosen set of fields. The total field-of-view is 30 degrees both horizontally and vertically.

Figure 18 shows a grid of spots 1800 traced from a virtual object depth of 200mm in front of the eye. Each spot 1802 shows the distribution of ray angles for a given field position and configuration of reflection paths within the image replicating combiner 1200. Each row in the grid corresponds to a field position. Each column of the grid corresponds to a configuration of reflections paths (i, j) within the image replicating combiner 1200. Most combinations of field and path configuration have no traceable rays. The circle on each spot diagram shows the Airy disk size - the minimum size of a diffraction-limited spot. This represents the minimum practical spot size of a real optical system, where diffraction effects are present. It is noted that there is at least one path for rays from any given field to reach the exit pupil. It is also noted that, with the exception of the fields at the corners of the field of view, all rays fall within the Airy disk. Rays traced from the corner fields lie within a small number of radii of the Airy disk. Comparable imaging performance is obtained from spot diagrams of virtual image plane depths greater than 200mm.

As a further validation of image quality, an eye lens (focal length 20mm) can be simulated focussing onto a retina. Figure 19 shows the fields focussed at the retina. It can be seen that the field positions are clearly resolved.

It can also be verified that there are discrete replications of the entire pupil across the eyebox. Figure 20A shows the same set of fields and the entire pupil replicated across the eyebox. Different configurations of reflections within the image replicating combiner 1200 ensure that we generate a 3x3 grid of replicated pupils, in which the full field of view is present. Figure 20B shows the intersection points of all 3x3 pupil replications at the eyebox. We see that we have clear separation of the replicated pupils. With a few millimetres of lateral entrance pupil steering we can allow only a single replicated pupil to be incident on the eye pupil.

As can be seen, the simulated eyebox of Figure 20B is remarkably similar to the eyebox shown in Figure 11. It can further be seen that an optimal exit pupil 2002 has good overlap of replications. Similarly, all other exit pupils have improved overlap over a display system without the optical system. While they are not necessarily as good as the optimal exit pupil 2002 it allows the viewing area to be expanded while acceptably preserving depth information across exit pupils (rays from each exit pupil are not straying into other exit pupils, which is the case without the optical system).

The image quality of the left-centre replicated pupil (as viewed by the eye) can be analysed. Figure 21 shows a grid of spots 2100 traced from a virtual object depth of 200mm in front of the eye through the left-centre replicated pupil at the eyebox. When compared with Figure 18, which shows a similar grid of spots through the optimal replication, it can be seen that there is comparable image quality. The image quality is good, but, in addition, many of the aberrations that are present may be corrected at the SLM (for example field curvature, and the small amount of field breakup in some regions where rays travel by different path configurations in the image replicating combiner). Figure 22 shows a simulated retinal image from the centre-left replication. In a similar manner to the simulated retinal image shown in Figure 20, it can be seen that the field positions are clearly resolved.

Figure 23 depicts a simulated output of the replications from a display system without an optical element according to this disclosure. It can be seen that replications from the image replicating combiner do not form an ordered pattern of exit pupils in the eyebox in contrast to the simulated output depicted in Figure 20B. Whereas in Figure 20B there is an optimal exit pupil with very good overlap of replications and eight exit pupils with acceptable overlap of replications, Figure 23 shows that without an optical element according to this disclosure, the replications are scattered over the eyebox, without significant overlap of replications.

The above embodiments are to be understood as illustrative examples of the invention. It has been shown how the optical system may be designed to give a set of rays which combine on extraction from the output surface of the waveguide combiner in various ways. This includes design methods working in both directions through the optical system, from input to output, and vice versa. Further embodiments of the invention are envisaged, for example using alternative design methods, such as methods using similar design constraints and assumptions. It is to be understood that any feature described in relation to any one embodiment may be used alone, or in combination with other features described, and may also be used in combination with one or more features of any other of the embodiments, or any combination of any other of the embodiments. Furthermore, equivalents and modifications not described above may also be employed without departing from the scope of the invention, which is defined in the accompanying claims.

Claims

26 CLAIMS

1. A display system comprising: an image generating unit configured to generate an image with depth information; an image replicating combiner comprising an input surface and an output surface, and configured to generate a plurality of replications of an image incident on the input surface at the output surface; and an optical system positioned in an optical path between the image generating unit and the input surface of image replicating combiner, wherein the optical system is arranged to transform a set of rays from the image generating unit having a first focal point into a plurality of sets of rays with respective different focal points, such that the plurality of sets of rays, when incident on the input surface, combine on extraction at the output surface to give each replication substantially the same focal point at a viewing position.

2. The display system of claim 1, wherein the optical system comprises: a replicating optical element to generate multiple copies of a ray; and a lens.

3. The display system of claim 2, wherein the replicating optical element is a diffractive optical element.

4. The display system of claim 2, wherein the replicating optical element is a further image replicating combiner.

5. The display system of any of claims 2 to 4, wherein the lens is positioned in an optical path between the replicating optical element and the input surface of the image replicating combiner.

6. The display system of any preceding claim, wherein the optical system is configured to generate multiple copies of a ray from the image generating unit, and wherein the optical system is arranged so that one copy from the multiple copies is incident on the input surface of the image replicating combiner.

7. The display system of any preceding claim, further comprising an adjustment system for adjusting the location of the focal point of each replication.

8. The display system of claim 7, further comprising an eye-tracking system configured to control the adjustment system based on a location of a viewer’s pupil.

9. The display system of any preceding claim, wherein the optical system is arranged to have an optical power in an x axis which is different from an optical power in a y axis.

10. The display system of any preceding claim, configured to display a computer-generated hologram image.

11. The display system of any of claims 1 to 9, configured to display a 4D light field.

12. A head-mounted display comprising the display system of any preceding claim.

13. A head-up display comprising the display system of any of claims 1 to 11.