GB2496108A - Image colour frequency-based pixel number allocation - Google Patents

Abstract

A method of operating a pixellated spatial light modulator (SLM) 350 to display data representing a polychromatic image, wherein a first subset of modulator pixels are allocated to first colour image information and a second subset of SLM pixels are allocated to second colour information. The numbers of pixels in the first and second subsets are determined by the respective first and second color wavelengths, to which the human eye has differing sensitivities. Also disclosed is a projector forming a 2D polychromatic image, using the same method of allocation of amount of pixels based on colour wavelength, and a liquid crystal on silicon (LCOS) SLM to display Fourier transform (FT) data. Based on different colours, different pixel subsets comprise differing, greater or lesser, pixel numbers. Phase-only, holographic diffractive image data (e.g., based on the Gerchberg-Saxton, FIENUP, FIDOC, direct binary search or genetic algorithms) may be displayed, and the SLM may modulate to apply a phase delay to incident light. Fourier data for plural colours may be displayed simultaneously on the SLM. The method cen be applied to a head-up display (HUD), 300.

Description

Iniaue Production The present disclosure relates to a method of operating a pixellated spatial light modulator and a projector. Embodiments use liquid crystal on silicon

Background

Light scattered from an object contains both amplitude and phase information. This amplitude and phase information can be captured on, for example, a photosensitive plate by well known interference techniques to form a hologram, or holographic recording, comprising interference fringes. The hologram may be reconstructed to form an image, or holographic reconstruction, representative of the original object by illuminating the hologram with suitable light.

Computer-generated holography is a technique to numerically simulate the interference process using Fourier techniques.

It has been proposed to use holographic techniques in a two-dimensional image projector.

The system may accept as an input "conventional" 2D video information -comprising a temporal sequence of 2D image frames -convert this into a real-time sequence of holograms which may be reconstructed, in real-time, and on a screen. Accordingly, there may be provided a real-time 2D video projector using computer-generated holograms.

Referring to Figure 1, there is shown a light source 1 00 which applies light via a Fourier lens (120) onto a spatial light modulator (140) in this case as a generally planar wavefront. The spatial light modulator is reflective and consists of an array of a large number of phase-modulating elements. Light is reflected by the spatial light modulator and consists of two parts, a first specularly reflected portion (known as the zero order) and a second portion that has been modulated by the phase-modulating elements to form a wavefront of spatially varying phase. Due to the reflection by the spatial light modulator all of the light is reflected generally back towards the light source (1 00) where it impinges on a mirror with aperture (160) disposed at 45° to the axis of the system. All of the image part of the light is reflected by the mirror towards a screen (180) that is generally parallel to the axis of the system. Due to the action of the Fourier lens (120), the light that impinges on the screen (180) forms a real

I

image that is a reconstruction of an image from which the information applied to the phase modulating elements was derived.

Data written to the spatial light modulator causes the light modulating elements to vary in state, in some cases to change phase states. The spatial light modulator (SLM). for example a LCOS (liquid crystal on silicon) SLM, forms an array of phase-modulating elements that has been derived in some way from a source image (i.e. an original image that is to be reproduced) to be displayed or is generated so as to display that image. Light representative of an object is transformable into a phase delay distribution in a number of ways, including algorithms such as Gercbberg-Saxton. The aim is to provide something related to a Fourier transform of the received light. Then the light modulating elements -sometimes referred to as "pixels" (note that there is not a 1:1 correlation between pixels of the SLM and any specific location in the image, or indeed the object) -can form variable kinoforms, for example where 2D representations are to be formed.

In order to create a colour image, a plurality of monochromatic holographic reconstructions may be formed and the respective reconstructions combined to create a composite colour reconstruction. For example, a red, a blue and a green light sources, such as lasers, can be used with a single SLM or multiple SLMs to project a composite colour reconstruction. The reconstruction may be referred to as an image.

When producing colour images, for example, in the display of a video, there are two main methods of achieving this. One of these methods is known as "frame-sequential colour" (FSC). In an FSC system, a hologram is computed for each colour (e.g. red, green and blue) and displayed in succession at the SLM, The colour light sources are synchronised with the holograms to project a sequence of colour reconstructions. The colours are cycled (red, green, blue, red, green, blue, etc.) at a fast enough rate such that a human viewer sees a polychromatic image from a combination of the colours. For example, in a video at 25 frames per second, the first frame would be produced by firing thc red laser for l/7S of a second, then the green laser would be fired for 1175th of a second, and finally the blue laser would be fired for 1175th of a second. The next frame would then be produced, starting with the red laser, and so on.

An alternative method, known as "spatially separated colours" (SSC) involves dedicating one SLM to each colour light source (e.g. red, green and blur) and forming the red, green and blue holograms, and hence red, green and blue reconstructions, substantially concurrently. Such an approach required three SLMs and three holograms to be calculated per frame.

The inventor of the present invention has identified various advantages and disadvantages of these two methods, in particular when space limitations are considered. An advantage of the frame-sequential colour (FSC) method is that the whole SLM is used for each colour. This means that the quality of the three colour images produced will not be compromised because all pixels on the SLM are used for each of the colour images. However, a disadvantage of the FSC method is that the overall image produced will not be as bright as a corresponding image produced by the SSC method by a factor of about 3, because each laser is only used for a third of the time. This drawback could potentially be addressed by overdriving the lasers, or by using more powerful lasers, but this would require more power to be used, would involve higher costs and would make the system less compact.

An advantage of the SSC (spatially separated colours) method is that the image is brighter due to all three lasers being fired at the same time. However, if due to space limitations it is required to use only one SLM, the surface area of the SLIVI can be divided into three equal parts, acting in effect as three separate SLMs. The drawback of this is that the quality of each single-colour image is decreased, due to the decrease of SLM surface area available for each monochromatic image. In other words, less information content is present in each hologram.

The quality of the polychromatic image is therefore decreased accordingly. The decrease of SLM surface area available means that fewer pixels on the SLM can be used, thus reducing the quality of the image.

There is thus a trade-off between quality and brightness for a given SLM surface area (i.e. the total useable SLM surface area available from all of the one or more SLMs in the system) and laser power.

Aspects of the invention are defined in the appended independent claims.

The present disclosure relates to an improved method of operating a pixellatcd spatial light modulator to provide an improved projection system such as a 2D video projection system.

The method is particularly suitable for SSC but may be applied to FSC.

S In embodiments, a single SLM is used and the device can be implemented in a modified SSC method. The advantage of the SSC method of high brightness is maintained, whije the disadvantage the SSC method of a lower quality than the FSC method is compensated for by the adaptation of the surface areas of the SLM used for each colour. Instead of using three equally sized portions of the SLM, one for each light source, the relative sizes of the three areas are changed.

In an embodiment, the number of pixels in a subset is proportional to the wavelength of the light associated with the subset.

The inventors have recognised that the varying sensitivity of the human eye across different wavelengths of visible light can be exploited to enable an image to be created that appears to the viewer to be at a higher quality. This is achieved by setting the qualities of the three single-colour images at different levels. In turn, this is achieved by allocating more pixels of the SLM to colour/s which the eye is more sensitive to. In this way, more information is dedicated to thc colour/s have the most bearing on the perceived image quality of the composite colour reconstruction.

There is disclosed a devic.e for producing a polychromatic image using light of at least first and second different wavelengths, the device comprising means for producing a first colour of the image at a first quality using the light of the first wavelength, and means for producing a second colour of the image at a second quality different from the first quality using the light of the second wavelength.

When conventional red, green and blue light sources are used, the quality of the green image is may be made higher than that of the red image, while the quality of the blue image may be made lower than that of the red image. Because the human eye is more sensitive to green than it is to red or blue, a change in quality of the green part of the image has a much more noticeable effect to the human viewer than a change in quality of the red or blue images. As the human eye is least sensitive to blue, changes in the quality of the blue image have a much smaller effect on the viewer's perception of the quality of the combined polychromatic image.

Therefore, the perceived quality of the combined polychromatic image can be increased by decreasing the quality of the blue image and increasing the quality of the green image. The precise qualities of the three images can be set to optimum levels based on the exact wavelengths of the three light sources and the human eye's sensitivity to those wavelengths.

In some embodiments, the qualities of the three single-coiour images produced are determined by the percentage of the overall SLM surface area that is dedicated to each of the light sources. A larger SLM surface area results in a higher quality image being able to be produced. In other words, the number of pixels of the SLM dedicated to displaying information relating to a particular colour may be varied according to the photopic eye response.

Brief description of the drawings

Specific embodiments, based on the technology described above, will follow, by way of example only. It will be appreciated that routine variations can be made to alter the specific details provided herein. The examples are described with reference to the accompanying drawings, in which: Figure 1 depicts a schematic view of a system for producing an image on a screen; Figure 2 depicts a schematic view of a system for producing a virtual image; Figure 3 depicts a graph of the normalised photopic eye response of the human eye across the visible spectmm; Figure 4 shows the sensitivity of the human eye as a function of wavelength; Figure 5 shows the structure of an example LCOS device.

In the figures like reference numerals referred to like parts.

Detailed description of the drawings

It is found that the phase information alone is sufficient to generate a hologram which can give rise to a holographic reconstruction of acceptable quality. That is, the amplitude information in the hologram can be discarded. This can reduce the power of the required laser light sources but has other advantages too. Fourier-based computer generated holographic techniques have therefore been developed using only the phase information.

The image reconstructed by a hologram is given by the Fourier transform of the hologram.

The hologram is therefore a phase-only pattern representative of the Fourier transform of the object whereas the reconstructed image (or holographic reconstruction) may contain both amplitude and phase information, Gerchberg-Saxton is one example of an iterative algorithm for calculating a phase only hologram from input image data comprising only amplitude information. The algorithm starts from a random phase pattern and couples this with amplitude data to form complex data. A discrete Fourier transform is performed on the complex data and the resultant dataset is the Fourier components, which are made up of magnitude and phase. The magnitude information is set to a-utiitm value representative of the illuminating light source, and the phase is quantised, to match the phase values available. An inverse discrete Fourier transform is then performed. The result is another complex dataset, where the magnitude information is overwritten by the target image and the process is repeated. The Gerchberg-Saxton algorithm therefore iteratively applies spatial and spectral constraints while repeatedly transferring a data set (amplitude and phase), between the spatial domain and the Fourier (spectral) domain.

The Gerchberg-Saxton algorithm and derivatives thereof are often much faster than other "non-Fourier transform" algorithms such as direct binary search algorithms. Modified algorithms based on Gerchberg-Saxton have been developed -see, for example, co-pending published PCT application WO 2007/131650 incorporated herein by reference.

These improved techniques are able to calculate holograms at a sufficient speed that 2D video projection is realised. Embodiments described herein relate to 2D video projection using a computer-generated hologram calculated using such a modified Gerchberg-Saxton algorithm Holographically generated 2D video images are known to possess significant advantages over their conventionally projected counterparts, especially in terms of definition and efficiency.

To display the phase only holographic data, a phase modulating device is required. Since these devices do not modulate the amplitude, they are optically transparent, in general.

Therefore no light is lost to absorption, for example. This has the-advantage that the majority of the reconstruction light is used in the creation of the holographic reconstruction. This translates to a more energy efficient display system.

The phase modulating device may be pixellated and each pixel will act as a diffractive element, The diffraction pattern from each pixel will give rise to a complex interference pattern at a screen referred to as a replay field. Due to this complex relationship, each pixel on the hologram contributes to all parts of the reconstructed image.

An example phase modulating device is a spatial light modulator (SLM). Typically a SLM has a field of addressable phase-modulating elements. In some SLMs the phase-modulating elements are a linear or one-dimensional array of elements; in others a two dimensional array are provided. For simplicity many SLMs have a regular 2D array of like, generally square, phase-modulating elements; it is however not necessary for the phase-modulating elements to be alike in size or shape.

Figure 2 shows an example of using a reflective SLM, such as a LCOS, to produce a holographic reconstruction at a replay field location, in accordance with the present

disclosure,

A light source (210), for example a laser or laser diode, is disposed to illuminate the SLM (240) via a collimating lens (211). The collimating lens causes a generally planar wavefront of light to become incident on the SLM. The direction of the wavefront is slightly off-normal (i.e. two or thiee degrees away from being truly orthogonal to the plane of the transparent layer). The arrangement is such that light from the light source is reflected off a mirrored rear surface of the SLM and interacts with a phase-modulating layer to form an exiting wavefront (212). The exiting wavefront (212) is applied to optics including a Fourier transform lens (220), having its focus at a screen (225).

The Fourier transform tens receives light from the SLM and performs a frequency-space transformation to produce a holographic reconstruction at the screen (225) in the spatial domain.

In this process, the light from the light source is generally evenly distributed across the SLM (240), and across the phase modulating layer. Light exiting the phase-modulating layer may be distributed across the screen in order to form an image. There is no correspondence between a specific image region of the screen and any one phase-modulating element.

The hirefringent nature of the liquid crystal is taken into account in that the axis of the laser and the LCOS are aligned, to ensure accurate phase modulation.

As can be understood from the foregoing, to provide colour, a plurality of monochromatic light sources may be used (e.g. red, green and blue) and a corresponding plurality of SLMs may be provided -or a single SLM may be sub-divided at the cost of a reduction in image quality.

The present disclosure relates to addressing that reduction in quality by provided the improved method as claimed in claim 1.

Referring to Figure 3, there is shown an embodiment in accordance a virtual image of the reconstruction is formed Figure 3 shows a head-up display (300) having an SLM based system (305) for providing a real image of a holographic reconstruction (310). The holographic reconstruction is formed at a so-called replay field. The spatial position of the replay field may be varied in accordance with embodiments described herein.

The display consists of an optical combiner (320) and a lens (330) disposed between the holographic reconstruction (310) and the combiner (320). The arrangement is such that a viewer (340) looking towards the combiner (320) will see a virtual image (350) of the holographic reconstruction (310) at a distance d from the viewer and behind the combiner (320). Such a system can be used for example in a head-up display or head-mounted display.

S

In summary the information that is applied to the phase modulating elements of the SLM (380) consists of two parts, a first part that comprises the information representative of the final image and a second part which has the effect of providing a negative lensing and adjustment characteristic. By varying this latter part it is possible to cause the position of the holographic reconstruction and therefore virtual image (350) to be varied.

In some embodiments, the SLM is arranged in a one phase per pixel configuration.

In general, a smaller pixel size is preferred as this results in a larger diffraction angle.

However, at present, SLMs with pixel sizes smaller than about 5 pin cannot be made to operate at adequate speeds. It is anticipated that future technological advances will enable smaller pixel sizes to be used.

In some embodiments, the device operates within an optimum temperature range of 40-50 °C.

The device can operate at temperatures outside this range, for example at about 10 °C, but the speed of the device is slower at this temperature. In some embodiments, the device is used in conjunction with temperature regulation means, such as a cooling device, for ensuring that the device remains within a preferred temperature range when in use.

In one particular embodiment, the red light source is a laser that emits light at a wavelength of 630 nm, the green light source is a laser that emits light at a wavelength of 532 urn, and the blue light source is a laser that emits light at a wavelength of 450 am.

Figure 4 shows a graph of the sensitivity of the human eye across the visible spectrum. The wavelengths of the three light sources used in this embodiment are marked with vertical lines in the Figure. As can be seen, the human eye is most sensitive to the green light source and least sensitive to the blue light source, In embodiments, the SLM used is an LCOS SLM comprising an array of about two million pixels in a 2x1 array. The pixel size is about 10 jim and the cell gap is about 2 p.m.

In embodiments, the spatial light modulator is a Liquid Crystal over silicon (LCOS) device.

The image quality is, of course, affected by the number of pixels and the number of possible phase levels per pixel.

LCOS devices are a hybrid of traditional transmissive liquid crystal display devices, where the front substrate is glass coated with Indium Tin Oxide to act as a common electrical conductor. The lower substrate is created using a silicon semiconductor process with an additional final aluminium evaporativc process being used to create a mirrored surface, these mirrors then act as the pixel counter electrode.

Compared with conventional glass substrates these devices have the advantage that the signal lines, gate lines and transistors are below the minored surface, which results in much higher fill factors (typically greater than 90%) and higher resolutions.

LCOS devices are now available with pixels between 4.5prn and 12 pm, this size is determined by the mode of operation and therefore amount of circuitry that is required at each pixel.

The structure of an LCOS device is shown in Figure 5.

A LCOS device is formed using a single crystal silicon substrate (402). It has a 2D array of square planar aluminium electrodes (401), spaced apart by a gap (4Ola), arranged on the upper surface of the substrate. Each of the electrodes (401) can be addressed via circuitry (402a) buried in the substrate (402). Each of the electrodes forms a respective planar mirror.

An alignment layer (403) is disposed on the array of electrodes, and a liquid crystal layer (404) is disposed on the alignment layer (403). A second alignment layer (405) is disposed on the liquid crystal layer (404) and a planar transparent layer (406), e.g. of glass, is disposed on the second alignment layer (405). A single transparent electrode (407) e.g. of ITO is disposed between the transparent layer (406) and the second alignment layer (405).

Each of the square electrodes (401) defines, together with the overlying region of the transparent electrode (407) and the intervening liquid crystal material, a controllable phase-modulating element (408), often referred to as a pixel. The effective pixel area, or fill factor, is the percentage of the total pixel which is optically active, taking into account the space between pixels (40.1 a). By control of the voltage applied to each electrode (401) with respect to the transparent electrode (407), the properties of the liquid crystal material of the respective phase modulating element may be varied, thereby to provide a variable delay to light incident thereon. The effect is to provide phase-only modulation to the wavefront, i.e. no amplitude effect occurs.

A major advantage of using a reflective LCOS spatial light modulator is that the liquid crystal layer is half the thickness that it would be if a transmissive device were used.

This greatly improves the switching speed of the liquid crystal (a key point for projection of moving video images). A LCOS device is also uniquely capable of displaying large arrays of phase only elements in a small aperture. Small elements (typically approximately 10 microns) result in a practical diffraction angle (a few degrees) so that the optical system does not require a very long optical path.

It is easier to adequately illuminate the small aperture (a few square centimetres) of a LCOS SLM than it would he for the aperture of a larger liquid crystal device. LCOS SLMs also have a large aperture ratio, there is very little dead space between the pixels (as the circuitry to drive them is buried under the mirrors). This is an important issue to lowering the optical

noise in the replay field.

The above device typically operates within a temperature range of 10°C to around 50°C, with the optimum device operating temperature being around 40°C to 50°C.

As a LCOS device has the control electronics embedded in the silicon backplane, the Fill factor of the pixels is higher, leading to less power in the higher diffraction orders.

Using a silicon backplane has the advantage that the pixels are optically flat, which is important for a phase modulating device.

A Peltier device may be used to keep the operating temperature of the device at about 45 °C.

In an embodiment, the surface area of the SLM is divided up for use by the three light sources such that the green light source uses 75% of the surface area, the red light source uses 20%, and the blue light source uses S%. The surface areas are thus generally in proportion to the human eye's sensitivity at these wavelengths. In this embodiment, the following equation is used to calculate the SLM surface area to be used by each light source: A(A. )= AT*P(2,fl) P(2) + R+1) + P(1) where A(A) is the SLM surface area to be used for light of a particular wavelength 2, A is the total SLM surface area, P is the photopic eye response curve, and ii is the number of specific wavelengths being used. In other embodiments, more complex algorithms are used, which consider, for example, how the area is distributed with regard to the the size of the tiles.

The device can also be used to provide 3D holographic video.

Additionally the quality of the reconstructed hologram from the LCOS SLM is also affect by the so-called zero order problem which is a consequence of the diffractive nature of the 1 5 reconstruction.

Such zero-order light can be regarded as "noise" and includes for example specularly reflected light, and other light that is unrefracted by the patterns on the spatial light modulator.

This "noise" is generally focussed at the focal point of the Fourier lens, leading to a bright spot at the centre of a reconstructed hologram. In a direct view application the zero order wou!d be a substantial distraction when looking at the virtual image.

Conventionally, the zero order light is simply blocked out however this would clearly mean replacing the bright spot with a dark spot.

However as the hologram contains three dimensional information, it is possible to displace the reconstruction into a different plane in space -see, for example, published PCT application WO 2007/131649 incorporated herein by reference.

It will be understood that the above description of specific embodiments is by way of example only and it is not intended to limit the scope of the invention. Many modifications of the described embodiments, some of which are now described, are envisaged and intended to be covered by the appended claims.

In some embodiments, instead of being a laser, each light source is a laser diode, or an LED, or any other suitable light source.

In some embodiments, a different algorithm from that used in the specific embodiment described above is used to calculate the phase pattern, such as the Gerchberg-Saxton algorithm, FIENUP, FIDOC, Direct binary search and its variants, or Genetic algorithm.

In some embodiments, more than one SLM is used, e.g. one for each light source, or one SLM for the green light source and one shared between the other two light sources.

In some embodiments, the surface area of the SLM is divided up such that the green light 1 5 source uses 65-75% of the surface area, the red light source uses 20-25% of the surface area and the blue light source uses 5-10% of the surface area. Other values are of course possible.

In some embodiments, the green light source uses a larger surface area of the SLM than either of the other two hght sources. In some embodiments, the blue light source uses a smaller surface area of the SLM than either of the other two light sources.

In some embodiments, instead of the red, green and blue light sources descrihed above, any combination of two or more light sources of at least two different wavelengths is used to create a polychromatic image. The respective surface areas used by each light source on the SLM is determined based on the human eye's sensitivity at those wavelengths to ensure that the image produced is of the highest possible perceived quality by the viewer. In some embodiments, only two light sources are used, while in other embodiments three or more light sources are used.

Although the above embodiments are described in relation to an LCOS SLM, in some embodiments, the device is another kind of imaging device that is capable of producing images at different wavelengths and different qualities. For example a MEMS (Microelectromechanical systems) SLM is used, or an LCD or an OASLM.

Some embodiments implement the technique of "tiling", in which the surface area of the SLM allocated to a particular light source is further divided up into a number of tiles, each of which is set in a phase distribution similar to that of the original tile. Each tile is then used to produce a copy of the image to be produced at that wavelength. The images produced via S each tile then combine to form a combined image. Each tile is therefore of a smaller surface area than if the whole allocated area of the SLM were used as one large phase pattern. The image is created within the first diffraction order, and it is preferred that the second and subsequent orders are obscured.

The reconstructed image produced by this method (whether with tiling or without) comprises spots that are akin to image pixels (not to be confused with the pixels of the SLM). The higher the number of tiles used, the smaller these spots become. If one takes the example of a Fourier transform of an infinite sine wave, a single vertical spike is produced. This is the optimum output. In practice, if just one tile is used, this corresponds to an input of a single phase of a sine wave, with a zero value extending in the positive and negative directions from the end nodes of the sine wave, Instead of a single spike being produced from its Fourier transform, main spike is produced with a series of spikes on either side of it, corresponding to different diffraction orders. The intensity of these spikes diminishes with distance away from the central spike. The use of tiling reduces the number and intensity of visible spikes to the sides of the main spike, thus resulting in an output that is more concentrated to a smaller area. As a direct result of this, less interference (constructive or destructive) occurs between adjacent image pixels, thereby improving the image quality.

Preferably, each tile is a whole tile, although it is possible to use fractions of a tile. These contain most of the information of a whole tile.

In some embodiments, tiling is not used and each light source is incident on only one instance of the phase pattern used to produce its respective image. In some embodiments, more tiling is used for a light source that uses a larger surface area of the SLM than for a light source that uses a smaller surface area. In some embodiments, the amount of tiling is proportional to the surface area used by each light source. In some embodiments, each tile is of an identical size.

There is disclosed a device for producing a first colour of the image at a first quality and a second colour of the image at a second quality, different from the first quality using the light of the second wavelength.

The first quality may be higher than the second quality, wherein a normal human eye is more sensitive to light of the first wavelength than light of the second wavelength.

Each of the means for producing may comprise a spatial light modulator surface area arranged to be used only by the respective means for producing to produce the respective colour of the image.

The spatial light modulator surface area may rearranged to be used by the means for producing the first colour of the image is larger than the spatial light modulator surface area arranged to be used by the means for producing the second colour of the image.

Each spatial light modulator surface area may comprise a respective portion of a single spatial light modulator.

Each spatial light modulator surface area may be located on an LCOS spatial light modulator.

Each of the means for producing may comprise a respective light source for emitting light at the respective wavelength.

The device may further comprise means for producing a third colour of the image at a third quality using light of a third wavelength different from the first and second wavelengths.

The third quality may be lower than the second quality, and a normal human eye is more sensitive to light of the second wavelength than light of the third wavelength.

The image that the device is arranged to produce may be a three-dimensional image.

There is also disclosed a method of producing a polychromatic image using light of at least first and second different wavelengths, the light of the first wavelength producing a first colour of the image at a first quality, and the light of the second wavelength producing a second colour of the image at a second quality different from the first quality.