GB2507469A

GB2507469A - Operating a spatial light modulator

Info

Publication number: GB2507469A
Application number: GB1215688.1A
Authority: GB
Inventors: Jamieson Christmas; Dackson Masiyano; William Alden Crossland; Neil Collings
Original assignee: Two Trees Photonics Ltd
Current assignee: Two Trees Photonics Ltd
Priority date: 2012-09-03
Filing date: 2012-09-03
Publication date: 2014-05-07
Anticipated expiration: 2032-09-03
Also published as: GB2507469B; GB201215688D0

Abstract

Operating a spatial light modulator or SLM 201 having a first input pattern on a first subset of pixels 221, a first filter pattern on a second subset of pixels 222, a second input pattern on a third subset of pixels 223, a second filter pattern on a forth subset of pixels 224, and performing a first and second optical correlation. The first input pattern is representative of a first input image. The first filter pattern is representative of a first reference image. The second input pattern is representative of a second input image. The second filter pattern is representative of a second reference image. The first optical correlation is performed between the first input image and the first reference image. The second optical correlation is performed between the second input image and the second reference image.

Description

A MULTICI-JANNEL OPTiCAL DEVTCF.

Field

The present disclosure relates to a method of operating a spatial light modulator.

Embodiments disclosed herein relate to a method of performing optical correlation. More specifically, embodiments relate to a method of performing multichannel optical correlation on a single spatial light modulator.

Backizround An optical correlator is a device for comparing a lirst image wIth a second image. An optical correlator may therefore be used to detect the presence of a reference object in a scene and/or determine the location of the reference object in the scene. The first image may be the reference object and the second image may be a input scene, such as a photoaph, which may or may not contain the reference obj ect.

There is interest in optical correlation techniques based on the use of Fourier transfonns because ii is possible to achieve an optical Fourier transform mslanLly, for example, using a simple converging lens. The most commonly used processing architectures lbr optical correlation arc the joint transform correlalor (JTC) and the Vander-Lugt corrclator (VLC).

Both approaches are based on a comparison between a largeL image and a reference image (for example, from a database) and simple detection of a correlation peak nieasuring the degree of similarity between the target and the reference. Both techniques require two Fourier trunsfomi (FT) lenses (each having a focal length of 0 arranged in a so-called 4f optical configuration resulting in a long optical path length.

Whilst die JTC is less aligiirnent sensitive, its detection efficiency is lower and it takes longer to perform the conelation, particularly for multi-target recognition. The VLC has a higher detection efficiency but is less robust against misalignment.

Figure 1 shows an example of a VLC including all input plane 101, a first FT lens 103, a Fourier plane 105, a second FT lens 107 and an output plane 109 respectively arranged on a common optical axis.

The VLC is based on the multiplication of the spectrum of a target image by a correlator filter made ibm a reference image. A linearly polarised collimated heani is used to illuminate the input plane 101 comprising of a target image displayed on a spatial light modulator (SLM). A first FT lens 103 carries out the Fourier transform of the object or target image. This results in the spectrum of the target image at the Fourier plane 105. The Fourier plane 105 is effectively the back focal plane of the first FT lens 103. A correlation filter is placed (displayed on a SLM) iii the Fourier plane 105 to coincide wilh the spectrum of the target image. A second FT lens 107 carries out the inverse transform of the Fourier plane 105. A CCI) located at the back focal lenglh of the second FT lens 107 is typically used Lo record the system output at an output plane 109. The similarity between the target image and the correlation filter is achieved by detection of a correlation peak at the output plane 109.

Classical VCL architectures are prone to alignment sensitivity of the optical components. For example, a relatively small defocus error in the Fourier plane can result in significant loss of correlation signal.

An overview of these correlation techniques may be found in "Understanding Correlation Techniques for Face Recognition: From Basics to Applications" by A. Allhlou and C. Brosscau, in Face Recognition, Tn-Tech, ISBN 978-953-7619-00-X.

Typical VLC devices use two spatial light modulators (SLMs) and two FT lenses. however, a lensless VLC using one phase-only SLM is disclosed in "Compact optical correlator based on one phase-only spatial light modulator" by Zeng ci aL, Optics Letters, Vol. 36, No. 8, 15 April 2011, page 1383-1 385. The system is folded by using a mirror and displaying the input and reference pattern on different halves of the same SLM. A lensless configuration is realised by adding a predetermined Fresnel lens pattern (ftP), and optionally a grating, to the input and the reference patterns.

The present disclosure provides an improved compact optical corrclator based on one SLM.

Summary

Aspects of the present disclosure are defined in the appended independent claims.

There is provided a method of operating a spatial light modulator comprising a plurality of pixels, the method comprising configuring the SLM in subsets of pixels and using pairs of subsets of pixels to perform optical correlations, optionally, at substantially the same time.

The present disclosure provides a multichannel system in which a single SLM may carry out multiple correlations by spatially separating different input images and utilising a reflective substrate possessing optical power. Each input image having a respective filter, or matched, pattern. The multichannel device allows for different correlations to be performed on different parts of the SLM. Certain correlations may therefore be selectively switched on and off A plurality of con-elalions may be performed concurrently or in parallel. The inventors have recognised that a SLM comprising a pluralily of pixels may be sub-divided to provide a inultichannel optical correlator.

Advaiitageously, a more compact device is provided by using a reflective substrate having optical power in conjunction with a Fresnel lens pattern to perform a Fourier transform necessary for optical correlation. Accordingly, a commercially-viable balance between processing demands and physical size is struck.

Brief description of the di-awings

Embodiments of the present invention will now be described with reference lo the accompanying drawings in which: Figure 1 is an all optical correlation setup for optical correlation; Figure 2 is a device in accordance with embodiments; and Figure 3 is an example liquid crystal on silicon spatial light modulator in accordance with cinhodimenls In the figures like reference numerals referred to like parts.

Detailed description

An "input image" may he an digital image of a scene, which may or may not contain a predetermined object of interest. The input image forms the input for an optical corrclator.

That is, the input image is probed to determine the likelihood thai the obj ect of interest is iii the image and/or where in the image the object is located. The input image may be a captured image such as a photograph or a sub-region of a photograph.

A "reference image" may be a digital image of a predetermined object of interest. l'hc reference image is effectively scamied across the input image to determine if the object of interest is anywhere in the input image. The reference image may be stored in a repository such as a database of reference objects.

Figure 2 shows a device in accordance with an embodiment.

There is shown a SLM 201 comprising an array of pixels 203. The SLM shown in figure 2 is configured to function in a reflective mode. That is, incident light is spatially modified by the SLM and output in reflection. SLM 201 may therefl re be termed a reflective SLM. There is also shown a reflective substrate 205 and a chargc-coupled device (CCD) 207. As shown in figure 2, the SLM is spaLially-separated and substantially parallel to the reflective substrate 205. The CCD 207 is arranged to receive output light alter two reflections off both the SLM 201 and the reflective substrate 205.

A first subset of pixels 221 of the SLM are arranged to represent a first input pattern. That is, a predetermined pixelated pattern is written to the first subset of pixels 221. The first subset of pixels 221 may be said to "display" the first input pattern. The first input pattern is representative of a first input image. In an embodiment, the input pattern may be a binary representation of the first input image.

A second subset of pixels 222 of the SLM arc arranged to represent a first filter pattern. That is. a further predetermined pixelated pattern is written to die pixels in the second subset of pixels 222. The second subset of pixels 222 may be said to "display" the first filter pattern.

The first filter pattern is representative of a first reference image. In an embodiment, the first filter pattern comprises a Fourier transform of the first reference image. In an embodiment, the first filter pattern ma.y be a binary representation of a Fourier transform of the first reference image.

A third subset of pixels 223 of the SLM are arranged to represent a second input pattern. That is, another predetermined pixciated pattern is written to the third subset of pixels 223. The third subset of pixels 223 may be said to "display" the second input pattern. The second input pattern is representative of a second input image. In an embodiment, the second input pattern may be a. binary representation of the second input image.

A fourth subseL of pixels 224 of thc SLM are arranged to represent a second filler pattern.

That is, a yet further predetermined pixolated pattern is written to the pixels in the fourth subset of pixels 224. The fourth subset of pixels 224 may be said to "display" the second filter pattern. 1'lie second filter pattern is representative of' a second reference image. In an embodiment, the second filter pattern comprises a Fourier transform of the second reference image. In an embodiment, the second filter pattern may be a binary representation of a Fourier transform of the second reference image.

In operation, a plane wave of light is incident on the first and third subsets of pixels 221, 223 of the SLM. The incident light is spatially mudulated by the pixels in the first and third subset o[ pixels 221, 223 and reflecied towards reflective substrate 205. the angle of' incidence between the plane wave and the nonnal of the SLM is greater than zero. the refleclive substrate 205 reflects the spatially modulated light back towards the SLM. More specifically, reflective substrate 205 reflects the spatially modulated light from the first and third subset of pixels 221, 223 towards a second and fourth subset of pixels 222, 224 of the SLM, respectively. The skilled person will therefore understand that the angle of incidence between the plane wave and the SLM, and the spacing between ihe SLM and the retlective substrate, is chosen such that the spatially modulaied light reflecting off the reflective substrate is directed to the second and fourth subsets of pixels 222, 224. That is, a different part of the SLM. The second and fourth subsets of pixels further spatially modulates the light. More specifically, a first portion of the original plane wave is modulated by the first subset of pixies 221 and then by the second subset of pixels 222. Likewise, a second portion of the original plane wave is niociulated by the third subset of pixels 223 and then by the fourth subset of pixels 224. Light from the second subset of pixels is directed onto a first region of CCD 207 optionally via a further reflection off the reflective substrate 205 as shown in ligure 2. Likewise, light from the fourth subset of pixels is directed onto a second region of CCI) 207 optionally via a further reflection off the reflective substrate 205 a.s shown in figure 2.

The filter pattern may be any suitable filter or matched pattern for optical correlation. In an cmbodiment, the filter pattern is a Fourier-transform bascd filter pattern. That is, a filter pattern based on a Fourier transform of a reference image. In an embodiment, the filter pattern is a Vander-Lugt matched pattern, The skilled person will know how to create a filter pattern from a reference image for optical correlation. The skilled person will also understand how to perform the necessary Fourier transforms for optical colTelation. Three alternative methods for performing the Fourier transforms are described below. In an embodiment, the filter pattern is a phase-only representation of the I'ourier transform of a reference image optionally including a FLP.

the skilled person will understand that using the correct filter pattern and two Fourier transforms, an optical correlation between the input images (represented in the input patterns) and the respective reference images (represented in the filter patterns) may be performed.

Optical correlation may be detected and measured using CCD 207. Correlation peaks appear at points in the received light beam where there is a. correlation between the input image and the reFerence image. The spatial position of correlation peaks indicates points in the input image whei-e there is a correlation with the respective reference image. The height of the correlation indicates the extent of the correlation. That is, high peale indicates strong correlation. In an embodiment, the correlator is a Vander-Lugt optical correlator.

It can be understood that the SLM is configured to define pairs of subsets u! pixels to perform optical correlations. In figure 2 there arc shown two pairs of subsets of pixels: the first pair being subset 221 and 222; arid the second pair being subset 223 and 224. the skilled person will understand that although embodiments relate to two optical correlations, the system is scalable to perform any number of optical correlations as allowed by the number of pixels on the SLM and the size of the images.

The number olpixels in the input pattern and reference pattern defines the "resolution" of the correlation. The more pixels, the more accurate the correlation result or the larger the area scanned. however, the more pixels, the more processing power is required to perform the correlation.

The skilled person will know how to represent the patterns on the SLM. The ski lied person will also know how to detect correlation peaks.

There is therefore provided a method of operating a spatial light modulator comprising a pluralily of pixels, the method comprising: representing a first input pattern, representative of a first input image, on a first subset of the pixels; representing a first filter pattern, representative oF a first reference image, on a second subset of the pixels; representing a second input pattern, representative of a second input image, on a third subset of the pixels; representing a second filler pattern, representative of a second reference image, on a fourth subset of the pixels; performing a first optical correlation between first input image and the first reference image and a second optical correlation between the second input image and the second reference image.

Ihe improved method described herein allocates different parts of the SLM to different correlations. Accordingly, this allows for certain correlations to be easily switched on and switched off Moreover, it allows for data to be written to one area of the SLM whilst a colTela.tlon is performed using a different area of the SLM. Therefore, more efficient use ol' time is made. That is, more correlations are provided per unit time. Accordingly, there is provided a multichannel optical eorrelator.

The present disclosure also allows multiple optical correlations to be performed in parallel -that is, at substantially the same time or concurrently. For example, the multieharmel optical eon-elator may be used to scan for more than one object in the same scene at the same time by using different reference images for each channel.

It can be understood that, in an embodiment, the first and second optical correlations are perfonned substantially concurrently. In an embodiment, the first input pattern is different to the second input pattern and/or the first filter pattern is different to the second filter pattern.

For example, different resolution correlations may be performed in parallel. Aeeordthgly, in an embodiment, the number of pixels in the first subset is different to the number of pixels in the third subset and/or the number of pixels in the second subset is different to the number of pixels in the fourth subset.

The process of determining whether a reference object is present in an unrelated image may he known as "cross-correlation". In another embodiment, a second opeiaton mode is provided in which the input image and reference image are of suhstantialiy the same scene hut at different points in time. Correlation iii the second operation mode may therefore be used to dctcct changes in a scene. For example, to determine that an object has appeared in a scene.

This second operation mode may be known as an "auto-correlation". In embodiments, auto-correjations and cross-correlations may be performed substantially concurrently.

In an embodiment, tile first correlation corresponds to an auto-correlation and the second correlation corresponds to a cross-correlation.

By performing a plurality of optical correlations at substantially the same time, time and resources are saved. For example, a multichannel device may perform a plurality of correlations of a captured image in parallel. This is particularly advantageous in real-time systems in which frames in a sequence of frames are scanned for objects. hi particular, the inventors have recognised that an SLM may be divided up as described to provided a multichannel optical correlator or "sensor".

The con-elation method requires a first Fourier transform: a Fourier transform of each input image. In an embodiment, the first and/or second input patterns both further comprise information representative of a Fresnel lens pattern (FLP) arranged to function as a Fourier transform lens to perfoim a Fourier transform of the first and/or second input image, respectively. This may be achieved by simple addition of FLP dath to the image data. The FLP may be considered a virtual Fresnel lens. The skilled person will understand how to calculate the required FLP and add it to image data. Optionally, gratings may be used in conjunction with the input pattern and/cr the FL1' to reduce the complexity of optical ahgnnient.

In a Further advantageous embodiment, at least a portion of the rellective substrate possesses optical power to function as a Fourier transform lens. In a yet further advantageous embodiment, optical power provided by the reflective substrate and the corresponding FLP are used in conjunction to perform the respective Fourier transforms. Advantageously, a hybrid system in which a fourier transform is performed by a i"LP in conj unction with optical power provided by the reflective substrate allows for the physical size of the device to be

B

decreased without impacting the detection sensitivity of the devicc. A hybrid system also allows for some software-based reconfiguration. In a fiart}ier embodiment, different parts oF the reflective substrate have different optical. powers. The skilled person will understand how to provide a reflective surface with optical power and align said si.irfiices with the beams of S the correlator as necessary. In an embodiment, the reflective substrate comprises a plurality ol reconfigurable reflective elements. In an embodiment, the reflective substrate is a MEMS device.

The correlation method requires a second Fourier transform: a Fourier transform of each flltcr pattern. In an embodiment, the first and/or second filter patterns both also further comprise a FLP. In an alternative enibodrnent, optical power at the reflective substrate may be used instead of the FLPs or be used in conjunction with the FLPs. Ihere is therefore provide means for performing a second Fourier transform for each correlation. Again, a hybrid system combining optical power at die rellective substrate with a FLP on the SLM is advantageous. Each filter pattern may also comprise adding a grating pattern.

Accordingly, in an embodiment, the first input pattern comprises a first input image, optionally, added to a Fresnel lens pattern and/or the second input pattern comprises the second input image, optionally, added to a Fresnel lens pattern. Likewise, in an embodiment, the Iirst and/or second filter patterns further comprise: adding a respective Fresnel lens pattern; and/or adding a respective grating pattern.

In an embodiment, the input and/or filter patterns comprise phase-only information. In an enibodinient, the input and filter patterns comprise binary phase-only information. In such embodiments, the SLM may therefOre be a phase-only SLM. That is, a SLM which spatially modulates only the phase (and not amplitude) of an incoming light beam. Advantageously, phase-only SLMs are more light efficient because light is not lost from the system by modulating the amplitude (intensity).

In operation, there is provided a method comprising: illuminating the first and third subset of pixels; receiving light from the first and third subset of pixels; redirecting the light onto die second and fourth subset of pixels respectively; receiving light from the second and fourth subset of pixels on spatial light detector; and monitoring the detected signal for correlation peaks.

It can be understood that embodiments utilising a reflective substrate having optical power arc equafly applicable to single-channel optical eorrc!ation. That is, whore there is only one input pattern and one filter pattern to perform one optical correlation. In an embodiment, the light is received from the first, and optionally third, subset of pixels by a reflective substrate having optical power and the method further comprises adding optical power to the received light.

Likewise, optical power may be added by the reflective substrate to the light received from the second and/or fourth subset of pixels. It can be understood that optical power provided at the reflective substrate may coi-nbinc with the optical processing provided by the 1 0 corresponding FLP to provide the necessary Fourier transfonu's.

In an embodiment, the light from the second subset of pixels is received on a first region of the spatial light detector and light from the fourth subset of pixels is received on a second region of the spatial light detector.

Whilst embodiments relate to a reflective substrate, the skilled person will understand that oilier reflective components, such as a mirror, may be equally suitable. Likewise, whilst embodiments relate to a CCD, it can be tLnderstood that any spatial light detector is suitable.

Analysis of the result of the optical correlation may include, for example, determining if there is at least one correlation peak having a height greater than a predetermined threshold. For example, il there is at least one correlation peak having a height greater than a predetermined threshold, it may be determined that an object (represented in the corresponding reference image) is present.

In accordance with the present disclosure, low resolution correlations may identify that an object may be present in an area of a scene. Higher resolution correlations -using more computational resourecs and a different area of the SLM -may then be switched on and used to perform a more accurate determination of the type of object and its precise location.

The first and second input images may be derived from the same image such as a photograph of a scene. For example, a first correlation may be performed on a first region of the scene and a second correlation may be performed on a second region of the scene. The first and second regions may or may not at least partially ovel-lap.

Accordingly, in an embodiment, the input pattern is representative of an input image corresponding to at least a first region of a captured image and the second input pattern is representative of a secon.d input image corresponding to at least a second region of the captured image. Tn an frirther embodiment, the first and sccond regions of the captured image at least partially overlap. In an embodiment, the second region of the captured image is a subset of the first region, or vice versa. In an embodiment, the first and second reference images are different.

Tn an embodiment, the SIM is a liquid crystal on silicon (LCOS) SLM. however, the skilled person will understand that other types of SIM may he equally compatible with the present disc] osure.

The structure of a stutahie LCOS device is shown in Figui-e 3.

A LCOS device is 1'ormcd using a single crystal silicon substrate (302). It has a 2D array of square planar aluminium electrodes (301), spaced apart by a gap (301a), arranged on the upper surface of the substrate. Each of the electrodes (301) can bc addressed via circiLliry (302a) buried in the substrate (302). Each of the electrodes forms a respective planar mm-or.

An alignmen.t layer (303) is disposed on the array of electrodes, and a]iquid erysLal layer (304) is disposed on the alignment layer (303). A second aliinent layer (305) is disposed on the liquid crystal layer (404) and a planar transparent layer (306), e.g. olglass, is disposed on the second a]igmnent layer (305). A single transparent electrode (307) e.g. of ITO is disposed between the transparent layer (306) and thc second alignment layer (305).

Each of thc square electrodes (301) derines, together with the overlying region of the transparent electrode (307) and the intervening liquid crystal material, a controllable phase-modulating element (308), often referred to as a pixel. The effective pixel area, or fill factor, is the percentage of the total pixel which is optically active, taking into account the space between pixels (3Ola). By control of the voltage applied to each electrode (301) with rcspeet to the transparent electrode (307), the properties of the liquid crystal material of the respective phase modulating element may be varied, thereby to provide a variable delay to ligh.t incidcnL thereon. The effect is to provide phase-only modulation to the wavefront, i.e. no amplitude effect occurs.

An advantage of using a reflective LCOS spatial light modulator is that the liquid crystal layer can be half the thickness than would be necessary if a transniissive device were used.

This greatly improves the switching speed of the liquid crysta' (a key point for projection of moving video images). A JCOS device is also uniquely capable of displaying large arrays of phase only elements in a small aperture. Small elements (typically approximately 10 microns or smaller) result in a practical diffi-action angle (a few degrees) so that the optical system does not require a very long optica.l path.

It is easier to adequately illuminate the small aperture (a few square centimetres) of a LCOS SLM than it would he for the aperture of a larger liquid crystal device. LCOS SLMs also have a large aperture ratio, there being very little dead space hei:wecn the pixels (us the circuitry to drive them is buried under the Inin-ors). This is an important issue to lowering the

optical noise in the replay field.

The above device typically operates within a. temperature range of 10°C to around 50°C, with the optimum device operating temperature being around 40°C to 50°C, depending however on the LC composition used..

Using a silicon backplane has the advantage that the pixels are optically flat, which is important for a phase modulathg device.

Whilst embodiments relate to a reflective LCOS SLM, the skilled person will Ltnderstand that any SLM can he used including transmissivc SLMs.

The invention is not restricted to the described embodiments but extends to the full scope of the appended claims.

Claims

Claims 1. A method of operating a spatial light modulator "SLM" comprising a plurality of pixels, the method comprising: representing a first input pattern, representative of a. first input image, on a first subset oftlie pixels; represenLing a first filter pattern, representative of a first reference image, on a second subset of the pixels; representing a second input pattern, representative of a second input image, on a third subset of the pixels; representing a second filter pattern, representative ola second reFerence image, on a fourth subset of the pixels; performing a first optical correlation between first input image mid the first reference image an.d a second optical con-elation between the second input image and the second reference image.
2. The method of claim 1 wherein the first and second optical correlations are perfonned substantially concurrently.
3. The method of any preceding claim wherein the step of performing the first and second optical correlation comprises: ilJuminating the first and third subset of pixels; receiving light from the first and third subset of pixels on a reflective substrate having optical power to add optical power to the received light; redirecting the light onto the second and fourth subset of pixels respectively; receiving light from die second and fourth subset olpixels on spatial light detector; and monitoring the detected signal for correlation peaks.
4. The method of any preceding claim wherein the number of pixek in the first subset is different to the number of pixels in the third subset and/or the number of pixels in the second subset is different to the number of pixels in the fourth subset.
5. The method of any preceding claim wherein the first input pattern is different to the second input pattern and/or the first filter pattern is different to the second filter pattern.
6. The method of any preceding claim wherein the first correlation corresponds to an S auto-correlation and the second correlation corresponds to a cross-correlation.
7. The method of any preceding claim wherein the lirst input pattern comprises the first input image, optionally, added to a Fresnel lens pattern and/or a grating function.
8. The method of any preceding claim wherein the second input pattern comprises the second input image, optionally, added to a Fresne! lens pattern and/or a grating function.
9. The mcthod of claim 7 wherein the first input image corresponds to at least a first region of a captured image and/or the second input image corresponds to at least a second region of the captured image.
10. The method as claimed in claim 8 wherein the first and second regions of the captured image at least partially overlap.
11. The method of claim 9 or 10 wherein the second region of the captured image is a subset of the first region, or vice versa.
12. The method of any preceding claim wherein the first filter pattern comprises the Fourier transform of th.e first reference image and the second filter pattern comprises the Fourier transform of the second reference image.
13. The method of any preceding claim wherein the fist and second reference images are different.
14. The method of claim 12 wherein the first and/or second filter patterns further comprise: adding a respective Fresnel lens pattern; and/or adding a respective grating pattern.
15. The method of any preceding claim wherein the input image and/or filter patterns comprise phase-only information.
16. The method of any preceding claim wherein the first and/or second filter pattern is a Vander-Lugt matched filter.
17. The method of claim 16 wherein the light from the second subset of pixels is received on a first 1-egion of the spatial light detector and light from the fourth subset of pixels is received on a second region of the spatial light detector.
18. An optical device for performing the method of any preceding claim.
19. A method of operating a spatial light modulator or an optical device substantially as hereinhefore described with reference to the accompanying drawings.