GB2507468A - Operating a spatial light modulator - Google Patents

Abstract

Operating a spatial light modulator or SLM 201 having a plurality of pixels 203 by representing an input pattern on a first subset of pixels 221, representing a first filter of a first reference image on a second subset of pixels 222 to receive the +n diffracted order from the input, representing a second filter of a second reference image on a third subset of pixels 223 to receive the n diffracted order from the input, and performing optical correlation between the input and the reference images on the second 222 and third 223 subsets of pixels.

Description

AN OPTICAL DEVICE USING CONJUGATE ORDERS

Field

The present disclosure relates to a method of operating a spatial light modulator, Enibudinients disclosed herein relate to a method of performing optical cm-relation. More specifically, embodiments relate to a method of performing plural optical correlations using conjugate orders.

Backgiound An optical correlator is a device for comparing a first image with a second image. An optical correlator may therefore he used to detect the presence of a. refercncc object iii a scene and/or determine the location of the refei-eiice object in the scene, The first image may be the reference object and the second image may be a input scene, such a.s a photograph, which may or may not contain the reference object.

There is interest in optical correlation techniques based on the use of Fourier transforms because it is possible to achieve an optical Fourier transform instant]y, for example, using a simple converging lens. The most commonly used processing architectures for optical correlation are the joint transform correlator (JIC) and the Vancler-Lugt correlator (VLC).

Both approaches are based on a comparison between a target image and a reference image (for example, from a database) and simple detection of a cot-relation peak measuring die degree of similarity between the target and the reference. Both techniques require two Fourier transform (FT) lenses (each having a focal length of 0 arranged in a so-called 4f optical configuration resulting in a long optical path length.

Whilst the JTC is less alignment sensitive, its detection efficiency is lower and it takes longer to perform the correlation, particularly for multi-target recognition. The VLC has a higher detection efficiency but is less robust against misalignment.

Figure 1 shows an example of a VLC including an input plane 101, a first FT lens 103, a Fourier plane 105, a second FT lens 107 and an output plane 109 respectively arranged on a common optical axis.

The VLC is based on the multiplication of the spectrum of a target image by a correlator filter, made from a reference image. A linearly polarised collimated beam is used to illuminate the input plane 101 comprising of a target image displayed on a spatial light modulator (SLM). A first FT lens 103 carries out the Fourier transform of the object or target image. This results in the spectrum of the targct image at the Fourier plane 105. The Fourier plane 105 is effectively [lie back focal plane of the first FT lens 103. A correlation Jilter is placed (displayed on a SLM) in the Fourier plane 105 to coincide with the spectrum of the target image. A second FT lens 107 carries out the inverse transform of the Fourier plane 105. A CCD located at the hack focal length of the sccond FT lens 1 07 is typically used to record the systcm output at an output plane 109. The similarity betwccn the target image and the correlation filter is achicvcd by dctection of a coi-relation peak at the output plane 109.

Classical VCI.. architectures are prone to alignment sensitivity of the optical components. For cxarnplc, a relatively small defocus error in the Fourier plane can result in significant loss of correlation signal.

An overview of these correlation techniques may be found in "Understanding Correlation Techniques for Face Recognition: From Basics to Applications" by A. Alfalou and C. Brosseau, in Face Recognition, In-Tech, ISBN 978-953-7619-00-X.

Typical VLC devices use two spatial light modulators and two FT lenses. However, a lcnslcss VLC using onc phase-only SIM is disclosed in "Compact optical correlator based on one phase-only spatial light modulator" by Zcng et af, Optics Lettcrs, Vol. 36, No. 8, 15 April 2011, page 1383-1385. The system is folded by using a mirror and displaying the input and reference pattern on diFferent halves of the same SIM. A lensless conFiguration is realised by adding a Fresnel lens pattern (FLP), and optionally a. grating, to the input and the reference patterns.

The present disclosure provides an improved compact optical correlator based on one Sl'M.

Summary

Aspects of the present disclosure are defined in the appended independent claims.

There is provided a method of operating a spatial light modulator in which the +n and -n (where n is any integer) diffracted orders are used to perform different optical correlations, optiortily, in parallel or at substantially the same time.

The present disclosure utilises binary modulation such that the Fourier transform will result in the formation of the far-field and its conjugate. Accordingly, each order may be treated independently allowing the matching of two different patterns simultaneously. The device therefore allows for different correlations to be performed on different parts of the SLM, without additional computational resources and, optionally, at the same time. Certain con-elations may be selectively switched on and off. A plurality of correlations may he 1 5 pei-Ib-rrned conew-rently or in parallel. The inventors have recognised that by using binary modulation of the input pattern, conjugate orders may be used to provide further correlations.

Further advantageous embodiments provide an improved device by using a combination of a physical Fourier transform lens and a Presnel lens pattern to perform Fourier transforms necessary for optical correlation. This can be optionally be combined with gratings to simplify the optical alignment during assembly. Accordingly, a commercially-viable balance between processing demands and physical size is struck.

Brief description of the drawings

Embodiments oF the present invention will now lie described with relerenee to the accompanying drawings in which: Figure 1 is an all optical correlation setup for optical correlation; Figure 2 is a device in accordance with embodiments; and Figure 3 is an example liquid crystal on silicon spatial light modulator in accordance with embodiments.

Tn the Figures like reference numerals referred to like parts.

Detailed description

An "input image" may he a digital image of a scene, which may or may not contain a predetermined object of interest. The input image forms the input for an optical correlator.

S That is, the input image is probed to determine the likelihood that the object of interest is in the image and/or where in the image the object is located. The input image may be a captured image such as a photograph or a sub-region of a photograph.

A "reference image" may be a digital image of a. predetermined object of interest. The 1.0 reference image is effectively scanned across the input image to determine if the object of interest is anywhere in the input image. The reference image may be stoied iii a repository such as a database of reference objects.

Figure 2 shows a device in accordance with an embodiment.

There is shown a SLM 201 comprising an array of pixels 203. The SLM shown in figure 2 is configured to function in a reflective mode. That is, incident light is spatially modified by the SLM and output in reflection. SLM 201 may therefore he termed a reflective SLM. There is also shown a mirror 205 and a charge-coupled device (ccl)) 207. As shown in figure 2, the SLM is spatially-separated and substantially parallel to mirror 205. The CCD 207 is arranged to receive output light after two reflections off both the SLM 201 and the mirror 205. Fourier transform lenses 209 and 210 perform respective Fourier transforms of the light before it reaches the CCD 207.

A first subset of pixels 221 of die SLM are arranged to represent an input pattern. That is, a predetermined pixelated pattern is written to the lirsi subset oF pixels 221. The iirst subset of pixels 221 may be said to "display" the input pattern. The input pattern is rcprescntativc of a input image. The input pattern is a binary representation of the input image.

A second subset of pixels 222 of the SLM are arranged to represent a first filter pattern. That is, a further predetermined pixelated pattern is written to the pixels in the second subset of pixels 222. The second subset of pixels 222 may he said to "display" the first filter pattern.

The first filter pattern is representative of a first reference image. In an embodiment, the first filter pattern comprises a Fourier transform of the first reference image. In an embodiment, the first filter pattern may be a binary representation of a Fourier transform of the first reference image.

A third subset of pixels 223 of the STM are arranged to represent a second filter pattern. That is, a yet further predetermined pixelated pattern is written to the pixels in the third subset of pixels 223. The third subset of pixels 223 may be said Lu "display" the second filter patient The second filter pattern is representative of a second reference image. in an embodiment, the second filter pattern comprises a Fourier transform of the second reference image. In an embodiment, the second filter pattern maybe a binary representation of a Fourier transform of the second reference image.

In operation, a plane wave of light 220 is incident on the first subset of pixels 221 of the SLM.

The incident light is spatially modulated by the pixels in the first subset of pixels 221 and diiTi-acied into a conjugate pair of diffracted orders because the input image is binary -that is, each pixel can take one of only two possible valLLes The conjugates are reflected towards mirror 205. The angle of incidence between the plane wave and the normal of the SLM is greater than zero. Mirror 205 reflects the spatially modulated conjugate pair back towards the SLM. More specifically, minor 205 reflects the -n order towards a second subset of pixels 222 and the +n order towards a third subset of pixels 223. The skilled person will therefore understand that the angle of incidence between the plane wave and the SLM, and the spacing between the SEM and mirror, is chosen such that the conjugate pair are directed to the second and third subsets of pixels 222. 223. That is, a. different part of the SLM. The second and third subsets of pixels 222, 223 thiTher spatially modulate the -n and n orders, respectively.

More specifically, the -ii order is modulated by the first subset olpixels 221 and then by the second subset of pixels 222. Likewise, the +n order is modulated by the first subset of pixels 221 and then by the third subset of pixels 223. Light from the second subset of pixels 222 is directed onto a first region of CCD 207 via a Fourier transform lens 210 and, optionally, a further reflection off mirror 205 as shown in figure 2. Likewise, light from the third subset of pixels 223 is directed onto a second region of CCD 207 via a Fourier transform lens 209 and, optionally, a further reflection off mirror 205 as shown in figure 2.

The filter patterns may he any suitable filter or matched patterns for optical contlation. In an embodiment, each filter pattern is a Fourier-transform based filter pattern. That is, a filter pattern based on a Fourier transform of a reference image. In an embodiment, the filter patterns are Vander-Lugi matched patterns. The skilled person will know how to create a filter pattern from a reference image for optical correlation The skilled person will also understand how to perform the necessary Fourier Iransforms for optical correlation. Figure 2 shows an embodiment using physical Fourier transform lenses to perform a second of two Fourier transforms. Alternative methods for performing the Fourier transforms are described below. In an embodiment, the filter pattern is a phase-only representation of the Fourier transform of a reference image.

The skilled person will understand that using the correct filter pattern and Iwo Fourier transforms, an optical correlation hetwccn an input image (represented iii the input pattern) and a respective reference image (represented in a filter pattern) may be performed. Optical correlation may be detected and measlLred using CCI) 207. Correlation pcaks appear at points in the received light bean) where there is a correlation between the input image and the reference image. The spatia.l position of correlation peaks indicates points in the input image where thei-e is a correlation with the respective reference image. The height of the correlation indicates the extent of the correlation. That is, high peak indicates strong correlation. In an embodiment, the eon-elator is a Vander.-Lugt optical correlator.

The number of pixels in the input pattern and reference pattern defines the "resolution" of the correlation. The more pixels, the more accurate the correlation result or the larger the area scanned.

The skilled person will know bow to represent the patterns on the SLM. The skilled person wilt also know how to detect correlation peaks.

The input pattern is binary that is, each pixel represents one of two possible values. Because of this, the output light is diffracted. The inventors have recognised that the conjugate of' a diffracted order may be used to perform a second correlation, such as a parallel cotTelation.

Whilst embodiments relate to performing two correlations using the +1 and -1 diffracted order, the skilled person will understand that providing there is enough optical energy. higher diffracted orders, and their conjugates, may equally be used for additional eorrcbtions.

There is therefore provided a method of operating a spatial light modulator "SLM" comprising a plurality of pixels, the method comprising: representing a binary input pattern, representative of an input image, on a first subset of the pixels; representing a first filter pattern, representative of a first reference image, on a second subset of pixels arranged to receive the +n diffracted order from the binary input pattern; representing a second filter pattern, representative of a second reference image, on a third subset of pixels arranged to receive die -n diffracted order from the binary input pattern; performing a first optical correlation between the input image and the first reference image on the second subset of pixels; and performing a second optical correlation between the input image and the second reference image on the third subset of pixels, where n is an integer equal to or greater than 1.

Tt can be understood that, in embodiments, multiples diffracted ordcis and their conj ligates may be used to perform a plurality of correlations, optionally, at substantially tile same time.

1 5 That is, in an embodiment, the method further comprises: representing a third filter pattern on a fourth subset of pixels arranged to receive the +m diffracted order liorn the binary input patlern; represeuiting a fourth 1111cr pattern on a fifth subset of pixels arranged lo receive the -m diffracted order from the binary input pattern, where m is an integer not equal to it The improved method described herein allocates different parts of the STM to different diffraction orders. Tn particular, it makes use of the conjugate orders. Accordingly, this allows for additional correlations to he performed on the same SLM without additional input images and optical parts being required.

The present disclosure also allows a piunlity of optical correlations to he performed iii paralle! -that is, at substantially the same time or concurrently. For example, the device may be used to scan for more than one object in the same scene at the same lime by using different reference images for each ehammel.

It can be understood that, in an embodiment, the first and second optical correlations are performed substantially concurrently. In an embodiment, the first filter pattern is different to the second filter pattern.

The process of determining whether a reference object is present in an unrelated image may be known as "cross-correlation". In another embodiment, a second operation mode is provided in which the input image and reference image are of substantially the same scene but at different points in time. Correlation in the second operation mode may therefore be used to detect changes in a scene. For example, to determine that an object has appeared in a scene.

This second operauion mode may be knowii as an "auto-correlation". In embodiments, auto-correlations and cross-correlations may be performed substantially concurrently.

In an embodiment, the first correlation corresponds to an auto-correlation and the second correlation corresponds to a cross-correlation.

By using the conjugate order, more cfflcient usc of the computational resources is made. By performing a plurality of optical correlations at substantially (lie same time, time and resourccs are saved. For exampEe, a multichannel device ma.y perform a plurality of correlations of a captured image in parallel. This is particularly advantageous in real-time systems in which frames in a sequence of frames arc scanned for objects. ln particular, the inventors have recognised that by using a binary input image, usc of the conjugate order or orders may be made to perform different correlations or multiple correlations at the same time,

for example.

The. correlation method requires a first Fourier transform: a Fourier transform of light after modulation by the input pattern. In an embodiment, the input pattern further comprises information representative of a Fresnel lens pattern (FLP) arranged to function as a FolLrier transform lens to perform a Fourier transform of the first input image. This may he achieved by simp!e addition of FLP data to the image data. The FLP may he considered a virtual Fresnel lens. Optionally gratings may used inconjunction with the input pallern and/or the FLP to reduce thc complexity of the optical alignment. The skilled person will understand how to calculate the required FLP and add it to image data. In an embodiment, a physical Fourier transform lenses is used instead of a FLP. In a further alternative embodiment, a physical Fourier transform lens and a FLP are used in conjunction to perform the respective Fourier transforms. There is therefore provided a first Fourier fransform of the input pattern.

Advantageously, a hybrid system in which the Fourier transforms are performed by a FLP in conjunction with a physical Fourier transform iens allows for the physical size of the device to he decreased without impacting the detection sensitivity of the correlator.

The correlation method requires a second Fourier transform: a Fourier transform of the light after spatial modulation by the filter patterns. In an embodiment, the first and/or second filter patterns both also further comprise a FLP to perform a. Fourier transforni. However, in thc embodiment shown in figure 2, a physical Fourier transform lens is used instead of a FLP. In an embodiment, a physical Fourier transforms lens is used in conjunction with a FLP. There is therefore provide means for performing a second Fourier transform 1r each correlation.

Again, a hybrid system using a physical Fourier transform lens and FLP may be advantageous.

Each filter pattern may also comprise adding a grating pattern to provide software based optical alignment.

Accordingly, in an embodiment, thc binary input pattern comprises an input image, optionally, added to a Frcsncl lens pattern. Likewise, in an embodiment, the first and/or second filter patterns further comprise: adding a respective Fresnel lens pattern; and/or adding a respective grating pattern.

In an cnibodimnent, the input and/or filter patterns comprise phase-only inlbrmation. In an embodiment, the filter patterns are also binary. In such embodiments, the SLM may therefore be a binary phase-only SLM. That is, a SLM which spatially modulates only the phase (and not amplitude) of an incoming light beam. Advantageously, phasconly SLMs are more light efficient because light is not lost from the system by modulating the amplitude (intensity).

Advantageously, the use of a binary SLM allows for the conjugate orders to be used For different correlafions.

In operation, there is provided a method comprises: illuminating the first and third subset of pixels; receiving light from the In-st subset of pixels; redirecting the +n diffracted light fl-am the first subset of pixel onto the second subset olpixels; redirecting the -ii diffracted light from the first subset of pixel onto the thirdsubset of pixels; receiving light from the second and third subset of pixels on spatial light detector; and monitoring the detected signal for correlation peaks.

In an embodiment, the light from the second subset of pixels is received on a first region of the spatial light detector and light from the third subset of pixels is received on a second region of the spatial light detector.

Whilst embodiments relate to a mirror, the skilled person will understand that other reflective components may be equally suitable. Likewise, whilst embodiments relate to a CCD, it can be understood that any spatial light detector is suitable.

Analysis of the result of the optical correlation may include, for cxarnpte, determining if there is at least one correlation peak having a height greater than a predetermined threshold. For example, if there is at least one correlation peak having a height greater than a predetermined threshold, it may be determined that an object (represented in the corresponding i-elerence image) is present.

In an embodiment, the first and second reference images are different.

In accordance with the present disclosure, low resolution con-elations may identify that an object may be present in an area ci a scene. Higher resolution correlations -. using more computarionai resources and a different area of the SLM -may then be switched on and used to perform a more accurate determination of the type of object and its precise location.

In an embodiment, the SLM is a liquid crystal on silicon (LCOS) SLM. However, the skilled person will understand that other types of SLM may be equally compatible with the present

disclosure.

The structure of a suitable LCOS device is shown in Figure 3.

A LCOS device is formed using a single crystal silicon substi ate (302). It has a 21) array of square planar aluminium electrodes (301), spaced apart by a gap (301a), arranged on the upper surface or the substrate. Each of the electrodes (301) ca.n bc addressed via circuitry (302a) buried in the substrate (302). Each of the electrodes forms a respective planar minor.

An alignment layer (303) is disposed on the array of electrodes, and a liqud crystal layer (304) is disposed on the alignment layer (303). A second alignment layer (305) is disposed on the liquid crystal layer (404) and a planar transparent layer (306), e.g. of glass, is disposed on the second alignment layer (305). A single transparent electrode (307) e.g. of ITO is disposed between the transparent layer (306) and the second alignment layer (305).

Each of the square electrodes (301) defines, together with the overlying region of the transparent electrode (307) and the intervening liquid crystal material, a controllable phase-modulating element (308), often referred to as a. pixel. The effective pixel area, or fill factor, is the percentage of the total pixel which is optically active, taking into account the space between pixels (301a). By control of the voltage applied to each electrode (301) with respect to the transparent electrode (307), the a director of the liquid crystal may be set in one of two orientations thereby imposing a binary delay to light incident thereon. The effect is to provide phase-only modulation to the wavefront, i.e. no amplitude effect occurs.

An advantage of using a reflective LCOS spatial light modulator is that the liquid crystal layer can he half the thickness than would be necessary if a transmissive device were used.

This greatly improves the switching speed of the liquid crystal (a key point for projection of moving video images). A TCOS device is also uniquely capable of displaying large arrays of phase only elements in a small aperture. Small elements (typically approximately 10 microns 1 5 or smaller) result in a practical diffraction angle (a few degrees) so that the optical system does not require a very long optical path.

It is easier to adequately illuminate the small aperture (a few square centimetres) of a LCOS SLM than it would he for the aperture of a larger liquid crystal device. LCOS SLMs also have a large aperture ratio, there being very little dead space between the pixels (as the circuitry to drive them is buried tinder the mirrors). This is an important issue to Lowering the

optical noise in the replay field.

The above device typically operates within a temperature range of I 0°C to around 50°C.

Using a silicon baclcplane has the advantage that the pixels are optically flat, which is important for a phase modulating device.

Whilst embodiments relate to a reflective TCOS SLM, the skilled person will understand that any SLM can be used including transmissive S1Ms.

The invention is not restricted to the described embodiments hut extends to the kill scope of the appended claims.

Claims (15)

1. Claims 1. A method of operating a spatial light modulator "SLM" comprising a plurality of pixels, the method comprising: S representing a binary input pattern, representative of an input image, on a first subset of the Pixels; representing a first filter pattern, representative of a first reference image, on a second subset of pixels arranged to receive the -f-n diffracted order from the binary input pattern; representing a second filter pattern, representative of a. second reference image, on a third subset of pixels arranged to receive the -n diffracted order from the binary input pattern; performing a first opticaJ correlation between thc input image and tile first reference image on the second subset of pixcls and performing a second optical correlation between the input image and the second reference image on the third subset olpixels, where ii is any integer.
2. 2. The method of claim 1 wherein the first and second oplical correlations are performed substantially concurrently.
3. 3. The method of any preceding claim wherein the first filter pattern is different to the second filter pattern.
4. 4. The method of any preceding claim wherein the first correlation corresponds to an auto-correlation and thc second correlation corresponds to a cross-correlation.
5. 5. The method of any preceding claim wherein in the binary input pattern comprises the input image, optionally, added to a Fresnel lens pattern and, further optionally, added to a grating function.
6. 6. The method of any preceding claim wherein the first filter pattern comprises the Fourier transform of a first reference image and the second filter pattern comprises the Fourier transform of a second reference image.
7. 7. The method of any preceding claim wherein the first and second reference images are different.
8. 8. The method of claim 7 wherein the first and/or second filter patterns further comprise: adding a respective Fresnel lens pattern; and/or adding a respective grating pattern.
9. 9. The method of any preceding claim wherein the input pattern and/or filler pat terns comprise phaseon1y information.
10. 10. The method of any preceding claim wherein the first and/or second filter pattern is a Vander Lugt matched filter.
11. 11. The method of any preceding claim wherein the step of performing the first and second optical correlation comprises: illuminating the first and third subset of pixels; receiving light from the first subset of pixels; redirecting the +n diffracted light from We first subset of pixel onto the second subset olpixels; redirecting the -ii diffracted light from the first subset of pixel onto the third subset of pixels; receiving light from the second and third subset of pixels on spatial light detector; and monitoring the detected signal for correlation peaks.
12. 12. The method of claim 11 wherein the light from th.e second subset of pixels is received on a first region of the spatial light detector and light from the third stibset of pixels is reecivcd on a second region of the spatial light detector.
13. 13. The method of any preceding claim further comprising: representing a third filter pattern on a fourth subset of pixels arranged to receive the +m diffracted order from the binary input pattern; representing a fourth filter pattern on a fifth subset of pixels arranged to receive the -m diffracted order from the binary input pattern, where m is any integer not equal to n.
14. 14. An optical device for performing the method of any preceding claim.
15. 15. A method of operating a spatial light modulator or an optical device substantially as hereinbefore described with reference to the accompanying drawings.S