GB2363886A

GB2363886A - Hybrid digital/optical system for pattern recognition

Info

Publication number: GB2363886A
Application number: GB0120301A
Authority: GB
Inventors: Meirion Francis Lewis; Rebecca Anne Wilson
Original assignee: Qinetiq Ltd; UK Secretary of State for Defence
Current assignee: Qinetiq Ltd; UK Secretary of State for Defence
Priority date: 1996-09-21
Filing date: 1996-09-21
Publication date: 2002-01-09
Anticipated expiration: 2016-09-21
Also published as: GB0120301D0; GB2363886B

Abstract

An initial Fourier Transform on input data is performed by a specialised fast Fourier Transform chip 2, the resulting spectral phase data being digitally added 4 to pre-stored template data 3 at high speed and loaded onto a phase modulating spatial light modulator 5. A subsequent optical Fourier transform 9 is used to generate a two-dimensional correlation between the input scene and template. A non-linear thresholding spatial light modulator 10 is used in the correlation plane to detect a strong correlation peak at a speed commensurate with the template search rate. An area photodiode 11 is employed to convert the optical correlation peak to an electrical signal independent of its location in the output plane. This hybrid correlator arrangement exploits the asymmetry in the processing time requirements between the initial Fourier transform performed on the input image, which needs only to be performed once per input cycle, and that of the stored reference image. This may require several iterations to achieve recognition and so must be performed between 10 and 1000 times faster than the initial Fourier transform to maintain the required system response time.

Description

2363886 - Hybrid digital/optical system for pattern recognition This

invention relates to pattern recognition correlators, particularly to the use of SL Ms as non-linear thresholding devices.

There is a wide and ever expanding need for the rapid identification and fault detection of manufactured products ranging from components in the automotive industry to food and confectionery products In this way, quality assurance and hence production efficiency can be enhanced Currently devices to achieve this are commonly designed for a very 1 o specific task, often permitting the use of very simple measurement techniques employing, for example, single element photosensors Present generation, digitally based visual inspection systems capable of a sub-second response time must also be designed for a specific task often requiring a specialised hardware implementation.

More flexible software solutions implemented on general purpose vector processor boards are restricted to the use of relatively simple pattern recognition algorithms such as area and perimeter measurements on constrained overhead views of binary images.

There is, however, a requirement for unconstrained orientation independent object recognition using grey-level imagery Algorithms capable of this are too numerically intensive for implementation on even specialised hardware when a recognition cycle time of the order of 10 -100 msec is demanded.

Thus there is a need for a visual recognition system that is sufficiently programmable to allow its flexible use in a wide range of pattern recognition tasks which also demand a high speed of response The close integration of current state-of-the-art digital signal processing technology with advanced analogue optical processing techniques offers this possibility.

For many years now it has been appreciated that the inherent Fourier transforming property of a lens, together with the capability of performing a complex multiplication between a holographically recorded function and a coherent wavefront, permits the rapid I S em two dimensional correlation between a reference template and an input scene containing an unknown image However, for the recognition of a 3-D object unconstrained in scale, rotation and orientation, many reference template searches may be required for recognition The success of early attempts at the application of optical correlator systems to this problem was greatly hindered by the number of reference templates that could be stored on a thin holographic medium Multiplexing filter techniques have ameliorated this problem to some extent by reducing the sensitivity of a single filter to variations in object orientation However, to realise the full flexibility required, an effective interface must be implemented between the optical processor and a digital computer The optical processor is used to implement at very high speed the numerically intensive basic pattern recognition algorithms, the search strategy being under software control from the host processor.

The optical-to-digital interface requires the use of an electrically addressed spatial light modulator This may be used in the space domain, the reference templates being input to the optical system as images The frequency domain filter is then realised by the non- linear interaction of the input and reference Fourier transforms in a photorefractive material Alternatively, the reference function Fourier transform may be input to the optical system using a phase modulating spatial light modulator.

Recently there has been considerable interest in the multi-level phase modulating capability of commercially available devices designed originally for use as miniature TV displays These devices are limited in their practical use in rapid correlator systems since they rely on a nematic liquid crystal for operation which has a response time of several tens of milliseconds However, over the last few years there has been a considerable effort in SLM development Of particular importance has been the fabrication of high resolution silicon active backplanes addressing a rapid response ( 100 microsec) chiral smectic C ferroelectric liquid crystal layer, permitting a binary phase modulation Much research has concentrated on filter realisations implementable with the limited quantisation possible with widely available SL Ms and shown that effective pattern recognition filters can be realised with these SL Ms.

Thus according to the present invention there is provided a pattern recognition correlator comprising a means for digitally capturing an input image and performing a Fourier transform thereon, means for digitally combining the Fourier transform of the input image with a Fourier transform of a reference image and displaying the combined image on an SLM, means for illuminating the SLM and passing the resultant light through an optical system, the optical system being arranged to perform an optical Fourier transform and a means for detecting a correlation peak in an output plane characterised in that the means for detecting a correlation peak in the output plane comprises a non- linear thresholding SLM, the thresholding SLM being pixellated, each pixel incorporating a photodetector and circuitry and being adapted such that only light above a certain intensity is transmitted or reflected, by that pixel, to a detector.

The non-linear thresholding SLM may include a liquid crystal layer The invention described herein, shown in Figure 1, involves the hybridisation of a digital processor ( 1-4, 12) to an analogue optical processing system ( 5-10) in order to realise a programmable correlator system This arrangement exploits the asymmetry in the processing time requirements between the initial Fourier transform performed on the input image, which needs only to be performed once per input cycle, and that of the stored reference image This may require several iterations to achieve recognition and so must be performed between 10 and 100 times faster than the initial Fourier transform to maintain the required system response time For this reason it is proposed to implement this portion of the correlation operation using analogue optical processing techniques.

The input image data is captured by a video camera ( 1) which may, or may not, be of a standard CCIR format The video data is digitised and Fourier transformed at video rates, or faster, by the dedicated digital signal processing (DSP) hardware ( 2) This means of implementing the initial Fourier transform has the advantage that signal degradations due to an input SLM are avoided and the transformation to frequency space can be effected with the high precision associated with digital techniques It also allows the phase modulating SLM to be placed in a collimated beam ( 7) rather than at the focus of a highly convergent beam with consequent improvements in the accuracy with which the phase modulation can be accomplished.

Reference template data, stored in rapid access VRAM memory ( 3), is readout at a multi- kilohertz rate Since it is stored as Fourier transform conjugated phase data, each frame is added to the current input frame Fourier transform phase Since this requires only a 1 to 5 bit addition, this can be accomplished at very high speed ( 5-10 k Hz) with field programmable gate array (FPGA) based digital hardware ( 4) The resulting data is down- loaded to the electrically addressed high frame rate ( 5-10 k Hz) phase modulating SLM ( 5) A coherent optical wavefront from a laser diode ( 6) addresses the SLM This wavefront is initially plane in phase and may be of uniform amplitude distribution.

Alternatively, it may have a Gaussian or difference of Gaussian amplitude profile ( 7).

The phase profile of the beam is modified upon reflection from the SLM so as to record the residual phase difference between the input and reference input data displayed on the SLM In the case of a perfect match between the two, there is no phase difference and the beam is reflected from the SLM as a plane wave If the input and reference are totally dissimilar, a random phase distribution is generated indicating that there is no match between the input and reference scene The complex multiply between the input and reference image Fourier transforms is thus realised with the phase modulating electrically addressed spatial light modulator ( 5) which provides an efficient coupling between the electronic and optical sub-systems The optical sub-system can thus be made compact, requiring only a modest power laser diode as the coherent source, and also robust since the phase modulation technique is non-interferometric The accuracy with which the optical Fourier transform is performed will be ensured by careful Fourier transform lens design ( 9) which, however, will incorporate only standard lenses available from optical suppliers in order to minimise final production costs.

The phase modulating SLM should ideally be capable of 4 bits of phase modulation allowing near 100 % diffraction into the first diffraction order upon Fourier transformation by the lens system ( 9) However, 1 bit of phase modulation can be used effectively in pattern recognition applications With this level of quantisation, a strong zero order term and higher orders of diffraction are produced The output window, in the Fourier plane of lens system ( 9), must be confined to a half plane to ensure the desired first order correlation peak is the only structure detected Alternatively, and in preference, a phase plate ( 8) designed according to the methods described in lM Neil, E Paige "Breaking of inversion symmetry in 2-level, binary Fourier holograms" Proc Holographic Systems, Devices and Applications, Neuchatel, ( 1993)l may be used to disperse the zero and minus one diffraction order, allowing the output plane to be of the same resolution as the input and Fourier transform planes.

Read-out of the correlation response is accomplished with a non-linear thresholding SLM ( 10) to detect the correlation peak generated by the matched template The SLM may incorporate a photosensor and non-linear thresholding circuitry on each pixel This array may address a liquid crystal layer or deflect a micro-mirror array to realise a non linear reflectivity A correlation peak that is above a predetermined threshold will be reflected from the SLM onto an area photodiode In this way, the correlation peak may be detected, whatever its location in the output plane, at a speed matching the high frame rate of the phase modulating SLM The template generating the match is determined with a simple timing circuit that measures the time at which the electrical pulse was generated by the photodiode This corresponds to the interval during which a given template was displayed on the phase modulating SLM Once the peak is detected, the individual correlation plane can be read-out in full with a rapid frame-rate CCD array ( 12) Thus, with this output detector scheme, the full processing rate of the optical processor may be exploited.

The close integration for the proposed hybrid system allows the rapid 2-D correlation of the input scene with stored reference templates These are selected wider control from software running on the host machine which enables an efficient search strategy to be implemented This can be readily re-programmed for new pattern recognition tasks with appropriate image templates being downloaded from a mass storage device to the fast access VRAM memory.

The speed of response for orientation independent object recognition can be enhanced by reducing the total number of template searches required This is to be accomplished with the use of filter multiplexing methods, particularly synthetic discriminant function filters.

It could be arranged that these are synthesised automatically, allowing the system to be put into a "teach" mode, during which it is shown representative items of the component to be recognised from which it automatically generates the required multiplexed filters In appropriate applications this may be achieved directly from a CAT) model.

q/f O O O I,

Claims

1 A pattern recognition correlator comprising a means for digitally capturing an input image and performing a Fourier transform thereon, means for digitally combining the Fourier transform of the input image with a Fourier transform of a reference image and displaying the combined image on an SLM, means for illuminating the SLM and passing the resultant light through an optical system, the optical system being arranged to perform an optical Fourier transform and a means for detecting a correlation peak in an output plane characterised in that;

the means for detecting a correlation peak in the output plane comprises a non- linear thresholding SLM, the thresholding SLM being pixellated, each pixel incorporating a photodetector and circuitry and being adapted such that only light above a certain intensity is transmitted or reflected, by that pixel, to a detector.

2 A correlator as claimed in claim 1 wherein the non-linear thresholding SLM includes a liquid crystal layer.