DE3635688A1 - Device and method for the optically coherent correlation of two images - Google Patents

Abstract

The invention relates to a device for the optically coherent correlation of two images, namely of an original image and of a reference image representing an object to be found, with the aid of at least two optically coherent Fourier transforms. To be able to carry out all correlations, even triple invariant and multi-spectral correlations, it is proposed to connect an optical Fourier system (22) for optical Fourier transformation with an electronic digital image and transformation system (23). The core of the image and transformation system (23) is a digital computer (31) followed by a digital transformation circuit (34). The computer can be supplied with the image data of the original image, the image data of the reference image, the image data of the image transformed in the optical Fourier system (22) and, if necessary, with further external image or correction data. The digital computer controls the digital transformation circuit, in which image coordinate transformations are carried out for achieving a position, scale and rotation invariance of the correlation. <IMAGE>

Description

The invention relates to a device and a Method for the coherent optical correlation of two images according to the preambles of claims 1 and 5.

Coherent-optical correlation means methods those performed with the help of several with coherent light Fourier transforms by optical means and an image comparison at the speed of light, d. H. Perform correlations and determine the locations where a searched object, specified by a reference image, is contained in an image to be examined.

So far there are two different coherent-optical correlation methods known.

One method uses adapted spatial filters, holographic comparison filters are used; see. such as DE-OS 32 34 123.

In the other methods, common transformation arrangements are used for both images to be compared. The principle of such a coherent-optical correlator is described in Topics in Applied Physics, Volume 23, 1978, page 12. A basic arrangement is shown in Fig. 1. The image g to be examined and the reference image h are arranged next to one another in an input plane P 1 , z. B. on a transparent film. Both images are irradiated together by parallel light λ 1 and Fourier transformed using a lens L 1 in a plane P 2 and there z. B. recorded on a film layer. To carry out the Fourier transformation, it is necessary that the planes P 1 and P 2 are each in the focal point in front of and behind the lens L 1 . If the film layer developed in the plane P 2, again placed in the plane P 2 and λ via a beam splitter BS with prallelem coherent light 2 irradiated, then formed by a second Fourier transform by a second lens L 2 in a further turn in the focal point behind the Lens L 2 arranged plane P 3 the correlation g * h and h * g of the images g and h . The autocorrelation signals g * g and h * h are also shown in the plane P 3 on the optical axis. The positions of the light points g * h and h * g each indicate the location of the object defined by the reference image h in the image g to be examined.

The principle arrangement described is of course not for to use a real-time correlation since film layers as image data carriers for this because of the necessary Film development time are not suitable.

However, image data carriers in the form of liquid crystal light modulators are available for real-time correlators. Such a light modulator is shown in Fig. 2 and labeled LM . This component can be described in a simplified manner as a plane-parallel disk with an input side I and an output side O. The input side consequently has a glass layer 1 , a transparent conductive electrode 2 and a photoelectric sensor layer 3 . This is followed by an optical barrier layer 4 and a dielectric mirror 5 , as a result of which the input side is completely optically isolated from the adjoining output side. The output side O adjoining the dielectric mirror 5 has a liquid crystal layer 6 , which is located between two insulating layers 7 . This is followed by a transparent conductive counterelectrode 8 , a glass layer and finally an anti-reflective coating 9 as a surface on the output side. Light L falling on the input side I of the light modulator LM modulates the photoelectric sensor layer 3 , but does not reach the output side O. An electrical flow between the two electrodes 2 and 8 transfers the exposure of the input side I to the liquid crystal layer 6 of the output side, thereby modulating the liquid crystal layer so that the local reflectivity of the output side irradiated with a projection light L _{p is} dependent on the local illuminance of the input side by the light L. This light modulator LM thus replaces the film layers mentioned above as image data carriers without any time-consuming intermediate steps, such as image development, being necessary.

FIG. 3 shows a known device 11 for coherent-optical correlation, in which the principle according to FIG. 1 is used. The original image g and the reference image h are output from a television camera 12 or a reference image memory 13 on a fiber optic display 14 (fiber optic cathode ray tube) next to one another and coupled directly via the fiber optics into the input side of a first light modulator LM 1 according to FIG. 2. The two images are Fourier transformed using coherent light λ 1 , a polarizing beam splitter 15 and a Fourier lens 16 . The intensity of the Fourier transformation is recorded at the input of a second light modulator LM 2 . With the aid of a further coherent light bundle λ 2 , the information on the output side of the light modulator LM 2 is read out with the aid of a further polarizing beam splitter 17 , and then Fourier transformed with a further Fourier lens 18 . The result in the correlation plane corresponding to P 3 in FIG. 1 is recorded by a second television camera 19 . A downstream correlation signal detector 20 determines the pixel coordinates of the correlation signals for locating the object sought.

This system is for fast optical correlation suitable in real time, but is only location-invariant. This means that the searched object in the examined picture is found, no matter where it is but only if the reference image is in the Original image with the same size and under the same Viewing angle is included. With a small change in scale already between 1 and 3% or even with small ones Rotation angles are no longer successful in locating objects. Accordingly, the system is neither scale nor rotation invariant.

This known system also has a significant one Overall length of a few meters and corresponding weight. In addition, the costs are very high and are currently included at least DM 400,000. This system can do so be expanded to scale and is rotationally invariant, d. H. a triple invariant System. This is done with the help of so-called Mellin Pixel coordinate transformations. This will however, the effort for the system is considerably greater.

The invention has for its object a device and a method for coherent-optical correlation of the Specify the type in question with which the usual  optical correlations, especially triple invariant optical correlations with minimal material expenditure with the greatest possible flexibility of the facility can be executed.

This object is according to the invention by the in the characterizing parts of claims 1 and 5 specified features solved.

The device according to the invention is therefore a hybrid arrangement comprising an optical Fourier transformation system and a digital image and transformation system. The optical Fourier transform system is essentially the same as the first Fourier transform device in FIG. 3, but instead of the second light modulator LM 2, a recording device, e.g. B. a high-resolution CCD camera or a high-resolution tube camera with a fiber optic imaging tube is used.

This optical system is coupled to an electronic one Data processing system, the core of which is fast digital computer and a digital transformation device are. The digital computer can be used as Input data Image data of the original image, e.g. B. about a television camera, the image data of the reference image, for. B. from a reference image memory, the image data of the Fourier optical system recording device and finally, if necessary, further external image or Correction data are supplied.

In the digital transformation facility, the Corresponding pixel coordinates of input images transformed a functional specification of the digital computer, with respect to a rotation of the pixel coordinates by a predetermined angle, a translation of the  Pixel coordinates around a given vector what corresponds to an image shift with respect to one Change in scale of the pixel coordinates, which one Corresponds to enlargement or reduction, and finally regarding a rotational invariance with the help of a Mellin transformation. There is also a 1: 1 transformation possible, d. H. the pixel coordinates are not changed.

The simple invariant correlation of two images is simplified so that the digital computer The original image and the reference image are entered at the same time will. In the transformation facility, this becomes one Image, e.g. B. the original image for rotation, Scale change and translation, the reference image transformed for translation. The two pictures are then displayed side by side on the fiber optic display output by the Fourier optical transformation system Fourier transformed and with the help of the television camera added. The image data of the Fourier transform Images are re-entered into the digital computer and usually without further pixel coordinate transformation or after a transformation regarding a change in scale again on the fiber optic display shown as picture. This picture is in the optical Fourier system again Fourier transformed, from recorded on the television camera and then to the Transfer digital computer. This image data is already the result of the coherent-optical correlation. This The result is evaluated in the digital computer. The correlation result is issued.

All considered correlations between two images, i. h too triple invariant correlations and multispectral Triple invariant correlations can be calculated using 5 Standard operations are performed.  

The main advantages of the specified in real time working hybrid correlator compared to the known Systems are flexible. It will be both the high computing speed of coherent optical processes used as well as the flexibility of digital electronic Computer with minimal optical effort Components.

Further advantages and refinements of the invention go from the subclaims.

The invention is explained in more detail in an exemplary embodiment with reference to FIG. 4, in which a device 21 for coherent-optical correlation is shown schematically.

The device 21 is composed of a device 22 for optical Fourier transformation, ie an optical Fourier system, and a digital image and transformation system 23 . The Fourier system 22 consists of a fiber optic display 24 , which is coupled to a spatial light modulator 25 according to FIG. 2, a polarizing beam splitter 26 , a light source 27 for coherent light L , a Fourier lens 28 and a high-resolution television camera 29 . A spatial frequency filter 30 can also be arranged directly in front of the target of the television camera 29 . This filter serves to attenuate or completely filter out certain frequency components of the Fourier-transformed image before the recording by the television camera 29 , for. B. the intensity of the spatial frequency zero. The spatial frequency filter 30 consists, for. B. from a film layer which has the desired blackening or opacity at the spatial frequency points to be filtered.

The digital image and transformation system 23 has a digital computer 31 with a correlation display 32 , an image memory 33 , a digital transformation circuit 34 and a correction stage 35 connected downstream thereof for correcting the image geometry with a correction memory 36 . The output of the correction stage 35 is connected to the input of the fiber optic display 24 .

On the input side, the digital computer 31 is connected to the output of the television camera 29 of the Fourier system 22 , a second television camera 37 , a reference image memory 38 and an additional input unit 39 for inputting external image or correction data.

The television camera 37 can be used to enter images to be examined in real time, that is to say original images, or else images of the object sought, and accordingly reference images. Image data of searched objects are stored and called up in the reference image memory.

The image data recorded by the digital computer are either passed on unchanged or after on the Image transformation performed after input image or after Implementation of digital image processing algorithms passed on.

On the output side, the digital computer has an image output 40 and a function output 41 . Image data is supplied via the image output 40 to the input of the image memory 33 , which in turn supplies the stored data to the transformation circuit 34 . The function output 41 of the digital computer 31 controls the transformation circuit 34 with regard to the image coordinate transformations to be carried out, which are explained below.

The specified image memory 33 can be omitted if image storage in the fiber optic display 24 or in the spatial light modulator 25 is possible. In this case, the image data of the digital computer 31 are entered directly into the transformation circuit 34 via the image output 40 , the corresponding transformation being indicated simultaneously via the function output 41 .

The digital computer 31 also determines the following correlation results after the correlation has been carried out:

• - searched object available? YES NO?
  - coordinates of the searched object
  - Size of the searched object
  - Rotation angle of the searched object

These correlation results are output on the correlation display 32 .

The digital transformation circuit 34 accepts the pixels of the image to be output on the fiber optic display 24 from the image memory 33 or - as indicated as a further possibility above - directly from the digital computer 31 and performs one or more of the following functions with the pixel coordinates in accordance with the function specification of the digital computer 31 :

F _O no change of pixel coordinates F _R rotation of the pixel coordinates by a predetermined angle F _G scale changes of the pixel coordinates, i. h enlargement or reduction F _T translation of the pixel coordinates, ie image shift on the fiber optic display by a predetermined vector F _M transformation of the coordinates (x, y) of a pixel into new coordinates (x ′, y ′) , where

These are Cartesian pixel coordinates (x ′, y ′) , obtained from the original Cartesian coordinates (x, y)

• 1) Transformation of the Cartesian coordinates (x, y) into polar coordinates ( ν , ϕ ) around the pole P (x ₀, y ₀) and
• 2) Mellin transformation of the coordinates ( ν , ϕ ) into the new coordinates ν ′ = ln ( ν ) ϕ ′ = ϕ
• 3) Interpretation of the polar coordinates ν ′, ϕ ′ as new Cartesian coordinates x ′ = ν ′ y ′ = ϕ ′

The new pixel coordinates (x ′, y ′) are passed on to correction stage 35 .

In the correction stage 35 , the entire area of the fiber optic display 24 is divided into n fields in a matrix. An average coordinate reproduction error Δ x _i , Δ y _i (1 ≦ i ≦ n) is stored for each of these fields. These coordinate reproduction errors arise from poor quality of the fiber optic display and the coupled fiber plates, e.g. B. image distortion, fiber disorder and the geometric distortion of the television camera 29 of the Fourier system, etc. The correction stage 35 determines from the coordinates x ', y' in which coordinate field i the pixel is located and then performs the error correction transformation

x ′ ′ = x ′ + Δ x _i y ′ ′ = y ′ + Δ y _i

to achieve a geometrically distortion-free reproduction on the fiber optic display by positioning the pixel on the coordinates (x ′ ′, y ′ ′) .

The correlation of two pictures with the one described Correlator is done using five standard operations OP1 to OP5 defined as follows are:

Standard operation OP1

• a) Input of an image X into the digital computer, from the first television camera 29 , the second television camera 37 , the reference memory 38 or the input unit 39 .
  • a1) Transformation of the image X into an output image X ' in the digital computer, this transformation can be a 1: 1 transformation or another transformation to improve the correlation properties.
• b) storing the output image X ' in the image memory 33rd

Standard operation OP2

• a) output of an image from the first television camera, the second television camera, the reference image memory, external input or image memory to the fiber optic display of the Fourier system after previous computer-controlled coordinate Transformation in the digital transformation circuit
• b) recording the coherent-optical Fourier transformation with the Fourier system television camera
• c) transfer of the Fourier-transformed image into the digital computer
  • c1) optionally a transformation corresponding to step a1) in OP1
• d) storing in the image memory

Standard operation OP3

• a) Simultaneous output of two images from the Television camera of the digital image and transformation system,  the reference image memory or the external input unit to the fiber optic display at a spatially predetermined distance after the previous one computer-controlled, possibly different Coordinate transformation of both images in the digital transformation circuit
• b) recording the common coherent-optical Fourier transformation with the television camera of the Fourier systems
• c) transfer of the Fourier-transformed image into the digital computer
  • c1) optionally a transformation corresponding to step a1) in OP1
• d) storing in the image memory

Standard operation OP4

• a) Output of an already Fourier-transformed image from the television camera of the Fourier system or the image memory or the external input unit of the digital image and transformation system the fiber optic display of the Fourier system previous computer-controlled coordinate transformation in the digital transformation circuit
• b) recording the coherent-optical Fourier transformation with the Fourier system television camera, so this recorded, now double Fourier-transformed image a transformation correlation represents  
• c) transferring the double Fourier transform Image to the digital computer
• d) Calculate the correlation results
• e) Output of the correlation results

Standard operation OP5 (for multispectral correlation)

• a) Input of two images X and Y with n × m pixels PX _{i, j} and PY _{i, j} (1 = i = n , 1 = j = m) from two image sources in the digital computer
• b) Calculating the pixel values PXY _{ÿ of} the image XY using the following formula PZ _ÿ = FT (PX _ÿ / PY _ÿ ) The value PZ _ÿ represents a measured value for the color of the pixel i, j in the case of partial images X, Y recorded in different spectral ranges, FT represents a color temperature conversion transformation which takes the color temperature conditions of the lighting into account when taking the image, and can also be the identical transformation. PXY _ÿ is defined as MIN : Value for the smallest pixel brightness
  MAX : Value for maximum pixel brightness
  F (PZ _ÿ , R ₁, R ₂, MIN, MAX) : transformation function for the brightness coding of the pixels PXY _ÿ . In the case of linear coding, e.g. B. be set, whereby
  R ₁, R ₂ are limit values for the selected and newly coded interval,
• c) storing the image XY in the image memory.

The following are correlation processes with monochrome Described images, namely the process steps of a triple invariant correlation and the process steps a simple invariant correlation:

A Preparatory work for the specified correlations

• 1. The standard operation OP1 is applied to the reference image, ie to the image of the object sought. This reference image can be fed from the television camera 37 , the reference image memory or the external input unit 39 to the digital computer 31 . If necessary, step a1) is also carried out in the digital computer in order to improve the correlation results.
• 2. The correction values Δ x _i and Δ y _i (1 ≦ i ≦ n) are stored in the correction memory 36 of the correction stage 35 .
• 3. The standard operation OP1 is on the original image applied, d. H. on the picture that you are looking for  Object to be examined.

B Process steps of the triple invariant correlation

• 1. The standard operation OP2 is applied to the original image, and a combination of the above-mentioned transformation functions F ₀ , F _R , F _G and F _T can be used in operation step a).
• 2. The standard operation OP2 is applied to the reference image, whereby an image coordinate transformation can be carried out with the functions F _O , F _R , F _G and F _T.
• 3. With the help of the standard operation OP3, the first step for the common Fourier transformation of the processed original image and the processed reference image now takes place, the transformation functions F _M and F _{T being} applied to both the processed original image and the processed reference image.
• 4. The Fourier-transformed image is treated with the standard operation OP4, wherein in operation step a) either no image coordinate transformation or a change in scale is carried out, ie the transformation functions F _O or F _G are used. A first result E1 of the correlation is then:
  • 1. The searched object is present in the searched image:
    Yes or no.
    If so:
  • 2. The rotation angle Δ ϕ by which the object sought in the original image is rotated relative to its reference image is displayed.
  • 3. In addition, the scale correction factor Δ m is displayed, by which the size of the searched object in the original image deviates from its reference image.
    If NO, that is to say the object sought is not present in the image sought, then the correlation is ended and, if necessary, a return is made to process step 1 .
• 5. The standard operation OP3 is applied again, but to the original image and the reference image, whereby for the original image the transformation functions F _R (rotation by the angle - Δ ϕ , F _G (enlargement / reduction by a factor 1 / Δ m) and F _T and the transformation function F can be applied to the reference image.
• 6. The standard operation OP4 is applied to the obtained Fourier transformation, the transformation F ₀ or F _G again being carried out in program step a).

The results E2 of this correlation are displayed, that is the location coordinates x, y of the object sought in the examined original image.

The triple invariant correlation is now complete

C process steps of the simple invariant correlation

• 1. The standard operation OP3 is applied to the original image and the reference image, the original image being treated with the transformation functions F _R , F _G and F _T and the reference image with the transformation function F _T ; again after the recording of the common coherent-optical Fourier transformation by the television camera in the digital computer, operation step c1 of the standard operation OP3 is possible, ie a 1: 1 transformation or a transformation suitable for improving the correlation.
• 2. The standard operation OP4 is applied to the Fourier-transformed image, the transformation function F _O or F _{G being applied} in operation step a), accordingly either no change or a change in scale.

As the result E of the correlation, the location coordinates x, y of the object sought and specified by the reference image are displayed in the examined original image if this object was present there.

D Multispectral correlation

The hybrid correlator can also be used to correlate with multispectral images using shape and color be carried out for object detection and location. At this method is based on the correlation of two monochrome Drawing files proceeded in different from each other Spectral ranges are included, both for the reference image of the searched object as well as for the original image to be examined, so that two There are drawing files. The intervals of the spectral ranges are chosen so that the best possible color characterizability the picture elements is possible.

The process steps for multispectral correlation are the following:  

• 1. Apply the standard operation OP5 to the two reference partial images, which are supplied by the television camera 37 , the reference image memory 38 or the external input unit 39 . R ₁ and R ₂ are the limit values of the color interval selected for the correlation in order to optimize the recognizability of the object sought. Either the identical transformation or the specified color temperature correction transformation F _{T is} carried out as the transformation.
• 2. Store the coordinate correction values Δ x _i , Δ y _i (1 = 1 = n) in the memory of the correction stage 35 .
• 3. Apply the standard operation OP5 to the partial original images provided by either the television camera 37 or the external input unit 39 . An identical or the color temperature correction transformation is carried out again.

E process steps of the multispectral and triple invariants correlation

• 1. Performing the preparatory work is described in point D.
• 2. Perform the triple invariant correlation accordingly section B above.

F process steps of the simply locally invariant multispectral correlation

• 1.Performing the preliminary work in accordance with Section D
• 2. Carrying out the process steps of the simple site invariants  Correlation according to section C.

From the foregoing it can be seen that all Correlations with just a few standard operations in Can be done in real time. The advantages of fast optical Fourier system and flexible electronic image and transformation system optimally used.

Claims (5)

1. Device for the coherent-optical correlation of two images, namely a first image to be examined (original image) and a second image representing a searched object (reference image) with the aid of at least second coherent-optical Fourier transformations, characterized by

• a) an optical Fourier system ( 22 ) consisting of a display device, preferably a fiber optic display ( 24 ) on which the two images can be displayed, and a spatial light modulator ( 25 ) assigned to the display, a light source ( 27 ) for emitting coherent light onto one Beam splitter ( 26 ) between the light modulator ( 25 ) and a Fourier lens ( 28 ) and a first recording unit ( 29 ) for recording the Fourier-transformed images;
• b) a digital image and transformation system ( 23 ) with
  • b1) a digital computer ( 31 ) which is connected on the input side to
    • a second recording device ( 37 ) which supplies image data of the original image and / or of the reference image to the digital computer ( 31 ),
    • a reference image memory ( 38 ), which supplies image data of the reference image to the digital computer ( 31 ) and accepts such data from the latter,
    • - The first recording device ( 29 ) which supplies image data of the Fourier-transformed images to the digital computer ( 31 ) and
    • - If necessary, an input unit ( 39 ) for the external input of further image or correction data into the digital computer
  • b2) at least one image memory ( 33 ) for storing processed image signals
  • b3) a digital transformation circuit ( 34 ), the image signals supplied by the digital computer ( 31 ) or the image memory ( 33 ) in accordance with a transformation predetermined by the digital computer ( 31 ), (F _O , F _R , F _G , F _T , F _M ) transformed with respect to the image coordinates (coordinate transformation) and delivers these transformed image signals to the display device ( 24, 25 ) of the Fourier optical system ( 22 );
• c) the measure that controlled by the digital computer ( 31 ), the images to be compared in a first cycle through the digital image and transformation system ( 23 ) and the Fourier optical system ( 22 ) Fourier transformed and then correlated in a second cycle, wherein the correlation results are evaluated in the digital computer ( 31 ) and displayed on a correlation display ( 32 ).

2. Device according to claim 1, characterized in that the two recording devices ( 29, 37 ) in the optical Fourier system ( 22 ) or in the digital image and transformation system ( 23 ) are high-resolution television cameras, preferably CCD cameras. 3. Device according to claim 2, characterized in that a spatial frequency filter ( 30 ) is provided directly in front of the target of the television camera ( 29 ) of the optical Fourier system ( 22 ). 4. Device according to one of the preceding claims, characterized in that the digital transformation circuit ( 34 ) is followed by a correction stage ( 35 ) for correcting the image geometry with a correction memory ( 36 ), and that the output of this correction stage ( 35 ) with the input of Display device ( 24 ) of the Fourier optical system ( 22 ) is connected. 5. Method for the coherent-optical correlation of two Images with a device according to one of the preceding Claims, characterized in that the Images captured electronically and as a sequence of image signals that are stored on the image signals electronic image coordinate transformations to achieve a geometric invariance of the correlation applied that the transformed images be optically Fourier transformed that the result the Fourier transform recorded electronically, saved and - if necessary after renewed Image coordinate transformation - again optically Fourier transformed and that the result of second optical Fourier transformation as a correlation result electronically recorded, saved and is evaluated.