RU2260847C2 - Method for detection, recognition and determining object coordinates and device for realization of said method - Google Patents

Abstract

FIELD: processing of images.

SUBSTANCE: method includes inputting signals, appropriate for alternating series of scene images, at which object can appear, first Fourier transformation of this signals series, recording Fourier spectrum signals, appropriate for one of images of series in form of synchronized Fourier filter, synchronized Fourier filtering of signals of current images with following second Fourier transformation of resulting signals of multiplication and change of response signals values, being responses to synchronized Fourier filtering, while at the beginning current images of scene are divided on fragments, and then synchronized Fourier filtering of signals of Fourier spectrums of each fragments are performed after which value of response signals is measured, being a result of synchronized Fourier filtering of current fragment of threshold processing of these signals is performed, while response signals being results of synchronized filtering of signals of fooling current images are used to determine coordinates of object.

EFFECT: higher efficiency, broader functional capabilities.

1 cl, 11 dwg

Description

The invention relates to image processing and can be applied in systems, for example, security, guidance, orientation of aircraft relative to the terrain, etc.

A known method of coordinated Fourier filtering for recognition and tracking of objects using a bank of coordinated Fourier filter standards, a priori made from images of predetermined objects [see, for example, 1, p. 443-455]. When using this method, the operations of the first Fourier transform consistent with the Fourier filtering (multiplying the Fourier spectrum of the current image of the object by the Fourier spectrum of the object registered as a matched filter) and the second Fourier transform of the multiplication result are sequentially performed on the image of the object.

As a result, a response signal is formed at the output, the amplitude of which is proportional to the degree of similarity of the input image with the image recorded as a matched filter-reference, and the coordinates are proportional to the coordinates of the object.

The disadvantage of this method is the inability to work in the inter-frame processing mode with the operational recording of a matched filter as reference information.

An analogue of the prototype is a method of interframe processing with operational recording of a consistent Fourier filter [see, for example, 2, p. 80-90].

In this method, the operations of the first Fourier transform matched by the Fourier filter (multiplying the Fourier spectrum of the current image by the Fourier spectrum of the reference image registered as a matched Fourier filter) and the second Fourier transform of the multiplication result are sequentially performed on the input current image. At the same time, one of the current images of the sequence of images of the phono-target environment, which is set as the reference, is operatively recorded as a matched Fourier filter.

As a result, a response signal is formed at the output with coordinates in the center of the frame, which is an autocorrelation function (see Fig. 1c).

This method allows you to detect arbitrary (and not pre-set, as in the analogue) objects on complex backgrounds for various signs, for example, by the factor of movement (see fig.1d).

The disadvantage of this method is the impossibility of recognition (the system only responds to changes in the background), image acquisition and determination of object parameters, since the reaction in the form of the amplitude of the response signal allows you to establish only the fact of the movement of a certain object.

A device is known, for example, a holographic correlator with long-term memory and a priori manufactured holographic matched Fourier filters (GSF) for a priori given objects [see, for example, 2, pp. 233-246; or 3, pp. 140-152], which provides the ability to recognize objects by their a priori specified image-standard, as well as track them.

Such a correlator contains a source of coherent illumination, a device for inputting and converting current information to a coherent carrier, for example, based on spatio-temporal light modulators (PVMS), lenses of the first and second Fourier transforms, long-term memory with a priori made GSF standards and position-sensitive photodetector, for example a CCD camera.

The response signal — the cross-correlation function — is generated as a result of the coordinated Fourier filtering, that is, multiplying the Fourier spectrum of the current image obtained with the lens of the first Fourier transform by the Fourier spectrum recorded on the GSF in long-term memory and the subsequent second Fourier -convert the result of multiplication. The amplitude of the response signal is determined by the similarity of the Fourier spectra of the current and recorded in the form of GSF images, and the coordinates of the object are proportional to the coordinates of the object in the field of view of the system.

The main disadvantage of such a correlator is the inability to quickly detect an unknown, a priori undefined object on an arbitrary background.

An analogue-prototype device for detecting, recognizing and determining the coordinates of an object is a holographic correlator with online recording of GSF [for example, 4, p. 53-55, see Appendix I].

This correlator includes a coherent illumination source, a device for input and conversion to a coherent carrier of current information, for example, based on spatio-temporal light modulators, a lens of the first Fourier transform and a position-sensitive photodetector, for example, a CCD camera. In addition, it includes a block of operational holographic memory, an optical divider of the backlight beam into a signal and reference beam and a mirror of the reference beam.

Such a correlator allows you to detect arbitrary objects on complex backgrounds, since the principles of inter-frame processing are used here.

However, this correlator does not allow receiving an image of a detected object and measuring its parameters. In addition, with its help it is impossible to determine the coordinates of the object, since the signal-response allows you to determine only the direction of movement [2, p. 80-85].

The objective of the invention is the ability to detect an object with the allocation of its image on a complex background, recognition and determination of the coordinates of the object.

The essence of the invention lies in the fact that in the known method, comprising inputting signals corresponding to a time sequence of images of a scene on which an object may appear, the first Fourier transform of this sequence of signals, registration of Fourier spectrum signals corresponding to one of the images of the sequence in the form of a consistent Fourier filter, consistent Fourier filtering of the current image signals by multiplying the signals of the Fourier spectra of the current images by the Fourier spectrum signals, zareg gated in the form of a matched Fourier filter followed by a second Fourier transform of the resulting multiplication signals and measuring the values of the response signals resulting from the matched Fourier filter, the current scene images are first divided into fragments, and then the matched Fourier filtering of the Fourier spectral signals of each fragment, after which they measure the value of the response signal resulting from the matched Fourier filtering of this fragment, and carry out threshold processing of these signals B-responses so that the signals of the current image, which correspond to fragments whose response signals are greater than or equal to the threshold signal, are blocked, and the signals of the current image corresponding to fragments whose response signals are smaller than the threshold signal are passed and recorded in in the form of a selected image of the object.

In this case, the signals of the selected image of the object are introduced and recorded as a reference and, after the first Fourier transform, registered as a matched Fourier filter, and the response signals resulting from the matched filtering of subsequent current images are used to determine the coordinates of the object.

In addition, matched Fourier filters are recorded for each image fragment specified as a reference, and image fragments are selected equal in area with a dimension of K pixels, where 2≤K≤N, where N is the number of pixels in the image.

The essence of the invention lies in the fact that in a device containing a sequentially optically coupled source of coherent illumination, an optical divider made in the form of a cube prism, a liquid crystal spatial light modulator, a lens of the first Fourier transform, an optical random access memory, a CCD camera, simultaneously through a cube - a prism, an optically coupled mirror of the reference beam, an optical random access memory and a CCD camera include an element of rotation of the plane of polarization of coherent radiation, for example p λ / 2 plate, optically coupled to a backlight and a cube-prism, a matrix of electrically controlled shutters, optically coupled to the lens of the first Fourier transform and a liquid crystal spatial light modulator, lens of the second Fourier transform, optically coupled to optical random access memory, and using a movable mirror installed in one position is optically connected to the optical input of the photodetector, and in another position, through an electrically controlled shutter, is optically connected to the optical input of the CCD amer, the photodetector is connected by an electrical output to the input of an electronic threshold device, the output of which is electrically connected to the input of an electrically controlled shutter, and the CCD camera is electrically connected to the first input of the switch, the output connected to the electrical input of the liquid crystal spatial light modulator, and the second input connected to the input devices.

Note: In contrast to the prototype, which uses a liquid crystal spatial light modulator operating \ on reflection \, the proposed scheme uses a similar modulator, but working \ on passage \. Therefore, it is turned on after the lens of the first Fourier transform. The schemes with the inclusion of the spatial modulator \ before \ and \ after \ the lens of the first Fourier transform are identical in functionality [4, p. 76].

The invention allows for combined intra-and inter-frame processing, which allows:

- recognition and tracking of a priori given objects;

- detection of unknown objects, selection of their images, registration as a reference image and transition to recognition and tracking mode, etc.

Figure 1 (a ÷ g) shows the results of computer simulation for the detection and extraction of images of the object of the proposed method. In this case, FIGS. 1a and 1b show the phono-target environment (FCO) at various positions of the object (\ airplane \). In Fig. 1c, the response signals of the fragments after dividing the FCO image into fragments (fragment size — 20 × 20 pixels) and performing coordinated Fourier filtering of the fragments. In Fig.1d - the result of threshold processing of the signal-response fragments and subsequent fragmentary filtering of images of the FCO.

Figure 2 (a ÷ e) shows the comparative results of modeling operations to determine the coordinates of the object (\ van \) by the prototype method (Fig. 2 a ÷ d) and the proposed method (Fig. E-f). On figa and 2B shows the reference frame and its corresponding signal response after matched filtering. On figb and 2g - the current frame and the corresponding result of the matched filtering - the reference and current frames. Fig.2g shows the detection of the object (signal response on the left), the direction of its movement and the distance traveled. The coordinates of the object (geometric center) do not correspond to the real ones. On fig.2d - the result of highlighting the image of the object of the proposed method, and on fig.2e - the result of the agreed Fourier filtering after recording the image of the object as a reference. The response signal corresponds to the coordinates of the geometric center of the object in figb.

Figure 3 shows a block diagram of a device made in accordance with the proposed method.

The device contains sequentially optically coupled coherent illumination source 1, plate λ / 2 (2), optical divider 3, made in the form of a cube prism, lens 4 of the first Fourier transform, matrix 5 of electrically controlled shutters, made, for example, in the form of a matrix of liquid crystal ( LCD) of shutters or LCD modulators operating in binary mode, an image input device 6, also made on the basis of a liquid crystal spatial light modulator (Appendix 2, 3), block 7 of optical random access memory, based on, for example, photothermoplastics [4, p. 54, see Appendix 1], a lens 8 of the second Fourier transform and a moving mirror 9, while at the same time the cube-prism 3 is sequentially optically connected to the polarizer 15, the mirror 16 of the reference beam, an optical RAM 7, the lens 8 of the second Fourier transform and a movable mirror 9, which is simultaneously optically coupled in one position with the optical input of the photodetector 10, and in another position through an electrically controlled shutter 12, made in the form of an LCD cell, with an optical input CCD camera 13, wherein the photodetector 10 is connected by an electric output to the input of an electronic threshold device 11 based on a comparator (Appendix 4), the output of which is electrically connected to the input of the electrically controlled shutter 12, and the CCD camera 13 is electrically connected to the first input of the switch 14 by the output connected to the electrical input of the liquid crystal spatial light modulator 6, and a second input connected to the input of the device.

The device operates as follows. A parallel light beam from a coherent illumination source 1 (collimating optics not shown) is launched through a λ / 2 plate (2) and the signal and reference beams are separated using a cube-prism 3. The signal beam is passed through the lens 4 of the 1st Fourier transform, the matrix 5 of the LCD shutters, the LCD modulator 6 and focus in the focal plane of the lens 4, where the input plane of the optical random access memory 7 is located.

When a video signal (for example, from a TV camera) is supplied to the input of the LCD modulator 6, an image of the observed scene is formed on it, and the coherent Fourier spectrum of this image is formed in the focal plane of the lens 4.

The reference beam after the cube-prism 3 passes through the polarizer 15 to the mirror 16 of the reference beam and, reflected from it, enters the input plane of the optical random access memory 7.

In operating mode, the input of the LCD modulator 6 through the block 14 of the video signal switches receives the current image from the TV camera.

To register a holographic matched Fourier filter (GSF), which is the reference information, in the input plane of the optical random access memory 7, the Fourier spectrum and the reference beam are coherently added and the resulting interference pattern recorded on the photosensitive layer (hologram filter) is formed. Before registering a GSF, the polarization of the backlight beam using the λ / 2 plate (2) is set so that it coincides with the polarization of the matrix LCD shutter 5 and polarizer 15, that is, that light passes through these devices without loss.

In the matched Fourier filtering mode (detection, selection and image acquisition of an object) using the λ / 2 plate (2), the polarization of the backlight beam is set so that the reference beam is turned off (the polarization of the beam and polarizer 15 are “crossed”), and the gates of the LC matrix -gates 5 "close" for the same reason.

When applying power pulses to the matrix of LCD shutters 5, the cells are opened one by one, highlighting individual fragments of the current images on the LCD modulator 6, which, using the lens 4, are converted to the "private" Fourier spectra of the corresponding image fragments. As a result of the coordinated Fourier filtering of these Fourier spectra, a response signal is formed, which after the mirror 9 is focused in the focal plane of the lens 8 and the intensity of which is proportional to the degree of similarity of a given fragment of the current image with the corresponding image fragment registered as GSF.

In addition to the response signal in the focal plane of the lens 8, a current image is also formed through the mirror 9 and received in the form of a video signal from the TV camera to the LCD modulator 6, but presented in a coherent form (the result of the double Fourier transform, “zero order”).

The optical response signal is supplied to a single-site receiver 10, the sensitive plane of which is installed in the focal plane of the lens 8, and is converted into an electrical signal, which, after threshold processing in the device 11, is converted into a binary signal (0 or 1) and is transmitted to the LCD shutter 12, which in the initial state is "closed".

The coherent current image after the mirror and the LCD shutter 12 is localized in the input plane of the CCD camera 13, since its photosensitive part is located in the focal plane of the lens 8.

If no changes occur in the observed scene, then the response signals of the fragments are converted using the comparator 11 into the control signals of the LCD shutter 12 equal to 0, and the shutter remains closed, and the image is not received on the CCD camera 13. If the background situation has changed (for example, a new object has appeared in the field of view), then the correlation of the reference (recorded on the GSF) and the current Fourier spectra is destroyed. The response signals for these fragments become lower than the threshold and signals are opened from the comparator 11 to the LCD shutter 12, which opens the shutter, which passes images of these fragments to the CCD camera 12, forming an image of an object selected from the background.

This image through the block 14 of the video signal switches can be supplied as a reference image to the input of the LCD modulator 6 to implement the recognition mode and tracking the object. To do this, using the mirror 9 and with the shutter 12 open, the optical response signal is transferred from the input plane of the photodetector 10 to the input plane of the CCD camera 13, and the current image, respectively, is output from it.

In addition, by entering reference images from long-term memory, a recognition mode with an a priori reference image is also realized.

Literature

1. COLLIER R. et al. Optical holography. Per. from English - M.: Mir, 1973, 686 p.

2. Aleshin B.S. and others. Optical image recognition. - GosNIIAS, 2000, 279 p.

3. VASILENKO G.I., TSIBULKIN L.M. Holographic recognition devices. - M .: Radio and communications, 1985, 312 p.

4. BARSKY A.G. et al. Holographic calculator of cross-correlation function with incoherent long-term memory. - M.: Optical Journal, 1993, No. 5, p. 53-55.

Claims (4)

1. A method for detecting, recognizing and determining the coordinates of an object, including inputting signals corresponding to a time sequence of images of a scene on which an object may appear, the first Fourier transform of this sequence of signals, registration of Fourier spectrum signals corresponding to one of the images in the form of a consistent Fourier filter, consistent Fourier filtering of the current image signals by multiplying the Fourier spectrum signals of the current images by the Fourier spectrum signals a, registered as a matched Fourier filter followed by a second Fourier transform of the resulting multiplication signals, and measuring the values of the response signals resulting from the matched Fourier filter, characterized in that the current scene images are first divided into fragments, and then the matched Fourier -filtering the signals of the Fourier spectra of each fragment, after which measure the magnitude of the response signal resulting from the matched Fourier filtering of this fragment, and carry out horn processing of these signals so that the signals of the current image, which correspond to fragments whose response signal values are greater than or equal to the threshold signal value, are blocked, and the current image signals corresponding to fragments whose response signal values are less than the threshold signal value are passed and recorded in the form of a selected image of the object, while the response signals resulting from the coordinated filtering of signals of subsequent current images are used to determine dividing the object coordinates. 2. The method according to claim 1, characterized in that the signals corresponding to the selected image are input and recorded as a reference and, after the first Fourier transform, are recorded as a matched Fourier filter. 3. The method according to claim 1 or 2, characterized in that the matched Fourier filters are recorded for each image fragment specified as a reference. 4. The method according to claim 1, characterized in that the image fragments are selected equal in area with a dimension of K pixels, where 2≤K≤N, and N is the number of pixels in the image.

Images