GB2242590A

GB2242590A - Visual tracking apparatus

Info

Publication number: GB2242590A
Application number: GB8926357A
Authority: GB
Inventors: Colin Eley
Original assignee: GEC Marconi Ltd; Marconi Co Ltd
Current assignee: BAE Systems Electronics Ltd
Priority date: 1989-11-22
Filing date: 1989-11-22
Publication date: 1991-10-02
Also published as: GB8926357D0

Abstract

Apparatus for visually tracking a moving object comprises an imaging device (1), such as a thermal imager, for providing an electronic signal comprising input video fields representing sequential images of the field of view of the imaging device, processing means for processing the signal provided by the imaging device, and for providing an output image signal, and a display device (19) for displaying the images represented by the output image signal. The processing means comprises designating means (14) such as a joystick control for designating an object in the images displayed by the display device, tracking means (6, 7) responsive to the designating means to identify, and to determine the position of, the designated object within each successive input video field, and image stabilising means (8) responsive to the position determined by the tracking means to reposition the image in each successive output video field so that the position of the designated object in the output video field is substantially constant. <IMAGE>

Description

VISUAL TRACKING APPARATUS This invention relates to apparatus for visually tracking a moving object.

Known visual tracking systems making use of imaging devices such as video cameras and thermal imagers may incorporate target designators to indicate an object which can then be tracked by the system as it moves relative to the system, or as the system moves relative to the object, as in, for example, the case where a fixed object on the ground is being tracked from a moving aircraft. The tracking may be carried out by known techniques such as correlation tracking, in which a portion of the image is correlated with a reference portion, and centroid tracking, in which the centroid of the area of an object defined by e.g. intensity, particularly in the case of the thermal imager, is determined in each field or frame of the image, and this point is then tracked through successive fields or frames.

A problem occurring with visual tracking systems is that with fast-moving objects, or with fast movement of the system relative to the object, the image of the object moves between one field or frame and the next, causing image breakup, in an interlaced scanning system, and blurring, with consequent difficulties in accurate tracking and identification. The present invention provides a system in which the tracking data is used to reposition the image in each successive field or frame so that it occupies the same relative position, and is thus stabilised.

According to one aspect of the invention, apparatus for visually tracking a moving object comprises an imaging device for providing an electronic signal comprising input video fields representing sequential images of the field of view of the imaging device, processing means for processing the signal provided by the imaging device, and for providing an output image signal, and a display device for displaying the images represented by the output image signal, the processing means comprising: designating means for designating an object in the images displayed by the display device; tracking means responsive to the designating means to identify, and to determine the position of, the designated object within each successive input video field; and image stabilising means responsive to the position determined by the tracking means to reposition the image in each successive output video field so that the position of the designated object in the output video field is substantially constant.

Preferably, the processing means comprise cueing means operable upon the signal provided by the imaging device to identify any objects in the input video fields which meet predetermined criteria and to mix with the output image signal a signal providing in the output images displayed by the display device a visible marker indicating the position of each such object, and the designating means is arranged to permit a selected one of the marked objects to be designated. The cueing means may determine, for example, portions of the image which have an intensity above a threshold value, which can be determined according to the intensity range of the image, and which have a predetermined temporal stability, thus excluding random high intensity pixels in the image arising from non-image causes.

Suitably, the processing means comprise an analogue-to-digital converter for converting the signal to digital data, and three video field stores, each store being capable of storing data representing a complete video field, means for feeding successive input video fields sequentially into the three video field stores, and control means for transferring data from a first video field store to an output, the start address for reading out data from the first field store being selected to take account of any change in position in the designated object, and at the same time inputting video field data to a second video field store and performing tracking on the data in the third video field store.

The tracking means may comprise means arranged to perform a centroid tracking method, means arranged to perform a correlation tracking method, or both. It may be desirable to be able to select one or other of the types of tracking performed according to such factors as target size, background intensity and target intensity, as is known in prior art apparatus. The selection may be automatic.

The processing means may also comprise masking means arranged to mask out those parts of the output display image which alternate between image and non-image output with a frequency greater than a predetermined frequency. Thus, for example, a band of the displayed image around the periphery of the field of view may be masked or blanked to avoid the possibility of psychological problems associated with flicker resulting from the stabilisation process.

Further processing stages may be incorporated to process the signal output by the image stabilising means to provide one or more of electronic zoom, edge enhancement and contrast stretching.

Reference is made to the drawing, which is a block diagram of an apparatus according to a preferred embodiment of the invention. A standard video signal is input from an imaging device 1, for example a thermal imager of a type well-known in the art and therefore not requiring detailed description.

The imaging device 1 could equally well be a conventional video camera or other video device.

A part of the signal is fed to a timing generator 2 which extracts from the signal the odd/even field and frame synchronisation pulses, while a synchronisation separation circuit 3 extracts line sync. and field sync. pulse signals.

The resulting line sync. signal is also used to lock up a clock generator 4 which generates a 1OMHz pixel clock signal.

The pixel clock signal is fed, together with the remainder of the video signal, to an analogue-to-digital converter 5, which is used to digitise the video signal for the subsequent processing stages.

The digital picture signal is fed to a tracking stage, comprising a centroid tracker 6 and a correlation tracker 7, and to a stabilisation circuit 8. The centroid tracker and the correlation tracker 7 are operable alternatively, according to the nature of the scene viewed by the imaging device. For example, if the area around the target is difficult to define, say when viewing an aircraft against cloud, centroid tracking might be used, because this operates to determine the centroid of area of a block of pixels having an intensity above a threshold value, and the precise shape of the block is therefore not important. Thus, the centroid tracker incorporates a thresholding stage 6a and an address accumulator stage 6b from which the centroid of area is determined. Control signals from a window enable circuit 7b forming part of the correlation tracker 7 hereinafter described may be used to reduce the processing required by the system's C.P.U. 13.

But if the area of the target in the image is relatively small, or the scene is cluttered, more satisfactory results may be obtained by using correlation tracking, which correlates the pixel contents of a reference block in one field with the next field to detect the location of the target. The correlation tracker 7 includes a correlation stage 7a, and window enable and sequencer circuits 7b and 7c respectively serving to define the block used by the correlation stage 7a and to handle the passage of the reference block over the scene during correlation. The products of the correlation process are accumulated in an accumulator 7d, and sent to the C.P.U. 13. The C.P.U.-processed results are used to handle addressing of the field stores, hereinafter described.

The stabilisation circuit 8 comprises three input video field stores 9a, 9b and 9c, each having a data capacity sufficient to store one complete field. Each store 9 has an associated address generator 10 controlling the input and output flow of data. For outputting data, the address generators are supplied with feature co-ordinate information by the C.P.U. 13 via a common data bus 12 which also serves the trackers 6 and 7.

A target designator 14, for example a "joystick" providing an output corresponding to co-ordinates in the image, and connected via a serial interface, for example of RS232 standard, is also connected to the common data bus 12. Optionally, a cueing unit 20 may also be connected to the bus 12. This may be of known form, for example a thermal cueing unit permitting potential targets to be identified by their intensity and temporal characteristics, or possibly by their shape.

Reposition data output from the stabilisation circuit 8 is fed to a digital-to-analogue converter 15, either directly or via further signal processing stages comprising an electronic zoom stage 16, a temporal recursive filter 17 to remove random noise from the output signal, and an edge enhancement and contrast stretching stage 18 to permit enhancement of the image to permit, for example, the target to be more clearly distinguished from the background and to be identified more readily. The analogue output from the digital-to-analogue converter 15 is fed to a display device 19, for example a cathode ray tube monitor.

Claims

1. Apparatus for visually tracking a moving object, comprising an imaging device for providing an electronic signal comprising input video fields representing sequential images of the field of view of the imaging device, processing means for processing the signal provided by the imaging device, and for providing an output image signal, and a display device for displaying the images represented by the output image signal, the processing means comprising: designating means for designating an object in the images displayed by the display device; tracking means responsive to the designating means to identify, and to determine the position of, the designated object within each successive input video field; and image stabilising means responsive to the position determined by the tracking means to reposition the image in each successive output video field so that the position of the designated object in the output video field is substantially constant.

2. Apparatus according to Claim 1, wherein the processing means comprise cueing means operable upon the signal provided by the imaging device to identify any objects in the input video fields which meet predetermined criteria and to mix with the output image signal a signal providing in the output images displayed by the display device a visible marker indicating the position of each such object, and the designating means is arranged to permit a selected one of the marked objects to be designated.

3. Apparatus according to Claim 1 or 2, wherein the processing means comprise an analogue-to-digital converter for converting the signal to digital data and three video field stores, each store being capable of storing data representing a complete video field, means for feeding successive input video fields sequentially into the three video field stores, and control means for transferring data from a first video field store to an output, the start address for reading out data from the first field store being selected to take account of any change in position in the designated object, and at the same time inputting video field data to a second video field store and performing tracking on the data in the third video field store.

4. Apparatus according to Claim 1, 2, or 3, wherein the tracking means include means arranged to perform a centroid tracking method.

5. Apparatus according to any preceding claim, wherein the tracking means include means arranged to perform a correlation tracking method.

6. Apparatus according to any preceding claim, wherein the processing means also comprise masking means arranged to mask out those parts of the output display image which alternate between image and non-image output with a frequency greater than a predetermined frequency.

7. Apparatus according to any preceding claim, wherein the processing means also comprise further processing stages to process the signal output by the image stabilising means to provide one or more of electronic zoom, edge enhancement and contrast stretching.

8. Apparatus for visually tracking a moving object, substantially as described with reference to the drawing.