IL181820A

IL181820A - Method for detecting and tracking punctual targets in an optoelectronic surveillance system

Info

Publication number: IL181820A
Application number: IL181820A
Authority: IL
Original assignee: Thales Sa
Priority date: 2004-09-21
Filing date: 2007-03-08
Publication date: 2011-03-31
Also published as: ES2335517T3; FR2875626B1; WO2006032650A1; ATE448530T1; FR2875626A1; EP1792278B1; KR101207225B1; KR20070067095A; IL181820A0; JP2008513868A; DE602005017646D1; EP1792278A1; MY140799A; JP4900611B2

Abstract

The invention concerns a method in an optoelectronic image processing for monitoring and tracking punctual targets, which consists in providing a quantization of a level of complexity of the local background on which a target is detected in an image, to adapt the criteria for associating blips with tracks based on said complexity level. Said quantization is based on a segmentation of the image into zones of homogeneous radiometric characteristics. The integration of the complexity level of the local background on which a punctual target is detected enables the hypothetical tracks for which the blips detected are highly contrasted to be more rapidly validated. Further, it enables phantom blips to be detected, exclusively for monitoring the validated tracks, by locally lowering the blip detection threshold in a zone of the complex image in which the target should be located.

Description

181820 |7'Ji I 453490 τηκ Method for detecting and tracking punctual targets, in an optoelectronic surveillance system THALES C. 173635 METHOD FOR DETECTING AND TRACKING POINTLIKE TARGETS, IN AN OPTRONIC SURVEILLANCE SYSTEM The present invention relates to a method of detecting and tracking pointlike targets, in an optronic monitoring system, based on the observations of images provided by optronic sensors, for example infrared sensors. The invention relates more particularly to a method of associating singular points and of creating tracks that is particularly suited to high speed sectorial or panoramic optronic surveillance.

In an optronic monitoring system, the objective of the digital processing of the images is to extract the useful signal that may correspond to a threat in a scene observed and to perform spatial and temporal tracking of this possible threat. It thus comprises in particular a detection module, which makes it possible to extract singular points, called plots, from the optronic scene observed and a tracking module which makes it possible to inter-associate plots extracted, on spatial and temporal criteria: a set of plots observed at various instants and associated by the tracking module, is called a track. Each track represents a potential target (a real object).

An example of such an optronic monitoring system is illustrated in Figure 1. It is composed of optronic sensor(s) 1 making it possible to cover the field of the scene 2 to be observed. This (these) optronic sensor(s) delivers (deliver) at regular time intervals, via a data link, images Data,N of the observation scene 2 to a digital signal processing computer 3. This computer applies algorithmic functions to these images in real time.

In addition to the algorithms for processing the images proper (filtering, restoration of defective pixels, etc) which are customarily provided, the algorithmic functions carried out in real time by the computer 3 are divided into two main modules, a detection module 4 and a tracking module 5.

We are more especially interested in the context of the search for a threat whose surface area and distances of appearance are such that it ought to generate an optical spot on the optronic sensor of the size of a pixel. Within this framework, the algorithmic functions of the detection and tracking modules are defined for pointlike targets, that is to say ones corresponding to a pixel of the image. In the case where a target were to correspond to several pixels of the image, reduction functions (zoom air) are used locally, in the detection module, to revert to the case of a pointlike target. These image processing features that are well known to the person skilled in the art do not form the subject of the invention proper, and will therefore not be detailed here.

Within the specified framework of searching for pointlike targets, the processing performed by the computer 3 in real time may be organized as follows: - The detection module 4 receives optronic images of the scene observed, provided by the optronic sensor(s) 1 , generally prefiltered. On the basis of an optronic observation image DATA,N applied as input, it extracts from this image the set of plots that may correspond spatially and radiometrically to a threat, a plot corresponding to a pixel (pointlike target).

- The object of the tracking module 5 is to provide information on a potential threat detected, called a track, to a higher order system, such as a man/machine interface or a weapon system.

The creation of tracks consists in solving the following problem: which are the plots detected arising from successive observed images which have a common origin, that is to say one and the same moving source? Thus, in practice, a track will be defined as a set of plots associated over time. From this set of plots is deduced a kinematic behaviour of the target, which makes it possible in particular to predict its movement in the scene observed.

More particularly, the detection module 4 is customarily based on analysis of the image. More precisely, the detection module may for example undertake a local analysis of the noise observed in the image (comprising the inherent noise of the sensor and the noise of the scene proper) so as to extract the plots whose signal-to-noise ratio exceeds a certain threshold. With each plot thus extracted is associated a certain number of characteristics or attributes, as, for example, its angular position and its signal level (radiometric level).

The use of the most recent optronic sensors allows analysis of the spatial and/or spectral characteristics of the image (according to the type of information delivered by the sensor). Thus, it is possible to associate with each plot, information on the spatial and spectral distribution of the background on which the plot has been extracted. For this purpose there is provided an image decomposition module whose objective is to identify the various parts of the image that may exhibit homogeneity features, to group them together into homogeneous zones and to determine the characteristics of each zone, in particular, a mean noise level over the zone and a standard deviation. It is this information which will subsequently be used to calculate at each point (pixel) of the image the signal-to-noise ratio at this point, referred to the image portion corresponding to the homogeneous zone in which this point lies.

The module for calculating the signal-to-noise ratio at a given pixel uses the characteristics of the associated homogeneous zone Z, which characterizes the local background of the pixel considered: mean noise level m2i and standard deviation σΖι .

More precisely, the calculation of the signal-to-noise ratio SNR at a point of the image consists in doing the following calculation: S-m7 RSB= =i- where S is the level of the radiometric signal (that is to say the grey level) of the pixel considered, and mz and σΖι the mean and the standard deviation of the radiometric signals (typically, the infrared levels) over the homogeneous zone Zj associated with this pixel.

The signal-to-noise ratio SNR calculated at each pixel of the image observed is compared with a predetermined detection threshold thd, so as to discriminate the plots for which we have SNR>thd. The plots which satisfy this relation are the detected plots. These detected plots may correspond either to contrasted objects of the scene, or to pixels marred by noise.

In practice, the predetermined detection threshold thd is calculated in such a way as to guarantee a given pair of probabilities of detection and of false alarms for a pixel exhibiting a signal-to-noise ratio equal to that of a target at the range limit. This value depends in practice on the target that one wishes to detect and on the range required for this target.

At the output of the detection module, output data DET0UT are provided. These output data DET0UT of the detection module are applied as input to the tracking module.

It is this that is represented diagrammatically in Figure 2. On completion of the step of image decomposition and plots search, we obtain detected plots ph, pl2, p for each of which the signal-to-noise ratio SNR calculated over the associated homogeneous zone Z, is greater than the detection threshold thd.

At the termination of this step, the detection module 4 transmits data DETouT comprising the list of plots detected to the tracking module 5, with for each plot, its respective attributes, in particular: position of the plot in the image, radiometric level of the plot S, means of the radiometric signals over the homogeneous zone ¾ associated with the plot, standard deviation of the radiometric signals over this homogeneous zone Z,.

These attributes are used in particular in the tracking module, to associate a plot with a hypothesis track or a validated track, and to predict the position of these tracks in the following image observed (position, signal level, speed, acceleration) are the basis of the attributes of each of the plots of the hypothesis or validated track considered.

Dealing with the tracking module 5, two processes for associating plots are implemented: the associating of plots with the aim of validating a track hypothesis, and the associating of plots with the aim of upholding the validated tracks. These two processes are ones which implement different algorithms. On completion of these two association processes, if there remains a plot which was unable to be associated either with a track hypothesis or with a validated track, a new track hypothesis is created with this plot.

The associating of a plot arising from the current observed image with other plots arising from previous observations, inter-associated and which form a hypothesis or validated track, generally relies on the existence of a temporal and spatial coherence between all these plots. The techniques of association are well known to the person skilled in the art. They are generally based on kinematic or spatial criteria, through which a model of motion common to the plots is determined.

This model of motion makes it possible to make a prediction of the position of the target. It is around this position that one will search for the plot corresponding to the detection of the target at the new observation. As the model of motion is not perfect and may change if the target manoeuvres, a certain latitude is allowed to the association process to search for a plot around the predicted position of the target. This plot search zone constitutes the zone of association. Its size depends in practice on the manoeuvrability of the targets sought, on the frequency of the system, on the maximum speed of the targets, etc.

The association and validation process consists in practice in performing the following operations: -at the first image, for each plot detected (whose signal-to-noise ratio SNR is greater than or equal to the detection threshold thd), a hypothesis track is initialized with a zero speed, and filtering operations are applied to predict the position of the target assumed in the next image; - at the next image, for each hypothesis track created, we look in the zone of association around the position predicted at the previous step for whether a plot has been detected by the detection module 3. If yes, the filtering operations are applied to the two plots associated with this hypothesis track, so as to update the model of motion and predict the position of the target assumed in the next image etc.

According to the state of the art, a hypothesis track is validated after Q successive observation images, if a number P of plots have been associated out of these Q observations. The triplet (P, Q, thd) therefore constitutes the validation criterion for a hypothesis track.

In practice, the values of P and Q and the detection threshold thd are determined so as to ensure a predetermined probability pair (creation of tracks, false tracks), for example equal to (90%, 10%): there is thus a 90% chance of a track created on this criterion representing a real object (a potential threat) present in the optronic scene, and a 10% chance of being a false alarm, due to poor association of plots.

Thus, at the first observation, we detect a first plot (SNR>thd), a corresponding hypothesis track is created. This hypothesis track will not be validated until after the Qth observation, if at this Qth observation, it has been possible to associate a Pth plot with this hypothesis track. Each of the P plots associated with the hypothesis track satisfies SNR≥thd.

If the hypothesis track is validated [criterion (P, Q, thd) satisfied], the data associated with this validated track (position of the plots, kinematic characteristics, etc) are transmitted to the higher order system (DATA0UT)- In practice, for each hypothesis or validated track, the history of the track is logged in a computer table, for example over the last H observations (with H>Q). Each time a new plot is associated, the filtering operations are applied to the whole set of plots of the updated table, so as each time to reinitialize the prediction parameters (kinematic, etc).

In the monitoring systems reflecting the state of the art, the detection threshold thd used in the detection module to discriminate the plots of the image observed and the numbers P and Q used in the tracking module to validate a track hypothesis, are fixed system parameters, predefined as a function of the characteristics of the surveillance system and of the threats to be monitored. The values of these system parameters P, Q and thd are determined so that the plots detected, but which would correspond in reality to noise, cannot serve to validate a hypothesis track or to uphold a track already created. These values P, Q and thd therefore depend on the probability FA of false alarms at the output of the detection for a target at the range limit of the optronic sensor.

The probability of creating k false tracks per hour follows a Poisson law. It may be written: P(k) = e~m— , where m is the mean number of false tracks . k\ created per hour.

This mean number m of allowable false tracks per hour is in practice linked to the pair (P, Q) and also to the value FA of the rate of false alarms arising from the detection.

Specifically, the means number m of false tracks per hour is equal to 3600.f times the mean number Nm of false tracks created per image (per scan), f being the frequency of scanning of the zone covered by the surveillance system, i.e: m = 3600.f.Nm. (eq.1) The mean number Nm of false tracks per scan is equal to the number Npi of pixels serving to cover the scan zone, multiplied by the probability PfP of this pixel giving rise to a false track, i.e. Nm=NPiXPfP.

The number NPi is known, and is dependent on the sensor used. The probability PfP that a pixel gives rise to a false track is the probability that this pixel gives rise to a false alarm and that this pixel is associated subsequently with at least P false alarms out of Q observations of the surveillance sector.

To determine this probability PfP, it is necessary to consider the probability FA of false alarms at the output of the detection, which is expressed as follows: Q-1 FA = ^ p(i false alarms out of Q observations) (eq.2) i = P-l The probability of getting an additional false alarm that is to be associated with an existing track is the probability of the following event: "On opening a zone of association of size R*R around the extrapolated position of the track, at least one false alarm is found". This event is the event contrary to "On opening a zone of association around the extrapolated position of the track, no false alarm is found", this also being expressable as follows: "None of the pixels of the zone of association has given rise to a false alarm", the probability of which event is (1-FAR*R), where R*R represents the size of the zone of association. From which it follows that the probability of getting an additional false alarm to be associated with an existing track is equal to 1-(1-FAR*R).

The probability of the event "obtain i false alarms out of Q-1 observations" is the probability of obtaining i times "an additional false alarm to be associated with an existing track" and (Q-1-i) times "no false alarm in the zone of association".

This probability is therefore equal to c^_j (ι-(ΐ- ν4)Λ*Λ)'.(ι- ν4)Λ*Λ(β~1~') , where CQ. designates the number of combinations of i elements out of Q-1.

Hence it follows that Pfp= .

Starting from m=360O.f.NPj.Pfp, we therefore have: This relation between the mean number m of allowable false tracks per hour and the pair (P,Q) as well as to the value F A of the rate false alarms arising from the detection, therefore makes it possible, by fixing Q as the maximum number of allowable observations before taking a track creation decision (this corresponding to the delay time allowable for the processing), to find the parameters FA and P which will make it possible to guarantee a given rate of creation of false tracks. This formula shows for example that if, for a given rate of false tracks, we choose to decrease the value of P, the constraints on the association of alarms for the creation of tracks decreases. Hence it will be necessary to decrease the value of the false alarm rate FA so as to decrease the false alarm rate at the output of detection. On the contrary, if we increase P, we will be able to allow ourselves to increase the false alarm rate, since we have become more demanding regarding the number of alarms necessary to create tracks.

A problem encountered in the tracking processing stage resides in the complexity of the optronic scene's background in which the target is moving. For example, in the case of a validated hypothesis track, if the target enters a complex zone, it might not be detected in the corresponding image. Specifically, the radiometric signal of a target is a priori constant. The signal-to-noise ratio of a pointlike target in a complex zone therefore tends to decrease. If it becomes lower than the detection threshold thd, the target will no longer be detected on a complex background such as this. However, if there is no detection of the pointlike target in the current image, the kinematic attributes (filtering operations) will be modified erroneously and the prediction will itself be erroneous. There is a risk of losing target lock-on. This problem of losing lock-on may also arise if the target goes to the range limit of the optronic sensor: its radiometric signal becomes weaker. The signal-to-noise ratio also decreases.

An object of the invention is thus to render the association process more robust so as to avoid the problems of loss of lock-on.

Another problem encountered with the tracking module resides in the time necessary to validate the hypothesis tracks. Specifically, it is desirable to validate a track as fast as possible, while still satisfying the requirements of false alarm probability, which must remain low. Now, for a predetermined detection threshold thd, if we consider a simple, uniform background, the false alarm probability is lower than in the case of a complex background. Now, the tracking module of the state of the art applies a fixed, predefined criterion which requires at least Q observations to be made, and P plots to have been associated out of these Q observations in order to validate a track.

If this number of observations can be reduced at least in certain favourable observation cases, then the temporal deviation between detection (of the first plot) and track setup is advantageously reduced: the reactivity of the sensor is thus improved. In the case where the target approaches the optronic sensor, this is equivalent to an improvement in the range of the monitoring system.

An idea on which the invention is based is to render the tracking adaptive.

According to the invention, at each new plot detected, we determine a comparison threshold or a plurality of comparison thresholds associated with this plot, that are greater than or equal to the detection threshold and we apply a criterion of validation of the associated hypothesis track, dependent on the comparison thresholds associated with the plots of the hypothesis track.

At each new plot associated with a track, we look to see whether a track validation criterion is satisfied in relation to the threshold or to the plurality of thresholds associated with each plot of the track. Thus, we go from a fixed criterion (P, Q, thd) to an adaptive criterion, in which, through the use of adapted thresholds, the values P and Q may vary according to the conditions of observation.

Thus, as characterized, the invention relates to a method of optronic processing of images, to ensure the detection of pointlike targets and the following of the track of each target detected in the said images, comprising a step of comparison at each pixel of an observed image, to compare the signal-to-noise ratio with a detection threshold (thd) and a step of creating hypothesis tracks, each hypothesis track comprising at least one plot detected, characterized in that for each plot detected, it comprises a step of determining a set of comparison thresholds associated with the said plot, the said set comprising one or a plurality of comparison thresholds, the said comparison thresholds being greater than or equal to the said detection threshold (thd), and a step of applying a criterion for validating the hypothesis track associated with the said plot, the said criterion of validation being dependent on the comparison thresholds of each of the plots of the hypothesis track.

According to an aspect of the invention, the said comparison thresholds are determined as a function of the complexity of the local background of the plot considered.

The introduction of an adaptive validation criterion as a function of the local background makes it possible to adapt the processing algorithm to the detection conditions encountered in the various zones of the images provided by the optronic surveillance sensors.

According to an aspect of the invention, the said comparison thresholds associated with the plot detected are determined as a function of the number of plots already associated with the hypothesis track considered.

Performance is further improved by providing for the adaptation of the criterion as a function of the number, to which it is applied, of plots of the track. A comparison threshold applicable as a function of the number i of plots, to which the validation criterion is applied, of the hypothesis track is thus determined for each plot.

If the hypothesis track comprises p plots, the criterion of validation consists in verifying whether there exist i plots, i=1 to p, such that for each plot out of the i plots, the signal-to-noise ratio is greater than the comparison threshold applicable for i plots.

In particular, if a detected plot has a very large SNR, greater than the comparison detection threshold for a plot that has been associated with it according to the invention, then the track will be validated by applying the validation criterion to this plot alone, independently of the signal-to-noise ratios of the other plots. The reactivity of a surveillance system implementing a method according to the invention is thus substantially improved.

Preferably, the complexity of the local background on which a plot is detected is modelled, and the sets of comparison thresholds are precalculated for each model. In this way, the implementation of a method according to the invention is facilitated. Furthermore, the precalculation of the detection sets to be applied allows a processing of a tracking algorithm according to the invention that is faster, adapted to high image rates.

According to another embodiment of the invention which may be combined with the previous one, an adaptive criterion is used in the process of associating plots for the following (upholding) of the validated tracks. According to this embodiment, at each new observation, this adaptive criterion will be used when the detection module has provided no plot in the predicted zone of association for the validated track considered, for this new observation. This criterion allows the detection of phantom plots, by lowering of the detection threshold in the association zone considered. These phantom plots will advantageously make it possible not to lose lock-on of the target in case of crossing of a complex image zone. Here, as a function of the characteristics of the local background on which the future position of the target corresponding to the validated track considered is predicted, the detection module is controlled with a detection threshold that is less than the nominal detection threshold thd, so as to obtain a probability of detection in this zone of close to 100%. This aspect of the invention relies on the fact that the track being validated, the false alarm probability has been decreased significantly.

Other advantages and characteristics of the invention will become more clearly apparent on reading the description which follows, offered by way of nonlimiting indication of the invention and with reference to the appended drawings, in which: -Figure 1 already described is a block diagram of an optronic surveillance system; -Figure 2 already described is a block diagram of a detection module 4 according to the state of the art regarding image data DATA,N applied as input, -Figure 3 illustrates the process of a tracking module 5 using adaptive criteria according to the invention, -Figure 4 depicts the adaptive processes according to the invention, in the detection and tracking modules; -Figure 5 illustrates a set of detection thresholds defined as a function of the level of complexity of the local background according to the invention; -Figures 6 and 7 are representations of the curves of detection probability density, in relation to the sensor noise and a target signal, according to the complexity of the local background over which the target is deploying.

According to the invention, each time a plot is detected in a current observation, a comparison threshold or plurality of comparison thresholds applicable as track validation criterion is (are) defined, making it possible at one and the same time to guarantee a target detection and false alarm rate, and to rapidly validate a track under favourable conditions, when the target is moving over a simple or fairly noncomplex background. Thus, the performance of the system in terms of reactivity is improved while still satisfying the system reliability criterion (detection rate, false alarm rate).

More particularly, according to an aspect of the invention, these comparison thresholds are defined, by calculation, as a function of the radiometric characteristics of the local background on which the plot has been detected, and curves of detection and false alarm probability for this plot.

Returning to Figure 2, provision is thus made for the detection module to provide at each new observation: -the set of homogeneous zones Z, of the image, with the attributes of each zone, typically at least its position, its size, the mean noise level mZi and the standard deviation signal σ2. of the radiometric levels over the zone Z,; -the set of plots detected with the attributes of each plot (position, radiometric level S, mean noise level m, and standard deviation signal σ,. of the radiometric levels over the zone ¾); Thus, each time a plot has been detected in a new observation, it is possible to determine a set of comparison thresholds (one threshold or a plurality of thresholds), as a function of the radiometric characteristics of the homogeneous zone over which the plot has been detected, and apply a validation criterion for the associated hypothesis track PH,, dependent on the comparison thresholds associated with each of the plots of this track. Typically, these thresholds are determined by utilizing equation (eq.3). For each track for which the number Q of observations and P of current detections are known, we calculate the false alarm rate necessary to ensure the required rate of false tracks. From this we deduce the threshold thd making it possible to obtain -this false alarm rate. If the detections that served to create the track have exceeded this threshold, then the decision to create the track is taken immediately.

More precisely the adaptive criterion according to the invention consists in looking to see whether there exist i plots, out of the p plots of the track, for which their signal-to-noise ratio is greater than or equal to a defined comparison threshold for i plots. Thus, the hypothesis track may be validated after 1 , 2, ...p plots, or not validated, according to the conditions of observation.

This criterion is adaptive and dynamic, since it is re-evaluated each time a new plot is associated with the hypothesis track: - as a function of the homogeneous zone on which it has been detected, the target moving it, from one observation to the next, we may not be on the same type of background at all; hence the complexity may vary. - as a function of the number q of observations performed, counted up. The thresholds are not as selective if we are at the 10th observation as if we are at the first, this being related to the statistical detection probability law. - as a function of the number p of plots associated with the hypothesis track considered: for a predetermined number q of observations, the thresholds are less selective depending on whether we consider 4 plots or a single plot, this being related to the statistical detection probability law.

This therefore makes it possible to increase the reactivity on the alarms whose signal-to-noise ratio is high and which therefore have more probability of exceeding thresholds thd associated with faster track setup.

We thus have an adaptive and dynamic method of validation, which allows fast track setup (i.e., hypothesis track validation) in the case of plots with large SNR, while still allowing track setup in the case of plots with small SNR. Stated otherwise, the track setup lag is dependent on the contrast of the pixel. The reactivity of the system is therefore improved. In particular, the method described allows fast track setup of objects that are closer to the sensor than the limit range.

This process for validating a hypothesis track according to the invention is triggered at each new observation, if in this observation, a new plot has been associated with this hypothesis track. This method necessitates the determination of the comparison thresholds associated with each plot detected in the image. These thresholds are determined from the radiometric characteristics associated with the homogeneous zone on which the plot has been detected, this being so for each plot and at each new image. They are also determined as a function of the number of plots that are considered for verifying the criterion.

In practice the determination of the radiometric characteristics of the local background to define the comparison thresholds applicable for each plot detected may be facilitated to a greater or lesser extent according to the mode of decomposition of the image into homogeneous zones.

A known method of decomposing an image into homogeneous zones uses as criterion for defining a homogeneous zone, one and the same radiometric level (grey level) at each pixel of the zone. According to this procedure, a sort of radiography of the image observed is compiled, as represented diagrammatically in the table below: We thus have a partition of the image into homogeneous zones, of like signal level (grey level), on which zones calculations of mean noise and standard deviation are carried out.

Next, for each plot detected, we look to see which homogeneous zone of this radiography this plot lies in, so as to calculate a corresponding signal-to-noise ratio.

According to the invention, we then define a threshold or a plurality of thresholds as a function of the local background defined by the homogeneous zone on which the plot has been detected. However, such a procedure requires additional calculation times to be implemented for each plot detected.

According to a preferred embodiment, the mode of decomposition of the image into homogeneous zones allows a modelling of the complexity of the local background. In this preferred embodiment, use is made of a criterion of larger uniform neighbourhood around each pixel. Thus, in this procedure, the homogeneous zones in the image are determined starting from the pixel.

A uniformity criterion may be defined on the basis of a radiometric distribution function. For example, it may be defined on the basis of the standard deviation: a uniform zone is then a zone over which the standard deviation of the radiometric level is constant, equal to the noise level of the optronic sensor. Concepts of level ramps may also be integrated into the definition.

According to such a decomposition into homogeneous zones, one then has to determine, at each pixel of the image, the homogeneous zone Z, which is the largest uniform neighbourhood zone around this pixel: this is the largest zone around the pixel, excluding the pixel itself, over which the uniformity criterion is satisfied. The algorithmic processing associated with this procedure for decomposing an image into homogeneous zones is well known to the person skilled in the art and will not be detailed.

In practice, it is possible to model this decomposition using sizes TFK of windows lying between a maximum size TFi and a minimum size TFt. The decomposition then consists in looking to see, at each pixel, which size of window meets the criterion of largest uniform neighbourhood. More precisely, the image decomposition procedure consists at each pixel in opening the window of maximum size (or optimum size) TFI around the pixel, and in applying thereto the calculations of uniformity criterion over the image zone in the window. If the uniformity criterion is satisfied in this zone, then it is the largest uniform local neighbourhood zone within the sense specified herein above. Otherwise, the size of the window around the pixel is reduced and so on and so forth, down to the smallest size of window TFt defined.

At the completion of this decomposition phase, each pixel is thus associated with a window size TFk. The homogeneous zone Z, of each pixel is thus defined by a window size TF^ and the calculation of the signal-to-noise ratio SNR at a given pixel of the image is performed in relation to this zone.

This decomposition based on a uniform radiometric distribution criterion makes it possible to match each size TFk of window, k varying from 1 to t, with a criterion k of complexity of local background. If we use t sizes of window, we thus have a complexity criterion varying from 1 to t. By convention, the largest size of window is denoted TFI and the smallest size of window TFt. With this convention, obtaining a window of maximum size TF-I , signifies that the pixel considered is situated in a simple region of the scene observed and obtaining a window of minimum size TFt, signifies that the pixel considered is situated in a complex region of the scene observed. In such a region, the probability of detecting plots is much lower than in a simple region.

At the conclusion of this splitting phase, each pixel is thus associated with a size TFk of window in the image observed. This size TFk of window defines the homogeneous zone Z, (Z, is equal to the window of size TFk centred on the pixel considered). It is understood that we will find the largest homogeneous zones in the "simple" regions of the scene observed corresponding to fields, forests, etc., and the smallest homogeneous zones, in the "complex" regions: roads, verges, etc.

The complexity of the background of the image has thus been modelled by sizes of window.

According to a preferred embodiment of the invention, this modelling makes it possible to predefine the comparison thresholds associated with the plots detected, as a function of the size of the homogeneous zone associated with each plot.

In a preferred implementation of the invention, an optronic monitoring system which uses a splitting of the image into homogeneous zones is used, based on a uniform radiometric distribution criterion, the said criterion being applied at each pixel of the image observed so as to determine the largest size of window applied around the said pixel and on which the said criterion is satisfied.

Within this framework, the level k of complexity of the background associated with a detected plot is given by the size of the window associated with this plot, in accordance with this splitting. It is thus possible to quantize the complexity of a local background over k levels, corresponding to the k sizes of window of the uniform distribution criterion.

Each window representing a uniform background, the probability density of detection over this window is related solely to the sensor noise and to the target signal.

The use of a model of "normalized" windows to represent the complexity of the background allows a priori knowledge of the curves of detection probability and false alarm density, for each size of window. Specifically, when we are on a uniform background, the detection probability density is a Gaussian whose parameters are related solely to the sensor noise and to the target signal. It can thus be determined statistically. This is what is used to predetermine a set Ek of comparison thresholds applicable for each quantized level k of complexity of the background.

The implementation of the adaptive criterion then consists in simply indexing the set of comparison thresholds that is applicable to the new detected plot, on the size TFK or on the criterion k of complexity of the associated homogeneous zone, then in applying the validation criteria to the corresponding hypothesis track by selecting the sets of comparison thresholds thus indexed of the plots of the hypothesis track.

Thus, it is possible to decide with regard to the validation of a track each time a new plot is detected, and to do so while ensuring the reliability of the system (detection rate and false alarm rate).

Figure 5 gives a representation of a set Ek of comparison thresholds determined as a function of the level k of complexity of the local background according to the invention.

As may be seen clearly in this Figure 5, each set Ek comprises Px.0 thresholds thu>k , with i = 1 to P and j = 1 to Q with 1 < j, and where P represents the maximum number of associated plots and Q the maximum number of observations. In practice we have P=Q.

Each set Ek is structured in the form of a table with two entries, which defines, for a number q of current observations (that is to say ones already performed, including the current observation), a subset Tq,k of q comparison thresholds thj,q,k, i=1 to q, with thi,q,k>th2,q,k>th3,qik,....>thi,q,k>...>thqiq,k (for a given number q of observations, the thresholds are less selective if we consider 4 plots than if we consider a single plot, this being related to the statistical detection probability law).

We may thus write: >th2 ,2,k · -Tj,k >■ · -thjj,k }>· · J g.it v -thQ ,Q,k }} , this corresponding in the array of Figure 5 to the columns.

In each set Ek, we also have (on each row, in the array of Figure 5): thPii,k≤ thPi2,k....< thp,Q,k, (for a considered number p of plots, the thresholds are not as selective if we are at the 10th observation as if we are at the first, related to the statistical detection probability law).

In this representation: - the level k of complexity of the zone on which a plot has been detected is used as pointer to the set Ek of thresholds to be used for this plot in the validation process; - the number q of observations performed since the first associated plot of the hypothesis track considered, is used as pointer for selecting the subset of thresholds Tq,k in the set Ek and - the number p of plots associated with the hypothesis track is used as pointer for selecting the first p thresholds of this subset, i.e.: .

In reality these thresholds form a validation multicriterion, since they are all to be considered in succession, beginning with the first, thi,q,k. All the thresholds are thus reviewed in succession until the last, unless a value of i thi,q,k for its associated value of k.

In an example given by way of illustration of the invention, the validation process implemented by the tracking module may be as follows, employing the following notations: we are at the q-th observation for the hypothesis track PH (that is to say the q-th observation since a first plot has been associated with the hypothesis track, i.e., since the hypothesis track was created) and during this q-th observation, a p-th plot is associated with this hypothesis track. We have p≤q .

An attribute of each of the p plots associated with the hypothesis track PH,, is the level k of complexity of the homogeneous zone on which this plot has been detected.

According to the invention, the tracking module consists: Φ-From the sets Ek of applicable comparison thresholds, indexed by the complexity levels k associated with the p plots of the hypothesis track considered, in selecting the subsets indexed by the number q of current observations, that is to say the subsets ©-In initializing a comparison loop with i=1 , this loop consisting in the following steps: -a). Selection of the thresholds thj,qik, of index i, in the selected subsets Tq,k and application of the following criterion: ■ If there exist i plots out of the p plots of the hypothesis track PH such that each plot out of these i plots satisfies the following inequality: SNR > thj q k for the value of the level of complexity k associated with this plot.

• Then the track is validated - end of the loop. ■ if not, step b). -b). i=i+1.

■ While i≤p, return to step a).

■ Else, step c). -c) . if i>p, Then the track is not validated - end of loop.

We assume for example that q=4. If we assume that the first two plots of the hypothesis track have an associated attribute k equal to 2, and that the other two plots of the hypothesis track have an associated attribute k equal to 3, we therefore select from the set E2, the subset T ,2 which comprises the following thresholds (Figure 5): thi,4,2> th2,4,2> th3,4,2> th4,4,2- We also select the subset T4, 3 which comprises the following thresholds (Figure 5): thi,4,3> th2,4,3> Vri3A,2> th ,4,3.

In an example, we have thi,4,2 *=8.3> th2,4,2-5.5 > th3,4,2 *4.4> th4,4,2-3.7. thi,4i3«9> th2l4,3 *6 > th3,4,3«4.6 > th4,4,3 *3.8.

The process for validating the track q=4-th observation is to look to see: -whether a plot out of the p associated plots satisfies SNR2, whether two plots out of the p associated plots each satisfy SNR3, whether three plots out of the p associated plots each satisfy SNR4, whether four plots out of the p associated plots each satisfy SNR Else, the track is not validated.

For example, in the case of the hypothesis track described, if the second plot (k=2) has an SNR of greater than 5.5 and if the third plot (k=3) has an SNR of greater than 6, both of them fulfil the conditions for a creation of track at P=2 on their respective complexity levels k, and hence the track can be created.

It is understood that the track establishment is obtained much faster than in the case of the fixed criterion (P,Q,thd) customarily used in the state of the art.

In a refinement of the invention, if during an observation, no plot is associated with the hypothesis track PH, we compare the pair (P,Q) associated with the track (that is to say the number p of observations performed since the first associated plot, and the number q of associated plots). If this pair is close to the maximum values (P,Q) compiled in the validation process, then we look at the complexity level associated with the prediction zone of the hypothesis track. If this prediction leads to a complex background, then the validation of the hypothesis track is decided. This will typically be the case if we have a complexity level k in the prediction zone equal to t or t-1 (corresponding to the minimum window size), according to the modelling convention adopted. Thus, stated otherwise, in the case where no plot is detected in the current image observed for a given hypothesis track, we make provision for a step of comparing the pair (p, q) (of the number p of plots of a hypothesis track and of a number q of observations performed since the first plot associated with the said hypothesis track, with an optimum pair (P,Q), so as to validate the said hypothesis track as a function of the difference between these two pairs (p,q) and (P,Q) and as a function of the complexity level k associated with the plot position prediction zone in the current image.

To illustrate this suggestion, if we have (P,Q)=(10,10), such a refinement will be implementable with (p,q)=(8,9) for example.

According to another aspect of the invention, the complexity level k is again used to render the process for associating plots with a validated track adaptive. Here this involves ensuring the following of the track, that is to say the locking-on of the target.

A given target has a radiometric signal which remains constant to the first order. Thus, if the trajectory of this target passes over simple and complex backgrounds, the signal-to-noise ratio of this target will vary strongly. In complex zones, it may be too weak to be detected by the detection module.

It is recalled that the detection module applies a fixed detection threshold thd. In practice this fixed threshold thd corresponds to the minimum threshold of the tables Ek, which will be given by the threshold thQiQ Λ of table Ei (corresponding to a simple background).

Figure 6 represents the detection probability densities dp(x), for a target and for the noise, as a function of the signal-to-noise ratio, for a given size window.

For constant size window, the detection probability is the area under the curve dp(x) of the target, delimited on the left by the vertical line corresponding to the detection threshold thd of the detection module, which may be written p(det) = . thd is chosen so that p(det)=90%.

The false alarm probability is given by the area delimited on the left by the detection threshold thd and the portion of probability density curve for the noise on the right.

Now, if the target passes over a more complex background, the size of the window varies. This is manifested as a target detection probability density curve which shifts leftwards as represented dashed in Figure 7. To maintain the same detection probability, it is then necessary to decrease the detection threshold. In the area calculation, this amounts to adding the portion "a" of shaded expanse under the target detection probability density curve, that is to say a = J dp(x)t arg et . This threshold is denoted thda in Figure 7. It is chosen so that the target detection probability remains 90%. It may be seen, this being so, that the false alarm probability increases over time.

The idea is therefore, in the case of a validated track, to instruct in the detection module a local lowering of the detection threshold, in the case of a complex local background, if the detection module which applies the fixed detection threshold thd has not made it possible to detect a plot corresponding to this validated track, in the prediction zone. By locally adapting the detection threshold, we allow the detection of a plot that is dubbed a phantom plot. This principle is entirely inapplicable to the detection of plots for the hypothesis tracks, since in the case of hypothesis tracks, we would then increase the false alarm probability, which is not what we seek.

In practice, an implementation of the method according to the invention comprises two aspects: a prediction aspect and a detection aspect.

In this implementation, when a new plot has been associated with the validated track considered PV,: -the prediction module calculates for this validated track PVj, the position that the plot of the target ought to have in the next image observed, but also, the image zone in which this plot ought to lie. It can make this prediction since it has available as input the position of the plot and the associated homogeneous zone in which it lies in the current image. It can on the basis of this prediction, and by using the attributes of the plot associated with the track validated in the current image, of the associated zone and of the prediction zone in the next image observed, deduce therefrom as the case may be the future loss of contrast of the target as a function of the kinematics associated with the target. If the target moves from a simple zone to a more complex zone, we will thus be able to predict the SNR of the plot that we shall have to detect in the next image. -at the next observation, the module for associating plots with the validated tracks, opens a zone of association around the plot position prediction for each validated track. It is recalled that the zone of association is the zone of search of a plot, dependent on the model of motion associated with the track. The model of motion in fact allows a prediction to be made of the future position of the target. It is around this position that we shall seek the plot corresponding to the new date (new observation). As the model of motion is not perfect and may change if the target manoeuvres, the association module is given a latitude for performing its search. It is the zone of association defined by a radius of association around the predicted position.

If in this association zone the association module does not find any plot detected by the detection module, it looks at the complexity level k in the zone in which the predicted position of the plot lies, as determined at the previous image by the detection module. This zone is given by the detection module's image decomposition process.

If k=1 , indicating a simple background, the process stops. There is no detected plot, and therefore no plot associated with the track PH for this observation.

If k is greater than 1 , it selects an adapted detection threshold thda, corresponding to the complexity level k of this zone, and applies this adapted threshold as input to the detection module, for local application to the association zone considered. The detection module sends back the result of the detection: zero, one or more phantom plots pf,.

Thus, the use of the criterion of complexity of the local background of the plots, advantageously makes it possible to render the module for associating plots with the validated tracks more robust. The risk of losing track lock-on is decreased.

The invention improves the performance of the tracking module of an optronic surveillance system, by utilizing the radiometric information on the images, to define complexity levels of homogeneous zones of the image.

In a more detailed manner, to determine the suitable detection threshold thda adapted to the conditions of the detection, the following process is applied: -The table of the plots associated with the validated track provides a mean value Sm of the radiometric signal of the corresponding target.

Specifically, for a plot pi,, we know the radiometric signal S, measured, transmitted by the detection module.

The mean signal Sm associated with the target is therefore given by: Sm =∑s, , where N is the maximum number of plots kept in N memory table for the track considered.

The value Sm is recalculated each time that a new plot is associated with the track.

This value may be used by the prediction module to predict the signal-to-noise ratio that a plot of this track ought to have in the next image. This prediction is denoted SNRP.

Specifically, the prediction module provides a position prediction. It is therefore capable of predicting the homogeneous zone ¾■ in which this prediction lies. As we know the mean value of the target signal Sm, and the mean and the standard deviation of the radiometric signal in the zone ¾·, from this we deduce the signal-to-noise ratio predicted in the future observation: SM x m7 · SNRD. =— ' °z The size of window (or the complexity level k) associated with this zone Zp is known from the previous observation.

From this various information it is possible to determine an adapted detection threshold thda so as to be capable of detecting a signal-to-noise ratio corresponding to the predicted signal-to-noise ratio SNRp of the target, by using the classical equations for detection probability density (Figures 7 and 8) that are customarily used for the determination of a detection threshold. The detection threshold thda is preferably determined so as to obtain a probability of detection of the sought-after blip of close to 100%, in the open association zone. The blip detected by this method is called a phantom blip.

Whenever necessary, the piloting module pilots the detection module so that locally it applies an adapted threshold thdai, in a determined association zone Zai, corresponding to the search for a blip in respect of a validated track PV,, (Figure 4).

The detection module thus piloted will look at the signal-to-noise ratio at each pixel of this association zone Zai so as to compare it, pixel after pixel, with the adapted detected threshold thdai. If it detects a phantom blip pf,, it sends the attributes of this phantom blip to the association module, which can associate this blip with the validated track PV,.

The principle of detection of phantom blips, for the validated tracks may again behave in the following manner: The zone ¾■ in which the detection is made is known. If this zone has a high complexity level, the detection is made in reality in a small uniform neighbourhood zone of the image, and hence the errors in the mean and the standard deviation that are calculated for this zone are large (due to the low number of pixels of the zone considered). The signal-to-noise ratio SNR calculated is therefore underestimated. To detect a blip allowing association with the validated track, it is therefore necessary to lower the detection threshold locally in this zone, so as to guarantee the same detection probability as that which would have been obtained for the same predicted SNRP, but on a simpler background.

Thus, the implementation of this method makes it possible via the prediction function to predict the type of background on which a track will lie at a later date (the next observation). By using this information, the tracking is capable of forecasting the future loss of detection probability on complex backgrounds, and to remedy same by locally piloting an adapted detection threshold, allowing the upholding of the tracks.

Thus in a first aspect of the invention, the complexity level associated with each of the zones of the image observed makes it possible to define a set of comparison thresholds which is applicable for validating a hypothesis track, which makes it possible to render the validation criterion adaptive and dynamic, as a function of the local conditions of detection of the blips, the number p of blips of the hypothesis track and the number q of observations.

In a second aspect of the invention, the complexity level associated with each of the zones of the image observed makes it possible to locally adapt the detection threshold, so as to detect a blip corresponding to a validated track, as a function of the zone complexity level on which the position prediction for this blip is situated.

Thus, the tracking module adapts to the detection conditions encountered in the various zones of the images provided by the optronic monitoring sensors. It takes into account the radiometric characteristics of the local background over which the blips extracted by the detection module are deploying, so as to adapt the criteria for validating hypothesis tracks, and for upholding the validated tracks.

The various aspects of the invention are presented diagrammatically in Figures 4 and 5.

In Figure 3, the three processing operations performed by the tracking module are presented: firstly, the tracking module deals with the tracks already validated PV,. It is in this stage of association with the validated tracks that the tracking module instructs the detection module in respect of the search for phantom blips by local lowering of the detection threshold, using the information provided by the prediction module.

Secondly, the tracking module deals with the hypothesis tracks PHj. It is in this stage that the tracking module implements the adaptive and dynamic criterion, using the sets of comparison thresholds determined as a function of the local complexity of the background on which the blips considered in this stage were detected, to decide on the validation of a hypothesis track.

Finally, thirdly, the tracking module deals with the detected blips not attributed to the previous two steps, so as to create new hypothesis tracks.

Furthermore, the tracking module comprises a prediction module, which performs the suitable processing operations (Kalman filtering) to determine for the hypothesis tracks and the validated tracks, their future position.

Figure 4 borrows the block diagram of Figure 3 relating to the detection module, and is supplemented with a simplified block diagram of the tracking module 5, indicating the inputs/outputs of each. In particular, the output data DET0UT of the detection module that are applied as inputs to the tracking module comprise the attributes of the blips but also the zones Ζ,.

The tracking module furthermore pilots the detection stage of the detection module 4, so as to locally apply an adapted detection threshold thdai, in a determined zone zai, with the aim of detecting phantom blips pf,.

Finally, the sets of thresholds Ek used in the tracking module, by the stage for associating the hypothesis tracks of the tracking module, for the validation of these tracks, are represented symbolically, in the form of tables.

The implementation of the invention in an optronic image processing, to detect and follow tracks of pointlike targets, makes it possible through the consideration of the level of complexity of the local background on which a pointlike target is detected, to more rapidly validate the hypothesis tracks for which the blips detected have strong contrast. Furthermore, it allows the detection of phantom blips, for the exclusive purposes of following the validated tracks, by the lowering of the blip detection threshold, locally, in a zone of the complex image in which the target ought to be situated.

An advantageous use of the invention applies the modelling of the complexity of the local background by means of the complexity criterion k, associated with a uniform radiometric distribution criterion, which makes it possible to precalculate the sets Ek. However, it applies more generally to all techniques for splitting an image into homogeneous zones.

Claims

1. 81820/2 29 CLAIMS: 1. Method of optronic processing of images, to ensure the detection of pointlike targets and the following of the track of each target detected in the said images, by creating and validating hypothesis tracks on the basis of plots detected in an observed image, a hypothesis track comprising at least one detected plot, the said method comprising for each observed image, a step of detecting plots in the image, comprising a partition of the image observed into a plurality of homogeneous zones, a homogeneous zone being a zone on which a uniformity criterion defined by a determined radiometric distribution function is satisfied, the calculation for each pixel of the image observed of a corresponding signal- to-noise ratio (SNR) relatively to the homogeneous zone in which the said pixel is situated, and the comparison of the signal-to-noise ratio calculated with a detection threshold (thd), such that a plot is detected for a pixel of the image if the signal-to-noise ratio calculated for this pixel is greater than or equal to the said detection threshold, and for each plot detected, the determination of a set (Ek) of comparison thresholds which is associated with the said plot, the said set comprising one or more thresholds, the said comparison thresholds being greater than or equal to the said detection threshold (thd), a step of validating hypothesis tracks comprising for each plot detected, a step of applying a criterion for validating the hypothesis track associated with the said plot, the said criterion of validation being dependent on the comparison thresholds associated with each of the plots of the hypothesis track.

2. Method according to Claim 1, characterized in that the said step of determining comparison thresholds associated with a plot is dependent on the local background defined by the homogeneous zone on which the said plot has been detected.

3. Method according to Claim 1 or 2, characterized in that the said step of determining comparison thresholds associated with a plot comprises the 01736354\27-01 181820/2 30 determination of a comparison threshold applicable as a function of a number i of plots of the hypothesis track to which the criterion of validation is applied. Method according to Claim 3, characterized in that if the hypothesis track comprises p plots, the criterion of validation consists in verifying whether there exist i plots, i=l to p, such that for each plot out of the i plots, the signal-to-noise ratio (SNR) is greater than the comparison threshold applicable for i plots. Method according to any one of Claims 1 to 4, characterized in that it comprises for each homogeneous zone (Z,) of the image, the calculation of a mean (mz ) and a standard deviation (σΖ[ ) of the radiometric levels of the pixels on the said zone, allowing the calculation of the signal-to-noise ratio at each pixel of the image observed relatively to the homogeneous zone in which it is situated, given S - m2 by RSB = '- , where S is the level of the radiometric signal of the pixel considered. Method according to any one of Claims 1 to 5, characterized in that the partition of the image into homogeneous zones comprises the application of a step of determining a homogeneous zone (Ζ;) for each pixel of the image observed, corresponding to a search of a zone of the image which is as large as possible around the pixel, apart from the pixel, on which the said radiometric distribution uniformity criterion is satisfied, the said homogeneous zone (¾) thus defined being called the largest zone of neighbourhood around the the said pixel. Method according to Claim 6, characterized in that the said step of determining the largest zone of neighbourhood around the said pixel, applies a modelling by size TFIC of window, corresponding to a level of complexity k of local background, the said modelling comprising the definition of t window sizes, the determination of the said largest zone of neighbourhood around a pixel consisting in determining the largest size of window on which the said uniformity criterion is satisfied, and the definition of a criterion of complexity k of local background associated with each size of window, k varying from 1 to t, X27-01 181820/2 31 such that it is equal to 1 for the window of largest size and to t for the window of smallest size, and in that with each plot detected is associated the level of complexity k corresponding to the size of window TFk of the said largest zone of neighbourhood thus determined. Method according to Claim 7, characterized in that a set of comparison thresholds (Ek) is determined for each level of complexity k of local background, PxQ each set (Ek) comprising—^- thresholds thi j k , with i = 1 to P and j = 1 to Q, with i < j, where P represents a maximum number of associated plots and Q a maximum number of observations. Method according to Claim 8, characterized in that each set of comparison thresholds (Ek) defines, for a number q of current observations, a subset of q comparison thresholds thjjq, , i=l to q, with thi,q,k>th2,q,k>th3,q,k >thj,q,k>...>thq,q;k. Method according to Claim 9, characterized in that it consists, when a p-th plot of a hypothesis track PH has been detected at the q-th observation: - from the sets of comparison thresholds (Ek) indexed by the levels of complexity k associated with the p plots of the hypothesis track (PHj), in selecting the subsets as a function of the number q of observations; . In initializing a comparison loop with i=l, this loop consisting in the following steps: -a), selection of the thresholds thj>q;k, of index i, in the selected subsets and application of the following criterion: ■ If there exist i plots out of the p plots of the hypothesis track PH such that each plot out of these i plots satisfies the following inequality: SNR > thj>qjk for the value of the level of complexity k associated with this plot. • Then the track is validated - end of the loop. ■ if not, step b). -b). i=i+l. '-01 181820/2 32 ■ While i≤p, return to step a). ■ Else, step c). -c).if i>p, Then the track is not validated 11. Method according to any one of the preceding claims, comprising a step of predicting a position of a plot of a track in the next observed image, the said prediction comprising the prediction of an associated homogeneous zone. 12. Method according to the preceding claim, in the case where no plot is detected in the current observed image for a given hypothesis track, the method comprising a step of comparison of the number p of plots of a hypothesis track and of a number q of observations performed since the first plot associated with the said hypothesis track, with predefined optimal values, respectively a maximum number of plots P to be compared with the value p and a maximum number of observations Q to be compared with the value q, so as to validate the said hypothesis track as a function of the deviation at the said optimal values and as a function of the level of complexity k associated with the plot position prediction zone in the current image. 13. Method according to Claim 11 or 12, comprising an additional step of associating plots with a validated track (PVj),wherein in the case where no plot is detected in the current observed image for a given validated track (PVj), the said step comprises the checking of the detection threshold in the detection step (4), so as to adapt it locally as a function of the level of complexity k associated with a plot position prediction zone for the validated track considered. 1

4. System of optronic surveillance comprising one or a plurality of optronic sensors able to provide images for observing a scene, the said system comprising means of processing of the said images implementing a method according to any one of the preceding claims. AND PARTNERS 01736354\27-01