DE4332753A1 - Process for the detection of moving objects - Google Patents

Abstract

A method which is used for optically detecting moving objects method comprises an image recording unit 1, a computing unit 2 with a memory 4, and a signal display 5. The computing unit compares an image recorded by the image recording unit with a reference image in relation to the measured variables: image signal variation, displacement and texture. For determining the comparison values the chronological mean value and the chronological variance are thus preferably used for describing the typical distribution of the measured variables in each case. If the comparison displays an atypical image signal variation and an atypical texture and an atypical displacement, a moving object is detected, a signal is emitted, and the actual image signal is stored as the new reference image signal and is used as the reference image signal for the next comparison. <IMAGE>

Description

State of the art

The invention is based on a method for detecting moving Objects according to the type of the main claim. It's already from J. Eliss et al., "A knowledge-based approach to automatic alarm inter pretation using computer vision, on image sequences ", ICCST, 1989, Page 215 ff., A method for the detection of moving objects in time Lich known successive images, but in which the image signal is compared with a specified reference image signal to detect a moving object. In addition, determined ver displacement vectors with given displacement vectors for detection comparison of a moving object.

Advantages of the invention

The inventive method with the features of the main claim has the advantage that the Refe renzbildsignal for each image area of the image depending on the evaluation result of an automatically running image analysis is updated adaptively. As a result, it is not necessary for ver different scenarios to be monitored different comparison thresholds for the image signal or various comparative movements  to specify vectors. By the method according to the invention due to the image analysis used, an adjustment of the Comparison thresholds at the different image areas reached. This leads to greater security in the detection of moving Objects regardless of the terrain to be observed and the Weather influences.

The measures listed in the subclaims provide for partial further training and improvements of the main claim specified procedure possible. It is particularly beneficial to Image area of the current image with the corresponding image area to compare the reference image in terms of texture features. In order to the security of recognizing a moving object is increased.

To determine texture features, it is particularly advantageous to to use two adjacent pixels. So that becomes a get enough level of information and at the same time that is too amount of data processed is low and it becomes fast processing information reached.

The method has the advantage that the object detection for a pair of images (reference image and current image) determined measured variables Image signal change, shift and texture are not one for that entire picture valid, fixed threshold are compared, but these thresholds from a separately for each image area performed measurement of the temporal statistical distribution of this measurement sizes - described by mean and variance - are determined.

The speed and security of moving object detection is optimized by first changing the image signal for detection  a moving object is used and, if a moving object ject was recognized, the texture features to detect the moving Object are used and if this comparison also reveals moving object, finally the displacement vector is considered to recognize the moving object and only then even if this comparison shows a moving object, a moving one Object is recognized as safe and a signal to indicate whether eject alarm is issued. This means that despite high security Detect moving objects with the shortest possible processing time reached.

The comparison of the measured variables image signal change, displacement vector and texture feature is done easily and quickly when the Current measured value of the measured variable of an image area minus the time mean value of this measurand with the product of time standard deviation of the measured variable evaluated with a weighting constant is compared. So a simple and quick ver equal to the measurands for object detection.

The estimate of the mean and the variance of the measurands image signal, displacement vector and texture features are advantageous ter way with a recursive first order low pass, a so-called called Infinite Impulse Respond Low Pass. This will make one simple estimate of mean and variance achieved.

drawing

An embodiment of the invention is shown in the drawing represents and explained in more detail in the following description. It demonstrate

Fig. 1 shows an arrangement for detecting a moving object,

FIG. 2 is an imaging unit with Click

Fig. 3 is a distribution like tefunktion for evaluating a measured variable and

Fig. 4 shows a schematic program flow of the method.

Description of the embodiment

In Fig. 1 illustrates an arrangement for detecting a moving object in successive images. An image recording unit 1 is connected to a computing unit 2 via a data line 7 . The computing unit 2 is in turn connected via a data line 7 to an input unit 3 , via a further data line 7 with a memory 4 , via an additional data line 7 with a screen 6 and via a further data line 7 with a signal display 5 .

Fig. 2 shows an image unit 31 , which consists, for example, of 5 × 5 pixels 32 of an image 30 . To determine texture features, so-called click features are used in the method described here. One click includes an arrangement of two locally adjacent pixels. The sum of the squared differences of the image signals of the pixel pairs of an image unit determined by the clicks is used as the measurement variable for the texture features. Three different types of clicks are used as clicks: a horizontal click 33 , a diagonal click 34 and a vertical click 35 .

Fig. 3 is a distribution density function P (vi) 40 shows the measured variable vi for a typical scene situation. The mean value m _{vi of} the measured variable vi is plotted. The "typical" range of the measured variable lies in the range of values of the temporal mean value m _vi ± (c _vi × s _vi ), c _{vi being} a fixed weighting constant and s _vi the temporal standard deviation of the measured variable vi. The range of values 41 in which an "atypical" measured variable is present is hatched in FIG. 4. An atypical measurement is a sign of a moving object. The range of values 42 , in which a typical measured variable is present, is not hatched. A distribution density function for each measurement variable (image signal change, texture features and displacement vector) is stored in the memory 4 for each image area.

Fig. 4 shows a schematic flow of the method for recognizing a moving object. At program point 11 , the image recording unit 1 takes an image and forwards it to the computing unit 2 . The computing unit 2 then stores an image signal at program point 12 for each pixel of the image. The image 30 is divided into image units 31 . In this exemplary embodiment, an image unit consists of an image area which comprises 5 × 5 pixels 32 .

In the following program point 13 , the computing unit 2 compares the change in image signal between the recorded image and a reference image which is stored in the memory 4 . The change in the image signal is compared with a distribution function of the change in the image signal which is valid for this image unit - described by mean and variance - and which represents a typical scene situation. The image units whose image signal changes are atypical are identified by the computing unit 2 in a first binary mask and stored in the memory 4 . A binary mask consists of a memory field that contains a memory location for each image unit. This memory location is assigned 0 or 1 for identification.

At program point 14, computing unit 2 determines texture features for each image unit in accordance with a predetermined method, which is stored in memory 4 , and stores them in memory 4 . At program point 20, computing unit 2 determines whether the texture features determined for the image units are atypical texture features. The image units, whose texture features are atypical, are identified by the computing unit 2 in a second binary mask and the mask is stored in the memory 4 .

Thereupon, the computing unit 2 at program point 15 determines the displacement vector of the image unit for each image unit from the comparison of the current image with the reference image stored in the memory 4 and stores it in the memory 4 . In a third binary mask, the computing unit marks the image units that have an atypical displacement factor and stores the third mask in the memory 4 . At program point 21 , the query is made as to whether the displacement vectors of the image differ from a typical displacement vector of the corresponding image unit.

If this is the case, program point 22 follows. At program point 22 , the computing unit 2 recognizes in the image, by comparing the binary masks, a moving object for the image unit, whose image signal change, its texture and its displacement vectors are atypical and displays this via the signal display 5 . The program then branches to program item 26 .

At program point 26 , the reference image is reworked. In the case of a detected, moving object, the computing unit 2 replaces the image signal of each image unit of the reference image with the image signal of the image unit of the current image. The program then branches to program item 11 , the next picture is taken and the program run again.

If the query at program point 21 shows that the displacement vectors of all image units are typical, then in program point 18 for the displacement vectors, in program point 17 for the texture features and in program point 16 for the image signal changes, each using the computing unit 2 with a low-pass filter according to the first order Formula (2), taking into account the current measured variables of image signal change, texture and shift, a new temporal mean value and a new temporal variance of the measured variables according to formulas (3) and (4) for the current description of the distribution function of the measured variables for each image unit of the image. In addition, at program point 16, the computing unit 2 replaces the image signal of the image units of the reference image with the image signal of the image units of the current image, the displacement vector of which is identical to zero. The image signal of the image units whose displacement vector is not equal to zero remains unchanged.

At program point 19 , the mean value and the variance of the measured variables image signal change, texture and displacement vector for each image unit are stored in the memory 4 and the previously stored mean values and variances for the image signal change, the texture and the displacement vector are deleted.

In this special process, one for each pixel Image unit the measured variables image signal change, texture and Displacement vector determined. An optimization of the computing time will achieved by determining the measurands for one image unit become.

A particularly fast image processing is achieved if, according to program item 13 in program item 14, text characteristics are only determined for the image units by the computing unit 2 , the image signal change of which is atypical. And at program point 15 , displacement vectors are determined by the computing unit 2 only for the image units, whose image signal change and their texture features are not typical.

Based on the schematic program flow of Fig. 4 and A of Figure 1 relationship., 2 and 3, a specific embodiment will be described below. At program point 11 , the image recording unit 11 , which in this special case is designed as a CCD camera, records an image in the form of an analog video signal. A typical resolution of an image is 756 × 288 pixels, an image signal being recorded for each pixel. The analog video signal is first low-pass filtered in the image recording unit 1 , digitized and passed on to the computing unit 2 . At program point 12 , computing unit 2 stores the current image in memory 4 . Then the stored image is subsampled by the computing unit 2 , the horizontal subsampling factor being four in this particular embodiment and the vertical subsampling factor being two. This combines 2 × 4 pixels into one pixel. The undersampled image is stored in the memory 4 by the computing unit. The entire object detection is performed on the subsampled image. This procedure considerably reduces the effort for the evaluation and, on the other hand, influences the detection ability only slightly. In the event of a detected moving object, the image is immediately available in full resolution and can be evaluated, for example, for perpetrator detection or alarm checking.

In the case of operation in the dark or at dusk, a local accumulation of the pixels in the scanning grid of the video signals as well as an accumulation of several in time the following pictures a virtual extension of the exposure time  beyond a field interval of 1/50 second led to increase the sensitivity of the camera. This will a current image signal statistics measured and from the image signal Statistics Contrast and average brightness of the camera image determined telt. If the lighting conditions are insufficient, the contrast by local and temporal accumulation of the pixels of the video signals increased and the average brightness adjusted appropriately. These measures lead under unfavorable lighting conditions to improve the image signal and thus also in the dark or dawn for reliable object detection.

The computing unit 2 then determines the difference to the image signals of the image points of a reference image, which is stored in the memory 4 , for each image signal of each image point of the subsampled image. The image signals of the reference image are assigned fixed initial values which are given to the computing unit 2 via the input unit 3 . After a few images, the image signals of the reference image are assigned values that correspond to a typical scene situation. In this exemplary embodiment, the square pixel differences between the subsampled current image and the reference image are accumulated for the acquisition of temporal image signal changes for each image unit. This square sum is used as a measure for describing the temporal change in the image signal of an image unit.

In object detection, i.e. H. when recognizing a moving object, it is examined whether the currently determined measured variables (Image difference signal, texture feature, displacement vector) one Describe a typical or atypical scene situation. These Evaluation is based on a statistical analysis of the measurands, with a temporal statistical distribution of the Measured variables is determined. When evaluating a measured variable checks whether the current measured value of the distribution of a typical Measured variable is sufficient or  Not. Since the measurement of the distribution function is complex to describe the distribution mean and variance estimated. This estimation is carried out whenever there is no object alarm was recognized, and thus the image unit of a typical scene situation corresponds. In this way describe the mean and the variance the distribution of the measured variables in typical scene situations ions. If a current measured value of a measured variable deviates significantly this description, this event indicates an atypical event Scene situation, d. H. towards a moving object.

The decision as to whether a measurand is significantly different from its statistical distribution deviates, is based on a threshold divorce. The following formula is used:

(v _i (x, y) -m _vi (x, y)) ² / (c _vi × s _vi (x, y)) ² <1 (1)

With

c _vi : weighting constant [0 <c _vi <50 depending on the measured variables examined]
v _i (x, y): current measured value of the measured variable vi at the location (x, y)
m _vi (x, y): average over time of the measured variable vi at the position (x, y)
s _vi (x, y): temporal standard deviation of the measured variable vi at the position (x, y).

The weighting constant c _vi depends on the measured variable being examined. It is a measure of how many times the standard deviation the current measured value may deviate from the estimated mean of the measured variable. The greater this weighting constant, the more the current measured value has to deviate from the estimated mean value in order to trigger an object alarm.

In Fig. 3 the inequality (1) is shown graphically. The distribution density function P (vi) of a measured variable vi at a location (x, y) of the camera image is described by the two parameters m _vi (mean) and s _vi (standard deviation). If the current measured variable vi lies within the interval [-t _vi , + t _vi ] (e.g. v _i ⁽¹⁾ , the probability density p (v _i ⁽¹⁾ ) for the measured value v _i ^{(1) is} greater as a required threshold p _min . This threshold p _min is the probability density of the measured value vi, for which the following condition applies:

(v _i -m _vi ) ² = (c _i × s _i ) ².

If the current measured value vi lies outside the interval [-t _vi , + t _vi ] (eg Bv _i ⁽²⁾ ), an object alarm is triggered. The remaining probability of error, ie the probability that an "atypical" scene situation is erroneously concluded, although the current measured value satisfies the distribution, is then the integral of all possible measured values outside the interval (-t _vi , + t _vi ] and in Fig. 3 drawn hatched.

To be able to make a threshold decision, the required statistical statistical parameters mean and Variance estimated separately for all image units and measured quantities.

This estimate comes with an IIR (Infinit Impulse Respond) low pass first order according to the following formula:

m _vi (x, y, t): r _* v _i (x, y) + (1-r) _* m _vi (x, y, t-1) (2)

respectively.

p _vi (x, y, t): r _* v _i ² (x, y) + (1-r) _* p _vi (x, y, t-1) (3)

and

s _vi ² (x, y, t): p _vi (x, y, t) - m _vi ² (x, y, t), (4)

With

m _vi (x, y, t), m _vi (x, y, t-1): temporal average of the measured variable vi at time t or t-1,
p _vi (x, y, t), p _vi (x, y, t-1): temporal output of the measured variable vi at time t or t-1,
_vi ² (x, y, t): temporal variance of the measured variable vi at time t,
v _i (x, y): value of the measured variable vi at the time t at the location (x, y),
r: recursion factor of the IIR low pass.

This filter is characterized in that the estimate of the only a few arithmetic steps per detector cell (Operations) required and with only one additional one at a time Memory per measurement and parameter can be performed.

Changes depending on the set recursion factor r the average settling time of the IIR low pass (number of One step increments at the input of the filter until the end filter has 0.9 times the amplitude of the input).  

This settling time is a measure of how many pictures at least have to be processed until the statistical parameters are determined with sufficient accuracy. In principle, the smaller the Re italic factor, the longer the settling time. On the The recursion factor determines whether the determined parameters are the Record long-term statistics of the measurand (recursion factor very small), or whether medium or short term changes in the Measured variables are taken into account in the statistics (Recursion factor as large as possible). It also fits Description of the distribution of the measurands through the recursive, Continuous estimation of the parameters on changing Scene situations.

At program point 13 , temporal image signal changes between the reference image and the current image are detected by a change detection. The image areas that have atypical image signals are identified in the first binary mask. Small image signal changes that correspond to a typical scene situation, ie the z. B. by the noise of the video sensor fen are suppressed for object detection. In the exemplary embodiment described here, the decision for a temporally preceding image or the image area is taken into account in addition to the image signal difference. Depending on whether an image area was recognized as changed or not changed during the previous change detection, two different thresholds are used in the evaluation for the current image signal change. This process is described in the unpublished patent application with the file number P 43 11 972.7.

The arithmetic unit 2 checks whether the square pixel differences of the image unit between the reference image and the current image deviate more from the mean value m _{vi of} the square pixel differences of the corresponding image unit than the temporal variance s _{vi of} the square pixel differences multiplied by the fixed and stored in the memory 4 Weighting constant c _vi , that is, it is checked whether the inequality shown in formula (1) is satisfied. The computing unit marks the image units that satisfy the formula (1) as atypical in the first binary mask.

In program point 14 , the computing unit 2 examines whether the atypical changes in the image signal were caused by a change in the image texture. A texture is described by clicking features. In the exemplary embodiment described here, two directly locally diagonally adjacent pixels are used as clicks, as shown in FIG. 2. By measuring diagonally adjacent pixels, local correlations in the diagonal direction are taken into account.

By analyzing the texture, the following scene situations can be recognized and object alarms can be suppressed:
Diffuse movements (e.g. rustling leaves, rain, snow), global changes in brightness (day, night, large shadows) and small camera movements (wobbling, shocks).

The computing unit 2 determines the texture for each image unit. One click describes the local arrangement of two adjacent pixels. Clicks in the form of diagonal clicks 34 are used in this exemplary embodiment. The sum of the squared differences of the pixel pairs of an image unit determined by the clicks is used as the measurement variable for the texture features. This measured variable depends on the type of click used and on the local correlation of the image signal within an image unit. The determined texture features are stored by the computing unit 2 in the memory 4 . Further clicks are horizontal clicks 33 and vertical clicks 35 , which can be used to determine the texture.

At program point 20 , the measured variables of the texture features are then checked by the computing unit 2 in accordance with the formula (1). The image units whose texture features are atypical, ie which fulfill the formula (1), are marked in the second binary mask.

At program point 15, computing unit 2 determines a displacement vector for describing movement information for each image unit. When the displacement vector is estimated, it is checked whether an image unit of the current image can be found in its local environment in the previous image. Suitable similarity measures are defined for this search, for example the absolute or square sum of the pixel differences of the image area defined by the image unit.

The location of the image unit in the previous image with the greatest similarity, that is to say, for example, with the smallest square sum of the pixel differences, is then assigned to the current image unit via a displacement vector. The determined displacement vector is stored in memory 4 . This describes the difference in the position of an image unit in two temporally successive images. In the method presented here, a "three step block matching" algorithm is used to estimate the displacement vectors, which is described by T. Kogy, K. Linuma, A. Hirano, T. Iÿiama and T. Ishiguro in "motion-compensated interframe coding for video conferencing ", IEEE, National Telecommunications Conference, New Orleans, pp. 65.3.1-65.3.5, December 1981. The "Three Step Blockmatching" algorithm is characterized by low complexity and moderate computing effort. The program then branches to program item 21 .

At program point 21 there is a query for each image unit of the current image as to whether the displacement vector corresponds to an atypical scene situation, ie whether the measured variable for the displacement vector fulfills the inequality (1).

If the measured variable of the displacement vector of all image units does not meet inequality (1), then displacement vectors are available for a typical scene situation and branching to program item 18 takes place .

At program point 18, the computing unit 2 for the displacement vectors of the image units uses the formulas (2), (3) and (4) to determine a new temporal mean, a new temporal power and a new temporal variance of the displacement vectors of the imaging units. Subsequently, at program point 17, the computing unit 2 uses the texture features determined for each image unit in order to use formula (2), (3) and (4) to generate a new time average, a new time performance and a new time variance of the texture features for each image unit to investigate.

At program point 16, the computing unit determines a new temporal mean value, a new temporal power and a new temporal variance of the quadratic table point differences for each image unit using the ascertained square pixel differences using the formulas (2), (3) and (4). The computing unit 2 then replaces the image signals of the image units of the reference image, the displacement vectors of which are zero, with the image signals of the corresponding image units of the current image.

At program point 19 , the arithmetic unit 2 replaces the previous temporal averages, the temporal performance and the temporal variance of the displacement vectors, the texture features and the pixel differences by the new temporal averages, the new temporal performance and the new temporal variance, which according to the formulas (2) , (3) and (4) are calculated by the computing unit.

If a displacement vector of an image unit of the current image at program point 21 fulfills the inequality (1), ie - due to the procedure described for the measurement variable acquisition - all measurement variables (image signal change, texture and displacement vector) of the image unit under consideration describe a non-typical scene situation, then according to program item 22 branches. At program point 22 , computing unit 2 detects a moving object and emits a signal using signal display 5 . At the same time, the current image is displayed in full resolution on the screen 6 . In the following program item 26 , the computing unit 2 replaces the reference image with the current image.

This ensures that the current scene with the moving object describes the typical scene situation used below. Changes the typical scene situation, so the reference picture is through adapted the object alarm to the current scene situation. These Adaptation through reinitialization is repeated until the Object comes to a standstill or leaves the picture.

The program then branches to program item 11 and runs through the program again.

The change in the image signals of the reference image becomes dependent decided by the size of the displacement vectors. In doing so differentiated the following cases:

The displacement vector is zero, ie there is no directional movement in the image. Temporal image signal changes cannot be described by movement in the viewed image area. Causes: many different movements within an image area, local changes in brightness.
Displacement vector not equal to zero: an object alarm is only triggered if there is significant movement. In order to be able to detect moving objects that move slowly and in a directional manner, the statistical significance test of the movement description is used in addition to object detection and for the post-processing of the reference image. The reference image is not reworked in all image areas in which there is a non-zero displacement vector which corresponds to a typical situation, ie the reference image remains unchanged in these image areas.

This procedure leads to small and directed movements with a motion estimation between the current camera image and Accumulate reference image over time. This way you can choose between an intruder moving slowly through the picture, and a tree that moves back and forth in the wind made a difference because, in the first case, the movement accumulates and therefore the amount of time grows, while in the second case the movement is mean-free since the tree is firmly anchored to the ground.

Claims (8)

1. A method for the detection of moving objects in chronologically successive images of a sequence, an image being divided into image areas and for each image area being compared by comparing the image signal with a reference image and / or a displacement vector being determined from the comparison and the change of the image signal and / or the size of the displacement vector are compared with a threshold and from the comparison a moving object is detected and, in the case of the detection of a moving object, a signal is emitted, characterized in that when a moving object is detected the image signal is new Image signal of the reference image is stored and used as the reference image for the next comparison. 2. The method according to claim 1, characterized in that the image area of the current image with the corresponding image area of the Reference image is compared in terms of texture features. 3. The method according to claim 1 and 2, characterized in that at the threshold value test of the measured variables the statistical distribution of the Measured variables used - described by mean and variance - be used.   4. The method according to claim 3, characterized in that the Mit determined and the variance for each image area separately and can be used for comparison. 5. The method according to claim 3 and 4, characterized in that for Determination of texture features the difference between two locally adjacent barter pixels can be used. 6. The method according to any one of claims 3 to 5, characterized in net that the image signal of the image unit with the corresponding Refe renzbildsignal is compared and that when the image signal change a set amount above or below a set amount Image signal threshold, the texture change of the image unit of the current image compared to the corresponding image unit of the Reference image is compared with a defined threshold and that if the texture change is more than a specified value of deviates from the defined threshold, the displacement vector of the Image area in relation to the corresponding image area of the Reference image is determined and that finally a moving Object is recognized and a signal is emitted when the Displacement vector is greater than a predetermined threshold. 7. The method according to any one of claims 3 to 6, characterized in that the following formula is used as a threshold value comparison for the measured variables image signal change and / or displacement vector and / or texture: (v _i (x, y) -m _vi (x, y)) ² / (c _vi × s _vi (x, y)) ² <1, whereby with c _vi a weighting constant, with vi (x, y) the current measured value of the measurand vi at the position (x, y) , with m _vi (x, y) the time average of the measured variable vi at the location (x, y) and with s _vi (x, y) the temporal standard deviation of the measured variable vi at the location (x, y). 8. The method according to claim 7, characterized in that the mean value and the variance of the measured variables image signal, Verschiebungsvek tor and texture for each image area with the aid of a low pass first order is estimated according to the following formula: m _vi (x, y, t) = r _* v _i (x, y) + (1-r) _* m _vi (x, y, t-1) or
p _vi (x, y, t) = r _* v _i ² (x, y) + (1-r) _* p _vi (x, y, t-1) and
s _vi ² (x, y, t) = p _vi (x, y, t) - m _vi ² (x, y, t), where with m _vi (x, y, t), m _vi (x, y , t-1) the time average of the measured variable vi at the time t or t-1,
with p _vi (x, y, t), p _vi (x, y, t-1) the temporal output of the measured variable vi at time t or t-1, with s _vi ² (x, y, t) the temporal output Variance of the measured variable vi at time t, v _i (x, y) denotes the value of the measured variable vi at time t at the local position (x, y) and r denotes a fixed recursion factor of the low pass.