CN102903120A - Time-space condition information based moving object detection method - Google Patents

Abstract

The invention discloses a moving target detection method based on spatio-temporal condition information. This method considers the saliency of human visual space-time domain to construct the space-time domain model of target detection, calculates the conditional probability that the detected image belongs to the reference background in the space-time domain, and uses the negative logarithm to check the conditional probability to perform nonlinear transformation to extract the space-time condition information, considering the local consistency of image features The image condition information in the neighborhood is weighted and summed, and used as a feature for target detection using a linear classifier. The color histogram is used to quickly calculate the conditional probability, and the image block is used instead of a single pixel for modeling and detection, which reduces the algorithm complexity and storage space requirements, and combines the image block differential pre-detection mechanism to improve the target detection speed. The invention has low algorithm complexity, small storage space requirement, and high real-time performance of the algorithm, can effectively suppress the influence of background disturbance interference and isolated noise, and realize real-time detection of moving targets on existing computers, and is suitable for embedded intelligent camera platform applications.

Description

A Moving Target Detection Method Based on Spatial-Temporal Condition Information

Technical field:

The invention relates to video moving target segmentation in computer vision, in particular to moving target detection in a video monitoring system. the

Background technique:

Video moving object detection is one of the basic problems in computer vision applications. It is the basic support for advanced applications such as moving object tracking, moving object recognition, man-machine interface, action recognition, and behavior understanding. It has been used in specific applications such as video surveillance and video retrieval. It will play an important role in military, transportation, security, culture and entertainment and many other fields. the

The intelligent video surveillance system can liberate people from heavy video surveillance tasks, reduce manual intervention, reduce the workload of surveillance personnel, automatically discover moving targets in the surveillance environment, automatically track and identify moving targets, and automatically discover suspicious events in the surveillance scene and extract information of interest. The intelligent analysis functions in the aforementioned intelligent video surveillance system all rely on the video moving object detection algorithm. The video moving object detection is to separate the moving object from the background in the video to extract the moving object. It is the basic algorithm of the intelligent video surveillance system. Algorithmic basis for subsequent target tracking, recognition, and suspicious event detection. the

The current mainstream moving target detection methods include background subtraction method and optical flow method. The complex calculation of the optical flow method makes it difficult to be practically applied; the background subtraction method is currently the most commonly used and most effective method for moving object detection. Target. Commonly used background subtraction methods include Gaussian Mixture Model (GMM), non-parametric model (Kernel Density Estimation, KDE), Code Book model (Code Book), etc., which model pixel brightness in the time domain to Detect moving objects. The challenge of moving target detection comes from how to overcome the influence of natural environment changes (light changes, shaking leaves, rain and snow, water surface fluctuations, etc.) and imaging equipment (electronic noise, camera shaking, etc.). The commonly used background subtraction method based on temporal domain modeling detects and locates moving targets through changes in image features (color, gradient, texture, edge, etc.) in the temporal domain. However, the image features at each pixel position in the image are not isolated, and there is a relationship between them. Therefore, it is difficult to deal with background disturbances in the scene by using time domain changes. It is difficult to suppress the influence of environmental noise. Although the method based on image segmentation (moving target segmentation based on random field) can suppress isolated noise, the method based on segmentation relies on the initial detection results, and it is difficult to obtain accurate segmentation results when the initial detection results are seriously wrong, and the algorithm has poor real-time performance. . The method based on the temporal-spatial domain model fully considers the spatial-temporal consistency of image color distribution, and performs joint modeling in the temporal-spatial domain for moving object detection, and shows good performance when dealing with background disturbances in dynamic scenes. Algorithms based on the time-space domain, due to the need to process a large amount of time-space domain data, have high computational complexity, large memory requirements, and poor real-time performance of the algorithm. Due to the influence of isolated noise interference, the final detection results need post-processing such as morphological filtering or image segmentation. In order to get better test results. the

With the development of video surveillance systems from the analog era to the network era, cameras are also developing towards intelligence. More and more intelligent video processing algorithms, including moving object detection algorithms, need to be transplanted to smart cameras and embedded in smart cameras. . However, the existing video moving object detection algorithms that can deal with environmental noise in dynamic scenes not only have high computational complexity, but also require a lot of memory, making it difficult to apply on embedded smart camera platforms. For this reason, for the practical application of intelligent video surveillance systems, we propose a moving target detection method in dynamic scenes based on spatio-temporal condition information, which can effectively suppress the detection of moving objects in dynamic scenes. In the environment noise interference, effectively detect moving targets, and use the image block strategy to accelerate target detection, reduce algorithm complexity, increase real-time performance, reduce memory requirements, so that the moving target detection method based on spatio-temporal condition information can not only be used in the existing Real-time detection of moving targets in dynamic scenes is realized on the PC platform, and it is also suitable for embedded smart camera platform applications. the

The essence of moving target detection is a binary classification problem, that is, using the background sequence as a reference condition, the pixels in the currently observed image are classified into foreground (also called target in the present invention) and background. Based on the consideration of algorithm complexity, the existing moving target detection algorithms mostly use linear classifiers to classify image pixels and segment the foreground in the current image. The background (fluctuating water surface, swinging leaves) and the foreground are often linearly inseparable. Taking the detection of floating objects in the water surface fluctuation scene (b in Figure 1) as an example, it can be found that the background difference, the mixed Gaussian model, and the non-parametric model all have the problem that the background and the foreground are linearly inseparable to a certain extent. the

Background difference moving target detection: Subtract the input image from the reference background image to obtain a difference image, which is used as a classification feature, and uses a binarization operation (the simplest binary classifier) to detect moving targets. As shown in Figure 1, the difference between the current input image (b in Figure 1) and the reference background (a in Figure 1) is obtained to obtain the background difference image (c in Figure 1), and then the color histogram of the background area in the difference image is counted separately and the color histogram of the foreground area, the degree of separation between the two histograms reflects the linear separability of the background and the foreground. Statistical histograms of background and foreground areas, the specific implementation method is: manually mark the target area in each frame image in advance to obtain the moving target mask template (d in Figure 1), and then count the images belonging to the target mask area The difference histogram of the foreground area, the difference histogram of the background area (b0 in Figure 2) is counted by the image in the non-masked area, and the difference image histogram of the entire video target area and the background area is obtained in the same way as e0 in Figure 2 shown. Partially zooming in on the lower half of b0 and e0 in Figure 2 (c0 and f0 in Figure 2), it can be seen that there is a large-scale overlap in the histogram of the difference image between the target area and the background area, that is, the separability of the background and the foreground is low , therefore, using the background difference image as the moving object detection feature, it is difficult to perform linear classification by a linear classifier. Therefore, the background difference image feature is linearly inseparable from the background when dealing with moving object detection in dynamic scenes. the

Mixed Gaussian model and non-parametric model are two typical video moving target detection algorithms modeled on the image color probability distribution. They both use the conditional probability of the image pixel to be detected to belong to the background as the classification feature, and then use a linear classifier for detection. . Since non-parametric models can represent arbitrary probability distributions, we take non-parametric models as an example to investigate the linear separability of foreground and background in background subtraction methods based on color probability distribution modeling. As shown in Figure 2, the non-parametric model is used to estimate that the current input image (b in Figure 1) belongs to the background b. The conditional probability feature image is a1 in Figure 2. According to the above method, the histogram of the target and background areas on the feature image is obtained as As shown in b1 in Figure 2, and the histogram of the target area and the background area of the entire video is shown in e1 in Figure 2. Partially zooming in on the lower half of b1 and e1 in Figure 2 (c1 and f1 in Figure 2), it can be seen that compared with c0 and f0 in Figure 2, the overlapping range of histograms between the target and the background area is reduced, and the linear separability Increased, but the linear interface between the target and the background is narrow, and the selection of the segmentation threshold is easily disturbed by noise, which affects the robustness of the algorithm. the

Non-linear transformation is performed on the conditional probability p(x|b) in the non-parametric model, and the feature image shown in a2 in Figure 2 can be obtained, and the corresponding histogram of the target and background regions can be obtained in the same way (in Figure 2 b2, e2). It can be found from the partial enlarged images of the corresponding lower part (c2, f2 in Figure 2): the linear interface between the target and the background has widened. That is, the nonlinear transformation enhances the linear separability of foreground and background. the

The distribution of image features has local consistency, that is, image pixels are not isolated, but there are connections between them and pixels in the neighborhood. The image features of the current pixel x will be affected by the image features of the pixels in the neighborhood. Therefore, the weighted sum of the nonlinearly transformed image features in its neighborhood can further suppress the isolated noise and increase the linear interface between the target and the background (Fig. 2 in b3, c3, e3, f3), increase classification robustness and reduce classification errors. the

As shown in Figure 2, d0 in Figure 2 is the detection result of the background difference algorithm, d1 in Figure 2 is the detection result of the non-parametric model, d2 in Figure 2 is the detection result of the conditional probability nonlinear transformation, and d3 in Figure 2 is the conditional probability nonlinearity The weighted sum detection results in the neighborhood of the transformed feature image. It can be seen from the detection results that the nonlinear transformation of the conditional probability and the weighted summation in the neighborhood can effectively suppress the background disturbance in the dynamic scene, reduce the isolated noise pollution, and obtain good target detection results. Therefore, the present invention intends to adopt a method of nonlinear transformation of image color probability distribution to enhance the linear separability of foreground and background in dynamic scenes, so as to improve the accuracy of moving object detection in dynamic scenes. the

Invention content:

Aiming at the problem that computational vision applications, especially dynamic scene-oriented moving target detection in intelligent video surveillance systems, are easily interfered by environmental noise such as background disturbances and cause false detections, the present invention aims to propose a dynamic scene-oriented detection system based on spatio-temporal condition information. A moving target detection method to suppress disturbing background interference in dynamic scenes and accurately detect moving targets. the

The solution proposed by the present invention is:

1 Construct a spatio-temporal domain model considering the saliency of visual spatio-temporal domain, use the non-parametric probability density estimation method to estimate the conditional probability p(x|b) of the detected image pixel x belonging to the reference background sequence b, and use the negative logarithmic kernel function The conditional probability p(x|b) is nonlinearly transformed to obtain the spatio-temporal condition information I(x|b) of x. Considering the influence of the pixels in the neighborhood of x on it, the weighted summation of the spatio-temporal condition information of the pixels in the neighborhood of x is obtained as This is used as a feature to classify the target and background through a linear classifier;

2 In the spatio-temporal domain model, the color histogram in the reference background domain of the pixel x in the current image is used as the reference background probability distribution to calculate the condition information;

3 Optimize the aforementioned method by using the image block method, and use image blocks (Image Block, abbreviated as IB) instead of single pixels for background modeling and detection;

4 Use the background color histogram of the image block as the background model to replace the background image sequence cached in the spatio-temporal domain model to reduce the demand for data storage space;

5 Use the reference background color histogram of the image block as a reference background model shared by all pixels in the image block, calculate the condition information and weighted summation, and perform image block detection;

6. Using the image block differential pre-detection mechanism, the image block that has changed in the image is detected in advance as a candidate detection area, reducing the data processing amount of the image block detection method based on conditional information;

7 When an image block is detected as the background, update the reference background color histogram of the image block with the color histogram of the current frame image block, and randomly select an image block from the neighborhood of the image block to update the model according to this method, the image block Does not update when detected as a target. the

The moving object detection method based on spatio-temporal condition information for dynamic scenes proposed by the present invention mainly has the following advantages:

1. Negative logarithmic nonlinear transformation is performed on the conditional probability, which increases the linear classification boundary width when using a linear classifier for target detection, enhances the linear separability of the target and the background, and improves the robustness of target detection. the

2. The negative logarithmic nonlinear transformation of the conditional probability has a clear physical meaning, that is, the conditional information I(x|y), which is the uncertainty of the variable x conditioned on y. In the video, conditional on the reference background b, the conditional information I(x|b) of the current observation x measures the ability of the reference background to determine the observation x. In a dynamic scene, the reference background b can completely determine the region without change, partially determine the region of change caused by background disturbance, and it is difficult to determine the region of change caused by object motion. Therefore, using conditional information as the classification feature for moving object detection in dynamic scenes can linearly classify disturbing backgrounds and moving objects. the

3. The conditional information is weighted and summed, which suppresses the influence of isolated noise, enhances the ability to resist disturbance background interference, further enhances the linear distinction between the target and the background, and reduces the classification error. the

4. Considering the visual spatiotemporal saliency to construct a spatiotemporal domain model, which conforms to the psychological characteristics of human vision and is easy to extract interesting motion information. the

5. Using image blocks instead of single pixels for detection reduces algorithm complexity and memory requirements. The image block difference pre-detection mechanism is adopted, and the non-changing area of the image is pre-filtered through a simple algorithm, which reduces the calculation amount of subsequent target detection and speeds up the algorithm speed. the

6. The model update method adopted can effectively adapt to the scene illumination changes without updating the target information to the reference background, which can effectively avoid the problem of missing the tail of the target when the target moves slowly in the sliding window update method. the

7. In general, the present invention not only realizes the effective detection of moving objects in dynamic scenes, but also overcomes the high algorithm complexity, poor real-time performance, large memory requirements, and difficulty in existing moving object detection methods for dynamic scenes. Due to the shortcomings of embedded implementation, it can realize real-time detection of moving objects in dynamic scenes on the existing computer platform, and is suitable for the application of embedded intelligent camera platform. the

Description of drawings:

Figure 1 is a schematic diagram of the background difference algorithm;

where a is the background image;

b is the input image;

c is the difference image;

d is the target mask template, the middle dark area is the foreground, and the remaining light areas are the background. the

Figure 2 is a schematic diagram of the comparison of foreground and background linear separability in the moving target detection algorithm;

Among them, a0-a3 are background difference feature image, conditional probability density feature image, conditional information feature image, and weighted conditional information feature image;

b0-b3 are the image feature distribution histograms of a0-a3, which are calculated with reference to the target mask template b in Figure 1;

c0-c3 are partial enlarged display of the bottom area of b0-b3 respectively;

d0-d3 are the linear classification results of a0-a3 feature images respectively;

e0-e3 are the background difference feature image, conditional probability density feature image, conditional information feature image, and weighted conditional information feature image corresponding to the foreground and background feature distribution histograms of the entire video;

f0-f3 are partial enlarged display of the bottom area of e0-e3 respectively. the

Figure 3 is the visual salience model around the center;

Where a is the image of floating objects on the water surface, where 1 is the central area and 2 is the surrounding reference domain;

b is the Gaussian difference model;

c is the visual saliency map extracted by a under the action of Gaussian difference model b, and the brighter the image, the higher the saliency. the

Figure 4 is the visual salience spatio-temporal domain model;

Among them, 1 is the central region, which corresponds to the central region 1 shown in Figure 3 a;

2 is a surrounding reference domain, which corresponds to the surrounding reference domain 2 shown in a in Figure 3;

3 is the pixel in the image;

4 is the background reference domain of pixel 3. the

Figure 5 is a schematic diagram of image block detection;

where 3 is the pixel in the image;

4 is the background reference domain corresponding to the image block, which corresponds to 4 in Figure 4;

5 is an image block;

6 is the entire image. the

In the above attached drawings:

1-center domain 2-surrounding reference domain 3-pixel 4-background reference domain 5-image block 6-image

Detailed ways:

The basic idea of the present invention to propose a moving target detection method based on spatio-temporal condition information for dynamic scenes is to construct a spatio-temporal domain model in consideration of visual spatio-temporal saliency, use the non-parametric probability density estimation method to estimate the detection image pixel x belongs to Referring to the conditional probability p(x|b) of the background sequence b, the conditional probability p(x|b) is transformed nonlinearly by using the negative logarithmic kernel function to obtain the spatiotemporal conditional information I(x|b) of x. Considering the Influenced by the pixels in the domain, the time-space condition information of the pixels in the x neighborhood is weighted and summed, and used as a feature to classify the target and the background through a linear classifier to complete the moving target detection. In order to reduce the complexity of the algorithm, improve the speed of the algorithm, and reduce the storage space requirements, we use the image block strategy to optimize the aforementioned method. After optimization, the method is tested on a dual-core Intel Pentium Dual CPU E2180 2.0GHz, 1GBRAM computer with a resolution of 640*480 The dynamic scene video rate can reach 26fps (frame per second), which meets the real-time application requirements. the

We consider visual spatiotemporal saliency to construct a spatiotemporal domain model to model the reference background for estimating the reference background color distribution probability and input image object detection. The visual saliency of the human visual system is manifested in spatial saliency and temporal saliency. Spatial visual saliency is reflected in the fact that human eyes pay attention to high saliency areas and ignore low saliency areas when observing images. The receptive field of human retinal ganglion cells is represented by a model around the center, that is, the Difference of Gaussians model (Difference of Gaussians, shown in b in Figure 3). Under the Gaussian difference model, the more obvious the difference around the center, the greater the visual response of the receptive field, and the higher the visual salience of the image in the corresponding area. Specifically, as shown in Figure 3, an image of floating objects on the water surface (a in Figure 3), under the action of the Gaussian difference model (b in Figure 3), obtains a spatial visual saliency map (shown in c in Figure 3), from which From c in 3, it can be seen that due to the obvious difference between the plastic bottle floating on the water surface and its surrounding area (water surface), the response of this area under the Gaussian difference model is large, and the visual significance is high, while the fluctuating water surface is different from the surrounding area (also for water surface), the difference is not obvious, the response of this position under the difference of Gaussian model is small, and the visual salience is low. The salience of time-domain vision is reflected in the fact that when the human eye observes, it is easy to ignore periodic changes (such as shaking leaves, fluctuating water surfaces), and pay special attention to novel (sudden) changes, such as moving targets in the disturbed background. Video is a sequence of images combined in chronological order. Therefore, in video, there are both spatial visual saliency (a single image itself has spatial visual saliency) and temporal visual saliency (changes in image content over time). reflected in temporal visual salience). In videos, frequently occurring changing images have low visual salience, while newly emerging changing images have high visual salience. Compared with the disturbing background (changing images that appear frequently), moving objects often appear as new changing images, which have higher visual salience. Therefore, in the task of video moving object detection, the time domain and spatial domain vision of human vision are considered. Significance, it can effectively filter out background disturbances and improve the detection effect of moving objects in dynamic scenes. the

As shown in Figure 4, the neighborhood 1 of pixel 3 in the input image (CurImg) is used as the central area of the visual attention model around the center, and the surrounding area 2 corresponding to neighborhood 1 is used as the surrounding area of the center surrounding attention model, and The outer boundary of the surrounding domain sets the reference background sequence spatial range 4, and takes N frames of the background sequence (BckSeq, marked as the obliquely shaded area 4 in Figure 4) images as the temporal and spatial domain reference background of pixel 3, and uses this as the reference condition to calculate pixel 3 space-time condition information. the

Calculating the spatio-temporal conditional information requires calculating the conditional probability p(x|b) that the pixel value x belongs to the reference background. We use the kernel density estimation (KDE) method in the non-parametric model to calculate the conditional probability. Its general form is shown in Equation 1. the

$p\left(x\right|S)=\frac{1}{\left|S\right|}\underset{\mathrm{the\; s}\∈S}{\mathrm{\Σ}}K(\mathrm{the\; s}-x)$ Formula 1

Where K is the kernel function, which satisfies ∫K(x)dx=1, K(x)=K(-x), ∫xK(x)dx=0, ∫xx ^T K(x)dx=I _|x| , x is the observation data, S is the reference data set, |S| is the normalization factor indicating the number of data contained in the reference data set S. We use the δ(sx) kernel function instead of the commonly used Gaussian kernel function for kernel density estimation, as shown in Equation 2.

$p\left(x\right|S)=\frac{1}{\left|S\right|}\underset{\mathrm{the\; s}\∈S}{\mathrm{\Σ}}\mathrm{\δ}(\mathrm{the\; s}-x)$ Formula 2

The δ(s-x) kernel function can be quickly calculated using the statistical histogram, therefore, the conditional probability p(x|b) of the pixel x belonging to the reference background b can be quickly calculated through the color histogram of the reference background, as shown in Equation 3, H is Referring to the background color histogram, H(x) represents the value of the pixel x in the color histogram H, and normalizing H(x) gives the conditional probability p(x|b) that the pixel x belongs to the background, where |H| is the normalization factor, which is obtained by summing all the values of the histogram H. the

$p\left(x\right|b)=\frac{1}{\left|h\right|}h\left(x\right)$ Formula 3

In order to avoid the distortion of the probability density estimation caused by the histogram aliasing, we use the Gaussian convolution kernel g to smooth the histogram (see formula 4, H is the reference background color histogram) to improve the probability density estimation accuracy. the

H=H*g Formula 4

The nonlinear transformation of the conditional probability p(x|b) of the pixel x belonging to the background in the detection image can increase the linear classification boundary between the object and the foreground in the dynamic scene and enhance the robustness of the algorithm. Nonlinear transformation methods include exponential transformation, triangular transformation, negative logarithmic transformation, etc. The purpose of nonlinear transformation is to increase the classification boundary between foreground and background. As far as c1 in Figure 3 is concerned, the low value on the left side of the histogram needs to be interval stretches nonlinearly, while intervals with high values on the right perform nonlinear compression. Negative logarithmic transformation just has a very good nonlinear stretching ability in the low-value interval and nonlinear compression ability in the high-value interval, so we use the negative logarithmic check conditional probability p(x|b) for nonlinear transformation. In information theory, the negative logarithmic transformation of the conditional probability p(x|b) is to calculate the conditional information I(x|b) of the variable x under the condition b. The conditional information has a clear physical meaning, which means that under the condition b Under the uncertainty of x. In the video, conditional on the reference background b, the conditional information I(x|b) of the current observation x measures the ability of the reference background to determine the observation x. In a dynamic scene, the reference background b can completely determine the region without change, partially determine the region of change caused by background disturbance, and it is difficult to determine the region of change caused by object motion. Therefore, conditional information is a very effective classification feature for detecting and classifying moving objects in dynamic scenes, and using conditional information can linearly classify disturbing backgrounds and moving objects. the

The conditional probability density p(x|b) is nonlinearly transformed by negative logarithmic check to obtain a new image feature I(x|b), as shown in formula 5. The distribution of image features has local consistency, that is, image pixels are not isolated, but there are connections between them and pixels in the neighborhood. The image characteristics of pixel x will be affected by the image characteristics of pixels in the neighborhood. As shown in Equation 6, we weight and sum the condition information I(x _kl |b) of all pixels x _kl in the neighborhood of x as the condition information of pixel x I'(x|b), and then use the linear classifier (Formula 7) to classify the foreground and background through the threshold τ (usually 5). In Formula 6, α _kl is the weighted weight, and you can choose a uniform weight α _kl =1/(BL*BL) (BL is the neighborhood width), or a Gaussian kernel weight, or the pixel x _kl in the neighborhood color distribution Proportions are used as weights, and we use uniform weights in the present invention.

I(x|b)=-logp(x|b) Formula 5

$I^{\′}\left(x\right|b)=-\underset{k=1}{\overset{\mathrm{BL}}{\mathrm{\Σ}}}\underset{l=1}{\overset{\mathrm{BL}}{\mathrm{\Σ}}}{\mathrm{\α}}_{\mathrm{kl}}\log \left(p\left(x_{\mathrm{kl}}\right|b)\right)$ Formula 6

$x=\left\{\begin{array}{cc}1& I^{\′}\left(x\right|b)>\mathrm{\τ}\\ 0& \mathrm{else}\end{array}\right.$ Formula 7

The aforementioned target detection method based on spatio-temporal condition information needs to calculate the reference background color histogram corresponding to each pixel, and perform weighted summation of the condition information in the neighborhood of each pixel, which has high computational complexity and poor real-time performance of the algorithm. Next, we will use the image block as a unit instead of a single pixel for modeling and acceleration to reduce algorithm complexity and storage space requirements, and improve algorithm speed. the

The moving target in the video shows local consistency in both the time domain and the space domain. Compared with a single pixel, the image block can better reflect the aforementioned local consistency. Therefore, using image blocks to model the reference background and detect moving objects in units of image blocks can not only reduce the complexity of the algorithm, reduce the amount of calculation, and reduce the storage space requirements, but also can better suppress isolated noise interference without affecting the target. Detection accuracy. the

Different from the previous method of constructing a reference background domain for each pixel, we divide the image into blocks, and use the background color distribution of the image block as the reference distribution shared by the pixels in the image block, which can reduce the number of color histograms and reduce the complexity of the algorithm . As shown in Figure 5, all pixels in the image block 5 share a reference background domain 4, and share the background color distribution calculation condition information of the reference background domain 4, and carry out weighted summation of the condition information of all pixels in the image block 5, as For the classification features of the image block 5, the image block is classified by a linear classifier, and the classification detection method is the same as the above (Formula 7). the

In the aforementioned spatio-temporal domain model (Figure 4), it is necessary to cache N frames of background sequence images, and calculate a new reference background color histogram for each detection. Caching N-frame image sequences requires a large storage space, which is difficult to apply on embedded platforms with limited storage space. The reference background color distribution does not change every frame, and there is no need to recalculate the reference background color histogram with N frames of cached images every time detection is performed. We use the image block as a unit to calculate the color histogram of the N-frame cache image, and use it as the background data of the image block to replace the original N-frame cache image, which can reduce the storage space requirement, and update the color histogram through the color histogram. Ability to adapt to scene lighting changes. During detection, the reference background distribution of the image block 5 is obtained by directly summing the color histograms in the reference background area 4 corresponding to the image block 5 . the

In practical video surveillance applications, most areas in the image appear as static scenes most of the time, that is, the image in this area is relatively stable and less likely to change, such as building surfaces, ground, and other immobile objects. For a large number of static scene areas in the image, it is not necessary to use a dynamic scene-oriented target detection method for detection, but the simplest background difference method can be used to determine whether the area has changed, reducing the amount of data processing based on the conditional information method , which can increase the speed of practical applications. the

Image difference is the simplest detection method of image change area. Through image difference, the changed area in the image can be pre-determined as a candidate area, and then further processed by the method based on conditional information to identify whether it is a disturbed background or a real moving target. Compared with single-pixel difference, image block difference can better suppress isolated noise, and it is easy to combine with image block condition information detection method. Image block differential pre-detection can quickly detect image blocks that have changed in the image as candidate detection areas, reducing the amount of data for image block condition information detection. The image block difference is to calculate the SAD (Sum of Absolute Difference) of each pixel block, as shown in formula 8, to calculate the absolute difference sum of the image blocks at the corresponding positions of the input image and the background image, where BL is the image block width, m, n represents the position of the image block in the image. By binarizing the image block difference results (Formula 9, T is the binarization threshold), the image change area can be detected in advance as a candidate detection area. the

$\mathrm{SAD}(m,\mathrm{no})=\underset{x=1}{\overset{\mathrm{BL}}{\mathrm{\Σ}}}\underset{\mathrm{the\; y}=1}{\overset{\mathrm{BL}}{\mathrm{\Σ}}}|I(m\×\mathrm{BL}+x,\mathrm{no}\×\mathrm{BL}+\mathrm{the\; y})-B(m\×\mathrm{BL}+x,\mathrm{no}\×\mathrm{BL}+\mathrm{the\; y})|$ Formula 8

$\mathrm{IB}=\left\{\begin{array}{cc}1& \mathrm{SAD}>T\\ 0& \mathrm{else}\end{array}\right.$ Formula 9

In order to adapt to the scene lighting changes, the background model needs to be updated. The model update includes two aspects, updating the image block background color histogram in the condition information detection method, and updating the reference background in the image block differential pre-detection algorithm. For the update of the color histogram, the specific update strategy is: only when the current detection area is the background, the histogram is updated (Formula 10). The background color histogram is optionally updated. Specifically, an image block is randomly selected from the neighborhood of the current image block with a certain probability, and the background color histogram of the selected image block is updated with the color histogram of the current image block in the current frame. The update method is the same as before (Formula 10) . The model is updated only when the current detection area is the background, which can adapt to the scene illumination changes, but will not update the moving target information to the background by mistake, and can effectively overcome the time sliding window update and integrate the target information into the reference background. The problem of missed detection. While updating the current area, selectively updating its neighborhood background can overcome the problem of false detection caused by only updating the background area and not updating the foreground area, because the area detected as the foreground is updated by its neighborhood background , after a period of accumulation, the background is updated when moving in and out of the target area. the

Formula 10 actually weights and sums the background color histogram H _mn of the image block and the color histogram H _mnc in the current frame as a new reference background distribution H _mn , where β ₀ is the weighted weight. Larger updates are faster.

H _mn =H _mn *(1-β ₀ )+H _mnc *β ₀ Formula 10

Different from the histogram update method in the conditional information method, we adopt the strategy of fast background update and slow foreground update to update the reference background in the patch differential pre-detection. Based on the final target detection result, it is updated according to formula 11. When the image block is detected as the foreground, it is updated slowly at the rate β ₁ , and when the image block is detected as the background, it is quickly updated at the rate β ₂ , where β ₁ <β ₂ . Since the image block difference is pre-detection, in order to better adapt to environmental changes, we not only update the background but also update the foreground here, but the update rate is very slow. The actual application test proves that this method will not cause the target Missed detection, and can better adapt to environmental changes and deal with the problem of moving objects in and out.

$b=\left\{\begin{array}{cc}x*(1-{\mathrm{\&beta;}}_1)+b*{\mathrm{\&beta;}}_1& \mathrm{if\; x}=f\\ x*(1-{\mathrm{\&beta;}}_2)+b*{\mathrm{\&beta;}}_2& \mathrm{else}\end{array}\right.$ Formula 11

The basic flow of the moving target detection method in dynamic scenes based on spatio-temporal condition information is as follows:

1 Model initialization

Obtain the background image as the reference background for image patch differential pre-detection. For the image block, on the reference background image, calculate the color histogram of each image block IB _mn in the background image as the initial background color histogram H _mn of the image block.

2 image block differential pre-detection

Use Formula 7 to calculate the absolute value sum of the block image, and use Formula 8 to perform thresholding to pre-detect the image change image block as a candidate detection area. the

3 Secondary detection of candidate image blocks

On the candidate detection area obtained by pre-detection of image block difference in step 2, the condition information is used for secondary detection. First calculate the reference background histogram H of the candidate image block, then calculate the condition information of all pixels of the image block and perform weighted summation, and finally perform binarization by formula 7 to obtain the target detection result image (BinImg). the

4Model update

The model is updated according to the target detection result (BinImg), including updating the reference background and the color histogram of the image block. According to formula 11, the reference background is updated. When updating the color histogram of the image block, it is first necessary to calculate the color distribution histogram H _mnc of each image block in the current frame, and then update the background color histogram of the image block according to the aforementioned update method.

Claims (8)

1. A moving target detection method based on spatio-temporal condition information, characterized in that: the conditional probability p(x|b) of the pixel point (3) in the detection image belonging to the reference background is subjected to nonlinear transformation, and used as the classification feature of the moving target detection, Foreground and background classification in dynamic scenes with a linear classifier. 2. The moving target detection method based on spatio-temporal condition information according to claim 1, characterized in that the non-negative logarithm is used to perform nonlinear transformation on the conditional probability p(x|b) to obtain the pixel point (3) in the reference The conditional information I(x|b) under the background condition is used as the image classification feature. 3. The moving target detection method based on spatio-temporal condition information according to claim 2, characterized in that the condition information I(x|b) of the pixels in the neighborhood of the weighted pixel (3) is used as the detection pixel (3) final classification features. 4. The moving target detection method based on spatio-temporal condition information according to claim 1 or 3, characterized in that the conditional probability p(x|b) is calculated using the spatio-temporal domain model around the center, and the pixel points (3) are calculated accordingly The weighted condition information of . The spatio-temporal domain model, with the current detection pixel point (3) as the center, constructs the central area (1), and determines the reference background with the surrounding area (2) corresponding to the central area (1): the outer boundary of the surrounding area (2) All N-1 frames of background sequence images BckSeq (4) within the range, together with the surrounding area 2 in the current detection image CurImg as the reference background b, calculate the conditional probability p(x|b). Take the central area (1) as the neighborhood of the pixel point (3), and calculate the weighted condition information. 5. The moving target detection method based on spatio-temporal condition information according to claim 4, characterized in that the image is divided into blocks, and the detection is accelerated by using image blocks (5) instead of a single pixel (3), and the image is divided into blocks , using the weighted sum of the conditional information of the pixel x in the image block as the feature of the image block for detection and classification. 6. The moving target detection algorithm based on spatio-temporal condition information according to claim 5, characterized in that, the reference background probability distribution is directly modeled by using the image block color distribution histogram, so as to refer to the color distribution histogram of the image block in the background sequence Figure H is used as the probability distribution p(b) of the reference background b, and the conditional probability p(x|b) is calculated directly using the histogram H. 7. The moving target detection algorithm based on spatio-temporal condition information according to claim 5, characterized in that, the image block difference pre-detection is used to pre-detect the changed image block in the image as a candidate image, and then the condition information is used for secondary detection. 8. The moving target detection method based on spatio-temporal condition information according to claim 6, wherein when an image block is detected as background, the reference background color histogram of the image block is updated, and the image block in the neighborhood is randomly selected The selected color histogram is updated with reference to the background color histogram. the

Images