KR101106448B1 - Real-Time Moving Object Detection For Intelligent Visual Surveillance - Google Patents

Abstract

The present invention relates to a real time implementation of a detection method in a moving object detection method.

Moving object detection refers to finding background and other foreground objects in the input image and is a necessary process in many image processing applications such as intelligent video surveillance, HCI, and object-based image compression. Existing object detection algorithms require significant computational complexity, making them difficult to use in multichannel video surveillance applications or single channel real-time applications in embedded systems. Moving object detection involves the following steps: Extract foreground mask, correct foreground mask, and segment blobs by connecting element labeling. Foreground mask correction, a process necessary for more accurate object detection, is usually performed through morphology operations such as open and closed. Linking element labeling distinguishes between blobs, which are sets of linked pixels in the foreground mask. Morphological operations are not computationally expensive and differ in processing methods, making them difficult to process in parallel with connection element labeling routines. In the present invention, first, unlike the morphology calculation, the foreground mask correction routine using the peripheral pixel checking process used in the connection element labeling procedure is developed, and the foreground mask correction and the connection element labeling can be performed in parallel using the developed routine. An improved real-time implementation algorithm is provided. As a result of the improvement, it was confirmed through experiments that the moving object detection method provided by the present invention is processed more accurately and quickly than the moving object detection method using the existing morphology calculation.

Moving object detection, blob detection, connection element labeling, foreground mask correction, intelligent video surveillance

Description

Real-Time Moving Object Detection For Intelligent Visual Surveillance}

TECHNICAL FIELD The present invention belongs to the field of moving object detection required for image processing applications such as intelligent video surveillance, HCI, and object-based video compression. The present invention provides an improved moving object real-time detection technique and an implementation method for detecting a moving object different from a background in an acquired video faster and more accurately than a conventional method.

Moving object detection refers to finding a background and other foreground object in the input image, which is a necessary process in various image processing applications such as intelligent video surveillance, HCI, and object-based image compression. For example, intelligent video surveillance requires analysis of the behavior of objects of interest. The object of interest is derived from moving object detection and it is well known that the performance of intelligent video surveillance depends on how fast and accurately it can detect moving objects [Ref. 1]. However, the existing morphological algorithm using a moving object detection algorithm requires a considerable amount of computation and is difficult to use for real-time application of a single channel in a multi-channel video surveillance application or an embedded system. Moving object detection goes through the following procedure [1, 2] [see Figure 2]: foreground mask extraction, foreground mask correction, blob segmentation. Foreground masks represent extracted foreground pixels. For accurate object extraction, a foreground mask correction step is required to correct incorrectly extracted or unextracted ones. A morphology paper operation such as open / close is usually performed as a preprocessing step. 4,5]. On the other hand, since the foreground mask may include a number of blobs, it is necessary to divide the blobs and divide them, and they are processed using a connection element labeling algorithm [5]. A blob is a set of connected foreground pixels. Subsequently, a minimum rectangular area including each of the divided blobs is calculated and these minimum rectangular areas are detected as the object area.

However, the morphology operation used in the foreground mask correction process is not small and the processing method is different so that it is difficult to be processed simultaneously with the connection element labeling processing routine, and is different from the method used in the connection element labeling processing routine. After the correction process, the connection element labeling process is performed sequentially. Since the morphology operation itself does not have a small amount of calculation and the foreground mask correction and the connection element labeling are sequentially processed, the conventional morphology calculation using moving object detection method takes a lot of time.

The present invention provides a new real-time moving object detection technique and an implementation method different from those provided by the existing methods [Documents 1 and 2]. The provided moving object detection technique and implementation method is faster and more accurate than conventional methods using morphology operations. The present invention first provides a foreground mask correction algorithm utilizing the surrounding pixel check. The foreground mask correction algorithm, which is provided by the present invention, uses eight peripheral pixel checks, and the algorithm is simpler than the foreground mask correction algorithm using morphology calculation. In addition, the eight peripheral pixel checks are also used in the connection element labeling routine, so that "peripheral foreground pixel propagation" can be included in the connection element labeling routine and processed. Therefore, foreground mask correction is a simpler algorithm, which is processed together in the link element labeling process. Therefore, foreground mask correction is performed in comparison with the existing object detection methods which process the foreground mask correction process using morphology operation first and then link element labeling process sequentially. Brings a great improvement of time. Through experiments, it is confirmed that the proposed moving object detection method is processed more accurately and faster than the existing method.

[1] T. Chen, H. Haussecker, A. Bovyrin, R. Belenov, K. Rodyushkin, A. Kuranov, V. Eruhimov, "Computer Vision Workload Analysis: Case Study of Video Surveillance Systems", Intel Technology Journal, May 2005.

-Paper content;

An analysis of the execution throughput of computer vision algorithms implemented through an example of a video surveillance application is described and described.

C. Stauffer C, W.E.L Grimson, "Adaptive background mixture models for real-time tracking", IEEE CVPR, 1999, pp. 244-252.

-Paper content;

We propose a mixed Gaussian model for moving object detection based on background difference technique in intelligent video surveillance, and describe the results confirmed through experimental results that the detection of moving object based on mixed Gaussian model is an improvement over the existing method.

P. Soille, Morphological Image Analysis: Principles and Applications, Springer-Verlag Telos, Berlin, 2003.

-Paper content;

This book provides a detailed explanation of morphology operations.

R. C. Gonzalez, R. E. Woods, and S. L. Eddins, Digital Image Processing using MATLAB, Prentice Hall, Upper Saddle River, New Jersey, 2003.

-Book content;

This book explains the theory and techniques of image processing, and provides a detailed explanation of the use of morphology computation in image processing.

[4] E. R. Dougherty and R. A. Lotufo, Hands-on Morphological Image Processing, SPIE Press, Bellingham, 2003.

-Paper content;

A detailed description of the morphology computation in image processing has been provided.

L. Shapiro and G. Stockman, Computer Vision, Prentice Hall, Upper Saddle River, New Jersey, 2001.

 -Book content;

An explanation of computer vision theory and techniques, this book provides a detailed description of the use of morphology operations and the labeling of connected elements in computer vision.

Existing moving object detection methods [Documents 1 and 2] use morphology operations for foreground mask correction. First, morphology operations are large in computational complexity. Secondly, since parallel processing cannot be performed because there is nothing in common with the operations required in the connection element labeling routine, which is a moving object preprocess, the existing moving object detection method using morphology operations is multi-channel. Or real-time implementation in embedded systems is difficult. The present invention provides a more rapid and accurate real-time moving object detection method that solves the problem of excessive execution time of the existing morphology-based moving object method. This paper aims to solve technical obstacles in developing a video surveillance system and real-time implementation of object-based video compression.

Currently, the connection element labeling process required for moving object detection is performed in two steps. Step 1 scans the binary image row-by-row, left to right, if it encounters a foreground pixel during scanning, checking if there are surrounding foreground pixels among the already scanned pixels, and if not, the largest label (number) ever assigned to that foreground pixel. Assign the label after the label, and assign the lowest label among the labels of the scanned foreground pixels, if any. At this time, the labels of the surrounding foreground pixels all belong to the same equivalent class and the found equivalent class is recorded. In step 2, the binary image is rescanned and the recorded equivalent label class is used to replace the label assigned in step 1 with the final label.

Developing and providing 'foreground pixel propagation' algorithm, a foreground mask correction technique with a small amount of computation that can be performed during the process of checking the presence of surrounding foreground pixels in the first step of linking element labeling, Replace the morphology calculation process that had to be performed sequentially before executing the routine. As a result, the foreground mask correction process is processed simultaneously with the connection element labeling operation with a small amount of computation without using a large amount of computational morphology operations, thereby enabling rapid real-time processing of moving object detection.

Experiments and results performed to clearly describe the effects of the present invention are described and the effects are summarized.

end. Comparison of the foreground mask correction capability of the present invention 'foreground pixel propagation' algorithm and the conventional morphology calculation method

The foreground mask correction capability is compared with the existing Morphoroll paper calculation and the 'peripheral foreground pixel propagation' algorithm provided by the present invention. 3 shows the results of this comparison. FIG. 3 shows that the 'peripheral foreground pixel propagation' algorithm is simpler than the morphology calculation method and has better foreground mask correction capability. The foreground mask in FIG. 3 was obtained using a mixed Gaussian model, and the morphology calculation used square structure elements of size 3x3 as close and open. If more appropriate structural elements are used, the use of morphology calculations can yield more explicit foreground mask corrections, as is the case with the 'foreground pixel propagation' technique. However, more computational computation is needed in this case.

I. Execution time time comparison

Two experiments were performed to compare the execution time of each case using the existing morphology calculation using the moving object detection algorithm (hereinafter, referred to as an existing method) and the present invention.

In the first experiment, the moving object detection time was calculated for one frame of the foreground mask of FIG. 3, and in the second experiment, the execution time of continuous moving object detection was calculated and compared for the video image. Two experimental environments were Windows PCs with 3GHz Pentium 4 cores and 3GB DDR2 memory. In addition, the resolution of each video image frame is 320x240, and the sizes of video videos 1 and 2 are 4.67 MB (221 frames) and 245 MB (1117 frames), respectively.

The PC specification used in this paper was Intel Pentium 4 (3GHz), 3GB DDR2 RAM, and the program integration environment used was visual studio 2008.

4 (a) and 4 (b) show the results of the first and second experiments, respectively.

The results of FIG. 4 confirm that the providing method is performed faster than the existing method.

All. Effects of the Invention

The present invention provides a moving object detection method using a 'foreground pixel propagation' technique, which is an improvement over the moving object detection method using a conventional morphology calculation. Experimental results show that the proposed moving object detection method improves execution speed and accurate object detection capability over the existing morphology-based moving object detection method. With the improved moving object detection method provided, it is possible to develop a multi-channel intelligent video surveillance system, to develop an embedded intelligent video surveillance system, and to improve the performance of the object-based video compression method.

The operation structure of the 'peripheral foreground pixel propagation' based moving object detection method provided by the present invention is shown in FIG. 1. The present invention provides a 'foreground foreground pixel propagation' algorithm, which is simpler than the morphology calculation, and employs the sequential processing of 'morphology calculation for foreground mask correction + connection element labeling for blob division' and utilizes the 'foreground mask correction' process and ' It provides a method to perform fast and accurate real-time object detection by allowing the connection element labeling 'process to be performed simultaneously. In order to understand the contents of the present invention, an understanding of the connection element labeling process is first required. For more information on the connection element labeling process, refer to [Reference 5], and after introducing the 'union-find structure based connection element labeling' algorithm related to the present invention, Explain.

end. Introduction to the Union-Find Structure-Based Linked Element Labeling Algorithm

The connecting element labeling process scans an image and divides the blobs based on the concatenation of the pixel pixels. A blob is a collection of concatenated (foreground) pixel pixels. In the foreground mask image, the connecting element is a collection of foreground pixels with the foreground pixel connected to it. In the present invention, the periphery of a pixel means eight peripheries of the pixel (see FIG. 5).

Once all connecting elements are extracted, each pixel is assigned a label (number) of the connecting element to which it belongs. Extracting separate connection elements from images and labeling them is one of the important tasks required in many computer vision applications, including intelligent video surveillance.

Of the many connection element labeling problem handling algorithms proposed in the literature [see reference 5], the most promising and efficient connection element labeling algorithms usually consist of a two-step procedure using a union-find structure. All.

The union-find structure is used to effectively build and manipulate equivalence classes in a tree structure. It stores a collection of disjoint sets, combines two sets into one, and finds To determine which set belongs to). Each set is stored in a tree structure. The nodes in the tree represent the labels of each set, and their values point to the parent node. The union-find tree structure is represented by the vector array PARENT. The index of the PARENT represents a possible label (node) and its value represents the label of the parent node. A node with a value of zero means the root node.

The find operation, given a label X and a PARENT vector array, continues to traverse the parent node of the node represented by label X to find the label of the root node of the tree with label X. The union operation transforms the tree structure and adds X and Y when given two arrays of labels X and Y and a PARENT vector.

Union operations look for the root node of each label, starting at labels X and Y respectively, and if the root nodes of X and Y are different, make one label (X or Y) the parent node of the other (Y or X). Figure 6 shows the union-find structure for two label sets.

Below is a brief description of the two-step procedural linking element labeling algorithm using a union-find structure.

In the first step, the algorithm scans the binary image row by row from left to right. If it encounters a foreground pixel during scanning, it checks whether there is a surrounding foreground pixel among the already scanned pixels (check the surrounding foreground pixel), and if not, assigns the next label with the largest label ever assigned to that foreground pixel, Assign the lowest label among the labels of the surrounding foreground pixels. At this time, the labels of the surrounding foreground pixels all belong to the same equivalent class. The found equivalent class is recorded in the union-find structure. At the end of the first step procedure, each equivalence class has a unique label that is fully determined and determined by the root of the tree of the union-find structure.

The second step procedure scans the image and replaces the temporary labels with the final labels. The final label is the label of the root node of each label.

Figure 7 shows an example input binary image. In FIG. 7, f denotes a foreground pixel, and other pixels are background pixels.

FIG. 8 shows the result of performing the first step procedure for [FIG. 7] of the 'connecting element labeling using union-find structure' algorithm.

FIG. 9 shows the final labeling result after the two-step procedure for [FIG. 7] of the 'connected element labeling using union-find structure' algorithm.

I. Surround foreground pixel propagation algorithm

Natural objects are less likely to have holes and less protruding parts. Also, isolated fractional pixel sets are less likely to be objects of interest. The blobs of the foreground mask image are the blobs of the natural object. Thus, the hole in the foreground area visible in the foreground mask image is very likely a foreground pixel that has not been extracted. Also, isolated foreground pixels or protruding foreground pixels are most likely incorrectly extracted as foreground pixels. Therefore, holes in the foreground mask image are more likely to have many foreground pixels around, and protruding foreground or isolated foreground pixels are more likely to have background pixels around.

Here, the foreground periphery probability P (X) of the pixel X is defined as the ratio of the foreground pixel among the eight peripheries of the pixel X (see FIG. 5).

From this observation, the present invention provides an 'peripheral foreground pixel propagation' algorithm, which is a foreground mask correction method.

Peripheral foreground pixel propagation algorithm

In a foreground mask image with two pixel values (value 1 means foreground pixel and value 0 means background pixel), foreground mask correction is performed as follows.

When scanning a foreground mask image row-by-row from left to right, the correction of that pixel is to the foreground pixel (value 1) if the foreground margin is greater than 0.5, and to the background pixel (value 0) if the probability is less than 0.5: do. In other words,

An example of performing the 'peripheral foreground pixel propagation' algorithm is shown in FIG. 10.

FIG. 11 shows an example of an image of the result of applying the 'peripheral foreground pixel propagation' algorithm to the foreground mask obtained using the adaptive Gaussian mixture model [Document 2]. We can see that the blobs in the foreground mask are more apparent and filled after the 'peripheral foreground pixel propagation' algorithm is applied.

The Peripheral Foreground Pixel Propagation algorithm demonstrates a simple but good foreground mask correction. In addition to this good foreground mask correction property, a more important feature is that it can be performed during the connection element labeling process routine processing. This is because both the Peripheral Foreground Pixel Propagation algorithm and the Connected Element Labeling algorithm both utilize surrounding foreground pixel checks.

These characteristics can reduce a lot of computation in moving object detection. Based on these characteristics, this paper provides a new moving object detection method. The proposed moving object detection method is faster and more accurate than the existing morphology calculation based moving object detection algorithm.

All. Real-time moving object detection method based on surrounding foreground pixel propagation

The surrounding foreground pixel propagation-based moving object detection method provided by the present invention combines a union-find structure-based connection element labeling method and a surrounding foreground pixel propagation method. The operation structure is as shown in FIG. 1.

The provided surrounding foreground pixel propagation based moving object detection method is performed in a two-step procedure as with the existing union-find connection element labeling. However, unlike the conventional method in which union-find connection element labeling is performed after preprocessing morphology calculation, 'peripheral foreground pixel propagation' algorithm, which is a simpler foreground mask correction method, is executed simultaneously in step 1 during union-find connection element labeling. The amount of computation is reduced. In the first step, the foreground mask is corrected using the connection element labeling, 'peripheral foreground pixel propagation', and the union-find structure is constructed, and the label of the temporarily allocated blobs is corrected using the union-find structure constructed in the second step. The minimum rectangular area containing the blobs of each label is then calculated.
la. Implementation method of real time detection of moving object for intelligent video surveillance of the present invention
The moving object real-time detection method for intelligent video surveillance provided by the present invention combines a union-find structure-based connection element labeling method and a surrounding foreground pixel propagation method as a moving object detection method based on surrounding foreground pixels. ] And is implemented in two steps (20 and 30 steps of [Fig. 1]).
First, if a binary foreground mask (10 of FIG. 1) extracted by adaptive mixed Gaussian modeling is input, in the first stage (20 of FIG. 1) of real-time detection of moving object for intelligent video surveillance of the present invention The pixels around 8 are checked (step 21 of FIG. 1), and the foreground mask is corrected using the 'peripheral foreground pixel propagation algorithm' described in step 2) (step 22 of FIG. 1). At this time, the union-find structure-based primary connection element labeling is performed (step 23 of FIG. 1). The label of the temporarily assigned blobs is finally determined using the union-find structure constructed in step 2 (30 in FIG. 1) (step 31 in FIG. 1). Then, the minimum rectangular area including the blob of each label is calculated (step 32 in FIG. 1). At this time, a list of moving objects is created and output ((step 40 of FIG. 1)).

The present invention can be applied to the development of multi-channel intelligent video surveillance system, embedded intelligent video surveillance system, object-based video image compression, image processing and computer vision application software and system development requiring moving object detection such as HCI.

1) FIG. 1 is a view illustrating an operation structure of a moving object detection method based on 'peripheral foreground pixel propagation' provided by the present invention.

2) FIG. 2 shows images of a moving object detection process, (a) an original image, (b) a foreground mask image, (c) a corrected foreground mask image, and (d) a detected moving object image

3 is a comparison result image of the foreground mask correction ability of the present invention 'foreground pixel propagation' algorithm and the conventional morphology calculation method

4 is a comparison result of execution time between the present invention-provided moving object detection method and the existing Morphoroll paper-based moving object detection method. FIG. 4 (a) shows the first test result, and FIG.

5) 8 peripheral pixels of the pixel

6) Figure 6 shows a union-find structure for two label sets.

7) Figure 7 is an example input binary image

8) FIG. 8 shows the result of performing the first step procedure for [FIG. 7] of the 'connection element labeling using union-find structure' algorithm.

9) FIG. 9 shows the final labeling result after the two-step procedure for [FIG. 7] of the 'connecting element labeling using the union-find structure' algorithm.

10) FIG. 10 shows an example of performing the 'peripheral foreground pixel propagation' algorithm provided by the present invention described in b) of [the detailed description of the invention].

11) FIG. 11 shows an example of an image of the result of applying the 'peripheral foreground pixel propagation' algorithm to the foreground mask obtained using the adaptive Gaussian mixture model [Document 2].

Claims (3)

In the moving object real-time detection method for intelligent video surveillance, Extracting a binary foreground mask; Checking surrounding pixels when the extracted binary foreground mask is input; Correcting the binary foreground mask using an ambient foreground pixel propagation algorithm; Performing primary connection element labeling; Determining a label of temporarily assigned blobs; And Calculating a minimum rectangular area including a blob of each label, and creating a moving object list; The surrounding foreground pixel propagation algorithm is When scanning a foreground mask image row by row in a foreground mask image having two pixel values, if the foreground margin probability is 0.5 or more, the pixel is corrected as the foreground pixel, and if the foreground margin probability is less than 0.5, the background pixel is corrected. Moving object real-time detection method for intelligent video surveillance. The method of claim 1, The performing of the primary connection element labeling may include performing connection element labeling using a union-find structure, and performing the connection element labeling using the union-find structure may include: Scanning the binary image row by row from left to right; If it encounters a foreground pixel during scanning, checking whether there is a surrounding foreground pixel among the already scanned pixels; If the surrounding foreground pixel does not exist, assigning a next label after the largest label that has already been assigned to the corresponding foreground pixel, and if the surrounding foreground pixel exists, assigning the lowest label among the labels of the scanned surrounding foreground pixels; And Recording the labels of the surrounding foreground pixels belonging to an equivalent class and recording the found equivalent class in a union-fine structure; Moving object real-time detection method for intelligent video surveillance comprising a. 3. The method of claim 2, The performing of the connection element labeling using the union-find structure further comprises the step of scanning an image and replacing the temporary label with the final label.

Images