KR100364582B1 - System tracking and watching multi moving object - Google Patents

Abstract

The present invention relates to a system for tracking / monitoring a moving object using a wide range surveillance camera and a local surveillance camera. The multiple moving object tracking / monitoring system uses a fixed camera to monitor a plurality of moving objects existing within a surveillance range. Simultaneously extracting movements and registering these information, selecting at least one or more moving objects to be tracked from among the plurality of registered moving objects, tracking the selected at least one moving object in a first surveillance region, and selecting the selected A wide area monitoring apparatus outputting a moving object tracking command when at least one moving object is out of the first monitoring area; And at least one local surveillance camera moving in a pan / tilt and monitoring independent second surveillance zones, wherein the tracking instructions for the at least one selected moving object are input from the global surveillance apparatus. And a regional monitoring device configured to sequentially drive the surveillance camera to track and zoom until the selected moving object leaves the second surveillance area, wherein the wide area monitoring device and the local monitoring device move the selected at least one moving object. Simultaneously extract the movements of objects to effectively separate / merge the moving areas of each moving object to perform tracking / monitoring in real time.

Description

System tracking and watching multi moving object}

The present invention relates to an object tracking / monitoring system, and more particularly, to a multiple moving object tracking / monitoring system which simultaneously tracks / monitors a plurality of moving objects in real time using a wide-range surveillance camera and a local surveillance camera.

The problem of the conventional object tracking system is that it is impossible to identify a multi-animal object such as a person, an animal, a vehicle, an aircraft, etc., and additional equipment and additional costs are required to change the surveillance area.

In addition, the sensor itself is a single function (line monitoring, screen monitoring), expensive to build and complicated system.

An object of the present invention is to provide a multiple moving object tracking / monitoring system for detecting and tracking a plurality of moving objects for building a real-time monitoring system.

1 illustrates a configuration of a multiple moving object tracking / monitoring system according to the present invention.

Figure 2 shows the configuration of a wide area monitoring device in a multiple moving object tracking / monitoring system according to the present invention.

Figure 3 shows the configuration of the area monitoring device in a multiple moving object tracking / monitoring system according to the present invention.

4 illustrates a motion region extraction (left) using an input image and a motion region extraction (right) using an filtered image.

5 illustrates an input image, a disturbance map, and a motion region extraction result over time.

6 shows a comparison of the before (left) and after (right) of binary filtering.

7 illustrates a comparison between the front (left) and the rear (right) of an object merge.

8 shows a flowchart of a Kalman filter.

9 shows the projection results in the vertical and horizontal directions for a two-dimensional image.

10 shows a vector representation of a projection signal and its vector sum.

11 illustrates object tracking by template matching.

12 shows an input image and a background image in the initial frame.

13 shows an input image and a background image after a certain time.

14 illustrates a motion extraction image by using a difference between an input image and a background image.

15A shows the edge image in the current incoming frame, and FIG. 15B shows the resulting image.

16 shows a template (upper) registered in the previous frame and a template (lower) found in the current frame.

17A to 17C show a comparison of motion extraction results before and after camera motion correction, FIG. 17A is an input image, FIG. 17B is a motion extraction result before camera motion correction, and FIG. 17C is a motion extraction result after camera motion correction. It is shown.

18A to 18B illustrate motion vector extraction using block matching. FIG. 18A shows a previous frame and FIG. 18B shows a current frame.

19 illustrates a result of extracting a motion region using block matching.

20A to 20E illustrate a motion extraction process using an input image and a background image. FIG. 20A shows an input image and a background image in an initial frame, FIG. 20B shows an input image and a background image after a predetermined time, and FIG. 20C The motion image extracted by using the difference between the input image and the background image, FIG. 20D illustrates an edge image in a current incoming frame, and FIG. 20E illustrates a result image.

21 shows a difference image between each frame.

FIG. 22 illustrates a contour image of a moving object extracted using a current image and a difference.

FIG. 23 shows a table showing camera movements frame by frame.

24 illustrates an image being input to the camera.

25 illustrates an independently moving object after correcting the movement of the camera.

The multiple moving object tracking / monitoring system according to the present invention for solving the above technical problem, extracts the movement of a plurality of moving objects existing in the monitoring range by using a fixed camera at the same time to register the information, and the registered Selecting at least one moving object to be tracked from among a plurality of moving objects, tracking the selected at least one moving object in a first surveillance region, and selecting the at least one movable object out of the first surveillance region A wide area monitoring device for outputting a moving object tracking command; And at least one local surveillance camera moving in a pan / tilt and monitoring independent second surveillance zones, wherein the tracking instructions for the at least one selected moving object are input from the global surveillance apparatus. And a regional monitoring device configured to sequentially drive the surveillance camera to track and zoom until the selected moving object leaves the second surveillance area, wherein the wide area monitoring device and the local monitoring device move the selected at least one moving object. Simultaneously extracting the movements of the object to effectively separate / merge the moving area of each moving object to perform tracking / monitoring in real time. The wide range monitor, the fixed camera, the fixed camera Image input unit for receiving an image from the input, the input image Image correction to correct the image through filtering, a motion detector for detecting a portion in which the motion is generated by separating the motion region and the background region from the corrected image, and motion region information of each moving object in the extracted motion generation portion. A moving object extraction unit for separating / merge and simultaneously extracting the movements of the at least one selected moving object by using a plurality of moving objects, and a prediction algorithm and an information matching technique using the Kalman filter It characterized in that it comprises a moving object tracking unit to track using.

In addition, the local monitoring device, at least one pan / tilt camera moving in a pan / tilt, an image input unit for receiving an image from the pan / tilt camera, the image correction to correct the input image through filtering, the correction A motion detection unit for detecting a region in which a motion is generated by separating a moving region from a background region in a captured image, and using the movement region information of each moving object in the extracted motion generating portion to move the selected at least one moving object A moving object extraction unit for separating / merge and simultaneously extracting the moving objects, a moving object tracking unit for tracking positions and speeds of the extracted plurality of moving objects using a prediction algorithm and an information matching technique using a Kalman filter, and the plurality of moving objects Camera that detects / corrects the movement of the camera according to the movement of moving object And it characterized in that it comprises a movement compensation.

In addition, the image corrector may use a noise removing filter to remove noise components of the input image.

In addition, the noise canceling filter is characterized in that any one selected from the median filter, Gaussian filter, smoothing filter, average filter.

In addition, the average filter is characterized by using the average value previously calculated while moving the block.

The moving object extracting unit may extract each moving object through segmentation and merging using the motion region information.

In addition, the moving object tracking unit is characterized by predicting and tracking the next position of the object by applying the speed and position information of the moving object to the prediction algorithm using the Kalman filter.

In addition, the information matching technique of the moving object tracking unit is characterized in that it is used when the tracking object is stopped and movement is lost or when several objects cross each other and it is difficult to accurately track.

The camera movement compensator extracts edges of image data continuously input in an auto-panning mode of the camera, projects the extracted edges on an xy axis, and compares the projected values. It is characterized by correcting the movement of the.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 illustrates a configuration of a multiple moving object tracking / monitoring system according to the present invention, and includes a wide area monitoring device 10 and a local monitoring device 20.

The wide area monitoring apparatus 10 is a device for monitoring a large area by using a fixed camera, and analyzes an input image to extract motion, and registers a moving object using the extracted motion information and tracks the trace. In addition, all moving objects within the surveillance range are registered and tracked.

The wide area monitoring apparatus 10 includes a camera 110, an image input unit 120, an image compensator 130, a motion detector 140, a motion object extractor 150, and a motion object tracker 160.

The camera 110 collects image data of the surveillance area by using a fixed camera.

The image input unit 120 inputs an image collected from the camera 110.

The image corrector 130 is a means for correcting an image input from the image input unit 120 and corrects the image through filtering.

Here, the image corrector 130 is a means for correcting an image for more accurate motion detection and moving object extraction by removing noise components of the input image. The image correction unit 130 removes noise of the input image using a noise removing filter.

The motion detector 140 is a means for detecting motion in the image corrected by the image compensator 130. The motion detection unit 140 uses a disturbance map as a process for separating the motion region and the background from the noise-removed image. To detect the part where the motion occurred. Disturbance map (Disturbance map) is to obtain the temporal average of the input signal in real time in order to create a background image that is not moving in time to separate the moving object and the moving background in a fixed camera.

The moving object extractor 150 is a means for separating an actual moving object from the extracted motion region by using the disturbance map. The motion object extractor 150 uses segmentation and merging using motion region information. Extract each moving object.

The moving object tracking unit 160 is a means for tracking the position and speed of the extracted moving object. The moving object tracking unit 160 uses the Kalman Filter to predict the speed and position of the moving object for fast and accurate tracking of the moving object. Apply to accurately predict and track the next position of the moving object.

In addition, the moving object tracking unit 160 makes it possible to track an object without using an information matching technique when the tracking object is stopped and movement is lost or when multiple objects cross each other and accurate tracking is difficult. The information matching technique finds an object using a technique of matching the most similar data with the given information on the target data by using a given template. In general, the matching is calculated by calculating a correlation. Choose the one with the largest value.

As shown in FIG. 3, the local monitoring apparatus 20 is a device for monitoring a moving object while moving the camera using a Pan / Tilt camera at a short range. In the auto-panning mode, the camera moves from side to side to monitor movement, and when movement is detected, zooms to detect an object and moves and tracks the camera according to the movement path of the object.

In addition, since the movement of the camera occurs in the regional monitoring apparatus 20, the process of correcting the movement of the camera in addition to the process performed by the global monitoring apparatus 10 and moving the camera along the object when the moving object is registered. Tracking is necessary.

The regional monitoring apparatus 20 includes a camera 210, an image input unit 220, an image compensator 230, a motion detector 240, a moving object extractor 250, a moving object tracker 260, and a camera move. Compensation unit 270 is made.

The camera 210 collects image data of the surveillance area by using a Pan / Tilt camera used at a short distance.

The image input unit 220 inputs an image collected from the camera 210.

The image corrector 230 is a means for correcting an image input from the image input unit 220 and corrects the image through filtering.

Here, the image corrector 230 is a means for correcting an image for more accurate motion detection and moving object extraction by removing noise components of the input image. The image correction unit 230 removes noise of the input image using a noise removing filter.

The motion detector 240 is a means for detecting motion in the image corrected by the image compensator 230. The motion detection unit 240 is a process for separating a motion region and a background from a noise-removed image and using a disturbance map. Detect the part.

The moving object extractor 250 is a means for separating an actual moving object from the extracted motion region by using the disturbance map. The motion object extractor 250 uses segmentation and merging using motion region information. Extract a single object. In this case, segmentation means dividing only desired information from an input image signal, and merging means merging feature values belonging to the same object into one when multiple feature points representing the same object are divided into one. it means.

The moving object tracking unit 260 is a means for tracking the position and speed of the extracted moving object. When the motion detection unit 20 detects a movement, the moving object tracking unit 260 moves the camera 210 after locking the moving object. The object is tracked by moving according to the movement of the object. As a method for quickly and accurately tracking a moving object, a method using color information, a matching method, or an optical flow may be used. The optical flow refers to a flow chart that finds the same position between n-1 frames and n frames and connects points calculated to the same position.

The camera movement correction unit 270 operates the camera 210 in the auto-panning mode in the area monitoring, which means that the camera 210 automatically moves from side to side, in which case the movement is extracted. In this case, the overall movement due to the camera movement is detected and it becomes difficult to extract the movement of the actual moving object. In order to solve this problem, the movement of the camera is detected and corrected, so that only the movement of actual moving objects can be detected.

Based on the above description, the algorithm used in the multiple moving object tracking / monitoring system according to the present invention will be described.

1. Noise Reduction Algorithm Through Filtering The noise reduction algorithm through filtering is performed by the image correction unit 130 of the wide area monitoring apparatus 10 and the image correction unit 230 of the local monitoring apparatus 20.

1.1. background

Image change caused by noise or climate change when acquiring an image using a CCD camera increases the possibility of causing an error in the motion detection step. Image filtering is needed to effectively remove these noise components.

1.2. Application method

As a filter for removing noise components of an image, a median filter, a Gaussian filter, a smoothing filter, a Gaussian smoothing filter, and a mean filter are used. Several filters are used. Among them, a mean filter that can be quickly implemented for real-time system implementation removes noise components of the image input from the camera. Here, the median filter sorts the target data and filters the median value. The Gaussian smoothing filter is a filter that blurs the image and has a vertical shape. The average filter is a filter for blurring an image. The average filter obtains an average in an n x n mask and returns the value as a filtering result.

A normal mean filter is implemented by applying a 3x3 mask to an image, but for faster implementation, the average value is quickly obtained by using a previously calculated average value while moving a block without performing a mask operation (ie, filtering). The method of calculation was used.

1.3. Application result

4 illustrates a motion region extraction (left) using an input image and a motion region extraction (right) using an filtered image.

As shown in FIG. 4, in the case of the image that has not been filtered, a motion region due to noise is extracted.

2. Motion Detection Algorithm The motion detection algorithm is performed by the motion detection unit 140 of the wide area monitoring apparatus 10 and the motion detection unit 240 of the local monitoring apparatus 20. background

In order to track the moving object from the input image, detecting and locating the movement of the moving object must be preceded. As a method of detecting a motion region, a method using a difference between two images, an optical flow method, a motion vector method, and the like are used.

2.2. Disasterburn map

Among the methods of detecting motion, the method using difference is simple and quick to calculate, but it is very sensitive to changes in illumination or noise, and the method using optical flow or motion vector requires a large amount of computation. It has a disadvantage.

In the present invention, a disturbance map is used to overcome this problem. The disturbance map is a method of generating a disturbance map using a current image and a background (average image of a previously input image) image and detecting a moving region of an object using the same. The disturbance map is simple and allows fast calculations while being robust against noise and light intensity changes.

Disturbance map (Disturbance map) is a method of using the temporal average concept (temporal average) to create a background image by giving a weight (historical weight) to the average up to the previous frame, and obtains the difference between this and the current frame.

Here, I _t is the current frame and A _t represents the time average up to the present. That is, Equation 1 is an equation for creating a background image A _t which is least affected by movement with an input image input up to a specific frame. Motion region extraction

The disturbance map has a value in an area where motion exists and is close to 0 in a background where there is no motion. Using this, in the disturbance map, a threshold division method is used to extract an area in which a real motion is located. Here, the threshold dividing method is for dividing desired data and undesired data from specific data. In general, the threshold division method sets the value to 1 if it is above the reference point and to 0 if it is below. As shown in Equation 2 below, In this case, the absolute value of the entire distribution map is classified into a motion region when the value is larger than a predetermined threshold, and a background region when the value is smaller than the threshold map.

FIG. 5 illustrates an input image, a disturbance map, and a motion region extraction result over time.

3. Moving object extraction algorithm The moving object extraction algorithm is performed by the moving object extraction unit 150 of the wide area monitoring apparatus 10 and the moving object extraction unit 250 of the local monitoring apparatus 20. background

The motion regions extracted by the disturbance map are separated into respective regions through a labeling process. At this time, the area whose size is smaller than the reference value is regarded as noise and is removed, and the center of gravity of each labeled object is obtained to determine the target position. In the labeling process, after thresholding a specific image, the pixels are numbered separately so that the pixels connected to each other can be recognized as a single object.

However, when a disturbance map is applied to an actual image, a plurality of motion regions are often extracted for one object. This is caused by the fact that one object is made up of different parts of different brightness, and each part is recognized as an individual object. In order to compensate for this, a merging process of merging one recognized object into several objects is required.

3.2. Binary Filtering

In order to combine the separated objects into one region, first, binary filtering is performed on the moving region. Binary filtering is a method of widening or narrowing a desired part of an image by using morphology technique.By binary filtering, a noise reduction effect can be obtained. ) Combined results can be obtained between objects.

6 shows a comparison of the before (left) and after (right) of binary filtering.

3.3. Object Merge

Binary filtering removes noise and combines objects, but not perfectly. Therefore, the objects estimated as one object are combined by using the position information of the object generated after labeling and the velocity and direction information used for tracking.

Currently, objects are merged using only the distance information of two objects.

FIG. 7 shows a comparison between the front (left) and the rear (right) of an object merge.

4. Position and Velocity Tracking Algorithm Using Kalman Filter The position and velocity tracking algorithm using Kalman filter is used for tracking the moving object tracking unit 160 of the wide area monitoring apparatus 10 and the local monitoring apparatus 20. In step 260 is performed.

4.1. background

The moving object extracted in the previous step contains only the positional information in the current frame. In order to track moving objects, a method for predicting the next position by grasping the correlation between them based on the location information of each object extracted from each frame and classifying them as a moving object and grasping the motion of the object to date It is necessary. Also, by identifying and predicting the movement characteristics of the moving object, the moving object can be tracked more quickly and accurately.

4.2 Kalman Filter

The Kalman filter is mainly used to track moving objects. In this study, Kalman filter was used to track the target from the measured image information. It is well known that Kalman filters have a linear structure and that their characteristics are optimal when the conditions given in the linear system are met. In the discrete linear system given by Equation 3 Time Is a state vector with any number of positions, velocities, accelerations, etc. Is a constant matrix obtained by modeling the equation of motion of an object. And, in the state of the object, especially in the vector that is the characteristic of the sensor to get the position We design a Kalman filter for a model where the measurement equation for obtaining the position is given by Equation (4).

From here Is the process noise added to the system, and the Refers to sensor noise generated when measuring the signal. They are not correlated with each other, and each covariance , And assume that each has a characteristic of white noise with an average of zero. time Measured value of state variable in If is obtained through a sensor, the estimated value of the filter that predicts the information of this measurement at time k-1. Should be reinforced. If the structure of the filter is linear here, then the estimated value is augmented with the information contained in the new measurement.

Can be expressed as a linear structure as shown in Equation 5, and the design of the filter is the gain of the filter in Equation 5. It turns into a matter of determining.

The augmented estimate is the difference from the actual state vector, or error (e = - generates a x _k) is set to a performance index of a filter to minimize the mean square of the estimation error, it is organized as a problem to minimize the sum of the diagonal elements in the covariance matrix of the estimation error is given by Equation 6

In equation (6) Is the measured value until the information of the measured value is reinforced in Equation 5 It can be expressed as Equation 7 as a covariance matrix for the estimation error of.

The cost function is the sum of the diagonal elements of the covariance matrix It is quadratic with respect to, and the quadratic coefficient always has a minimum because it is a property of the amount of covariance between the state variable and the measured noise. therefore Let the first derivative of The value is expressed as in Equation 8. By substituting this value into Equation 6, the covariance matrix of the estimated error after performing the measurement reinforcement can be expressed as Equation 9.

The estimated value of the state variable given by Equation 5 may continue to develop into an optimal estimated value according to the equation of motion of the object, which is the information the filter has until the next measured value is input. Equation 10 shows an equation of the state variable which is progressed by the equation of motion of the object until a new measurement value of the state variable is obtained. Estimate Shows the covariance matrix of.

The estimated value and its covariance matrix obtained using Equations 10 and 11 are the filter gains of the next state given by Equation 8, and the following measured values If is input, the estimated value of the state variable as shown in Equation (8) Find the covariance matrix Is obtained as shown in Equation 9.

In this way, the Kalman filter, the optimum filter, can be found cyclically with every measurement. Kalman filter as described above can be arranged in the order of the flow chart as shown in FIG.

8 shows a flowchart of a Kalman filter.

5. Template Matching Algorithm

5.1. background

In general, when a moving object stops while tracking an object using motion detection, the moving object is not detected and the tracking object is missed. In order to solve such a case, an information matching method of directly searching for an object using an input image is used.

5.2. Information matching algorithm using projection vector sum

The general template matching method requires a large amount of computation and is difficult to apply to a real-time tracking system. To solve this problem, we used a fast template matching algorithm using the projection vector sum.

In the conventional projection method, a method of comparing the similarities of two-dimensional image signals is performed by comparing only the similarities of one-dimensional signals obtained by projecting two-dimensional images.

9 shows the projection results in the vertical and horizontal directions for a two-dimensional image.

Although the speed is greatly improved by the projection method, there is a problem that the total calculation time becomes longer when the number of stationary objects increases. In order to solve this problem, a vector sum method is introduced.

As shown in FIG. 10, as shown in FIG. 10, the projected one-dimensional signal is unfolded in a vector form, and the sum is obtained to express the sum as a single vector.

10 shows a vector representation of a projection signal and its vector sum.

5.3. result

FIG. 11 illustrates object tracking by template matching. FIG. 11 illustrates a result of registering a moving car as a template and finding a position by matching in the next frame. Matching techniques allow the object to be located in the image regardless of whether it is stationary or moving. This result can also be used to track moving objects detected in the local surveillance system.

6. Motion Segmentation

The motion segmentation algorithm is performed by the moving object extraction unit 150 of the wide area monitoring apparatus 10 and the moving object extraction unit 250 of the local monitoring apparatus 20. 6.1. purpose

The conventional method of extracting motion using a disturbance map has a disadvantage in that one moving object is divided into several moving objects, and thus, an additional process of combining the divided objects into one is required. This method is a new motion extraction method to solve this problem. The method is still under study for the purpose of accurately extracting and segmenting moving objects from images obtained from fixed cameras.

6.2. Main algorithm

Create Background Image: Creates a background image from the input image.

Difference Image Generation: Extracts the part with change by using the difference between the input image and the background image.

Image Edge Extraction: Detects the contour of a moving image. Differential filters are mainly used.

6.3. Characteristic

It derives the background image from the input image and separates the area of the moving part using the difference from the background image in the current image. It is a system that detects and displays the outline of the separated area. The method used in this study overcomes the phenomenon that the background image changes less than that of the previous disturbance map, so that the object is separated into two for a moving object. It is a system suitable for gradual change of lighting, and it can be usefully used as a road driving inspection system or a monitoring system of a certain space.

6.4. Detailed algorithm description

6.4.1 Create background image

The creation of a suitable background image is an important part of this system for finding moving objects. The proposed method minimizes the change of the background image so that the background image is not shaken even by the sudden change of the image.

The background image generation algorithm is as follows.

a. Enter the first incoming image as the background image.

b. Since there may be moving objects in the first image, it may give a certain amount of change in the image up to a certain initial frame.

c. After a certain time, the change of the background image is minimized to reduce the influence on the currently input image.

12 shows an input image and a background image in the initial frame.

13 shows an input image and a background image after a certain time.

6.4.2 Create Difference Image

The difference between the background image and the background image is used to indicate the change. In case the brightness value between the background image and the moving object is small, a small morphology technique is applied to enlarge the area. The difference image thus created is shown in FIG. 14. The morphology technique scans an image with an element structure and changes the center value of the image element to 1 if the image element has any desired value, or to zero if all the desired values are not filled. How to change.

14 illustrates a motion extraction image by using a difference between an input image and a background image.

6.4.3 Create contour image

Create an image that has edge-detected the current input image. Only the outlines included in the area image are extracted from the edge image generated in this way and displayed. Edge detection was generated using a differential filtering method.

15A shows the edge image in the current incoming frame, and FIG. 15B shows the resulting image.

6.5. summary

Disturbance is commonly used to create a background image using input frames. However, this method has a sensitive response to recent frames, so it has a split effect on one object. In order to secure these shortcomings, this study focuses on smoothing the segmentation of one object by limiting the amount of change after creating the background image. In addition, efforts have been made to increase the effect of segmentation by using edge information of an image.

7. Camera Motion Compensation Algorithm

7.1. background

When operating in the automatic monitoring mode in the local surveillance system, the camera automatically rotates to monitor a large area. When there is a movement of the camera, it is difficult to detect the movement of an object by a conventional motion detection method. In order to solve this problem, there is a need for a method of detecting a motion by estimating a camera motion and correcting an image.

7.2. Motion Estimation Algorithm Using Template Matching and Its Positional Relationship

In order to correct the movement of the camera using the input image, information (template) is registered at a predetermined position and interval of the image, and the position is found in the next frame. By using the difference between the positions of the templates found in the next frame and the position of the previous frame, the motion of the camera can be estimated.

In this case, in order to avoid incorrect camera motion estimation due to a matching error, a large error is generated by using a positional relationship between templates in the previous frame and a template found in the current frame. By excluding the position from the motion estimation, more accurate camera motion can be estimated.

16 shows a template (left) registered in a previous frame and a template (right) found in the current frame.

As shown in FIG. 12, the template (hatched portion) whose position of the template found in the current frame has a large error with the position set in the previous frame is excluded from the motion estimation, and the motion templates are calculated using the remaining templates. The camera motion is corrected using the obtained motion vector.

FIG. 13 shows a result of comparing a result of extracting a motion after calculating a motion of a camera and correcting the motion of the camera through the present method by simply detecting a motion with respect to a moving image of an actual camera. When the algorithm is applied, the movement of the camera is corrected a lot so that the movement of the moving objects is detected.

17A to 17C show a comparison of motion extraction results before and after camera motion correction, FIG. 17A is an input image, FIG. 17B is a motion extraction result before camera motion correction, and FIG. 17C is a motion extraction result after camera motion correction. It is shown.

8. Motion Detection Algorithm Using Block Matching The motion detection algorithm using block matching is performed by the camera movement correction unit 270 of the local monitoring apparatus 20.

8.1. background

In general, a method of detecting a motion using a difference of an input image has a problem in that it is difficult to apply the motion of a camera. In this case, if you use the block matching to divide the image into small blocks and extract the motion vector of the block, the motion vector of the camera and the motion vector of the object will appear differently. It becomes possible.

8.2. algorithm

The block matching algorithm divides an input image into small blocks of N X N and then searches where each block is moved in the next input image.

18A to 18B illustrate motion vector extraction using block matching. FIG. 18A shows a previous frame and FIG. 18B shows a current frame.

8.2.1 Global Search Algorithm

Although the optimal motion vector can be found by searching all the points included in the search area based on the position of the previous frame block having the same coordinate as the current frame block, the motion vector can be found.

8.2.2 Fast Search Algorithm

8.2.2.1 three-step search algorithm

A simple and efficient algorithm that searches for motion vectors according to a certain search pattern, reduces computations, and if the first search goes wrong, it can fall into local optima.

8.2.2.2 unrestricted center-biased diamond search algorithm

It increases the probability of finding the correct motion vector by using the central characteristic of the motion vector. It is not suitable for the image with the abrupt motion.

8.2.2.3 Prediction search algorithm

If the correlation of the motion vectors of the adjacent blocks is reduced by using the information of the motion vectors of the adjacent previous blocks, the performance decreases.

8.2.3 Adaptive Prediction Directional Search Algorithm

Algorithm that prevents computational problem in global search algorithm and local search problem due to lack of information in fast search algorithm. Temporal association with motion in consecutive frames and space between blocks within current frame. It is a method of calculating the motion vector by using the correlation.

8.3. result

19 illustrates a result of extracting a motion region using block matching. Currently, experiments are performed only when there is no camera movement, but when there is camera movement later, the motion vector of the camera and the motion vector of the object are distinguished. You will extract the movement.

9. Adaptive Background Method

9.1 Create Background Image

The creation of a suitable background image is an important part of this system for finding moving objects. The proposed method minimizes the change of the background image so that the background image is not shaken even by the sudden change of the image. The background image generation algorithm is as follows.

a. Enter the first incoming image as the background image.

b. Since moving objects may exist in the first image, the change of the image is regarded as the background image more than a certain amount until the frame for a certain time.

c. After a certain time, the change of the background image is minimized to reduce the influence of the currently input image on the background image.

20A illustrates an input image and a background image in an initial frame, and FIG. 20B illustrates an input image and a background image after a predetermined time.

9.2 Create Difference Image

The difference between the background image and the background image is used to indicate the change. In case the brightness value between the background image and the moving object is small, a small morphology technique is applied to enlarge the area. The difference image thus created is as follows.

FIG. 20C illustrates a motion image extracted by using a difference between the input image and the background image.

9.3 Create contour image

Create an image that has edge-detected the current input image. Only the outlines included in the area image are extracted from the edge image generated in this way and displayed. Edge detection was generated using a differential filter.

20D shows the edge image in the current incoming frame, and FIG. 20E shows the resulting image.

9.4 Summary

Disturbance is commonly used to create a background image using input frames. However, this method has a sensitive response to recent frames, so it has a split effect on one object. In order to secure these shortcomings, this study focuses on smoothing the segmentation of one object by limiting the amount of change after creating the background image. In addition, efforts have been made to increase the effect of segmentation by using edge information of an image.

10. 3-Different Image Method

10.1 3-different image

Changes between the image of the previous frame and the image of the next frame in the current frame. At this time, the moving object is detected in the comparison of each image, and the stationary object is not detected. Using this, the moving object in the current frame is segmented using the image difference detected in the comparison with the previous frame and the image difference obtained in the comparison with the next frame.

21 shows a difference image between each frame. FIG. 22 illustrates a contour image of a moving object extracted using a current image and a difference.

10.2 Summary

This segmentation technique of the moving object overcomes the residual phenomena generated by the adaptive background technique described above and can be easily implemented with an algorithm that can be adapted to a fast moving object. It is a strong separation method.

11.Global motion detection and moving object segmentation

11.1 Global Motion Detection

There are many difficulties in grasping the camera movement only with the input image. Therefore, researches that accurately compensate for motion are still being carried out in numerous research institutes. In this study, we used the correlation method for each image line as an algorithm for inferring camera movement. This method detects the movement of each line by comparing the previous frame and the current frame for each line and detects the movement of each line based on this. That is, the image is moved left and right for each line, and the moving range of the line is set where the most correlation value appears. In this way, the most-moved number of movements taken by line is set as the entire movement range. This method is applied in the horizontal and vertical directions.

11.2 Moving Object Segmentation

If you know the camera's movement and then correct the movement inversely, you can assume two images without camera movement. When the camera motion is corrected for each incoming frame and segmentation is performed using the 3-difference image method mentioned above, the moving camera appears differently from the moving path of the moving camera. (independent moving objects) can be detected.

FIG. 23 shows a table showing camera movements frame by frame. 24 illustrates an image being input to the camera. 25 illustrates an independently moving object after correcting the movement of the camera.

11.3 Summary

Algorithms for correcting the movement of cameras and detecting moving objects in the screen using them are studied. This is an algorithm that can be used for many applications in the future and it is required to supplement algorithms for accuracy and real-time calculation.

The present invention may have various application fields as a real-time moving object detection and tracking system using a camera input image.

Military applications

It is possible to construct a 24-hour unmanned surveillance system for various military facilities.

Eg missile base surveillance, ammunition and arsenal surveillance

B) Private application field

It is possible to construct an unmanned surveillance system of various public and private facilities.

Ex) Intake and water purification plant monitoring, port monitoring, traffic flow monitoring

The drawings and specification are merely exemplary of the invention, which are used for the purpose of illustrating the invention only and are not intended to limit the scope of the invention as defined in the appended claims or claims. Therefore, those skilled in the art will understand that various modifications and equivalent other embodiments are possible from this. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

According to the present invention, multiple moving object simultaneous identification function, multiple moving object simultaneous tracking / monitoring function, automatic tracking and zoom in / out, monitoring area and monitoring method setting, speed calculation and analysis, multi-object tracking situation only Recording function, excellent compression rate, bad weather or night tracking, diversification of monitoring targets such as indoor, outdoor, dangerous goods. Automated surveillance (withdrawal from human surveillance) and video surveillance combined functions (automated detection-tracking-alarm and image analysis). Monitor load reduction and cost reduction. Rescuing life as a partner in environmental conservation, such as poor environmental monitoring. While moving the surveillance image is effective, such as automatic detection of multiple animals.

Claims (10)

Simultaneously extract movements of a plurality of moving objects existing in the surveillance range using a fixed camera and register the information, select at least one moving object to be tracked from among the plurality of registered moving objects, and select the selected A wide area monitoring apparatus tracking at least one or more moving objects in a first surveillance region, and outputting a moving object tracking command when the selected at least one or more moving objects are out of the first surveillance region; And At least one local surveillance camera moving in a pan / tilt and monitoring a second independent surveillance zone, wherein the plurality of local surveillance is input when the tracking command for the at least one selected moving object is input from the wide area surveillance apparatus; A camera that sequentially drives the camera to monitor and zoom while the selected moving object leaves the second surveillance zone; The wide area monitoring device and the regional monitoring device extracts the movement of the at least one selected moving object at the same time to effectively separate / merge the moving areas of each moving object to perform tracking / monitoring in real time. Object tracking / monitoring system. The method of claim 1, wherein the wide area market value A fixedly installed camera; An image input unit which receives an image from the fixed camera; An image correction unit for correcting the input image through filtering; A motion detector which detects a part in which a motion occurs by separating a motion area and a background area from the corrected image; A moving object extracting unit for separating / merge and simultaneously extracting movements of the at least one selected moving object by using motion region information of each moving object in the extracted motion generating portion; And And a moving object tracking unit for tracking the positions and speeds of the extracted plurality of moving objects using a prediction algorithm and an information matching technique using a Kalman filter. The method of claim 1, wherein the regional supervisory value At least one pan / tilt camera moving with pan / tilt; An image input unit which receives an image from the pan / tilt camera; An image correction unit for correcting the input image through filtering; A motion detector which detects a part in which a motion occurs by separating a motion area and a background area from the corrected image; A moving object extracting unit for separating / merge and simultaneously extracting movements of the at least one selected moving object by using motion region information of each moving object in the extracted motion generating portion; A moving object tracking unit which tracks the positions and velocities of the extracted plurality of moving objects using a prediction algorithm and an information matching technique using a Kalman filter; And Multiple moving object tracking / monitoring system comprising a camera movement correction unit for detecting / correcting the movement of the camera according to the movement of the plurality of moving objects. The method of claim 2 or 3, wherein the video correction unit A multiple moving object tracking / monitoring system using a noise removing filter to remove noise components of an input image. The noise canceling filter of claim 4, wherein Multiple moving object tracking / monitoring system characterized by selecting one of median filter, Gaussian filter, smoothing filter, and average filter. The method of claim 5, wherein the average filter Multiple moving object tracking / monitoring system characterized in that the calculation is performed using a previously calculated average value while moving a block. According to claim 2 or 3, wherein the moving object extraction unit A multiple moving object tracking / monitoring system characterized by extracting each moving object through segmentation and merging using motion region information. According to claim 2 or 3, wherein the moving object tracking unit A moving object tracking / monitoring system, characterized by predicting and tracking the next position of an object by applying velocity and position information of a moving object to a prediction algorithm using the Kalman filter. The multiplexing method of claim 2 or 3, wherein the information matching technique of the moving object tracking unit is used when the tracking object stops and the movement is lost or when several moving objects cross each other, so that accurate tracking is difficult. Moving object tracking / monitoring system. The method of claim 3, wherein the camera movement correction unit In the auto-panning mode of the camera, the edges of the image data which are continuously input are extracted, the extracted edges are projected on the xy axis, and the projected values are compared to compensate for the camera movement. Moving object tracking / monitoring system.