KR20030024354A - Global motion compensation apparatus and method for detecting a motion of an object - Google Patents

Abstract

PURPOSE: A global motion compensation apparatus and a global motion compensation method are provided to achieve a significant reduction in the search area for detection of motion, while avoiding a morphological opening. CONSTITUTION: A global motion compensation apparatus comprises an image acquiring unit(200); a pre-processing unit(210) including a low pass filter, and which removes spatial high frequency components by filtering the image obtained by the image acquisition unit through the use of the low pass filter; a global motion estimation unit(220) for receiving the current image and previous image from the image acquisition unit, and estimating motion vector of the background; a block based motion compensation unit(240) for receiving the current image and previous image where spatial high frequency components are removed, from the pre-processing unit, obtaining a compensated image by performing a block based motion compensation through the use of the motion vector estimated by the global motion estimation unit, and obtaining a motion compensation error image from the difference between the compensated image and the current image where spatial high frequency components are removed; a motion detection unit(250) for comparing the sum of motion compensation error values of each block with a predetermined threshold value, and judging the block having the sum of motion compensation error values exceeding the threshold value, as an area where an object exists; and a moving object variable extraction unit(260) for obtaining variables of the object by calculating the center and area of the moving object in the block judged by the motion detection unit.

Description

Global motion compensation apparatus and method for detecting a motion of an object

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a background compensation apparatus and method for high speed motion detection in an active camera environment, and more particularly, a global motion and a local motion simultaneously generated by a camera motion. Background of the Invention The present invention relates to a background compensation apparatus and method for quickly detecting the motion of an object by effectively predicting the motion of the background with a small amount of computation in a limited search area for fast motion detection of an object.

With the widespread use of the Internet, technologies in related fields have been rapidly developed, and various forms of high-speed broadband transmission technology have already allowed Internet users to stream large amounts of multimedia data including voice and video through streaming services. While being able to be provided, the application fields of the streaming service vary widely from internet broadcasting to video service through a web camera.

The web camera is generally installed at a predetermined position to provide an image in a fixed area, which is a problem in actively observing an object of interest. Therefore, in order to obtain a wider image, a method in which a plurality of cameras are installed and an observer is selectively used, which has a disadvantage in that a high cost problem associated with installing a plurality of cameras occurs. In addition, when the web camera is applied to a security and surveillance system, there is a problem that the camera becomes useless when the location of the intruder is installed.

Accordingly, an improved system for tracking a moving object while the camera is active has been provided in various forms, hereinafter, the prior art for detecting the movement of an object in such an active camera environment is described.

Conventional motion detection techniques include a method using a difference image and a method using a motion vector.

The method using the difference image obtains a difference image by obtaining a difference in brightness between the obtained current image and the previous image, and compares the brightness value of each pixel of the difference image with a predetermined threshold to increase the movement in the current image. It is a way of recognizing that an object appears. This method has the advantage of being simple to implement and taking less time to detect motion, while the difference generated by the difference in the brightness of the background when the brightness of the background changes by more than a certain value over time. There is a problem that an error may occur in motion detection because an image is mistaken as a difference image by motion detection.

The method using the motion vector is a method of detecting motion using motion vectors obtained by performing motion estimation from the acquired current image and the previous image. According to this method, when the background brightness values are similar, the sum of absolute difference (SAD), which is one of the measures for motion estimation, is similar. need. For example, one method may be to perform motion estimation by placing a weight on the SAD in favor of zero motion.

In this regard, when there is a bit stream compressed by a standard compression scheme such as MPEG series or H.263, these information are already calculated values and do not need to be separately calculated for motion estimation. Good for motion detection. Most compression schemes including the above technique are stored in the bit stream after estimating and compensating for the motion vector, so that the movement of an object in the corresponding image can be easily detected by using the same.

Meanwhile, since the global motion of the background and the motion of the object are simultaneously present in the image acquired from the moving camera, it is common to detect the motion of the object by calculating the global motion for each image. Block Matching Algorithm, Frame Matching Algorithm, and Line Matching Algorithm.

The block matching technique divides the obtained current image and the previous image into small blocks of a predetermined size, predicts motion for each block, obtains a motion vector, and calculates a background vector based on a histogram generated using the frequency of each motion vector. As a method of determining the motion and the movement of the object, there is an advantage that the amount of calculation is small, and thus it is essential for realizing an image in real time.

In the frame matching technique, a global motion vector is obtained by considering a current frame as one block and block matching with a previous image under the assumption that the global motion can be represented by one motion vector. A compensation image is obtained by compensating the background area using the obtained motion vector, an edge image is obtained from the current image, and an image is obtained by performing an AND operation on the two images, that is, the compensation image and the edge image.

However, the assumptions used above do not take into account the geometry of the camera, and thus there is a problem in that an edge portion corresponding to the background region of the current image may be erroneously detected in the compensation image. Therefore, in order to minimize such noise components, after performing a morphological opening operation, the non-zero pixels are projected in the horizontal and vertical directions with respect to the image obtained by the AND operation. Investigate the number of times to calculate and track the position and area of a moving object.

In addition, the frame matching technique has a disadvantage in that a correct global motion vector cannot be obtained when the object occupies a larger area than the background region.

The line matching technique starts from the assumption that global motion can be represented by one global motion vector like the frame matching technique. First, an edge image is obtained from each of the acquired previous image and the current image, and the two edge images are lined. Match to get global movement. If it is assumed that the camera moves only in the horizontal direction, a motion vector corresponding to each line may be calculated only by line matching of each horizontal line constituting the image. At this time, the motion vector for each horizontal line is calculated and the most frequent motion vector is determined as the global motion vector. Therefore, there is an advantage that the error caused by the moving object inside can be minimized in the global motion estimation. In addition, even if the camera moves only in the vertical direction, a line motion can be obtained for each vertical line to obtain a global motion vector in the same manner. However, this method also assumes that global motion can be represented by a single global motion vector, so a computationally intensive mathematical morphological open operation is required. Therefore, in some cases, the amount of computation may be larger than block matching.

Hereinafter, with reference to the drawings will be described in detail an example of the configuration of an apparatus for detecting the movement of the object by employing the prior art described above.

FIG. 1 is a block diagram of an apparatus for tracking a motion using a conventional background compensation technique. As shown in FIG. 1, an image acquirer 100, a frame / line image matcher 110, and a background compensator 120 are illustrated. And a post-processing unit 130 and a moving object variable extraction unit 140.

The image acquisition unit 100 is a means for acquiring a current image and a previous image, and the frame / line image matching unit 110 uses a frame matching or line matching to obtain a motion vector of a background. The background compensator 120 is a means for estimating a difference from the current image by compensating for the motion of the background, and the post processor 130 removes a noise component from the frame difference. In order to perform a morphological opening operation (morphological opening) operation, the moving object variable extraction unit 140 is a means for obtaining the variable of the moving object by calculating the center area and the area of the moving object.

The above means operate as follows. That is, the image acquisition unit 100 obtains two consecutive images, and then the frame / line image matching unit 110 obtains a sum of absolute difference (SAD) for a search area having a predetermined size by frame or screening. The motion of the background is predicted by finding the smallest displacement, that is, the motion vector. Subsequently, the background compensator 120 obtains a difference image from the current image by compensating a frame or line of the previous image using the motion vector, and in this process, the size of the pixel value forming the difference image is threshold. value) is considered to be a pixel on a moving object. At this time, the post-processing unit 130 performs a mathematical morphological open operation to remove noise in the difference image generated in the process of approximating the motion of the background as one motion vector. Finally, the moving object variable extracting unit 140 clusters a portion considered as a pixel on the moving object to obtain coordinates and an area of the center of the object.

As described above, in order to predict the motion of the background, the SAD (Sum of Absolute Difference) is obtained for a certain search area by frame or screening, and the displacement is found to have the smallest value. In this relatively large case, there is a disadvantage in that there is a high probability that there is an error in the background motion prediction.

In addition, since the mathematical morphological open operation, which is essential for removing noise in the difference image generated in the process of approximating the background motion as one motion vector, is repeatedly performed for each pixel, a large amount of computation is required and therefore high speed motion is required. Makes detection difficult.

Accordingly, there is a need for a background compensation apparatus and method for quickly predicting only motion region of an object by effectively predicting background motion in a very limited search region for fast motion detection.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and in tracking the movement of an object in an active camera environment, a repair that requires a large amount of computation by removing the noise of the difference image by precisely finding a background movement with a small amount of computation. By avoiding morphological opening operations and screening out areas where moving objects are expected to be located, it is possible to reduce the probability of falsely predicting background movements while reducing the search area required to find background movements. It is an object to provide a background compensation apparatus and method.

1 is a block diagram of a background compensation apparatus for detecting a motion of an object according to the prior art;

2 is a block diagram showing the configuration of an apparatus according to the present invention;

3 is a block diagram showing the configuration of a global motion estimation unit according to the present invention;

4 is a block diagram showing the configuration of the method according to the present invention;

5 is a block diagram showing the configuration of a method for predicting a global motion vector according to the present invention;

6 is a block diagram showing the configuration of a method for selecting a block according to the present invention.

<Brief description of the symbols of the main parts of the drawings>

200 image acquisition unit, 210 preprocessing unit,

220 ... global motion estimator, 221 ... block selector,

222 ... global search motion estimator, 223 ... predicted motion vector selector,

230 ... block based motion compensation, 240 ... motion detection,

250. Moving object variable extraction unit.

In order to achieve the above object, the present invention provides an apparatus for detecting a high speed motion of an object in an active camera environment, the apparatus including an image acquisition unit for acquiring an image, a low pass filter, and a low pass for the current image acquired by the image acquisition unit. The preprocessing unit removes the spatial high frequency components by filtering through the pass filter, the global motion estimation unit which receives the current image and the previous image from the image acquisition unit, and predicts the motion vector of the background, and the spatial high frequency component through the preprocessor. After receiving the removed current image and the previous image, performing a block-based motion compensation using the predicted motion vector obtained from the global motion estimation unit to obtain a compensation image, and then removing the compensation image and the spatial high frequency component. Block based obtaining motion compensation error image Compares the sum of the motion compensation error value of each block of the motion compensation error image obtained from the block-based motion compensation unit with a predetermined threshold value to determine a block whose sum of motion compensation error values exceeds a threshold value; And a moving object variable extracting unit configured to calculate a center and an area of the moving object, and to obtain a variable of the object, based on the motion detecting unit determining that the region exists.

In the apparatus, the global motion estimator may include a block selector which is a means for selecting a block required for motion estimation from a current image divided into small blocks of a predetermined size, and a motion estimation for the selected block. A global search motion estimator for calculating a backward motion vector and a predicted motion vector selector for determining a predicted motion vector (PMV) used in estimating motion vectors of the remaining unselected blocks from the estimated motion vectors; It is another feature.

Meanwhile, in the high speed motion detection method of an object in an active camera environment, a first step of acquiring a current image and a previous image, and removing spatial high frequency components through a low pass filter, and global motion from the acquired current image and the previous image A second step of predicting a vector, and performing a block-based motion estimation and motion compensation using the current image, the previous image, and the predicted motion vector from which the spatial high-frequency components are removed, generate the compensation image, and then compensate the image. A third step of obtaining a motion compensation error image that is a difference image from the current image from which the spatial high frequency component has been removed; and comparing the sum of the motion compensation error value of each block of the motion compensation error image and a threshold value to obtain a motion compensation error value. The motion of the moving object is detected by judging the block whose sum exceeds the threshold as the area where the object exists. Is that it comprises a fifth step for obtaining parameters of an object by calculating the center of the area, etc. of the moving object to the target and a fourth step of the detected block to the other features.

In the above method, the second step may include: a twenty-first step of dividing the current image into small blocks having a predetermined size and selecting a block necessary for motion estimation from the blocks; and searching globally from the selected block and the previous image. And a twenty-second step of calculating a motion vector by estimating motion and a twenty-second step of determining a predicted motion vector (PMV) to be used in estimating a corresponding motion vector of all remaining blocks not selected from the estimated motion vector. do.

In the above method, in the twenty-first step, the second step of obtaining the first block from the current image divided into small blocks of a predetermined size, and the image of the obtained block through a high pass filtering (HPF) After filtering, it is determined whether the edge component is sufficiently contained as a result, and it is determined whether it is an edge. If it is not an edge as a result of determination, step 212 is performed, and if it is an edge, the motion compensation error value is determined. If the sum is greater than the threshold and the sum of the motion compensation error values is large, the step 213 of performing step 215 is performed. If the sum of the motion compensation error values is small, the position of the current block is determined. A step 214 of storing the buffer in the buffer; a step 215 of determining whether there is an empty space in the buffer; If there is a liver, it is determined whether a residual block exists, and if the residual block does not exist, step 216 is terminated, and if the residual block exists, the next block is obtained, and if the residual block does not exist, the operation is terminated. It is another feature that the step 217 is made.

EMBODIMENT OF THE INVENTION Hereinafter, the Example by this invention is described in detail with reference to an accompanying drawing.

FIG. 2 is an overall block diagram of a device configuration according to the present invention. As shown in FIG. 2, the device to which the present invention is implemented includes an image acquisition unit 200, a preprocessor 210, and a global motion estimation unit 220. , A block-based motion compensator 230, a motion detector 240, and a moving object variable extractor 250.

The image acquisition unit 200 is a means for acquiring an image, and the preprocessor 210 includes a low pass filter (LPF), and receives the current image from the image acquisition unit 200 and moves it. The current image obtained in order to minimize the background motion compensation (MC) error, which is the detection criteria of the area in which an object exists, is detailed and changed in a spatial high-frequency component, that is, a checkered pattern, through a low pass filter. It is the means by which the component is removed. The motion compensation error will be described later in the related section.

The global motion estimator 220 may generate a global motion vector (Motion Vector) from two images, the previous image and the current image, which are consecutively inputted through the image acquisition unit 200. As a means of predicting MV), its internal configuration is shown in FIG.

3 is a block diagram illustrating an internal configuration of the global motion estimation unit 220. As shown in the drawing, the global motion estimation unit 220 includes a block selector 221 and a full search (FS) motion estimation. And a predictor 222 and a predicted motion vector (PMV) selector 223.

The block selector 221 is a means for selecting a block necessary for motion estimation (ME) of a background from among a plurality of blocks received by receiving the current image from the image acquisition unit 200 and divided into a predetermined size. The global search motion estimator 222 is a means for calculating a motion vector after performing sophisticated motion estimation such as global search on the selected block, and the predicted motion vector selector 223 is for all remaining unselected blocks. Means for determining a corresponding motion vector, i.e., the predicted motion vector (PMV), from the estimated motion vector.

2 again, the block-based motion compensator 230 receives the current image and the previous image from which the spatial high frequency components are removed from the preprocessor 210 and obtains the motion vector predicted by the global motion estimator 220. Through the block-based motion estimation and motion compensation, a motion compensation error image is obtained, which is a difference image between the current image from which the spatial high frequency components are removed and the motion compensated image. Since local search is performed on a very small search area during the motion compensation, independent motion prediction is possible for each block, thereby eliminating errors generated when compensating the entire background with one background motion vector. The morphological opening operation performed for the sake of communication becomes unnecessary.

The motion detector 240 compares the sum of the motion compensation error values of the respective blocks of the motion compensation error image with a predetermined threshold value to determine whether the object is a block whose sum of the motion compensation error values exceeds the threshold value. It is a means of judging the existing area.

The moving object variable extractor 250 is a means for obtaining a variable of the moving object by calculating a center and an area of the moving object.

FIG. 4 is a flowchart illustrating an embodiment of the method according to the present invention. Hereinafter, embodiments of the method according to the present invention will be described in detail with reference to the flowchart of FIG. 4.

First, the image acquisition unit 200 obtains the previous image and the current image (S300).

Subsequently, the preprocessing unit 210 removes the spatial high frequency components by filtering the acquired previous image and the current image through a low pass filter in order to minimize the motion compensation error of the background, which is a detection criterion of the region where the moving object exists. S310).

Subsequently, the global motion estimation unit 220 predicts the global motion, ie, the background motion vector, from the previous image and the current image (S320). An embodiment of a method of predicting the global motion vector is shown in FIG. 5.

FIG. 5 is a flowchart illustrating an example of a method of predicting a background motion vector. First, the block selector 221 divides a current image into small blocks having a predetermined size and selects a block to use for motion estimation among the blocks. (S321). An embodiment of the block selection method is shown in FIG. 6.

FIG. 6 is a flowchart illustrating an embodiment of a method for selecting a block for motion estimation. First, the block selector 221 obtains a first block from a current image divided into small blocks of a predetermined size (S321-1). The image of the obtained block is filtered through High Pass Filtering (HPF), and as a result, it is determined whether the edge component is sufficiently included, that is, whether it is an edge (S321-2). In this case, step S321-6 is performed. At this time, the reason for determining whether the edge is to select the block that is the edge so that the motion of the actual background can be obtained when the motion is estimated.

If the determination result is an edge, the sum of the motion compensation error values of the obtained block is compared with a predetermined threshold value (S321-3). As a result of the comparison, if the sum of the motion compensation error values is large, step S321-6 is performed. In this case, the reason for comparing the two values is to select a block having a small sum of motion compensation error values so as to avoid the block on the moving object and increase the probability of selecting the block on the background area.

As a result of the comparison, if the sum of the motion compensation error values is small, the position of the current block is stored in the buffer (S321-4), and it is determined whether an empty space exists in the buffer (S321-5). If no empty space exists, the operation for block selection is terminated (S321-8).

As a result of the determination, when an empty space exists in the buffer, it is determined whether a residual block exists (S321-6). If there is no residual block, the operation of selecting a block is terminated (S321-8).

If the residual block exists, the next block is obtained (S321-7). If the residual block does not exist, the operation of selecting the block is terminated (S321-8). In this way, a block selection operation is performed to obtain blocks for estimation of about 3 to 4 motions.

5, in step S321, the global search motion estimator 222 estimates the motion by precisely globally searching the selected block and the previous image, and then calculates a motion vector (S322). In this case, even if the global search is performed on just a few blocks, the search area is large and it is a burden on the calculation. Therefore, the search area is reduced by using the correlation of the previous and current global motion vectors to obtain the motion vector quickly. It is desirable to be.

Finally, the PMV selector 223 selects the predicted motion vector to be used when estimating the corresponding motion vector of all remaining blocks not selected from the estimated motion vector (S322). At this time, it is important to obtain an accurate predicted motion vector that can describe the motion of the background. If several blocks selected to determine the predicted motion vector are located in the background region with homogeneous brightness, Since the random motion far from the motion is set as the predicted motion vector, even if only the background exists, the motion compensation error value is largely calculated after the background compensation. For this reason, when selecting a block to obtain a predicted motion vector as described above, an edge component is included and a motion compensation error value is selected to be small.

4, in step S320, the block-based motion compensation unit 230 performs block-based motion compensation on the image from which the spatial high frequency component is removed using the predicted motion vector (S330). In this process, a motion compensation error image is generated which is a difference image between the current image from which the spatial high frequency components are removed and the motion compensated image.

The motion compensation is performed for all blocks, and by performing a local search having a very small search area using a global motion vector that well describes the motion of the background, the motion compensation error value is minimized and moved for the background area. The motion compensation is performed for the object area by maximizing the motion compensation error value.

In addition, as described above, motion compensation is performed for all blocks, and is actually independent of each block by performing a local search having a very small search area using a global motion vector that describes the background motion well. Because motion prediction is possible, it is possible to avoid morphological opening operations that require a large amount of computation to eliminate the error that occurs when compensating the entire background with one background motion vector.

Subsequently, the motion detector 240 compares the sum of the motion compensation error values of the respective blocks of the motion compensation error image with a threshold value, and determines that the block in which the sum of the motion compensation error values exceeds the threshold value is an area in which the object exists. By detecting the movement of the moving object (S340). Since the sum of the motion compensation error values is a value already obtained in the above-described background motion estimation and motion compensation process, no additional calculation is necessary.

Finally, the moving object variable extracting unit 250 calculates the center and the area of the object to obtain the variable of the object (S350).

As described above, the present invention can effectively predict the movement of the background with a small amount of computation in a limited search area, thereby quickly finding the motion of the object.

According to the present invention, in a background compensation apparatus and method for fast motion detection in an active camera environment, a local search having a very small search area is predicted using a fast prediction of a global motion vector that describes a background motion well. Accordingly, the search region for motion detection can be greatly reduced by minimizing the motion compensation error in the background region and maximizing the motion compensation error in the moving object region.

In addition, according to the present invention, since independent motion prediction is possible for each block, it is performed to eliminate an error occurring when compensating the entire background with one background motion vector, and a mathematical morphology that requires a large amount of computation ( morphological opening operations can be avoided.

In addition, according to the present invention, it is possible to implement real-time video tracking in a relatively low-cost system due to the small amount of computation, and to implement an independent system using a commercial digital signal processor (DSP).

Claims (5)

In a high speed motion detection apparatus of an object in an active camera environment, An image obtaining unit obtaining an image; A preprocessing unit including a low pass filter and removing spatial high frequency components by filtering a current image obtained by the image acquisition unit through a low pass filter; A global motion estimator that receives a current image and a previous image from the image acquirer and predicts a motion vector of a background; After receiving the current image and the previous image from which the spatial high frequency components are removed through the preprocessor, a block-based motion compensation is performed by using the predicted motion vector obtained from the global motion estimation unit, and then the compensation image is obtained. A block-based motion compensation unit for obtaining a motion compensation error image that is a difference image from a current image from which spatial and high frequency components are removed; Compares the sum of the motion compensation error values of each block of the motion compensation error image obtained from the block-based motion compensator with a predetermined threshold to move a block whose sum of motion compensation error values exceeds the threshold to an area where an object exists. A motion detecting unit to determine; And Moving object variable extraction unit for obtaining the variable of the object by calculating the center and area of the moving object for the block detected by the motion detector Background compensation apparatus for the high-speed motion detection of the object in the active camera environment, characterized in that it comprises a. The method of claim 1, wherein the global motion estimation unit, A block selecting unit which is a means for selecting a block necessary for motion estimation from a current image divided into small blocks of a predetermined size; A global search motion estimator configured to calculate a motion vector after performing motion estimation on the selected block; And Predicted motion vector selection unit for determining a predicted motion vector (PMV) used in estimating motion vectors of the remaining unselected blocks from the estimated motion vectors. Background compensation device for the high-speed motion detection of the object in the active camera environment characterized in that it comprises a. In a high speed motion detection method of an object in an active camera environment, Obtaining a current image and a previous image, and removing spatial high frequency components through a low pass filter; A second step of predicting a global motion vector from the acquired current image and the previous image; After generating a compensation image by performing block-based motion estimation and motion compensation using the current image and the previous image from which the spatial high frequency component is removed, and the predicted motion vector, the current image from which the compensation image and the spatial high frequency component are removed A third step of obtaining a motion compensation error image that is a difference image with the second image; Comparing the sum of the motion compensation error values of the respective blocks of the motion compensation error image with a threshold value and determining a block whose sum of the motion compensation error values exceeds a threshold value as an area where an object exists to detect the movement of the moving object. The fourth step; And The fifth step of obtaining the variable of the object by calculating the center and area of the moving object with respect to the detected block Background compensation method for the high-speed motion detection of the object in the active camera environment characterized in that it comprises a. The method of claim 3, The second step, A twenty-first step of dividing the current image into small blocks of a predetermined size and selecting a block necessary for motion estimation from the blocks; A twenty-second step of estimating motion by globally searching from the selected block and the previous image to calculate a motion vector; And A twenty-second step of determining a predicted motion vector (PMV) to be used in estimating a corresponding motion vector of all remaining blocks not selected from the estimated motion vector; Background compensation method for the high-speed motion detection of the object in the active camera environment, characterized in that consisting of. The method of claim 4, wherein The twenty-first step, Obtaining a first block from a current image divided into small blocks of a predetermined size; The image of the obtained block is filtered through a high pass filter (HPF), and as a result, it is determined whether the edge component is sufficiently included to determine whether it is an edge. If the result is not an edge, step 215 is performed. step; If the result of the determination is an edge, the sum of the motion compensation error values is compared with a threshold; and if the sum of the motion compensation error values is large, step 215 is performed; If the sum of the motion compensation error values is small as a result of the comparison, step 214 of storing a location of a current block in a buffer; Determining whether empty space exists in the buffer, and if the empty space does not exist, ending (215); If it is determined that the empty space exists, it is determined whether a residual block exists; and if the residual block does not exist, step 216 is finished; And If the residual block exists, the next block is obtained; and if the residual block does not exist, step 217 is completed. Background compensation method for the high-speed motion detection of the object in the active camera environment, characterized in that consisting of.