KR101508310B1 - Apparatus and method for tracking multiple moving objects in video surveillance system - Google Patents

Abstract

The present invention relates to a method and an apparatus for tracking multiple moving objects in an image surveillance system, the method comprising the steps of: tracking, by an image surveillance apparatus, a moving area of at least one moving object using a RGB background modeling from the input image; imparting labels to each of the moving areas and merging the moving areas according to the imparted labels; identifying at least one moving object; searching for the location of a tracking object using the identified moving object; and tracking the object in the image based on the searched location.

Description

[0001] The present invention relates to a method and apparatus for tracking multiple moving objects in a video surveillance system,

The present invention relates to a method for tracking multiple moving objects in a video surveillance system, and more particularly, to a method for extracting a moving area by inputting an image, identifying an object, and tracking multiple moving objects.

The intelligent video surveillance includes a process of extracting a moving object after separating the background area from the input image, recognizing and tracking the object, and generating an event suitable for the monitoring situation from the identified object or the object being tracked. Here, in order to identify and track the moving object, it is necessary to be able to separate moving objects existing in the image. Typically, there are methods such as a Gaussian mixture model (GMM) and a background subtraction. The Gaussian mixture model is a useful method for image processing using probability distributions. It is effective when processing input images with various environmental changes, but has a disadvantage that it is slow. In addition, the method using the background difference can extract the moving object quickly after generating the background image, but the performance greatly depends on the background generating method. Kim et al. Proposed a background modeling method that extracts moving objects by proposing a background modeling method using RGB color model when extracting a moving object. The problem of extracting moving objects with similar colors to the background, The problem of extracting the image is effectively solved.

In the process of identifying the extracted moving object, when a plurality of extracted moving objects are extracted, or when one moving object is extracted as a plurality of moving areas, it is identified as a moving object that can be easily tracked. Contour-based classification is a classification method that uses data sets of points, box shapes, contours, and blobs in the extracted moving object. , Lipton et al. Used the extent and size of blobs to classify blobs for human, vehicle, and all other moving objects. Kim et al. Used moving vectors of blobs of each moving object to identify moving objects I have extracted it. However, these studies have a fatal disadvantage that each extracted moving object region must have one moving region for one moving object.

The process of tracking the moving object is to find the moving object identified in the previous image in the current image. Especially, the problem that tracked objects overlap each other or are blocked by background images is a big issue of tracking research, and various methods have been proposed by many researchers. Tracking techniques can be divided into four areas: region-based tracking, active contour-based tracking, feature-based tracking, and model-based tracking . Here, the area-based tracking algorithm is a method using a change in image or a structural characteristic corresponding to a moving object (usually, a tracking object is a human and uses a structure such as human head, body, and lower body) Depending on how you model your traceability, tracking performance can be influenced. In addition, the active contour-based tracking algorithm is a method of tracing an object by expressing an outline with an outline boundary and dynamically updating an outline in a succession of frames. Since the outline must be extracted, And it is very sensitive to the initialization of the tracking, so that there is a problem that automatic tracking is difficult. In addition, the feature-based tracking algorithm recognizes and traces an object by matching the features of the features with the features of the images through high-level clustering. It is easy to track when many features of the moving object are secured. (Size, color combination, etc.) of a moving object image for extracting a lot of computation amount and feature point. The model base is a method for measuring the structural characteristics of an object in advance and then matching it.

 Representative methods of using the color information of the moving object include Mean Shift and CAMshift algorithms. These algorithms extract objects at high speed using the color density distribution of the selected search window. The CAMshift algorithm, which adjusts the size of the search window itself to apply the mean shift to the continuous image, has been proposed and has the advantage of real-time processing. Although the CAMshift algorithm has been studied mainly as a method for human face area recognition, in recent years, it is possible to track an object by selecting a moving object area extracted using a car image as a search window, or to search for a moving object A method using invariant feature points, a Kalman filter, and a method for enabling more robust tracking using a particle filter. However, methods such as feature points and particle filters have a problem in that when the number of tracking objects increases, the amount of computation increases and real-time processing is not guaranteed. Also, there is a problem that the method using the image difference and feature information may fail to track when the image of the identified object does not include all of the area of the moving object.

In the meantime, the prior art cited below discloses a method for tracking multiple objects using a particle filter, and a method for tracking multiple objects by estimating multiple types of post-distribution using a nonlinear estimation method, particle filter algorithm The However, as described above, the prior art has a fatal disadvantage in that when the number of tracking objects increases, the amount of computation increases and real-time processing is not guaranteed.

From this point of view, it can be seen that there is a need for a technical means to solve the problem of object tracking using CAMshift and the identification of divided moving objects.

Published Patent Application No. 10-2009-0043016 (May 5, 2009)

Accordingly, a first problem to be solved by the present invention is to extract a moving area of a moving object using RGB background modeling from an input image, identify the moving object using a labeling technique, The present invention provides a method of tracking a moving object in a video surveillance system capable of tracking a position of a tracking object in a current image using a shift algorithm and tracking the position using a Kalman filter.

A second problem to be solved by the present invention is to extract a moving region of a moving object using RGB background modeling from an input image, identify a moving object by using a labeling technique, The present invention provides a device for tracking multiple moving objects in a video surveillance system capable of tracking the position of a tracking object in a current image and tracking the position using a Kalman filter.

There is also provided a computer-readable recording medium having recorded thereon a program for causing a computer to execute the above-described method.

According to an aspect of the present invention, there is provided a method of controlling a moving object, the method comprising: extracting a motion region of at least one moving object using RGB background modeling from an input image of the image monitoring apparatus according to an embodiment of the present invention; Identifying the at least one moving object by labeling the respective moving area by the video surveillance device and merging the moving area according to the assigned label; And tracking the moving object in the video surveillance system, wherein the video surveillance apparatus searches for the position of the tracking object using the identified moving object, and tracking the object in the image based on the detected position .

According to an embodiment of the present invention, the step of tracking an object may include searching for a location of the object to be tracked by performing a CAMshift algorithm according to the number of the identified moving objects and the number of tracking objects step; And tracking an object in the image using a Kalman filter at a position of the searched tracking object. The present invention also provides a method of tracking a moving object in a video surveillance system.

The step of tracking an object in the image according to an embodiment of the present invention includes the steps of: when the number of the objects to be tracked is larger than the number of the identified moving objects, And updating the search window by estimating the location of the tracked object if the location of the tracked object exists inside the image, Lt; / RTI >

The step of tracking an object in the image according to an embodiment of the present invention may further include the step of, when the number of the identified moving objects is greater than the number of the objects to be tracked, And registering the plurality of moving objects in the video monitoring system.

A rotated neighboring ellipse including the rotated minimum square area is generated from the one or more moving objects according to the embodiment described above and the rotated neighboring ellipses are re-extracted to minimize loss in area selection of the moving object Step < / RTI >

The step of identifying the at least one moving object according to an embodiment of the present invention may include: providing a label to each of the extracted one or more moving areas; Grouping motion regions including adjacent labels among the assigned labels to generate at least one motion region group; And identifying one or more moving objects using the label merging technique for the one or more motion area groups.

The method may further include deleting a label that does not exceed a predetermined size of the label-attached movement area according to the above-described embodiment, thereby deleting the meaningless area, thereby tracking multiple moving objects in the video surveillance system have.

In order to achieve the second object, an extraction unit extracts a motion region of at least one moving object using an RGB background modeling from an input image according to an exemplary embodiment of the present invention. An identification unit for identifying one or more moving objects by merging the moving areas according to a label given to each of the moving areas; And a tracking unit for searching for the position of the tracking object using the identified moving object and for tracking an object in the image based on the searched position.

The tracking unit according to an embodiment of the present invention performs a camshift algorithm according to the number of the identified moving objects and the number of tracking objects to search for the position of the object to be tracked, And the object is tracked in the image using a Kalman filter. In the video surveillance system, the moving object may be tracked.

If the number of the objects to be tracked is greater than the number of the identified moving objects, if the position of the tracking object exists in the boundary region of the image using the Kalman filter, And updates the search window by estimating a position of the tracked object if the position of the tracked object exists inside the image.

If the number of the identified moving objects is greater than the number of the objects to be tracked, the tracking unit according to the embodiment further includes registering a moving object not used for tracking the object as the tracking object The moving object may be a device for tracking multiple moving objects in a video surveillance system.

The identification unit according to an embodiment of the present invention assigns labels to the extracted one or more motion areas and generates one or more motion area groups by grouping the motion areas including adjacent labels among the assigned labels, Further comprising the step of identifying one or more moving objects using the label merging technique for the one or more motion area groups, but deleting a label that does not exceed a predetermined size of the labeled moving area, The moving object may be a device for tracking multiple moving objects in a video surveillance system.

A rotated neighboring ellipse including the rotated minimum square area is generated from the one or more moving objects according to the embodiment described above and the rotated neighboring ellipses are re-extracted to minimize loss in area selection of the moving object The moving object may be a device for tracking multiple moving objects in a video surveillance system.

According to another aspect of the present invention, there is provided a computer-readable recording medium having recorded thereon a program for causing a computer to execute a method for tracking multiple moving objects in the video surveillance system.

According to the present invention, it is possible to effectively combine moving objects composed of a plurality of regions and effectively track the position of an object even when the tracking objects overlap each other, by extracting a moving region by inputting an image, It is effective.

1 is a flowchart illustrating a method of tracking multiple moving objects in a video surveillance system adopted by embodiments of the present invention.
2 is a diagram illustrating a process of extracting a moving region using a background modeling method according to an embodiment of the present invention.
FIG. 3 is a diagram showing a labeling result of a moving region according to an embodiment of the present invention.
4 is a diagram illustrating the outline extraction result of a moving object according to an embodiment of the present invention.
FIG. 5 is a diagram illustrating a result of label merging according to an embodiment of the present invention.
6 is a diagram illustrating a result of generating a proximity ellipse of a label according to an embodiment of the present invention.
7 is a flowchart illustrating an object tracking method using a moving object and a camshift according to an embodiment of the present invention.
FIG. 8 is a block diagram illustrating an apparatus for tracking multiple moving objects in a video surveillance system adopted by embodiments of the present invention.
9 is a block diagram illustrating a method of tracking multiple moving objects in a video surveillance system according to an embodiment of the present invention.
FIG. 10A is a diagram illustrating a result of an identification and tracking experiment of a moving object according to an embodiment of the present invention.
10B is a graphical representation of the number of identified objects in accordance with an embodiment of the present invention.
FIG. 10C is a graph illustrating an object tracking error result according to an exemplary embodiment of the present invention. Referring to FIG.
FIG. 10D is a graph showing the time-required results according to an embodiment of the present invention.

Before describing the embodiments of the present invention, in order to solve the problems occurring in the conventional video surveillance method mentioned in the background of the above-mentioned invention, the technical means adopted by the embodiments of the present invention is introduced in an overview .

The present invention proposes a three-step method for solving the problem of object tracking using the divided moving object identification and camshift. First, a background modeling method using an RGB color model is used to extract moving objects in an image. Second, a labeling method is applied to each blob to identify moving objects extracted from a plurality of divided blob sets. At this time, a label-merging method of identifying a moving object using the contour distance information of the label is proposed. Finally, we propose a method of tracking the detected position of the camshift using the Kalman filter after selecting the identified moving object region as the search window of the camshift. Finally, in order to prove the applicability of the method proposed by the present invention, a video surveillance system is constructed in an indoor environment and it is proved by analyzing several experiments and results

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the following description and the accompanying drawings, detailed description of well-known functions or constructions that may obscure the subject matter of the present invention will be omitted. It should be noted that the same constituent elements are denoted by the same reference numerals as possible throughout the drawings.

FIG. 1 is a flowchart illustrating a method of tracking multiple moving objects in a video surveillance system adopted by an embodiment of the present invention. Referring to FIG. 1, Extracting a region, labeling each of the movement regions, merging the movement regions according to a given label, identifying one or more moving objects, and searching for the position of the tracking object using the identified moving object , And tracks the object in the image based on the searched position.

More specifically, in step S110, the motion monitoring unit extracts a motion region of one or more moving objects using the RGB background modeling from the input image.

More specifically, background modeling based on a color model was proposed by Mapiglu et al. As a method of modeling the background by learning the pixel change of the input image color. In this method, the maximum and minimum values of pixels are extracted from each gray image to generate a background in the input image, and a background modeling parameter is selected as the intensity value of each pixel having different maximum-minimum values. This method has the advantage of extracting moving objects at very high speed after background modeling. Background modeling for a single image channel such as a gray image can be expressed as follows.

는 z번째 입력 영상의 해당 픽셀 위치

에서의 그레이 값,

와

는 모든 입력 영상의 해당 픽셀에서의 중간값 및 표준 편차를 나타내며, M, N은 해당 픽셀 위치

의 최대, 최소 값을 D는 이전 영상과 현재 영상 간의 최대 픽셀 변화 값일 수 있다.here,

Is the pixel position of the z-th input image

The gray value at < RTI ID =

Wow

Represents the median value and standard deviation of the corresponding pixel of all the input images, M and N denote the corresponding pixel positions

And D may be a maximum pixel change value between the previous image and the current image.

However, the background model using the gray image is sensitive to noise such as shadows when extracting a motion region. Especially, when the background is white, such a problem becomes more conspicuous. FIG. 2 shows a result of extracting a moving object using a background image using a gray image. Referring to FIG. 2, in the lower left corner of FIG. 2, an area denoted by an ellipse is extracted in addition to the extracted motion area. This noise may not be a problem because there are several ways to simply remove some small noises. However, if such noise continues to occur repeatedly and the size of the area becomes large, it is recognized as a meaningful moving object in the image, Becomes difficult. In order to solve this problem, Kim et al. Proposed a method of extracting a moving area by using an RGB color model. In Fig. 2, a left side (23) Effectively removing noise other than the area. Accordingly, in the embodiment of the present invention, the moving area is extracted using the RGB background modeling method.

In step S120, the video surveillance apparatus identifies one or more moving objects by labeling the respective moving areas and merging the moving areas according to the provided labels. In other words, labels are assigned to the extracted one or more motion areas, and one or more motion area groups are generated by grouping the motion areas including adjacent labels among the assigned labels, One or more moving objects may be identified using a merge technique.

More specifically, in order to recognize the moving area extracted in step S110 as a meaningful object, a label should be given to each of the blobs. Labeling is a typical method of labeling a region. Here, the labeling technique may delete a meaningless area by discarding a label (noise or moving area) that does not exceed a predetermined size by a typical connected component technique of labeling the areas. Here, the predetermined size may be arbitrarily set by the user depending on the situation.

FIG. 3 is a diagram showing a labeling result of a moving area according to an embodiment of the present invention. In FIG. 31, the upper left image 31 and the upper right image 32 are images of the input moving area, (33) and the lower right picture (34) may be labeled results. Here, it can be seen that the moving region inputted in the lower left figure 33 is identified as 4 (red square), and in the right lower figure 34, it is identified as 3.

In the labeling result of FIG. 3, there may be an important problem for tracking a moving object. In other words, it is a matter of how to select a moving area as a representative area because of the problem of moving objects represented by two or more. In an embodiment of the present invention, a label merging (LM) method is proposed in which a moving object composed of a combination of two or more regions is created as one moving object to solve this problem.

라 하고, 각

의 외곽선을

라 하면, 병합된 라벨 결과

다음과 같은 병합 과정으로 표현할 수 있다.More specifically, the labeling results of the identified m moving regions are

Respectively,

Outline of

, The merged label result

It can be represented by the following merging process.

으로

은 추출된 이동 객체 번호, k는 새로운 라벨 번호,

는 a,b 위치 간의 유클리디언(Euclidean) 거리,

는 사용자 정의 병합 거리(픽셀 단위)일 수 있다.here,

to

Is the extracted moving object number, k is the new label number,

Is the Euclidean distance between a and b positions,

May be a user-defined merge distance (in pixels).

으로 표현되며, 추출된 외곽선은 도 4와 같을 수 있다. 다음, 각 라벨의 모든 외곽선들의 위치를 비교하여 사용자가 정의한 거리

보다 작은 거리를 갖는 라벨은 새로운 라벨

로 정의하여 과정을 완료한다. 그러면, 작성된 새로운 라벨은 병합된 라벨의 모든 외곽선을 포함하며, 도 5에서와 같이 병합된 하나의 라벨을 포함할 수 있다.First, the contour extraction of the moving object is performed by extracting a set of all pixels located at the outermost side of the label image acquired in the labeling process

And the extracted outline may be as shown in FIG. Next, the positions of all outlines of each label are compared and the user defined distance

Labels with smaller distances will have a new label

And the process is completed. Then, the new label created includes all the outlines of the merged label, and may include one label merged as in FIG.

In addition, a rotated melee ellipse including the rotated minimum square area is generated from the one or more moving objects, and the loss in area selection of the moving object can be minimized by re-extracting the rotated neighboring ellipses.

More specifically, a moving object tracking method using a camshift (CAMshift) and a Kalman filter (to be referred to in step S130 of FIG. 1) has a step of setting an area of the extracted moving object as a search window of the camshift . Therefore, it is necessary to select an area that can minimize the loss of the moving object. Looking at the rectangular area shown by merged label 1 53 and merged label 2 54 in FIG. 1, the selected area of merged label 2 54 is relatively well selected for the area of the moving object, but the merged label 1 (53), it can be seen that the moving object region is selected only up to the upper half of the person. This is because the moving object is obscured by obstacles or similar to the background color.

Therefore, in the embodiment of the present invention, a proximity ellipse (Fitellipse) method can be applied to minimize the loss when selecting the area of the moving object. The proximity ellipse method is a method of generating a rotated neighboring ellipse including a rotated minimum square area of a moving object and re-extracting a rectangular area including the neighboring ellipse. Here, a rectangular area including the extracted adjacent ellipses is selected as an area of the LM, and can be expressed as follows.

는 선택된

의 사각형 영역,

는 근접 타원 추출 함수,

는

의 외곽선일 수 있다. 도 6은 수학식 3을 통해 계산된 회전 사각형 및 근접 타원과 근접 타원을 포함하는 사각형 영역일 수 있다.here,

Is selected

A rectangular area of <

A proximity ellipse extraction function,

The

As shown in FIG. 6 may be a rotated rectangle calculated by Equation (3) and a rectangular region including a proximity ellipse and a proximity ellipse.

Returning now to FIG. 1, step S120 and subsequent steps will be described.

In step S130, the video surveillance apparatus searches for the location of the tracking object using the identified moving object, and tracks the object on the basis of the searched location. In other words, a CAMshift algorithm is performed according to the number of the identified moving objects and the number of tracking objects to search for the location of the object to be tracked, and a Kalman filter ) Can be used to track an object in the image. If the number of the objects to be tracked is greater than the number of the identified moving objects, the tracking object is deleted if the position of the tracking object exists in the boundary region of the image using the Kalman filter. If the location exists inside the image, the location of the tracking object can be estimated to update the search window. Also, when the number of the identified moving objects is greater than the number of the objects to be tracked, a moving object that is not used for tracking the object may be registered as the tracking object.

More specifically, the camshift algorithm, which is often used for object tracking, has several problems as follows. Since the size of the initial search window and the position of the initial search window can not be changed automatically, the user can directly set the search window so that accurate tracking performance can be obtained. In addition, in the case of a high-speed object, the object moves out of the search window, and CAMshift can not converge the search window to the position of the object. In order to solve these problems, in the conventional research, by detecting the object by separating the background and object, the position and size of the search window are designated, or the object tracking algorithm such as the Kalman filter is applied to generate the prediction model, And the like are continuously being carried out.

Therefore, in order to solve the problem of tracking the irregular moving object and changing the search window of the moving object, the area of the object using the proximity ellipse of the extracted moving object is set as the search window, We propose a method to track the position of a found object using a Kalman filter.

FIG. 7 is a flowchart illustrating an object tracking method using a moving object and a camshift according to an embodiment of the present invention. Referring to FIG. 7, step S130 in FIG. 1 will be described in more detail.

는 입력 영상,

는 추적 객체,

은 이동 객체,

는 추적 객체 수,

는 이동 객체 수,

캠시프트 알고리즘,

는 캠시프트 알고리즘이 탐색한 추적 객체의 위치,

는 추적 객체의 위치,

는 유클리디언 거리,

는 입력 영상 V의 크기(너비와 높이),

는 추적 객체의 메모리 할당,

는 메모리 해제를 의미할 수 있다.More specifically, the tracking method proposed in the present invention has an object tracking 71, an object deletion 72, and an object registration 73 according to the number of moving objects entering a video and the number of moving objects being tracked, A tracking process 74 using a filter can be performed. Before describing the flowchart of FIG. 7, the meaning of the symbols used in FIG. 7 will be described.

The input image,

A tracking object,

A moving object,

The number of trace objects,

The number of moving objects,

Camshift algorithm,

The position of the tracking object searched by the camshift algorithm,

The location of the tracking object,

Euclidian Street,

(Width and height) of the input image V,

The memory allocation of the tracking object,

May mean memory release.

7, when the number of moving objects and the number of tracking objects are equal, the object track 71 searches for the tracking object in the input image using the camshift algorithm, and the position of the tracking object It can be tracked using a Kalman filter.

In addition, the object deletion step (Object delete) 72 is a case in which the tracking object is not visible or out of the image when the number of tracking objects is larger than the number of moving objects. If the tracking object exists in the boundary of the image after performing the same tracking process as the object tracking step, it can delete the corresponding tracking object, estimate the position when it exists inside, and update the search window.

When the number of moving objects is greater than the number of tracking objects, the object registration step (73) performs a tracking process identical to that of the object tracking step in a state where a new object enters the moving area. You can register it as a trace object.

와 속도

를 상태 변수로 나타내며 다음과 같이 표현할 수 있다.Here, the process of searching for the position of the moving object in the image using the camshift and tracking it using the Kalman filter may be as follows. First, the pixel position of the moving object in the image

And speed

Is expressed as a state variable and can be expressed as follows.

는 추적 객체의 위치로

일 수 있다.here,

Is the location of the tracking object.

Lt; / RTI >

, 제어 입력과 상태 변화와의 관계를 나타내는 행렬 B, 프로세서 잡음

로 나타내며 다음과 같이 표현할 수 있다.If the state at time k is represented by the state variable at time (k-1), then the transition matrix A, the control input

A matrix B representing the relationship between the control input and the state change, a processor noise

And can be expressed as follows.

The measured value is the detected position of the tracking object detected by the camshift and can be represented by the state variable X and the measurement noise v as shown in Equation (6) below.

로 탐지된 객체의 위치만을 가지며,

는 측정오차로 가우시안 분포

를 따른다고 가정한다.

는 크기를 갖는 공분한 행렬이다. 이때, 영상 내에서 탐지된 객체의 속도는 알 수 없다. 따라서, 행렬

는 이하에서 수학식 7과 같은 형태가 되고, 수학식 5 및 수학식 6을 통해 칼만 필터의 처리 과정을 수행할 수 있다.here,

Only the position of the detected object is detected,

Gaussian distribution with measurement error

. ≪ / RTI >

The It is a coarse matrix with dimensions. At this time, the speed of the detected object in the image is unknown. Therefore,

Can be expressed as Equation (7) below, and the processing of the Kalman filter can be performed through Equations (5) and (6).

FIG. 8 is a block diagram illustrating an apparatus for tracking multiple moving objects in a video surveillance system adopted by an embodiment of the present invention. In an apparatus 80 for tracking multiple moving objects in a video surveillance system, 1, respectively. Therefore, in order to avoid duplication of explanation, the function is outlined mainly in the detailed configuration of the system.

The extracting unit 81 extracts a moving region of one or more moving objects from the input image using RGB background modeling.

The identifying unit 82 identifies one or more moving objects by labeling the respective moving areas and merging the moving areas according to the given label.

The tracking unit 83 searches for the position of the tracking object using the identified moving object and tracks the object in the image based on the detected position.

The tracking unit 83 searches for the position of the object to be tracked by performing a camshift algorithm according to the number of the identified moving objects and the number of tracking objects, So that an object can be tracked in the image.

If the number of the objects to be tracked is greater than the number of the identified moving objects, the tracking unit 83 deletes the corresponding tracking object if the position of the tracking object exists in the boundary region of the image using the Kalman filter. And if the position of the tracking object exists in the image, the position of the tracking object can be estimated to update the search window.

If the number of the identified moving objects is greater than the number of the objects to be tracked, the tracking unit 83 may register the moving object that is not used for tracking the object as the tracking object. Multiple moving objects can be tracked in the system.

In addition, the identification unit 83 assigns labels to the extracted one or more motion areas, and generates one or more motion area groups by grouping the motion areas including adjacent labels among the assigned labels, Further comprising the step of identifying one or more moving objects using a label merging technique for a moving area group, and deleting a label that does not exceed a predetermined size of the labeled moving area, An apparatus for tracking multiple moving objects in a system.

In addition, a rotated melee ellipse including the rotated minimum square area is generated from the one or more moving objects, and the loss in area selection of the moving object can be minimized by re-extracting the rotated neighboring ellipses.

On the other hand, a flowchart showing a method of tracking multiple moving objects in the projection of FIG. 1 can be expressed as a block diagram of FIG. 9 below.

FIG. 9 is a block diagram illustrating a method of tracking multiple moving objects in a video surveillance system according to an embodiment of the present invention. The block diagram of FIG. 9 includes a configuration corresponding to each process of FIG. 1 described above The detailed description of FIG. 9 and FIG. 9 will be replaced with the detailed description of FIG. 1, and each block will be briefly described here.

9 shows an example of a moving object extraction (MO-E) 91 that receives an image from a camera installed in a monitoring space, identifies the object as an object (MO-I: Moving Object-Identification) And Moving Object-Tracking (MO-T) 93, as shown in FIG. Here, the processing contents and goals for each step may be as follows. The MO-E 91 generates a background using an RGB background modeling method and extracts a moving region that has entered the monitoring space. In addition, the MO-I 92 can identify each moving object by assigning a number to the extracted moving area using a labeling technique, and then using a label merging method of merging adjacent numbers. In addition, the MO-T 93 can search the position of the tracked object in the current image using the camshift algorithm and track the position of the identified moving object or the tracked object (Track object) using the Kalman filter have.

Hereinafter, differences between the prior art and the present invention will be described with reference to FIGS. 10A to 10D.

For the experiment of the method proposed by the present invention, a system as shown in FIG. The above system obtained RGB color image of 640 * 480 from one IP camera (Samsung Techwon, SNP-1000A) and installed C / C ⁺⁺ on a desktop PC (CPU: i7, 8GB RAM, SSD 128GB) , And OpenCV.

FIG. 10A is a diagram illustrating a result of tracking and experimenting a moving object according to an exemplary embodiment of the present invention. FIG. 10A shows a result of a tracking experiment with a moving object entering a monitored space, The results of tracking the identified ellipse results by using the camshift and the Kalman filter on the right image.

In the experimental results, the tracking numbers are given to each moving object as the moving object enters the picture (a) (101), the picture (b) (102), and the picture (c) (104) and (e) (105) show that the tracking of the moving object is successfully performed even when the moving objects overlap each other. Detailed comparison data according to the experimental result of FIG. 10A is shown in FIGS. 10B to 10D.

10B is a graphical representation of the number of identified objects in accordance with an embodiment of the present invention.

More specifically, the number of objects identified using labeling method objects in the extracted moving area is compared with the number of objects identified using label-merge method objects proposed by the present invention. The number of ID objects is effectively reduced through the label merging method proposed in the present invention. However, the number of ID objects can be effectively reduced by using the label merging method proposed in the present invention .

FIG. 10C is a graph illustrating an object tracking error result according to an exemplary embodiment of the present invention. Referring to FIG.

More specifically, the position (GT), the camshift search position, and the position traced by applying the Kalman filter (the detected position of the moving objects in the image to the end of the tracking Respectively. The results of accumulating the tracking errors from the frame at which the tracking was started to the end of the tracking with respect to the tracking error of FIG. 10C are shown in Table 1 below.

Table 1 shows that the tracking method using the camshift and the Kalman filter proposed by the present invention had a smaller error than the tracking using only the camshift according to the prior art and showed 3.69%, 4.74%, and 1.67 %, It can be confirmed that the error is small.

FIG. 10D is a graph showing a time required result according to an embodiment of the present invention, illustrating the processing time of the method proposed by the present invention.

More specifically, the initial background modeling process performed 10 images at about 150 msec (66 frames / sec), and the label merging method after moving objects showed an average processing speed of 23 msec. The camshift operation time increased by about 10msec per object as the number of trace objects increased, and the Kalman filter had a maximum operation speed of 3ms (maximum number of trace objects: 3) with a computation time of about 2msec. The total time required using the method proposed by the present invention (Label-Merge + CAMshift + Kalman filter) is 16 msec when there is no tracking object, 35 msec / sec and 41 msec / , And 51 msec (20 frames / sec), and it can be seen that the method proposed by the present invention is applicable to a video surveillance system having real-time processing.

According to the present invention, a method of detecting a moving object extracts a region of a moving object by using an RGB background modeling method capable of effectively extracting a moving object having a color similar to a background. The moving object identification method proposed a label merging method for identifying a moving object represented by a plurality of divided area sets as one or a plurality of objects when extracting a moving object. Finally, the moving object tracking method proposed a method of selecting the search window of the camshift algorithm and a method of tracking the position searched by the camshift using the Kalman filter according to the number of detected moving objects and the number of tracked objects. Experimental results show that the method proposed by the present invention can effectively merge moving objects composed of a plurality of regions and effectively track the position of the object even when the tracking objects overlap each other and can be processed in real time .

Meanwhile, the embodiments of the present invention can be embodied as computer readable codes on a computer readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored.

Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device and the like, and also a carrier wave (for example, transmission via the Internet) . In addition, the computer-readable recording medium may be distributed over network-connected computer systems so that computer readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the present invention can be easily deduced by programmers skilled in the art to which the present invention belongs.

The present invention has been described above with reference to various embodiments. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

80: Device for tracking multiple moving objects in a video surveillance system
81:
82:
83:

Claims (14)

Extracting a moving region of at least one moving object from an image input by the video monitoring apparatus using RGB background modeling;
Identifying the at least one moving object by labeling the respective moving area by the video surveillance device and merging the moving area according to the assigned label; And
Wherein the video surveillance apparatus searches for a position of a tracking object using the identified moving object and tracks an object in the video based on the detected position,
Generating a rotated neighboring ellipse including a rotated minimum square area in the at least one moving object and re-extracting the rotated neighboring ellipse, thereby minimizing loss in area selection of the moving object A method for tracking multiple moving objects in a video surveillance system. The method according to claim 1,
Wherein tracking an object in the image comprises:
Searching for a location of the object to be tracked by performing a CAM SHIFT algorithm according to the number of the identified moving objects and the number of tracking objects; And
And tracking an object in the image using a Kalman filter at a location of the searched tracking object. 3. The method of claim 2,
Wherein tracking an object in the image comprises:
Wherein if the number of the objects to be tracked is greater than the number of the identified moving objects, the tracking object is deleted if the position of the tracking object exists in the boundary region of the image using the Kalman filter, And updating the search window by estimating a position of the tracked object if the tracked object exists inside the image. 3. The method of claim 2,
Wherein tracking an object in the image comprises:
And registering, as the tracking object, a moving object not used for tracking the object, when the number of the identified moving objects is larger than the number of the objects to be tracked. Way. delete The method according to claim 1,
Wherein identifying one or more moving objects comprises:
Assigning labels to the extracted one or more motion regions, respectively;
Grouping motion regions including adjacent labels among the assigned labels to generate at least one motion region group; And
And identifying one or more moving objects using the label merging technique for the one or more motion area groups. The method according to claim 6,
Further comprising deleting a label that does not exceed a predetermined size of the label-attached movement area, thereby deleting a meaningless area. A computer-readable recording medium storing a program for causing a computer to execute the method according to any one of claims 1 to 4 and 6 to 7. An extracting unit for extracting a moving region of one or more moving objects from an input image using RGB background modeling;
An identification unit for identifying one or more moving objects by merging the moving areas according to a label given to each of the moving areas; And
And a tracking unit for searching for the position of the tracking object using the identified moving object and for tracking the object in the image based on the searched position,
Generating a rotated neighboring ellipse including a rotated minimum square area in the at least one moving object and minimizing loss in area selection of the moving object by re-extracting the rotated neighboring ellipse, For tracking multiple moving objects. 10. The method of claim 9,
The tracking unit includes:
A camshift algorithm is performed according to the number of the identified moving objects and the number of tracking objects to search for the position of the object to be tracked, and the object is tracked using the Kalman filter Wherein the moving object is a moving object. 11. The method of claim 10,
The tracking unit includes:
Wherein if the number of the objects to be tracked is greater than the number of the identified moving objects, the tracking object is deleted if the position of the tracking object exists in the boundary region of the image using the Kalman filter, And updates the search window by estimating the position of the tracked object if it exists inside the image. 11. The method of claim 10,
The tracking unit includes:
And registering, as the tracking object, a moving object not used for tracking the object, when the number of the identified moving objects is larger than the number of the objects to be tracked. Device. 10. The method of claim 9,
Wherein,
And labeling the at least one motion area group by a label merging technique. The method of claim 1, wherein the at least one motion area group includes at least one of a plurality of labels, To identify one or more moving objects,
Further comprising deleting a label that does not exceed a predetermined size of the label-attached movement area, thereby deleting the meaningless area. delete

Images