KR101210866B1 - An object tracking system based on a PTZ(Pan-Tilt-Zoom) camera using Mean-shift algorithm - Google Patents

Abstract

In the object tracking system using a zoom camera-based average movement algorithm that tracks an object to be monitored in real time using a rotation zoom camera and tracks the object in real time, and obtains the position of the object of the next frame from the previous frame by an average movement algorithm. An image input unit configured to receive a captured image of a surveillance target area from a rotation zoom camera; A preprocessor for preprocessing the captured image; An object extracting unit extracting an object from the captured image; An object tracking unit which calculates a moving point to which the object has moved using an average moving algorithm; And a camera controller for setting the focus of the rotation zoom camera to the moving point.
By the system as described above, it is possible to use a rotating zoom camera for fast tracking, real-time processing, continuous object tracking in a large surveillance area, and can be utilized in an efficient video surveillance system.

Description

An object tracking system based on a PTZ (Pan-Tilt-Zoom) camera using Mean-shift algorithm}

The present invention monitors an object to be monitored in real time by using a zoom camera to track an object in real time, and tracks an object using a zoom zoom camera-based average movement algorithm that obtains a position of an object of a next frame from a previous frame using an average moving algorithm. It's about the system.

In general, although the technology of detecting and tracking image objects has been studied for many years, it is still difficult to expect accurate, stable and high performance.

Tracking techniques vary greatly in how they define objects and their surroundings. It can vary depending on the shape of the object you want to track, how unique the color is, and how long you keep the features, and how the object moves or changes. And most of the objects that are tracked are affected by the environment because they have movement. That is, it is very sensitive to camera movement, object obstruction, change of lighting and surrounding environment.

In particular, in the Pan-Tilt-Zoom (PTZ) camera environment, which has recently been widely used as a surveillance camera, this problem is more prominent. In the PTZ camera surveillance system, a detailed image can be obtained by zooming in to the camera's gaze at an object to be viewed in detail in the surveillance area. PTZ cameras have the advantage of being able to detect all directions and have a uniform resolution compared to a single fixed camera.

However, in current PTZ camera surveillance systems, the object must be almost stationary as a prerequisite. Because zooming in makes the camera's field of view very narrow, there is a concern that intense image shaking may occur as the object moves, or the object may move out of the camera's field of view. Therefore, in PTZ camera surveillance system, it is always necessary to adjust the direction of PTZ camera according to the movement of the object.

Therefore, there is a need for a technology for implementing a system that monitors an area to be monitored in real time using a PTZ (Pan-Tilt-Zoom) camera and enables real-time tracking of an object. In particular, such a tracking system must be capable of fast tracking for real-time tracking.

An object of the present invention is to solve the problems described above, by monitoring the monitoring target area in real time using a rotation zoom camera to track the object in real time, the average moving algorithm of the position of the object of the next frame from the previous frame It is to provide an object tracking system using the average zoom algorithm based on the zoom camera.

That is, the present invention proposes a system for detecting and tracking an object in real time by applying a mean shift tracking algorithm which is partially modified with a kernel function. The proposed system uses the PTZ protocol and RS-485 communication to control the camera using the camera so that the position of the object on the screen is located in the center of the screen for real-time tracking.

In order to achieve the above object, the present invention relates to an object tracking system using a rotation zoom camera-based average moving algorithm that tracks an object in real time by monitoring a monitoring target area in real time using a rotation zoom camera. An image input unit configured to receive a captured image of a surveillance target area; A preprocessor configured to preprocess the captured image; An object extracting unit extracting an object from the captured image; An object tracking unit for calculating a moving point to which the object has moved using an average moving algorithm; And a camera controller for adjusting the focus of the rotation zoom camera to the moving point.

In addition, the present invention is an object tracking system using a rotation zoom camera-based average moving algorithm, the pre-processing unit is characterized by applying a background difference method to the image of the captured image, and performs a morphology calculation and labeling on the applied image .

In addition, the present invention is an object tracking system using a rotation zoom camera-based average moving algorithm, the average moving algorithm is the position where the similarity between the object model tracked in the previous image of the captured image and the candidate object model in the next image is the largest It is characterized by using an average shift vector for the purpose of finding.

In addition, the present invention is an object tracking system using a rotation zoom camera-based average movement algorithm, the average movement vector is characterized in that [Equation 1].

[Equation 1]

In addition, the present invention is an object tracking system using a rotation zoom camera-based average movement algorithm, the kernel function used in the average movement algorithm is characterized in that [Equation 2].

[Equation 2]

Where σ is a constant of the kernel function.

As described above, according to the object tracking system using the rotation zoom camera-based average movement algorithm according to the present invention, it is possible to quickly track and real-time processing using the rotation zoom camera, continuous tracking of objects in a large surveillance area Possible and effective effects are obtained for the video surveillance system.

That is, according to the object tracking system using a rotation zoom camera-based average movement algorithm according to the present invention, by using a mean shift tracking algorithm, it is suitable for real-time tracking with fast and stable performance, and also to move or rotate objects A robust tracking effect is obtained.

1 is a diagram showing a configuration of an overall system for carrying out the present invention.
2 is a block diagram of a configuration of an object tracking system using a rotation zoom camera-based average moving algorithm according to an embodiment of the present invention.
3 is an exemplary diagram of a format and a protocol of a transport packet for controlling a motor of a rotation zoom camera according to an embodiment of the present invention.
4 is a view illustrating a rotation zoom camera and a development environment used in the experiment of the object tracking system of the present invention.
5 is a screen of a PTZ control program for transmitting a packet to control a PTZ camera according to an embodiment of the present invention.
6 is an object tracking experiment image on a PTZ camera according to an embodiment of the present invention.

Description of the Related Art [0002]
10: zoom camera 20: computer terminal
30: object tracking system 31: video input unit
32: preprocessing unit 33: object extraction unit
34: object tracking unit 35: camera control unit
40: recording video

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the drawings.

In addition, in describing this invention, the same code | symbol is attached | subjected and the repeated description is abbreviate | omitted.

First, the structure of the whole system for implementing this invention is demonstrated with reference to FIG.

As shown in Figure 1, the whole system for implementing the present invention is composed of a zoom camera 10, a computer terminal 20, and the object tracking system 30, the zoom camera 10 and the The computer terminals 20 communicate data with each other via a communication line.

The rotation zoom camera 10 is a camera capable of horizontally or vertically rotating such as a PTZ (Pan-Tilt-Zoom) camera to shoot all directions, the computer terminal 20 is a computer function such as a PC, notebook, netbook Terminal device.

The object tracking system 30 may be implemented by software for image control (S / W) and installed and executed in the computer terminal. In another embodiment, the object tracking system 30 may be integrated into a general purpose microprocessor and a program, or may be implemented as an integrated semiconductor such as an ASIC.

The object tracking system 30 receives an image through one pan-tilt-zoom (PTZ) camera, extracts an object on a program of a computer terminal (PC, etc.), and detects movement. The object tracking system 30 sends a command packet to the camera to move the motor of the camera so that the object is centered on the screen. Command in PC program converts RS-232 communication into RS-485 communication using serial converter and transmits it.

Next, the configuration of the object tracking system 30 according to an embodiment of the present invention will be described in more detail with reference to FIG. 2.

As shown in FIG. 2, the object tracking system 30 according to an exemplary embodiment of the present invention includes an image input unit 31, a preprocessor 32, an object extractor 33, an object tracker 34, and Camera control unit 35.

The image input unit 31 receives the captured image 40 of the surveillance region from the rotation zoom camera 10.

The preprocessor 32 preprocesses the photographed image. In particular, the preprocessor 32 applies a background difference method to the image of the captured image, and performs morphology calculation and labeling on the applied image. In this way, the shape of the object can be complemented.

The object extractor 33 extracts an object from the captured image. In particular, the object extracting unit 33 performs an AND operation on the pixel value of the current image, which is a binary image representing the area of the object, to extract the object from the captured image.

The object tracking unit 34 calculates a moving point to which the object moves by using an average moving algorithm.

The object tracker 34 may track the color based, the motion based, the model based, etc. according to the selection of the feature value of the object.

As one embodiment, the proposed object tracking unit 34 is characterized by the color distribution of the object to represent the color distribution of the object as a histogram and apply it repeatedly with a mean shift algorithm. That is, the method finds the candidate region most similar to the object in the probability distribution of the next frame.

Mean shift algorithm is a method of finding the mode of non-parametric probability density function pdf to find the main mode (mode) in the distribution of sample points. Here, the probability density function means a color distribution of image pixels in the color space. Calculate the average position and average value in the color space of pixels with similar color distribution to the current pixel among the color values around the given pixel (space). If there is a maximum in (kernel window size, bandwidth), it will converge.

If we define q as the pdf model of the object we want to find and p as the pdf model of the candidate object we want to find in the next frame, we can define it as [Equation 1].

[Equation 1]

는 다음 프레임의 위치 y에서의 후보 pdf모델을 나타낸다. 객체 모델 q와 후보 모델 p간의 유사도를 측정하기 위하여 유사도를 측정함수를 [수학식 2]와 같이

로 정의 한다. 유사도 측정함수

는 Bhattacharyya Coefficient를 사용한다. 다음 프레임에서 객체 모델과 가장 유사한 후보 모델을 찾는 것이 목적이므로

값을 최대로 갖는 위치 y를 찾아야 한다.

Denotes a candidate pdf model at position y of the next frame. In order to measure the similarity between the object model q and the candidate model p, the similarity measurement function is expressed as [Equation 2].

It is defined as Similarity Measurement Function

Uses the Bhattacharyya Coefficient. The goal is to find the candidate model that most closely resembles the object model in the next frame.

We need to find the position y that has the maximum value.

&Quot; (2) "

는 객체 모델과 후보 모델간의 거리(distance)를 정의한다. 이는 [수학식 3]과 같이 나타낼 수 있으며,

값을 최대화 하는 것은 d(y)값을 최소화 하는 문제로 나타낼 수 있다.Similarity measurement function between two models

Defines the distance between the object model and the candidate model. This can be expressed as [Equation 3],

Maximizing the value can be seen as a problem of minimizing the d (y) value.

&Quot; (3) "

는 Bhattacharyya coefficient를 사용하고 계산의 복잡도를 줄이기 위하여 [수학식 4]와 같이 Taylor expansion을 수행한다. Similarity Measure Function

Uses the Bhattacharyya coefficient and performs Taylor expansion as shown in [Equation 4] to reduce the complexity of the calculation.

&Quot; (4) "

에서 새로운 위치

으로의 이동을 [수학식 5]와 같이 정의할 수 있다. 여기서 커널

이다. 기울기 벡터는

가 되고

이

보다 작을 때까지 반복하여 객체를 추적한다. In Equation 4, the term associated with y is the second term, and the position y where the value of the term is maximum is the position of the candidate model whose maximum similarity with the object model. This is also the position to minimize the distance d between the two models. In the second term, we apply the mean shift algorithm that repeatedly searches for the local maximum at each step.

New location in

The movement to can be defined as shown in [Equation 5]. Where kernel

to be. Slope vector

Become

this

Iterate through the objects until they're smaller.

[Equation 5]

Commonly used kernel functions use the Epanechnikov kernel, which is a continuous and simple reduction function. This is defined as shown in [Equation 6].

&Quot; (6) "

이므로 [수학식 7]과 같이 나타낼 수 있고

는 Uniform 커널이 된다.

Since it can be expressed as [Equation 7]

Becomes the Uniform Kernel.

[Equation 7]

If this is applied to [Equation 5], it can be expressed as a simple weighted average form as shown in [Equation 8].

[Equation 8]

In summary, the objective is to find the position y with the greatest similarity between the object model q to be traced and the candidate model p in the next frame, and find it using a mean shift vector that repeatedly searches for the local maximum.

Meanwhile, representative kernel functions using the Parzen window are as follows.

As shown in [Equation 9], there is a normal function.

&Quot; (9) "

As shown in Equation 10, there is an Epanechnikov function.

&Quot; (10) "

The kernel function used in the present invention is as shown in [Equation 11].

&Quot; (11) "

Where σ is a constant of the kernel function.

As shown in Equation 11, it can be seen that it is a linear function of x in the exponential function of the kernel function. In other words, since it is a linear function, the amount of calculation can be reduced considerably, while the accuracy can be reduced.

As shown in [Equation 5], using the kernel of normal or Epanechinikov, since x in the exponential function is a quadratic function, it is difficult to calculate in real time because the calculation amount is very large. Therefore, the present invention can be calculated in real time by reducing the amount of calculation by using the kernel function as a linear function of x.

On the other hand, as described above, since the pre-processing, such as excluding the background, etc., it is possible to solve the accuracy problem that may occur by using the first order function. In addition, since the main purpose is to record and store the image of the object moving through the surveillance camera, even if there is a slight error does not matter.

The kernel function in Equation (11) is exponential and bilaterally symmetric, and is less sensitive than the normal and Epanechnikov functions, but the time required to track an object can be reduced and the characteristics of the radially converging kernel can be reduced. Since the computation is smaller than the conventional kernel function in real time object tracking, it is advantageous in terms of speed.

The camera controller 35 controls to adjust the focus of the rotation zoom camera 10 to the moving point. That is, when the object moves, the camera control unit 35 controls the motor of the PTZ camera 10 so that the camera moves according to the movement (or moving point) of the object.

When the extracted moving object moves, the object tracking system 30 determines this and transmits a command packet to the camera through RS-485 communication to move the motor of the camera 10. By moving the motor of the camera 10 to position the object in the center of the screen it is possible to track the object.

3A and 3B are examples of formats and protocols of a transport packet for controlling a motor of the PTZ camera 10, respectively. When the command shown in FIG. 3B is transmitted to the rotation zoom camera 10, the fan moves to the right at a speed of 34.

Next, the results of the object tracking experiment performed by the object tracking system 30 according to the present invention will be described in more detail with reference to FIGS. 4 to 6.

As shown in Figure 4a, the experiment used a Samsung Techwin SPD-1000 PTZ dome camera in a variety of indoor environments. The size of one frame of video is 320 * 240 video data. In addition, the experimental PC development environment is shown in Figure 4b. 5 is a screen of a PTZ control program for transmitting a packet to control a PTZ camera.

Experimental data shooting was performed by changing the targets of several test objects in a PTZ camera environment. 6 is an object tracking experiment image on a PTZ camera.

6A illustrates frame data of an image stored at five frame intervals. The frames are arranged in order from left to right, top to bottom.

FIG. 6B is a diagram illustrating a result of processing the image of FIG. 6A. In other words, the background is clustered after removing the background and performing a preprocessing step. Each result of FIG. 6B corresponds to the image of FIG. 6A. That is, the image sequence of 6b is also arranged from left to right and top to bottom.

In Figure 6b, the large red arrow is the center of gravity of the cluster. As shown in FIG. 6B, when the mean shift algorithm is applied, several clusters are produced. For example, the neck part, the body part and the like. In this case, leave clusters larger than a fixed size and remove the remaining clusters.

FIG. 6C is a diagram of tracking the trajectory of the movement of an object using the preprocessed image of FIG. 6B. That is, FIG. 6C shows that the object moves in the two-dimensional image plane and moves from left to right. In other words, as the pan of the camera moves from left to right, the tilt value is slightly reflected.

As mentioned above, although the invention made by this inventor was demonstrated concretely according to the Example, this invention is not limited to an Example and can be variously changed in the range which does not deviate from the summary.

The present invention is useful for developing an object tracking system using a rotation zoom camera-based average moving algorithm that tracks an object in real time by monitoring a monitoring target area in real time using a rotation zoom (PTZ) camera.

Claims (5)

In an object tracking system using a rotation zoom camera-based average movement algorithm that monitors an object to be monitored in real time using a rotation zoom camera,
An image input unit configured to receive a captured image of a surveillance target region from the rotation zoom camera;
A preprocessor configured to preprocess the captured image;
An object extracting unit extracting an object from the captured image;
An object tracking unit for calculating a moving point to which the object has moved using an average moving algorithm; And
A camera controller for focusing the rotating zoom camera to the moving point;
The average shift algorithm uses an average shift vector for the purpose of finding a position having the greatest similarity between the object model tracked in the previous image of the captured image and the candidate object model in the next image.
Kernel function K _p (x) used in the average movement algorithm is an object tracking system using a rotation zoom camera based average movement algorithm, characterized in that [Equation 2].
[Equation 2]

Where σ is a constant of the kernel function.
The method of claim 1,
The preprocessing unit applies a background difference method to the image of the photographed image, and performs a morphology calculation and labeling on the applied image object tracking system using a mean zoom algorithm based on a zoom camera.
delete The method of claim 1,
The average movement vector is an object tracking system using a rotation zoom camera-based average movement algorithm, characterized in that [Equation 1].
[Equation 1]

delete

Images