KR20140114594A - Auto-Camera Calibration Method Based on Human Object Tracking - Google Patents

Abstract

The present invention relates to a method to automatically calibrate a camera based on human object tracking. The method comprises: a video input step of receiving a video from a camera or the like; an object tracking determining step of determining whether to track objects based on the video received in the video input step; a moving object tracking step of tracking moving objects to obtain bounding box coordinates when a tracking instruction starts in the object tracking determining step; a single human object identifying step of identifying a single human object among the moving objects tracked in the moving object tracking step; a tracking termination determining step of determining whether a tracking termination instruction is generated when the identification of the single human object is completed in the single human object identifying step; and a camera parameter calculating step of terminating the tracking of the single human object, and calculating a camera parameter when the tracking termination instruction is generated in the tracking termination determining step. Therefore, human objects can be sensed and tracked from a video obtained by a CCTV camera, and a camera parameter value can automatically be calculated using the tracking result of the human objects.

Description

[0001] The present invention relates to an automatic camera calibration method,

 The present invention relates to an automatic camera correction method using human object tracking, and more particularly, to a method of automatically tracking a human object from a camera image and automatically correcting the camera using the tracking result.

In recent years, interest in an intelligent CCTV system that detects and tracks moving objects by analyzing real-time video images input from a CCTV camera and automatically detects meaningful events based on the moving objects is increasing. In the early days, it was simply used as a kind of "advanced motion detector" which detects an event that a moving object intrudes into a specific area from the image. However, recently, as the application range of the intelligent CCTV system is increased, There is a growing demand for data extraction. Accordingly, not only the detection and tracking technology of moving objects but also the classification and identification techniques of objects have become important. In order to classify and identify an accurate object, it is necessary to measure the size, position, and moving speed of the object in the actual 3D space, not the size, position, and moving speed of the object in the image. In order to obtain 3D spatial information from 2D image information, it is necessary to know the geometrical information (i.e., camera parameters) for the " camera " that projects 3D spatial information into 2D image information. Typical camera parameters include focal length, installation height, and rotation angle. The task of finding out these camera parameter values is called camera calibration.

   A conventional camera correction method is a method of accurately capturing a 3D artificial pattern (e.g., a plaid pattern) having a known structure, using a camera, and then calculating 2D pixel coordinates (e.g., corner points) of feature points (e.g., corner points) Value, the 3D space coordinate value for the feature points in the corresponding 3D artificial pattern, and the camera projection value using the camera projection model. Korean Patent Registration No. 10-0748719 entitled " 3-D Modeling Apparatus Using Multi-Stereo Camera and Method Thereof " The camera correction method requires a 3D artificial pattern to be installed and photographed every time the camera is installed or changed, and the feature points extracted from the photographed image must be matched with the minutiae points of the 3D artificial pattern. Also, the use of 3D artificial patterns may be restricted depending on the camera installation environment.

   Korean Patent Registration No. 10-0969576 entitled " Apparatus and Method for Calibrating Camera Parameters " discloses an apparatus and method for performing camera calibration by mutual operation of a user. When the user inputs a value for the three camera parameters (FOV, installation height, tilt angle), the device displays a virtual 3D model (e.g., a virtual 3D model for the person standing on the floor and the floor) 3D model) is projected onto the virtual camera image, and the virtual image is superimposed on the actual camera image in a translucent manner. The user compares the virtual image with the actual camera image to determine whether the inputted camera parameter value is correct, and repeats the operation of changing the camera parameter value until the virtual image matches the actual camera image. The apparatus and method are advantageous in that camera correction can be performed without a separate 3D artificial pattern, but it is inconvenient to perform camera correction by repeated manual operations of the user.

Korea Patent Office Registration No. 10-0969576

 It is an object of the present invention to provide a method of automatically performing camera correction without additional 3D artificial patterns or repetitive manual operations of a user under a conventional CCTV surveillance environment, Specifically, the present invention provides an automatic camera correction method using human person tracking, which detects and tracks moving human objects from a CCTV camera image and automatically calculates camera parameter values using tracking results of human objects .

According to an aspect of the present invention, there is provided a method for controlling an object, comprising: inputting a video image from a camera or the like; determining whether an object is traced through a video input in the inputting step; A moving object tracking step of tracking a moving object so as to obtain coordinates of a bounding box, a single human object identification step of identifying a single human object among the moving objects tracked in the moving object tracking step, A step of determining whether a termination command for tracking is generated when the identification of the single person object is completed in the single person object identification step; A camera parameter calculation unit for terminating the tracking for the camera and calculating the camera parameters The method of claim 1, further comprising:

The automatic camera correction method according to the present invention as described above performs camera correction automatically without additional 3D artificial patterns or repetitive manual operations of the user under a conventional CCTV surveillance environment, Detects and tracks human objects moving from the image, and automatically calculates camera parameter values using the tracking result of the human object.

1 is a view for explaining a method of measuring an actual size and position of an object from a single camera,
2 is a flowchart of an automatic camera correction method according to an embodiment of the present invention,
3 is a photograph showing an example of tracking movement objects in a video captured by a surveillance camera under a normal CCTV surveillance environment,
4 is a flowchart of an automatic camera correction method according to another embodiment of the present invention,
5 is a photograph showing an example of displaying the actual size and the actual moving speed of the tracking object calculated by the automatic camera correction method according to another embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings in order to facilitate a person skilled in the art to easily carry out the present invention.

1 is a view for explaining a method of measuring an actual size and position of an object from a single camera. However, in order to measure the actual size and position of an object from a single camera, the environment must satisfy the following conditions.

(1) The ground is flat.

(2) The column where the camera 101 is installed is perpendicular to the ground.

(3) The installed camera 101 does not have a rotation component about the optical axis.

(4) The lowermost end of the object 102 touches the ground.

라고 하면, 지면상의 특정 포인트의 좌표(X,Y)와 이에 대응하는 카메라(101) 영상에서의 픽셀 좌표(x,y) 사이에는 하기 <수학식 1>과 같이 주어진다.The focal length of the camera 101 is f, the installation height of the camera 101 is H, and the tilt angle of the camera 101 Theta, the center coordinates of the image

(X, y) of the specific point on the ground and the pixel coordinates (x, y) in the corresponding image of the camera 101 are given by Equation (1) below.

Equation 1

, 이에 대응하는 지면 좌표를

, 영상에서 물체(102)의 최상단 중심의 픽셀 좌표를

, 이에 대응하는 지면 좌표를

라고 하면, 물체(102)의 실제 높이 h는 하기 <수학식 2>와 같이 주어진다.In Equation (1),? Has an arbitrary value other than zero. The pixel coordinates of the lowermost center of the object 102 in the image

, And the corresponding ground coordinates

, The pixel coordinates of the uppermost center of the object 102 in the image

, And the corresponding ground coordinates

, The actual height h of the object 102 is given by Equation (2) below.

Equation 2

와

를 알면, 상기 <수학식 1>과 <수학식 2>에 의해 물체(102)의 실제 높이 h와 실제 위치

를 구할 수 있다.Therefore, in the image, the coordinates of the lowermost end of the object 102 and the coordinates

Wow

The actual height h of the object 102 and the actual position h of the object 102 can be calculated from Equations (1) and (2)

Can be obtained.

To use Equation (1), the camera parameters f,?, And H must be known. The task of obtaining these camera parameter values is called camera calibration. In the present invention, a method of automatically calibrating a camera using a tracking result of a human object in an image is proposed.

FIG. 2 is a flowchart illustrating an automatic camera calibration method according to an exemplary embodiment of the present invention. FIG. 3 is a photograph illustrating an example of tracking moving objects in an image captured by a surveillance camera under a conventional CCTV monitoring environment.

In one embodiment of the present invention, a method for calculating camera parameters based on tracking results of a &quot; single person object that knows an average actual height (key) &quot; A step S102 of determining whether or not to trace an object through an image input in the image input step S101; and a step of determining whether the object is traced through the image input step S101 A single-person object identification step (S104) for identifying a single-person object among the moving objects tracked in the moving object tracking step (S103); a single-person object identification step (S103) (S105) of judging whether a termination command for tracking is generated when the identification of the single person object is completed in step S105, When the track end command is generated in step (S105) and finishes the tracking for a single person object and a camera parameter calculating step (S106) for calculating the camera parameters.

The video image input step S101 is a step of receiving a video image from a camera or the like.

In step S102, it is determined whether or not an object tracking start command has been issued by the user. If the object tracking start command is issued by the user, the tracking object determining step S102 is performed. In step S103, If there is no start command, only the video image can be received continuously.

The moving object tracking step S103 is a step of tracking a moving object from the video image input in the video image input step S101. As a result of tracking the moving object, a bounding box of a tracking object Box) coordinates. This shows the coordinate values of the bounding box for the moving objects being tracked, as shown in Fig.

The single-person object identification step (S104) identifies objects corresponding to one person among the moving objects tracked in the moving object tracking step (S103). The identification of a single-person object may be performed using an object- Height / width ratio values are used. Therefore, if the height / width ratio of the object bounding box is more than a certain value (for example 2.0), it is identified as a single person object and the bounding box coordinates are collected accordingly.

In step S105, it is determined whether a tracking termination command is issued by the user. If the tracking termination command is issued, the object tracking is terminated and the camera parameter calculation step S106 is performed. If this does not happen, keep track of a single object continuously.

, 최상단 중심의 좌표를

라고 하고, 상기 <수학식 1>에 의한

에 대응되는 지면 좌표를

,

에 대응되는 지면 좌표를

라고 하면, 하기 <수학식 3>의 비용 함수(Cost Function) C(f,θ,H)를 최소화하는 카메라 파라미터 f,θ,H를 구한다.The camera parameter calculation step (S106) is a step of calculating camera parameters using the bounding box coordinate values of the tracked single-person object. The camera parameter calculation step calculates coordinates of the lowermost center of the i-th bounding box among the collected N bounding boxes

, The coordinates of the top center

Quot; and < RTI ID = 0.0 >

The ground coordinates corresponding to

,

The ground coordinates corresponding to

The camera parameters f,?, And H that minimize the cost function C (f,?, H) of Equation (3) below are obtained.

Equation 3

In Equation (3), h is an average height (key) value of a person given by the user. The camera parameter value for minimizing the cost function C (f,?, H) in Equation (3) can be obtained by a numerical analysis technique such as Powell's method.

Although all of the bounding box coordinates basically collected in Equation (3) are used, in practice, using all of the collected bounding box coordinates may not be suitable for obtaining accurate camera parameter values.

Therefore, since the information of the bounding boxes adjacent to each other spatially increases only the amount of calculation in the calculation of the camera parameter value, it is not helpful. Therefore, rather than using all of the bounding box coordinates obtained per each video frame from the specific tracking object, The collected bounding box information is preferably used when the object region is not correctly detected at the time of object tracking, the single-person object identification fails, or a person (for example, a child) who is away from the average key of the person given by the user, , It is preferable to use Robust Estimation method such as RANSAC rather than camera parameters by simple Least Squares method.

FIG. 4 is a flowchart of an automatic camera correction method according to another embodiment of the present invention, FIG. 5 is a flowchart illustrating an automatic camera correction method according to another exemplary embodiment of the present invention, This is a photograph showing an example.

Another embodiment of the present invention is to perform camera correction continuously by using the results obtained during tracking of a human object rather than collectively performing camera correction after collecting human object tracking results for a certain period of time as shown in FIG. (S201) for receiving a video image from a camera or the like, a moving object tracking step (S202) for tracking an object moving through an image input in the image inputting step (S201), a moving object tracking step A single person object identification step (S203) of identifying a single person object among the moving objects tracked in the object tracking step (S202); and a step of collecting the bounding box coordinates of the single person object identified in the single person object identification step (S203) And calculating a camera parameter (S204).

 The image input step S201 is a step of receiving a video image from a camera or the like.

The moving object tracking step S202 is a step of tracking the moving object from the video image input in the image input step S201. As a result of the tracking of the moving object, a tracking object's bounding box ) Coordinates.

The single-person object identification step (S203) identifies the objects corresponding to one person among the moving objects tracked in the moving object tracking step (S202). The identification of the single-person object is performed using the object bounding box Height / width ratio values are used.

The camera parameter calculation step (S204) is a step of calculating and updating camera parameters based on the bounding box coordinate values of the single human object collected in the single human object identification step (S203).

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be taken by way of illustration, It will be readily apparent that various substitutions, modifications, and alterations can be made herein.

S101: Video image input step
S102: Step of judging whether or not to trace
S103: Moving object tracking step
S104: single person object identification step
S105: Judging whether the tracking is finished or not
S106: camera parameter calculation step

Claims (4)

A step S102 of inputting a video image from a camera or the like, a step S102 of determining whether an object is traced through an image input in the image input step S101,
(S103) of tracking a moving object to obtain a bounding box coordinate when a tracking command is started in the tracking determination step (S102)
A single person object identification step (S104) of identifying a single person object among the moving objects tracked in the moving object tracking step (S103)
A step S105 of judging whether a termination command for tracking has been generated when the identification of a single person object is completed in the single person object identification step S104,
And a camera parameter calculation step (S106) of terminating the tracking for the single person object and calculating the camera parameter when the tracking end command is generated in the tracking end determination step (S105) Correction method. The method according to claim 1,
The camera parameter calculation step S106 calculates the camera parameters f,?, And H that minimize the cost function C (f,?, H) by the following equation (3) Automatic camera calibration method using object tracking.
&Quot; (3) &quot;

The coordinates of the lowermost center of the ith bounding box among the N bounding boxes collected,

Is the coordinates of the top center, and h is the average height (key) of the person specified by the user. An image input step S201 for inputting a video image from a camera or the like,
A moving object tracking step (S202) of tracking an object moving through the image input in the image input step (S201)
A single person object identification step (S203) of identifying a single person object among the moving objects tracked in the moving object tracking step (S202)
(S204) of calculating the camera parameters by collecting the bounding box coordinates of the single-person object identified in the single-person object identification step (S203), thereby immediately using the result obtained during tracking of the human object, Wherein the automatic camera calibration is performed continuously. The method of claim 3,
The camera parameter calculation step S204 calculates the camera parameters f,?, And H that minimize the cost function C (f,?, H) according to the following equation (3) Automatic camera calibration method using object tracking.
&Quot; (3) &quot;

The coordinates of the lowermost center of the ith bounding box among the N bounding boxes collected,

Is the coordinates of the top center, and h is the average height (key) of the person specified by the user.

Images