KR20110021500A - Method for real-time moving object tracking and distance measurement and apparatus thereof - Google Patents

Abstract

PURPOSE: A method for real-time moving object tracking and distance measurement, and an apparatus thereof are provided to accurately separate a region of interest and reduce the left/right region of interest. CONSTITUTION: A region of interest is separated from the left/right images obtained from a camera(101). The characteristics that the region of interest has are used in a color structure. The noises generated from the internal and external regions of interest are removed. Depth is estimated by using the difference value of absolute coordinate between the regions of interest in the left/right images(103). In order to control a system, the region of interest is estimated and an operation is recognized in real time(105).

Description

METHOD FOR REAL-TIME MOVING OBJECT TRACKING AND DISTANCE MEASUREMENT AND APPARATUS THEREOF}

The present invention relates to a method and apparatus for real-time tracking and distance measurement of a moving object, and more particularly, by using a color structure feature of an object obtained from a binocular camera to separate a region of interest from a left and right image, A method and apparatus for obtaining depth using a difference in coordinate values are provided. The present invention is derived from the research conducted as part of the IT industry source technology development project of the Ministry of Knowledge Economy and the Ministry of Information and Communication Research and the cooperation project of industry-university-industrial cooperation in Seoul [Task management number: 2009-F-208-01, Signal processing element technology and SoC development for the implementation of digital hologram integrated service system, task management number: NT080528]

Computer vision is a field with the goal of realizing a human visual system as a computer, and researches on a technology of recognizing a moving object in real time and tracking a movement have been conducted. The important thing in human visual system is to recognize and track the target in 3D space and to measure the distance to the target. By measuring the distance to the target, decisions about future behavior can be made. The important thing in applying these characteristics to the computer is the distance to the target, that is, the depth information.

There are many ways to acquire depth information. Active methods, such as ultrasonic or laser sensors, are sensitive to the surrounding environment and difficult to obtain accurate information due to interference, whereas passive methods are relatively insensitive to changes in the surrounding environment. It can be effectively used in a noisy environment due to its low cost. Stereo vision is a typical passive method that finds matching points between left and right images obtained from binocular cameras in three-dimensional space and estimates depths using the differences between the matching points. Stereo vision has been applied to various fields because of its advantages that it can target the general natural environment without any special environmental constraints.

Image matching is an important process of finding the correspondence between the points projected on two images in a three-dimensional space in stereo vision. This process is called stereo matching, and the core of this process is to determine whether the selected points in the two images have corresponding points and calculate the variation that is an indicator of depth information at the points. Stereo matching can be simplified by finding the matching points of the left and right images, and it is very important to choose the matching method to deal with the matching point problem. The matching method is largely divided into area-based, feature-based, and energy-based matching, and there are cases where the three are mixed for better results.

Area-based matching is a method of finding a correspondence point that uses a correlation with another image with a fragment of a certain region including a feature in one image. Since the pixel brightness values of the two images are used for direct matching, they are very sensitive to changes in the characteristics of the pixels used for matching and are simple to calculate, but they are time consuming and vulnerable to image distortion and noise.

Feature-based matching is a matching method that obtains one-dimensional or two-dimensional feature values using differential operators on the brightness of pixels, selects the average value of contrast and the direction of the boundary, and uses them for matching. to be. The features of the images used include zero crossing points, boundaries, edges, gradients, corners, ridges, valleys, and cone curves. More accurate matching is possible, but the algorithm is complex.

 An energy-based matching match is an energy-based matching match that uses variations of the Bayesian inference or uses the iteration method to find the variation by recursion. Among these, the area-based matching method is widely used to obtain correlations with other images for all the pixels included in the ROI to be matched in the image. However, this method can obtain the entire variation at once, while the computational complexity is high. The disadvantage is that it takes considerable time.

The present invention has been proposed to solve the above-mentioned shortcomings of the region-based matching method. The present invention proposes a method for estimating depth of a moving object and a series of depth estimation methods using a matching method in which the region of interest is accurately separated and the left and right regions of interest are reduced. It is an object of the present invention to provide a method and apparatus for recognizing a movement of an acquired object for controlling a system.

The present invention for achieving the above object is a region of interest separation step of separating the region of interest from the left and right images obtained by the binocular camera; A depth information estimating step of estimating depth using a difference value of absolute coordinates between regions of interest in a left and right image; The present invention provides a method for measuring a depth of a moving object including a region of interest tracking and recognition for tracking a region of interest in real time and recognizing a motion for controlling the system.

According to the present invention, an accurate ROI can be obtained from left and right images acquired by a binocular camera, and depth can be measured relatively accurately and quickly by using only the feature points of each ROI. Therefore, the depth of the moving object can be obtained in real time.

In addition, the operation of the region of interest may be recognized and used as an execution condition of the series of operations.

DETAILED DESCRIPTION Hereinafter, the most preferred embodiment of the present invention will be described with reference to the accompanying drawings so that those skilled in the art may easily implement the technical idea of the present invention.

1 is an overall flowchart illustrating a method for real-time tracking and depth measurement of a moving object according to an exemplary embodiment of the present invention.

Referring to FIG. 1, in the real-time tracking and depth measuring method of a moving object according to the present invention, a step 101 of separating a region of interest, a depth estimation and coordinate estimation stage 103 of a region of interest, and a motion recognition stage 105 may be performed. ).

2 is a detailed flowchart illustrating a region of interest separation step 101 according to the present invention.

First, in the separation of the ROI, only the ROI is separated from the left and right images 201 obtained by the binocular camera. The present invention designates and separates a hand region as a region of interest, and extracts a skin region by using a feature of only skin color on a color difference plane. The camera provides an image in an RGB color structure, which is sensitive to the ambient lighting environment, which causes a problem of detecting similarly reflected light together. Therefore, the pixels of the image are represented by a luma component and a chrominance component, respectively, to convert the color structure of the image into a YC _b C _r color structure that has a more stable value even when the lighting environment is changed. use. After converting the RGB color structure of the left and right images obtained from the binocular camera into the YC _b C _r color structure, the blue color difference component, C _b (blue chrominance) and red color difference component, C _r (red chrominance) Using the distribution of), separate hand regions.

FIG. 3 is a diagram illustrating an image of a hand region separated from an image through an ROI separation step 203 subject to a chrominance component according to the present invention.

Although the YC _b C _r color structure, which is less susceptible to changes in the lighting environment, is used in the region of interest separation step 203, the noise region 305 is present in and around the region of interest in the image according to the ambient lighting environment.

In this way, the hand region separated by the lighting environment and the noise region occurring in the background region are compensated through the noise region removing step 205. The noise domain removal step consists of hole processing 207 and smoothing filtering 209.

The hole processing searches for an image in two directions, horizontally and vertically, and determines, compensates, and removes a hand region or a noise region based on the size of the interval between pixels determined as the hand region.

FIG. 4 is a diagram illustrating an embodiment of a hole process in which a gap between pixels for determining a region of interest is set to two; FIG.

When the distance between the pixels determined to be the hand region is 2 or less, the noise is determined and compensated for in the hand region. However, when the distance between the pixels exceeds 2, the noise is not judged as noise and is not compensated.

5 is a diagram illustrating an embodiment of smoothing filtering.

Smoothing filtering searches for an image on a pixel-by-pixel basis, and determines and compensates whether a pixel is a hand region or a background region according to a sum of pixel values in a predetermined size block surrounding the pixel.

FIG. 6 is a diagram illustrating an embodiment of an image that has passed the ROI separation step 101 including the noise removing step 205 according to the present invention.

The noise region generated by the influence of the lighting environment on the hand region image 601 obtained using the characteristic of the skin color in the YC _b C _r color structure is the image 605 from which the noise is removed through hole processing and smooth filtering. Is compensated.

7 is a flowchart showing in detail the depth estimation and coordinate estimation step 103 of the ROI in accordance with the present invention.

Referring to FIG. 7, the depth estimating and coordinate estimating step of the ROI includes a displacement estimation step 701 using the center coordinates of the ROI and a three-dimensional spatial coordinate acquisition step.

The left and right regions of interest separated through the region of interest separation may cause differences in coordinate values for the same object. This is called disparity and the depth of interest is estimated using this. In general, the region-based matching technique for estimating the variation using the image patch of a certain region of the left and right images using the correlation between each other is used for the estimation of the variation, which is sensitive to noise and image distortion. There is a disadvantage in that the calculation amount is large.

In order to reduce the large amount of computation of the region-based disparity estimating technique and to improve the above-mentioned problem, the depth estimation technique 701 which reduces the left and right regions of interest used for the disparity is performed.

FIG. 8 illustrates a depth estimation technique in which the left and right regions of interest used in the variation are reduced in order to reduce a large amount of calculation of the region-based variation estimation technique.

The center coordinates of the hand region that are averaged by adding all the coordinate values separated by the hand region from the separated hand region image 801 by minimizing the influence of the external environment such as noise through the region of interest separation. Obtain (803) and use it for estimating the variance to greatly reduce the amount of computation. The depth of the hand region is estimated using the difference in the center coordinates of the left and right hand regions. The three-dimensional coordinates 707 of the hand region in consideration of the center coordinates and the depth used for the variation are obtained in real time.

9 is a flow chart illustrating in detail the recognition step 105 of the ROI operation in accordance with the present invention.

Referring to FIG. 9, the ROI recognition step 105 includes a thinning boundary image acquisition step 901, a circle showing a circle passing through a finger, and a number of intersection points between the boundary and the circle ( 907).

10 shows a step 903 of counting the number of fingers to recognize hand gestures.

In order to obtain the number of fingers, a circle centered on the coordinates of the center of the hand region used for the disparity estimation is drawn on the boundary image of the hand region, and the number of intersection points is used. In order to determine the size of the circle through the finger properly, search for 22.5 degrees in 16 directions from the center coordinate of the hand region to find the intersection with the boundary of the hand region. The radius of the circle is set to the average of the size of the 12 points except the two points having the largest and the smallest of the 16 points and the distance between the center coordinates.

FIG. 11 shows an edge thinning step 901 of an image for stabilizing the number of intersections of a circle and a boundary via a finger.

The image 1101, which has undergone the Sobel filter, has a boundary part partially composed of one or more pixels, causing two or more pixels to overlap at an intersection with a circle passing through a finger. The number of intersections can be stably counted through the thinning process so that the boundary image is composed of one pixel. A boundary image 1101 is obtained through a Sobel filter on the hand image 1103 which has been separated, and a subtraction operation is performed with the hand region split image 1103 before passing through the Sobel filter. After the subtraction operation, the inner pixels corresponding to the hand region are erased from the boundary of the Sobel filter boundary image 1101, leaving the image 1105 having only a thin boundary surrounding the ROI.

On the other hand, the tracking and depth estimation method of the moving object according to the present invention as described above can be prepared by a computer program. And the code and code segments constituting the program can be easily inferred by a computer programmer in the art. The written program is also stored in a computer-readable recording medium (information storage medium), and read and executed by a computer to implement the method of the present invention. The recording medium may include any type of computer readable recording medium.

Although the invention has been described by way of limited embodiments and drawings, the invention is not to be limited thereto and is intended to be equivalent to those of ordinary skill in the art and claims by those skilled in the art to which the invention pertains. Various modifications and variations are possible within.

1 is a flowchart illustrating a method for tracking a moving object, estimating depth, and estimating coordinates.

2 is a flowchart illustrating a method of separating a region of interest in detail;

FIG. 3 is a diagram showing an image of a result obtained by extracting a hand region based on an area occupied by skin color on a YC _b C _r color structure in different lighting environments. FIG.

4 is a diagram for explaining hole processing;

5 is a diagram for explaining smoothing filtering.

FIG. 6 is a diagram illustrating a resultant image of removing noise of an image through hole processing and smoothing filtering. FIG.

7 is a flowchart illustrating a method of estimating depth and a coordinate of a region of interest in detail;

8 is a diagram illustrating an operation of reducing an area used for disparity estimation to the center coordinates of a region of interest in order to reduce the calculation amount of the region-based disparity estimating technique.

9 is a flowchart illustrating a method of recognizing a motion of a region of interest.

10 is a view for explaining an operation of counting the number of fingers to recognize a hand gesture.

FIG. 11 is a view for explaining a thinning process for accurately counting an intersection between a circle passing through a finger and a boundary of a hand region; FIG.

Claims (8)

A region of interest separation step of separating the region of interest from the left and right images acquired by the binocular camera; A depth information estimating step of estimating depth by using a difference value of absolute coordinates between regions of interest in the left and right images; And Tracking and recognizing a region of interest for tracking the region of interest in real time and recognizing motion for control of the system Real time tracking and depth measurement method of the moving object comprising a.        The method of claim 1 The region of interest separation step A separation step of using features of the region of interest in the color structure; And Noise Canceling Step Eliminates Noise Occurring in and Outside the Region of Interest Real time tracking and depth measurement method of the moving object comprising a.        The method of claim 2 The noise region removing step A hole processing step of removing the noise area according to the size of the gap between the separated pixels, determined as the hand area; And Smoothing filtering step of compensating pixel values according to the sum of sum of all pixel values in a block including pixels Real time tracking and depth measurement method of the moving object comprising a.       The method of claim 3 Each step above Performed pixel by pixel Real time tracking and depth measurement of moving objects.       The depth information estimating step Disparity estimator that obtains the center coordinates of the hand region and estimates the variation by dividing the sum of all coordinate values in the left and right region of interest through the separation step by the total number of regions of interest Real time tracking and depth measurement method of the moving object comprising a.       The ROI recognition step Subdividing the hand region boundary for stabilization of acquisition of the intersection with the circle via the hand region; And Showing a circle for recognizing a hand gesture via all of the fingers Real time tracking and depth measurement method of the moving object comprising a.      The method of claim 6 The direction of searching for the intersection point between the circle and the hand region boundary is Search in 16 directions by 22.5 degrees around the center of the hand area Real time tracking and depth measurement of moving objects. The method of claim 6 To size a circle via all of your fingers. Use the average of 12 intersection points except the maximum and the minimum of 16 intersection points between the circle and the boundary of the hand region. Real time tracking and depth measurement of moving objects.

Images