KR20090068474A - Method for position estimation and apparatus for the same - Google Patents

Abstract

An apparatus for estimating position and an estimating method are provided, which improve the efficiency of system and reduce expense. An apparatus for estimating position using the stereo vision comprises the stereo matching part(320), the accumulation cross section stoker(340), the accumulation cross section processing unit(350), and the location estimate part(360). The stereo matching part stereo-matches the photographed image and obtains the depth map image. The accumulation cross section processing unit obtains the line which shows outline and background image of the cross section image by making use of the accumulation cross section image. The location estimate part estimates the location of object by using the horizontal location information and vertical position information of object.

Description

Position estimation apparatus and estimation method

The present invention relates to a position estimating apparatus and a position estimating method, and more particularly, to an apparatus and a method for estimating a position of an object included in an image using a captured image.

There are several ways to measure the distance and position of an object away from the viewing side. Among them, a vision technique may be used as an example of a technique of photographing an image of the object and measuring a distance and a position to an object included in the image by using the photographed image.

In the object position detection method using vision technology, preprocessing is performed using only an image captured by one image capturing apparatus, and two-dimensional feature information such as an outline, a color, and a template is extracted. The location of the required object is detected using the extracted information and a matching filter.

There are two main methods for extracting distance information of an object. A method of extracting accurate distance information uses a depth map information of stereo vision, and a perspective using only one image (mono image) as a method of extracting approximate distance information. There is. The above-mentioned perspective method is a method of measuring distance by using an approximate proportional relationship with the property that the object appears smaller as the distance is larger and the object becomes larger as it is closer. However, an error occurs depending on the size of the object, especially on hills and downhill roads where the object and the camera's vertical position are not the same.

As described above, when only one image (mono image) is used to detect the position of an object, it is difficult to accurately extract an object because a lot of two-dimensional feature information of several backgrounds and obstacles is similar. In addition, the perspective using a single image in extracting a distance from an object has a high error, and thus uses stereo vision. Therefore, the use of different systems for the position detection and the distance estimation of the object is inferior in utility and inferior in reliability in object detection.

The present invention has been made to solve the above problems, and an object of the present invention is to provide a position estimating apparatus and a position estimating method capable of estimating an exact position of an object.

According to an embodiment of the present invention, a method of estimating a depth map image by stereo matching a photographed image, accumulating the depth map image in a horizontal direction, obtaining a cumulative cross-sectional image, and extracting an outline of the cumulative cross-sectional image Obtaining a line representing a background image by using the outline, obtaining a vertical position information of the object by comparing the outline with a line representing the background image, extracting a depth map image corresponding to the vertical position of the object, Accumulating the extracted depth map image in a vertical direction to obtain a cumulative value, and obtaining horizontal position information of the object using the accumulated value accumulated in the vertical direction.

In another aspect, an apparatus for estimating position according to an embodiment of the present invention includes a stereo matching unit for stereo matching a photographed image to obtain a depth map image, a cumulative cross-sectional unit for accumulating the depth map image in a horizontal direction to obtain a cumulative cross-sectional image, and the accumulation. A line representing the outline of the cross-sectional image and a background image is obtained by using the cross-sectional image, the vertical position information of the object is obtained by comparing the outline and the line representing the background image, and the depth map image corresponding to the vertical position is vertically obtained. And a cumulative cross-section processing unit for obtaining horizontal position information of the object using a cumulative value accumulated in a direction, and a position estimating unit for estimating a position of the object using horizontal position information and vertical position information of the object.

According to the position estimating apparatus and the position estimating method of the present invention, there is an effect of reducing the position estimation error and estimating the exact position and distance of the target object. In addition, since the position and distance of the object can be estimated using stereo vision, the efficiency of the system can be improved and the cost can be reduced, and the accuracy and reliability of the position and distance estimation of the object can be improved.

The estimation method as described above has an effect of estimating an accurate position and distance by applying to various fields. For example, when applied to a robot or a vehicle, it can be used for driving systems such as collision prevention of cars, driving, safety driving systems, and the like, and can also be used in industrial factories or general homes.

The objects, features and advantages of the present invention will become apparent from the following detailed description of embodiments taken in conjunction with the accompanying drawings. In addition, the terms used in the present invention was selected as a general term widely used as possible now, but in certain cases, the term is arbitrarily selected by the applicant, in which case the meaning is described in detail in the corresponding description of the invention, It is to be clear that the present invention is to be understood as the meaning of terms rather than names.

Operation of the position estimating apparatus and the position estimating method according to the present invention configured as described above will be described in detail with reference to the accompanying drawings.

1A is a diagram illustrating a left image according to stereo vision according to an embodiment of the present invention, and FIG. 1B is a diagram illustrating a right image according to stereo vision according to an embodiment according to the present invention. . 1C is a diagram illustrating a depth map made by using the left and right images of FIGS. 1A and 1B according to an embodiment of the present invention.

The left image as shown in FIG. 1A may be photographed using the left image capturing apparatus using stereo vision, and the right image as shown in FIG. 1B may be photographed using the right image capturing apparatus. The depth map of the image as shown in FIG. 1C may be obtained by stereo matching the left image and the right image photographed as described above. Before obtaining the depth map, the left image and the right image may be rectified using a calibration parameter according to the characteristics of the imaging apparatus. For example, lens distortion correction, horizontal and vertical line distortion correction, and the like can be performed. In the depth map of FIG. 1C, the cone-shaped obstacle object (object) A and the spherical obstacle object (object) B can be seen. Hereinafter, as an example according to the present invention, a method of estimating the positions of the obstacle A and the obstacle B will be described.

In the depth map of the image, the close distance has a high brightness value and the far distance has a low brightness value. Accordingly, when the brightness value of the depth map is horizontally accumulated, the cumulative value of the relatively close distance part is high and the cumulative value of the relatively long distance part is low. Therefore, the position of the object can be estimated using the accumulated value. For example, the background part has a feature that the cumulative value decreases proportionally. When the object is placed, the cumulative value increases because of this. Therefore, the position of the object is estimated using the part where the cumulative value increases. can do. In addition, since the brightness value of the depth map is distance information, distance information of an object may also be known using the depth map.

2A is a diagram illustrating a part of horizontally accumulating and cross-sectionalizing brightness values of the depth map according to one embodiment of the present invention. The x-axis of FIG. 2A represents a horizontal cumulative value, and the y-axis represents a vertical line of the depth map image. FIG. 2A illustrates a cumulative value of a partial region including an obstacle based on the depth map of FIG. 1C.

It can be seen that the lower part of the image of FIG. 1C has a high brightness value and the middle part of a long distance has a low brightness value. As a result of horizontally accumulating the brightness of the depth map image for each vertical line, as shown in FIG. 2A, the horizontal accumulation value of a region having a low vertical value (the lower portion of the depth map image) is high, and as the vertical value increases, the horizontal accumulation value decreases. . However, in the case where the obstacle A and the obstacle B are present, the cumulative value is higher than the increase or decrease ratio of the peripheral area. In this case, before calculating the horizontal cumulative value to increase the reliability of the depth map, the noise of the depth map may be removed through filtering.

FIG. 2B is a diagram illustrating extracting an outline of a horizontal cumulative cross-sectional image according to an embodiment of the present invention. FIG. When the horizontal cumulative value as shown in FIG. 2A is calculated, an outline of a horizontal cumulative cross-sectional image is extracted using the horizontal cumulative value image. This is because a cumulative value is required to estimate the position of the object, and the cumulative value is the same as the outline of the cumulative cross-sectional image. When the outline of the cumulative cross-sectional image is extracted, filtering is performed to remove high frequency components of the extracted outline. The extracted outline may include many high frequency components, and the high frequency components may be removed through high frequency elimination filtering because they act as noise to extract a line representing a background image.

Since the outline of the high frequency rejection filtered cumulative cross-sectional image has a gray value, the image is binarized using a set threshold value. In other words, in order to clearly distinguish the outline from which the high frequency component is removed from the surrounding image, the threshold value is binarized to recognize a portion that is above the threshold value as a line. The threshold may vary depending on implementation.

2C is a diagram schematically illustrating extracting a line representing a background image according to an embodiment of the present invention. When the outline becomes clear, a line representing a background image is extracted from the outline to estimate the position of the object. In the case of the background image, as the distance increases, the brightness value is lowered. A line representing the background image may exhibit a tendency of decreasing the brightness value. The line representing the background image is similar to the line representing the overall tendency of the outline.

Therefore, the position of the object may be estimated by comparing the outline with the line representing the background image. The line representing the background image may be extracted in various ways. For example, when a certain number of consecutive pixels form a line, the line may be extracted using an algorithm that recognizes the line as a line. An example of the algorithm is a Houth transform. When a plurality of lines are extracted through the algorithm or the like, the lines representing the background image may be extracted by interpolating the extracted lines.

2D is a diagram schematically illustrating a method of estimating a vertical position of an object according to an embodiment of the present invention. By comparing the outline line and the outline line extracted from FIG. 2C, the position of the region where the difference between the outline line and the line feature background is out of a predetermined threshold value may be estimated as the position of the object. The threshold value may vary depending on the implementation example, and may vary in consideration of system reliability or system performance requirements. In FIG. 2D, the vertical position of the obstacle A (object) A and the vertical position of the obstacle B (object) B may be estimated.

FIG. 2E is a diagram illustrating an example of extracting a depth map image corresponding to an estimated vertical position, and FIG. 2F illustrates a value of vertically projecting the extracted depth map image. 2G is a diagram illustrating estimating a horizontal position of an object using a vertically projected value.

An image of a part including the obstacle is extracted from the depth map image of FIG. 1C by using the vertical positions of the obstacle A and the obstacle B estimated in FIG. 2D. Then, the extracted depth map image is vertically projected to accumulate the brightness values of the extracted image vertically. In FIG. 2F, the x axis represents a horizontal line of the depth map image, and the y axis represents a vertical cumulative value of the extracted depth map image as shown in FIG. 2E. When the vertical cumulative value along the horizontal line is calculated as shown in FIG. 2E, the object may have a higher depth value than the background area, and thus the cumulative value may be high. Therefore, the horizontal position of the region having a higher accumulation value than the background region accumulation value may be estimated as the horizontal position of the object. For example, when a cumulative value of a specific horizontal area is compared with a surrounding cumulative value, the position of the horizontal area may be estimated as the horizontal position of the object. 2G shows that the horizontal positions of the obstacles A and B are estimated according to the estimation method.

FIG. 2H illustrates an example in which an object position is estimated using a depth map, and FIG. 2I illustrates an object position in a captured image by using estimated position information of the object. Drawing.

As shown in FIG. 2H, the vertical position (vertical position of the obstacle A and the obstacle B) of the object estimated in FIGS. 2D and 2E and the horizontal position of the object estimated in FIG. 2G (the horizontal of the obstacle A) are estimated. Location, the horizontal position of the obstacle B) can be used to estimate the position of the object in the depth map image. Also, by applying the obtained vertical position information and the horizontal position information of the object to the captured image, it is possible to indicate the position of the object in the captured image as shown in FIG. As illustrated in FIG. 2I, the image representing the position of the object may be applied to a left image, a right image, or a stereo matched image.

3 is a block diagram schematically illustrating a configuration of a position estimating apparatus according to an embodiment of the present invention. The position estimating apparatus includes a left image capturing apparatus 300, a right image capturing apparatus 302, a stereo preprocessor 310, a stereo matcher 320, a stereo postprocessor 330, a cumulative cross-sectionalizing unit 340, and a cumulative cross-section. It includes a cross-sectional processing unit 350 and the position estimation unit 360.

The left image capturing apparatus 300 and the right image capturing apparatus 302 respectively capture an image and output the captured image to the stereo preprocessor 310. The image capturing apparatus may use, for example, a camera. The left image captured by the left image capture device 300 and the right image captured by the right image capture device 302 are input to the stereo preprocessor 310. The stereo preprocessor 310 corrects and outputs the input left and right images. For example, the stereo preprocessor 310 may rectify the left image and the right image using a calibration parameter of each imaging apparatus. The stereo preprocessor 310 may not be used depending on implementation.

The left image and the right image output from the stereo preprocessor 310 are stereo matched by the matching filter in the stereo matching unit 320. The stereo matching unit 320 calculates a depth map image with respect to the matched image. The depth map image output from the stereo matching unit 320 is output to the stereo post processing unit 330 and post-processed filtered. The stereo postprocessor 330 may remove noise by using a thresholding or smoothing technique to increase the reliability of the depth map image. The stereo post-processing unit 330 may also not be used depending on implementation.

The depth map image output from the stereo post processor 330 is output to the cumulative cross sectioning unit 340. The cumulative cross-sectionalizing unit 340 calculates a horizontally accumulated cross-sectional image by accumulating the brightness values of the depth map image horizontally. The calculated horizontal cumulative cross-sectional image is output to the cumulative cross-section processor 350.

4 is a block diagram schematically showing an accumulating cross-sectional processing unit as an embodiment according to the present invention. The cumulative cross section processor 350 includes an outline extractor 352, a line extractor 354, a vertical position estimator 356, and a horizontal position estimator 358. The cumulative cross-section processor 350 may estimate the vertical position and the horizontal position of the object by using the horizontal cumulative cross-sectional image.

An outline extractor 352 extracts an outline of the cumulative cross-sectional image, and a line extractor 354 filters an outline of the cumulative cross-sectional image to remove high frequency components and binarizes the outlines from which the high frequency components are removed. To clarify the outline. The line representing the background image is extracted from the binarized outline using a line extraction algorithm. As an example of the line extraction algorithm, a Hough transform may be used. As described above, the vertical position estimator 356 may estimate the vertical position by comparing the outline of the image with a line representing the background image, and the horizontal position estimator 358 may estimate the depth map image of the estimated vertical position. Using to calculate the vertical cumulative value and to estimate the horizontal position.

The position estimator 360 may estimate the position of the object corresponding to the depth map image or the captured image using the horizontal position information and the vertical position information of the object estimated by the cumulative cross-section processor 350. Although the location estimation apparatus of FIG. 3 is described based on the location estimation of the object, a configuration of estimating the distance of the object using the stereo vision image may be further added.

5 is a flowchart schematically illustrating a method of estimating a position according to an embodiment of the present invention.

Stereo matching is performed using the left and right images captured by the left and right image capturing apparatus of the stereo vision, and a depth map image of the stereo matched image is extracted (S500). According to an embodiment, the left image and the right image may be filtered before the depth map image is extracted using the correction value of the image capturing apparatus. When the depth map image is extracted in step S500, the depth map image is filtered to remove noise of the depth map (S502). For example, noise may be removed by using a thresholding or smoothing technique, and the filtering process may not be used depending on the implementation.

In the depth map image, a close distance is represented by a high brightness value and a far distance is represented by a low brightness value, so that the depth map image is horizontally accumulated to obtain a cumulative cross-sectional image (S504). Using the cumulative cross-sectional image obtained in step S504 to extract the outline of the cumulative cross-section (S506), the extracted outline is filtered to remove the high frequency components of the outline (S508). The filtered outline is binarized (S510) to clarify the distinction between the outline and the surrounding image, and a line representing the background image is extracted using the outline (S512). The line representing the background image may represent a feature of the background image. The method of extracting the outline, the filtering method, the method of extracting the lines of the outline, etc. are as described above.

When the line representing the background image is extracted in step S512, the vertical position of the object may be extracted using the extracted line (S514). When the information about the vertical position of the object is extracted, the depth map image corresponding to the vertical position is extracted, and the extracted depth map image is projected in the vertical direction (S516). In operation S516, an accumulated value is calculated by accumulating the extracted depth map image in a vertical direction. In operation S516, the horizontal position of the object may be estimated using the accumulated value accumulated in the vertical direction (S518). For example, in operation S518, an area having a higher value than that of the background area may be estimated as the horizontal position of the object. The object may be detected from the captured image, the stereo matched image, or the depth map image by using the vertical position information of the object obtained in step S514 and the horizontal position information of the object obtained in step S518 (S520).

In FIG. 5, the position estimation order of the object has been described, but the distance estimation of the object may be performed in parallel or sequentially together with the position estimation of the object using the stereo vision image.

The present invention is not limited to the above-described embodiments, and as can be seen in the appended claims, modifications can be made by those skilled in the art to which the invention pertains, and such modifications are within the scope of the present invention.

1A is a diagram schematically showing a left image according to stereo vision according to an embodiment of the present invention.

1B is a diagram schematically illustrating a right image according to stereo vision according to an embodiment of the present invention.

1C is a diagram illustrating a depth map made using the left and right images of FIGS. 1A and 1B according to an embodiment of the present invention.

2A is a diagram illustrating a part of horizontally accumulating and cross-sectionalizing brightness values of the depth map according to one embodiment of the present invention.

FIG. 2B is a diagram illustrating extracting an outline of a horizontal cumulative cross-sectional image as an embodiment according to the present invention. FIG.

2C is a diagram schematically illustrating extracting a line that is a feature of a background image according to an embodiment of the present invention;

2d is a diagram schematically illustrating a method of estimating a vertical position of an object according to an embodiment of the present invention.

2E is a diagram schematically illustrating extracting a depth map image corresponding to an estimated vertical position according to an embodiment of the present invention.

FIG. 2F illustrates a value of vertically projecting the extracted depth map image. FIG.

2G illustrates estimating the horizontal position of an object using vertically projected values

2H illustrates an example of estimating the position of an object using a depth map according to an embodiment of the present invention.

2i is a diagram illustrating the position of an object in a captured image by using estimated position information of the object.

3 is a block diagram schematically showing a configuration of a position estimating apparatus according to an embodiment of the present invention.

4 is a block diagram schematically showing an accumulated cross-sectional processing unit as an embodiment according to the present invention.

5 is a flowchart schematically illustrating a method of estimating a position according to an embodiment of the present invention.

* Explanation of symbols for the main parts of the drawings

300: left video recording device 302: right video recording device

310: stereo preprocessing unit 320: stereo matching unit

330: stereo post-processing unit 340: cumulative cross-section

350: cumulative cross-section processing unit 360: position estimation unit

Claims (15)

In the position estimation method, Obtaining a depth map image by stereo matching the captured image; Accumulating the depth map image in a horizontal direction to obtain a cumulative cross-sectional image, and extracting an outline of the cumulative cross-sectional image; Obtaining a line representing a background image by using the outline, and obtaining vertical position information of the object by comparing the outline to a line representing the background image; Extracting a depth map image corresponding to a vertical position of the object, and accumulating the extracted depth map image in a vertical direction to obtain a cumulative value; And And obtaining horizontal position information of the object using the accumulated value accumulated in the vertical direction. The method of claim 1, And filtering the photographed image by using a calibration parameter. The method of claim 1, And filtering the depth map image obtained by the stereo matching to remove noise of the depth map image. The method of claim 1, And detecting an object from any one of a photographed image or a depth map image by using the vertical position information and the horizontal position information of the object. The method of claim 1, Obtaining a line representing the background image, Filtering the extracted outline to remove high frequency components; Clarifying the outline by binarizing the outline from which the high frequency component has been removed; And Obtaining a line representing a background image from the binarized outline using a line extraction algorithm. The method of claim 5, wherein The line extraction algorithm, Position estimation method using stereo vision using Hough transform algorithm. The method of claim 1, Obtaining vertical position information of the object, And estimating an area in which the line representing the background and the outline differ by more than a threshold value as a vertical position of the object. The method of claim 1, Obtaining horizontal position information of the object, A method for estimating position using stereo vision, which estimates an area having a cumulative value higher than a cumulative value of a surrounding area above a set threshold value as a horizontal position of the object compared with a surrounding cumulative value. In the position estimation device, A stereo matching unit configured to obtain a depth map image by stereo matching the captured image; A cumulative sectioning unit configured to accumulate the depth map image in a horizontal direction to obtain a cumulative sectioned image; Using the cumulative cross-sectional image, a line representing an outline of the cross-sectional image and a background image is obtained, and vertical position information of the object is obtained by comparing the outline and a line representing the background image, and a depth map image corresponding to the vertical position is obtained. An accumulation section processing unit for obtaining horizontal position information of the object using an accumulation value accumulated in a vertical direction; And And a position estimator for estimating the position of the object using the horizontal position information and the vertical position information of the object. The method of claim 9, And a stereo preprocessor for filtering the photographed image using a calibration parameter and outputting the filtered image to a stereo matching unit. The method of claim 9, And a stereo post-processing unit which filters the depth map image output from the stereo matching unit, removes noise of the depth map image, and outputs the noise to the cumulative sectioning unit. The method of claim 9, The cumulative cross-section processing unit, An outline extracting unit extracting an outline of the cumulative cross-sectional image; A line extracting unit configured to binarize and remove the high frequency components of the outline and obtain a line representing a background image from the binarized outline using a line extraction algorithm; A vertical position estimator for estimating a vertical position of an object by comparing an outline with an outline representing the background image; And And a horizontal position estimator for estimating a horizontal position of the object by using a cumulative value accumulated in a vertical direction in the depth map image corresponding to the estimated vertical position. The method of claim 12, The line extraction unit, An apparatus for estimating position using stereo vision to obtain a line representing a background image by using a Hough transform algorithm. The method of claim 9, The cumulative cross-section processing unit, And a stereo vision estimating an area where the line representing the background and the outline differ by more than a threshold from the vertical position of the object. The method of claim 9, The cumulative cross-section processing unit, And a stereo vision for estimating an area having a cumulative value higher than a cumulative value of a peripheral area as a horizontal position of the object by using the accumulated value accumulated in the vertical direction.

Images