KR101907852B1 - Method and Apparatus for Processing Stereo Images - Google Patents

Abstract

The present invention relates to a stereo image processing method and apparatus for efficiently performing stereo matching on a stereo image.
The present invention provides an apparatus for processing a stereo image of a vehicle, comprising: a stereo camera including a first camera and a second camera for acquiring a stereo image; And a stereo matching unit configured to perform stereo matching using the stereo image, wherein the stereo matching unit adjusts the stereo matching using the maximum slope between the first camera and the second camera, And performs parallelization between the first image and the second image of the second camera.

Description

[0001] Stereo image processing method and apparatus [0002]

The present invention relates to a stereo image processing method and apparatus. More particularly, the present invention relates to a stereo image processing method and apparatus for efficiently performing stereo matching on a stereo image.

A method of calculating a distance to a specific point included in an image using an image obtained from a plurality of cameras is known. As a representative method, stereo matching is a method of finding arbitrary patterns in an image obtained from any one of the cameras in an image of another camera mapped to the same point on the three-dimensional space, To determine the position on the three-dimensional space. Here, the pattern may be various elements such as a pixel in an image, a window block, or a feature (edge, trajectory).

1 is a diagram for explaining an example of calculating a distance using a stereo matching technique.

When the focal distance f of the lens is obtained from the two images captured by two cameras disposed at a certain distance from the base line b and the focal length f of the lens is known, The distance can be obtained through the geometry of FIG. 1 as: < EMI ID = 1.0 >

In Equation (1), r is the distance between the subject and the camera, b is the distance between the two cameras, f is the focal length of the camera lens, d is the disparity between the same objects ) As d _l -d _r .

In performing the above stereo matching, it is assumed that the patterns related to a specific object in two images exist on the same line in the vertical direction (assuming disparity only in the horizontal direction). Accordingly, the corresponding points on the two images are searched on the same line in the horizontal direction. To do this, the epipolar lines in the two images must be matched and this operation is called rectification. In general, a feature is searched to determine the degree of skew or distortion between the left and right images, and then the image is corrected using a rotation / correction matrix.

However, in the conventional method, there is a problem that a matching point changes and an actual disparity changes as another image is rotated based on one image. Also, there is a problem that a delay occurs in searching for a minutia in an image and confirming a tilt. This delay is undesirable especially in vehicle image stereo matching where immediate processing is required.

In order to solve the above problems, it is an object of the present invention to minimize errors in the process of rectifying an image for stereo matching on an image from the stereo camera 110, And to provide a stereo image processing method that can be applied.

It is another object of the present invention to provide a stereo image processing apparatus for performing the above-described stereo image processing method.

The present invention provides an apparatus for processing a stereo image of a vehicle, comprising: a stereo camera including a first camera and a second camera for acquiring a stereo image; And a stereo matching unit configured to perform stereo matching using the stereo image, wherein the stereo matching unit adjusts the stereo matching using the maximum slope between the first camera and the second camera, And performs parallelization between the first image and the second image of the second camera.

In one embodiment, the stereo matching unit may include: a pixel extracting unit that extracts a first extraction area and a second extraction area for parallelization from the first image and the second image using the maximum slope; A feature point matching unit for extracting feature points corresponding to the first extraction region and the second extraction region; A slope calculating unit for calculating a slope between the first extraction area and the second extraction area using the feature points; And an image converting unit for parallelizing the first image and the second image using the slope and performing a final stereo matching.

In addition, the first extraction region may extract the Xth row in the vertical direction in the first image.

The second extraction region may be a plurality of rows including the Xth row in consideration of the maximum slope in the second image.

The present invention also provides a stereo image processing method for a vehicle, comprising the steps of: (a) acquiring a first image and a second image from a stereo camera of a vehicle; (b) a pixel extracting unit of the stereo matching unit extracts a predetermined region from the first image and the second image by using a maximum slope between the first camera and the second camera set in consideration of the state of the vehicle, Extraction into a second extraction region; (c) calculating a slope between the first image and the second image by calculating a feature point corresponding to the first extraction region and the second extraction region in the stereo matching unit; And (d) parallelizing the first image and the second image using the slope in the stereo matching unit and performing a final stereo matching.

According to the present invention, in performing stereo matching on an image acquired from a stereo camera of a vehicle, a parallelization process between stereo images is performed on only a part of an image in consideration of a special environment, The processing time can be shortened and the system can be simplified.

1 is a diagram for explaining an example of calculating a distance using a stereo matching technique.
2 is a block diagram of a stereo image processing apparatus according to a preferred embodiment of the present invention.
3 is a diagram showing an example of processing of a stereo image in a stereo image processing apparatus according to a preferred embodiment of the present invention.
4 is a flowchart illustrating a stereo image processing method according to a preferred embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to designate the same or similar components throughout the drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. In addition, the preferred embodiments of the present invention will be described below, but it is needless to say that the technical idea of the present invention is not limited thereto and can be variously modified by those skilled in the art.

FIG. 2 is a block diagram of a stereo image processing apparatus according to a preferred embodiment of the present invention, and FIG. 3 is a diagram illustrating an example of processing of a stereo image in a stereo image processing apparatus according to a preferred embodiment of the present invention.

A stereo image processing apparatus 100 according to a preferred embodiment of the present invention includes a stereo camera 110, a stereo matching unit 120, and a storage unit 130 provided in a vehicle. In addition, the stereo image processing apparatus may be connected to an electronic control unit (ECU) 140 that performs various control of the vehicle using a stereo image.

The stereo camera 110 is provided in the vehicle and is usually provided in front of the vehicle. The stereo camera 110 includes a first camera 110a and a second camera 110b and the first camera 110a and the second camera 110b may be installed to acquire images of the same point have. However, it is needless to say that the stereo camera 110 may be installed laterally or rearwardly of the vehicle, not in front of the vehicle.

In one embodiment, the stereo matching unit 120 includes a pixel extracting unit 122, a feature point matching unit 124, a slope calculating unit 126, and an image converting unit 128.

The pixel extracting unit 122 performs a function of extracting a block to be matched from the first image 150a acquired from the first camera 110a and the second image 150b acquired from the second camera 110b . Referring to FIG. 2, the pixel extracting unit 122 sets a maximum slope A that can occur between the first camera 110a and the second camera 110b in consideration of the driving environment of the vehicle. For example, when the first camera 110a and the second camera 110b are installed in front of the vehicle, the first camera 110a and the second camera 110b mutually occur due to impact or vibration applied to the vehicle Set the maximum value of the angular difference to the maximum slope (A). The maximum slope A can be set to a small value in the case of a passenger car expected to run on a paved road and to a large value in the case of an off-road vehicle expected to run on an unpaved road. This maximum slope A may be set at the time of vehicle departure, but it may also be set in real time from the actual running condition of the vehicle.

The pixel extracting unit 122 extracts the row of the Xth pixel in the vertical direction from the first image 150a into the first extraction area. If the first image 150a and the second image 150b are M × N pixels, the first extraction region will have M × 1 pixels.

와 같은 수식을 이용하여 산출할 수 있다. 픽셀 추출부(122)는 상기 Y를 고려하여 제 2 영상(150b)으로부터 ±Y에 해당하는 영역을 제 2 추출영역으로 추출한다. 제 2 추출영역은 제 2 영상(150b)의 X번째 행의 상하 Y개만큼의 영역을 포함하는 것으로서 M×(1+2Y)의 픽셀을 갖게 될 것이다. On the other hand, in the second image 150b, considering the maximum slope A, the number Y of pixels to be considered in the vertical direction can be calculated. Y is

Can be calculated using an equation such as The pixel extracting unit 122 extracts a region corresponding to 占 from the second image 150b into the second extraction region in consideration of the Y. [ The second extraction region will have M × (1 + 2Y) pixels including the upper and lower Y regions of the Xth row of the second image 150b.

In the embodiment of the present invention, X may be set as a portion located at the center in the vertical direction in the first image 150a and the second image 150b. However, the X may be shifted up and down rather than the longitudinal center of the first image 150a and the second image 150b.

The information of the first extraction region and the second extraction region extracted by the pixel extracting unit 122 may be temporarily stored in the storage unit 130. [

The feature point matching unit 124 performs stereo matching using the first extraction region and the second extraction region to calculate a matching point. In one example, the feature point matching unit 124 may use a stereo matching algorithm such as Sum of Squared Differences (SSD), Sum of Absolute Differences (SAD), or Normalized Cross Correlation (NCC) The symmetric point of the region can be calculated. However, in the practice of the present invention, the stereo matching algorithm is not limited to the above, and various other algorithms for stereo matching may be used.

In the present invention, since the pixels of the first extraction region constituted by one row are compared with the pixels of the second extraction region, the symmetry point is calculated.

The slope calculating unit 126 calculates the slope between the image of the first extraction region and the image of the second extraction region using the minutiae calculated by the minutia matching unit 124. [

The image transform unit 128 rotates another image based on any one of the first image 150a or the second image 150b using the slope calculated by the slope calculating unit 126, And performs stereo matching between the first image 150a and the second image 150b.

The storage unit 130 temporarily stores the information of the first image 150a and the second image 150b and the information of the first extraction area and the second extraction area.

The electronic control unit 140 of the vehicle performs functions such as Forward Collision Warning (FCW) and Lane Departure Warning System (LDWS) by using the final stereo matched result in the image conversion unit 128.

4 is a flowchart illustrating a stereo image processing method according to a preferred embodiment of the present invention.

The stereo camera 110 of the vehicle acquires the first image 150a and the second image 150b using the first camera 110a and the second camera 110b (S200).

The pixel extracting unit 122 of the stereo matching unit 120 calculates the slope between the first image 150a and the second image 150b in consideration of the maximum slope A of the vehicle, The extraction region is extracted from the first image 150a and the second image 150b, respectively (S210).

The feature point matching unit 124 compares the first extraction region and the second extraction region to calculate feature points (S220).

The slope calculating unit 126 calculates a slope between the first extraction area and the second extraction area using the minutiae calculated from the minutia matching unit 124 (S230).

The image transform unit 128 rotates the other image based on the first image 150a or the second image 150b using the slope (S240).

The image converting unit 128 performs stereo matching between the first image 150a and the second image 150b using the rotated first image 150a and the second image 150b, (S250).

The final stereo-matched result in the image converting unit 128 is transmitted to the electronic control unit 140 of the vehicle, and the electronic control unit 140 performs control of the vehicle using the stereo matching result.

It will be apparent to those skilled in the art that various modifications, substitutions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims. will be. Therefore, the embodiments disclosed in the present invention and the accompanying drawings are intended to illustrate and not to limit the technical spirit of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments and the accompanying drawings . The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

100: Stereo image processing device
110: Stereo camera
120: stereo matching unit
122: Pixel extraction unit
124:
126: tilt calculating section
128:
130:
140: Electronic control device

Claims (7)

An apparatus for processing a stereo image of a vehicle,
A stereo camera including a first camera and a second camera for acquiring stereo images; And
And a stereo matching unit configured to perform stereo matching using the stereo image, wherein the stereo matching unit adjusts the stereo camera using the maximum slope between the first camera and the second camera, 1 image and a second image of the second camera,
Wherein the stereo matching unit comprises:
Extracting a first extraction region and a second extraction region for parallelization from the first image and the second image, each of which is composed of M × N pixels, and the first extraction region extracts X And the second extraction region includes a pixel number Y to be considered as a vertical direction in consideration of the maximum slope in the second image,

A pixel extracting unit that includes M x (1 + 2Y) pixels including upper and lower Y regions of the X th row;
A feature point matching unit for extracting feature points corresponding to the first extraction region and the second extraction region;
A slope calculating unit for calculating a slope between the first extraction area and the second extraction area using the feature points; And
And an image conversion unit for parallelizing the first image and the second image using the slope and performing a final stereo matching. delete delete delete A method of processing a stereo image of a vehicle,
(a) acquiring a first image and a second image from a stereo camera of the vehicle;
(b) a pixel extracting unit of the stereo matching unit extracts, from the first and second images of the M × N pixels using the maximum slope between the first camera and the second camera set in consideration of the state of the vehicle, Wherein the first extraction region includes Mx1 pixels obtained by extracting an Xth row in a vertical direction in the first image, and the second extraction region includes Mx1 pixels extracted in the vertical direction from the first image, Considering the maximum slope in 2 images, let Y be the number of pixels to be considered in the vertical direction

(1 + 2Y) pixels including upper and lower Y regions of the Xth row;
(c) calculating a slope between the first image and the second image by calculating a feature point corresponding to the first extraction region and the second extraction region in the stereo matching unit; And
(d) parallelizing the first image and the second image using the slope in the stereo matching unit and performing final stereo matching
/ RTI > delete delete

Images