WO2019009579A1

WO2019009579A1 - Stereo matching method and apparatus using support point interpolation

Info

Publication number: WO2019009579A1
Application number: PCT/KR2018/007496
Authority: WO
Inventors: 박상현
Original assignee: 전자부품연구원
Priority date: 2017-07-07
Filing date: 2018-07-03
Publication date: 2019-01-10
Also published as: KR101920159B1

Abstract

Provided are a stereo matching method, a stereo matching device, a recording medium, and an intelligent driver assistance system using the same. According to the present stereo matching method, as it is possible to extract an edge image of an input stereo image and calculate matching and disparity values thereof, memory usage and processing time required to obtain a three-dimensional image from a stereo image can be reduced. Therefore, it is possible to easily apply stereo matching to electrical and electronic systems for vehicles.

Description

Stereo matching method and apparatus using support point interpolation method

The present invention relates to a stereo matching method and apparatus, and more particularly, to a stereo matching method and apparatus using a support point interpolation method for extracting depth information of a two-dimensional image.

Conventional stereo matching techniques are used to obtain a three-dimensional image from a stereo image and are used to obtain a three-dimensional image from a plurality of two-dimensional images photographed at different photographing positions in the same line with respect to the same subject. The stereo image thus refers to a plurality of two-dimensional images taken at different photographing positions with respect to the subject, i.e., a plurality of two-dimensional images in a pair relationship with each other.

Generally, in order to generate a three-dimensional image from a two-dimensional image, a z coordinate, which is depth information, is required in addition to the x and y coordinates, which are horizontal and vertical position information of the two-dimensional image. To obtain the z-coordinate, a stereo matching technique using a stereo image is used.

However, the conventional stereo matching technology is a technology that can be operated on a CPU of a general computer, and when it is implemented as an FPGA / ASIC, it requires a lot of memory, which makes it difficult to apply it to a mass-production FPGA / ASIC hardware.

Accordingly, it is required to search for a method of generating a three-dimensional image using a two-dimensional stereo image using less memory.

It is an object of the present invention to provide a stereo matching method, a stereo matching apparatus, and a stereo matching method for extracting an edge image of an input stereo image and calculating matching and disparity values for the edge image, A recording medium, and an intelligent driver assistance system using the same.

According to an aspect of the present invention, there is provided a stereo matching method including receiving a stereo image including a first image and a second image simultaneously captured by two cameras; Extracting a first horizontal edge image and a first vertical edge image for the first image, and extracting a second horizontal edge image and a second vertical edge image for the second image; Matching a first edge pixel included in the first image with a second edge pixel included in the second image with respect to some or all of the edge pixels included in the first image; Calculating a disparity value that is a distance difference between the first edge pixel and the second edge pixel matched with respect to a part or all of the edge pixels included in the first image.

The matching step may include comparing a first horizontal edge image for a plurality of first peripheral pixels disposed in the periphery of the first edge pixel and pixel values in the first vertical edge image to a second horizontal edge image and a second vertical It may be determined as a second edge pixel to be matched with the first edge pixel by comparing the pixel value of the edge image with the pixel values of the edge image.

The matching step includes matching the pixel values of the first horizontal edge image and the plurality of first peripheral pixels in the first vertical edge image with the pixel values of the second horizontal pixel in the second horizontal pixel, The pixel having the smallest sum of the difference values of the edge image and the pixel values of the second neighboring pixels in the second vertical edge image may be determined as the second edge pixel matching the first edge pixel.

And, the first peripheral pixels may include more objects included in the first vertical edge image than those included in the first horizontal edge image.

Also, the first peripheral pixels may be divided into four regions arranged around the first edge pixel, and peripheral pixels arranged in the same shape may be selected for each region.

Generating a support point map by designating matched pixels as support points; And removing unnecessary support points from the support point map.

In addition, the removal step may remove unnecessary support points by removing all the support points in the area when the support point includes a certain number or less in the area of the specific size.

And generating a first disparity map by performing a first-order interpolation on a disparity value of the support point map, wherein the step of generating the primary disparity map comprises: ; Interpolating a disparity value by assigning pixels having no disparity value assigned in the reduced support point map to a smaller value among disparity values of pixels located on the left or right side; And expanding the reduced support point map to its original size if a disparity value is assigned to all pixels of the reduced support point map through interpolation.

The method may further include generating a second disparity map by performing a second interpolation on the first disparity map, wherein the step of generating the second disparity map comprises: The disparity value of the pixel to which the disparity value is not allocated is interpolated using the disparity values of the pixels having the four surrounding disparity values and the distances to the corresponding pixels, The second disparity map may be generated by interpolating disparity values of the unassigned pixels.

According to an embodiment of the present invention, there is provided a computer-readable recording medium having a computer readable recording medium having a computer readable recording medium configured to receive a stereo image including a first image and a second image simultaneously captured by two cameras; Extracting a first horizontal edge image and a first vertical edge image for the first image, and extracting a second horizontal edge image and a second vertical edge image for the second image; Matching a first edge pixel included in the first image with a second edge pixel included in the second image with respect to some or all of the edge pixels included in the first image; Calculating a disparity value that is a distance difference between a first edge pixel and a second edge pixel matched with respect to a part or all of the edge pixels included in the first image; A computer program may be included.

According to another aspect of the present invention, there is provided a stereo matching apparatus comprising: a camera unit for capturing a stereo image including a first image and a second image simultaneously captured by two cameras; An edge extraction unit for extracting a first horizontal edge image and a first vertical edge image for the first image and extracting a second horizontal edge image and a second vertical edge image for the second image; And a first edge pixel included in the first image is matched with a second edge pixel included in the second image with respect to some or all of the edge pixels included in the first image, And a video processing unit for calculating a disparity value, which is a distance difference between the first edge pixel and the second edge pixel matched for some or all of them.

An intelligent driver assistance system for recognizing objects around a vehicle using a stereo matching device includes a camera unit for photographing a stereo image including a first image and a second image simultaneously captured by two cameras; An edge extraction unit for extracting a first horizontal edge image and a first vertical edge image for the first image and extracting a second horizontal edge image and a second vertical edge image for the second image; And a first edge pixel included in the first image is matched with a second edge pixel included in the second image with respect to some or all of the edge pixels included in the first image, And a video processing unit for calculating a disparity value, which is a distance difference between the first edge pixel and the second edge pixel matched for some or all of the pixels.

According to various embodiments of the present invention, there is provided a stereo matching method, a stereo matching device, a recording medium, and an intelligent driver assistance system using the stereo matching method for extracting an edge image of an inputted stereo image and calculating matching and disparity values therefor So that the memory usage and processing time required to obtain a three-dimensional image from a stereo image can be reduced, which makes it easy to apply stereo matching to an automotive electrical and electronic system.

1 is a block diagram showing the configuration of a stereo matching apparatus according to an embodiment of the present invention;

Figure 2 is a flow chart provided to illustrate a stereo matching method, in accordance with an embodiment of the present invention;

FIG. 3 illustrates a process of a stereo matching method according to an embodiment of the present invention; FIG.

FIG. 4 is a diagram illustrating a process of finding matching pixels according to an exemplary embodiment of the present invention. FIG.

5 is a diagram showing peripheral pixels of a horizontal edge image and peripheral pixels of a vertical edge image according to an embodiment of the present invention;

6 is a diagram for explaining a process of calculating a disparity value and a depth value using the disparity value according to an embodiment of the present invention;

Figure 7 illustrates a support point map, in accordance with one embodiment of the present invention;

Figure 8 illustrates an algorithm for eliminating unnecessary support points according to an embodiment of the present invention;

9 is a diagram illustrating a support point map in which unnecessary support points are removed, according to an embodiment of the present invention;

10 is a diagram illustrating a process of performing a first-order interpolation according to an embodiment of the present invention.

11 is a view showing some pixels of a support point map before reduction according to an embodiment of the present invention;

12 is a diagram illustrating a process of reducing a support point map at a certain rate according to an embodiment of the present invention;

13 is a diagram illustrating a process of interpolating a disparity value of a reduced support point map and enlarging the disparity value back to its original size according to an embodiment of the present invention;

Figure 14 illustrates a method of performing a quadratic interpolation according to an embodiment of the present invention;

Figure 15 illustrates a post-processed disparity map, in accordance with one embodiment of the present invention;

16 is a diagram illustrating a disparity map for an input stereo image according to an embodiment of the present invention;

17 is a diagram illustrating a parallel processing process using a line unit buffer according to an embodiment of the present invention;

Figure 18 illustrates a pipeline for a matching process, according to one embodiment of the present invention;

19 is a diagram illustrating a hardware parallel processing structure according to an embodiment of the present invention.

Hereinafter, the present invention will be described in detail with reference to the drawings.

Stereo matching technology is a technique used to obtain a three-dimensional image from a stereo image, which is used to obtain a three-dimensional image from a plurality of two-dimensional images photographed at different shooting positions in the same line for the same subject. The stereo image means a plurality of two-dimensional images taken at different photographing positions with respect to the subject, that is, a plurality of two-dimensional images having a pair relationship with each other. For example, have.

Generally, in order to generate a three-dimensional image from a two-dimensional image, a z coordinate, which is depth information, is required in addition to the x and y coordinates, which are horizontal and vertical position information of the two-dimensional image. In order to obtain the z coordinate, disparity (disparity) information of the stereo image is required. Stereo matching is a technique used to obtain such a disparity. For example, if the stereo image is left and right images taken by two left and right cameras, one of the left and right images is set as the reference image and the other is set as the search image. In this case, the distance between the reference image and the search image for the same point in space, that is, the difference between the coordinates, is called disparity, and the disparity can be obtained using the stereo matching technique.

The stereo matching apparatus 100 obtains depth information for each pixel using the disparities between the reference image and the search image obtained as described above for all the pixels of the image, and generates a disparity map expressed by the three-dimensional coordinates .

A stereo matching technique used to calculate the parallax between the reference image and the search image according to the present embodiment will be schematically described as follows. First, the stereo matching apparatus 100 sets a plurality of windows around a reference pixel of a reference image, and sets a window of the same size around the search pixels with respect to the search image. The reference pixel means a corresponding point in the search image among the edge pixels of the reference image, i.e., a pixel to which the corresponding point is to be currently searched. The search pixel means a pixel to be currently identified as a corresponding point to the reference pixel among the pixels of the search image. The plurality of windows are in the form of a plurality of matrices each including a center pixel and surrounding pixels surrounding the center pixel.

As described above, the stereo matching apparatus 100 calculates the similarity between some of the surrounding pixels in the set reference pixel windows and some of the surrounding pixels in each of the searching pixel windows, And the search pixel of the search pixel window having the value corresponding to the corresponding point is matched with the corresponding pixel corresponding to the corresponding point. The distance between the reference pixel and the corresponding pixel is obtained as a disparity.

Through this process, the stereo matching apparatus 100 can generate a disparity map by performing a stereo matching method on the inputted stereo image.

1 is a block diagram showing the configuration of a stereo matching apparatus 100 according to an embodiment of the present invention. 1, the stereo matching apparatus 100 includes a camera unit 110, an edge extraction unit 120, and an image processing unit 130. [

The camera unit 110 includes a first camera 111 and a second camera 112. The camera unit 110 photographs a stereo image including a first image and a second image simultaneously captured by two cameras. Specifically, the first camera 111 and the second camera 112 may be a left eye camera and a right eye camera, and the first image may be a left eye image and the second image may be a right eye image. Also, the first image may be the reference image and the second image may be the search image.

The edge extracting unit 120 extracts a first horizontal edge image and a first vertical edge image for the first image and extracts a second horizontal edge image and a second vertical edge image for the second image. Specifically, the edge extracting unit 120 includes a Sobel filter, and extracts a horizontal edge image and a vertical edge image using a Sobel filter.

The image processing unit 130 processes various images, processes first and second images and corresponding edge images, and generates a disparity map for generating a three-dimensional image. To this end, the image processor 130 may match a first edge pixel included in the first image with a second edge pixel included in the second image, for a part or all of the edge pixels included in the first image, A disparity value which is a distance difference between the matched first edge pixel and the second edge pixel is calculated for part or all of the edge pixels included in one image. The image processing unit 130 generates a support point map composed of matched pixels, generates a disparity map through the primary interpolation and the secondary interpolation, and performs correction so as to further refine the disparity map .

The specific operation of the stereo matching apparatus 100 having such a configuration will be described below with reference to FIG. 2 and FIG. 2 is a flow chart provided to illustrate a stereo matching method, in accordance with an embodiment of the present invention. 3 is a diagram illustrating a process of a stereo matching method according to an embodiment of the present invention.

First, the stereo matching apparatus 100 receives a stereo image including a first image and a second image simultaneously captured by two cameras (S210). The original left image 310 and the original right image 320 of FIG. 3 correspond to the first image and the second image, respectively.

Then, the stereo matching apparatus 100 extracts a first horizontal edge image and a first vertical edge image for the first image, extracts a second horizontal edge image and a second vertical edge image for the second image, (S220). At this time, the stereo matching apparatus 100 can extract an edge image using a Sobel filter and generate a first horizontal edge image 311, a first vertical edge image 312, The second horizontal edge image 321, and the second vertical edge image 322, as shown in FIG.

Thereafter, the stereo matching apparatus 100 matches the first edge pixels included in the first image with the second edge pixels included in the second image, for some or all of the edge pixels included in the first image, A disparity value, which is a distance difference between the matched first and second edge pixels, is calculated for some or all of the edge pixels included in the first image (S230). In this case, the first image becomes the reference image and the second image becomes the search image.

In the matching step, the stereo matching apparatus 100 outputs a first horizontal edge image for a plurality of first peripheral pixels disposed around the first edge pixel and pixel values within the first vertical edge image to a second horizontal edge And determines the second edge pixel to be a second edge pixel to be matched with the first edge pixel by comparing the pixel value with the pixel values of the image and the second vertical edge image. Specifically, the stereo matching apparatus 100 calculates the pixel values of the first horizontal edge image and the plurality of first peripheral pixels in the first vertical edge image, among the pixels of the second image in the row corresponding to the first edge pixel, And the second edge pixel which matches the first edge pixel is determined as the pixel having the minimum sum of the difference values of the pixel values of the second horizontal edge image and the second surrounding pixels in the second vertical edge image. At this time, the first peripheral pixels divide the four regions arranged around the first edge pixel, and peripheral pixels arranged in the same shape for each region are selected.

More specifically, the selection process of the neighboring pixels for matching and the matching process will be described with reference to FIGS. 4 and 5. FIG.

4 is a diagram for explaining a process of finding matching pixels according to an embodiment of the present invention. In Fig. 4, Left Image represents a first image which is a reference image, and Right represents a second image which is a search image.

First, a first edge pixel which is a reference pixel in the first image corresponds to I_desc (x, y) in FIG. As shown in FIG. 4, the stereo matching apparatus 100 selects 5x5 window regions in the upper left, upper right, lower left, and lower right regions based on the first edge pixel (x, y). In Fig. 4, each window region is represented by Lf1, Lf2, Lf3, and Lf4. Then, the stereo matching apparatus 100 selects four surrounding pixels for the horizontal edge image for each window area, selects 12 surrounding pixels for the vertical edge image, and selects 16 peripheral pixels for each window area .

At this time, the first peripheral pixels are set to include more objects included in the first vertical edge image than those included in the first horizontal edge image. This is because the vertical edge has a more significant effect on matching than the horizontal edge. Accordingly, the stereo matching apparatus 100 selects more peripheral pixels of the first vertical edge image than the number of peripheral pixels of the first horizontal edge image, and considers the matching. A method of selecting 16 peripheral pixels for one 5x5 window region is shown in detail in Fig.

5 is a diagram showing peripheral pixels of a horizontal edge image and peripheral pixels of a vertical edge image, according to an embodiment of the present invention.

In FIG. 5, the image on the left represents a horizontal edge image, and in the horizontal edge image, four peripheral pixels around the center point of the 5x5 window region are selected.

In FIG. 5, the right image represents a vertical edge image. In the vertical edge image, a total of 12 peripheral pixels including 10 peripheral points and 2 central points of the 5x5 window region are selected. Here, in the case of the vertical edge image, since the center point plays an important role in matching, the center point is selected twice by weighting.

5, the stereo matching apparatus 100 selects four surrounding pixels for each horizontal edge image for each 5x5 window region, selects 12 surrounding pixels for the vertical edge image, A total of 16 peripheral pixels are selected for each region.

4, the stereo matching apparatus 100 selects 16 first peripheral pixels for each of the four window regions Lf1, Lf2, Lf3, and Lf4.

The stereo matching apparatus 100 then calculates the pixel values of the pixels (xd, y) of the second image in the row corresponding to the first edge pixel (x, y) of the first image (Left image) The second peripheral pixels are selected in the same manner. 4, the stereo matching apparatus 100 obtains a matching cost value, which is the sum of the difference values of the pixel values of the first and second neighboring pixels . Then, the stereo matching apparatus 100 determines a pixel having a minimum matching cost value as a second edge pixel to be matched with the first edge pixel. Since the sum of the difference values of neighboring pixels, that is, the matching cost value, is the pixel of the second image having the second neighboring pixels most similar to the first neighboring pixels of the first edge pixel, And the second edge pixel of the second image corresponding to the second edge.

Sum1 represents the sum of the difference between the pixel value of the peripheral pixels in the Lf1 window region and the pixel value of the peripheral pixels in the Rf1 window region, Sum2 denotes the sum of the pixel value of the peripheral pixels in the Lf2 window region and Rf2 Sum3 represents the sum of pixel values of neighboring pixels of the Lf3 window region and neighboring pixels of the Rf3 window region, and Sum4 represents the sum of the difference values of the pixel values of neighboring pixels of the Lf4 window region And a difference value between pixel values of neighboring pixels of the Rf4 window region. Accordingly, the sum of the sum values of Sum1 + Sum2 + Sum3 + Sum4 represents the matching cost value, which is the sum of the difference values of the pixel values of the second peripheral pixels corresponding to the first peripheral pixels.

In FIG. 4, all matching cost values are calculated for 245 pixels in the y-th row of the second image, and a pixel having the lowest matching cost value among the 245 pixels in the y- 2 edge pixels.

Through this process, the stereo matching apparatus 100 can determine a second edge pixel that matches the first edge pixel.

The stereo matching apparatus 100 calculates a disparity value, which is a distance difference between the first edge pixel and the second edge pixel, and will be described with reference to FIG. 6 is a diagram for explaining a process of calculating a disparity value and a depth value using the disparity value according to an embodiment of the present invention.

The disparity is a value indicating how the positions of the pixels representing the same point differ from each other in the two images. Referring to the formula shown in FIG. 6, the disparity value (x _R -x _T ) Can be obtained.

Referring back to FIGS. 2 and 3, the stereo matching apparatus 100 designates each of the matched pixels as a support point to generate a support point map 330 (S240). 7 is a diagram illustrating a support point map according to an embodiment of the present invention. Referring to FIG. 7, it can be seen that a support point map (Initial SPOINT) which is a collection of pixels matched from the left image and the right image is displayed.

Thereafter, the stereo matching apparatus 100 removes unnecessary support points from the support point map (S250). Specifically, when the support point includes a specific number or less within a specific size area, the stereo matching apparatus 100 removes all the support points in the corresponding area, thereby eliminating unnecessary support points, Thereby generating a map 340. This will be described with reference to Figs. 8 and 9. Fig.

Figure 8 is a diagram illustrating an algorithm for eliminating unwanted support points, in accordance with an embodiment of the present invention. In FIG. 8, when there are five or less support points in a 40x40 area, it can be confirmed that the algorithm is implemented by deleting the support points of the corresponding area. However, it goes without saying that the size of the area and the number of support points can be changed flexibly.

9 is a diagram illustrating a support point map in which unnecessary support points are removed according to an embodiment of the present invention. In the initial SPOINT, the support points map (Refined SPOINT) from which unnecessary support points are removed shows that unnecessary support points disappear in the sky area.

2 and 3, the stereo matching apparatus 100 performs a first-order interpolation on the disparity value of the support point map to generate the primary disparity map 350 (S260). Specifically, the stereo matching apparatus 100 reduces the support point map at a certain ratio, and assigns the pixels to which the disparity value is not allocated in the reduced support point map to a smaller value among the disparity values of the pixels located on the left or right The disparity value is interpolated. When disparity values are assigned to all the pixels of the support point map reduced through interpolation, the reduced support point map is expanded to the original size, thereby performing the primary interpolation.

The specific primary interpolation process will be described with reference to FIGS. 10 to 13. FIG. 10 is a diagram illustrating a process of performing a first-order interpolation according to an embodiment of the present invention. As shown in FIG. 10, the stereo matching apparatus 100 reduces the support point map to a 1/16 ratio, and assigns the pixels to which the disparity value is not allocated in the reduced support point map to the left or right And the disparity value is allocated to all the pixels of the support point map reduced through the interpolation, the reduced support point map is enlarged to the original size, The process of performing the interpolation can be confirmed.

11 is a diagram showing some pixels of a support point map before reduction according to an embodiment of the present invention. As shown in FIG. 11, the stereo matching apparatus 100 selects pixels (black and oblique lines) at regular intervals when reducing the image at a constant rate. In the case of FIG. 11, it can be confirmed that, in order to reduce to 1/16, a pixel for only the uppermost pixel in the 4x4 region is selected as the pixel to be removed and the remaining pixel is selected as the pixel to be removed.

FIG. 12 is a diagram illustrating a process of reducing a support point map at a predetermined ratio according to an embodiment of the present invention. If you reduce it to 1/16, you can see that only black and oblique pixels remain.

13 is a diagram illustrating a process of interpolating a disparity value of a reduced support point map and enlarging the disparity value back to its original size according to an embodiment of the present invention. The disparity value assigned to the black pixel is applied to the oblique pixel to which the disparity value is not assigned (for example, the disparity value of the smallest black pixel among surrounding black pixels is applied). Then, if it is increased to 16 times and then enlarged to the original size, it can be confirmed that the disparity value is allotted to the uppermost pixel in the 4x4 matrix.

In this manner, the stereo matching apparatus 100 performs a first-order interpolation on the disparity value of the support point map to generate the primary disparity map 350. [ The primary disparity map can confirm that a disparity value is allocated to only some pixels at predetermined intervals.

2 and 3, the stereo matching apparatus 100 performs a secondary interpolation on the primary disparity map to generate a secondary disparity map (S270). Specifically, the stereo matching apparatus 100 uses the disparity values of the pixels having the four surrounding disparity values and the distances to the pixels to which the disparity values are not allocated, A disparity map is generated by interpolating disparity values of pixels to which all disparity values are not allocated in a method of interpolating disparity values of pixels to which no values are assigned. This will be described in detail with reference to Fig.

FIG. 14 is a diagram illustrating a method of performing a second-order interpolation according to an embodiment of the present invention. As shown in FIG. 14, the secondary interpolation is performed using four peripheral support points.

Through this process, the stereo matching apparatus 100 can generate a secondary disparity map to which disparity values for all pixels are allocated. In addition, the stereo matching apparatus can perform noise elimination and hole filling using a median filter to generate a more accurate secondary disparity map, which is shown in FIG. 15 is a diagram illustrating a post-processed disparity map, in accordance with an embodiment of the present invention. As shown in FIG. 15, the stereo matching apparatus 100 may generate a more precise second-order disparity map by post-processing the second disparity map by a median filter.

16 is a diagram illustrating a disparity map for an input stereo image according to an embodiment of the present invention. Through the above-described process, the stereo matching apparatus 100 can generate the disparity map using the original image photographed through the two cameras as shown in FIG.

In addition, the stereo matching apparatus 100 may optimize the memory size and processing speed through parallel processing. This will be described with reference to FIGS. 17 to 19. FIG.

17 is a diagram illustrating a parallel processing process using a line unit buffer according to an embodiment of the present invention. As shown in FIG. 17, the stereo matching apparatus 100 may perform parallel processing using two buffer lines in an edge image generation process.

18 is a diagram illustrating a pipeline for a matching process according to an embodiment of the present invention. As shown in FIG. 18, the stereo matching apparatus 100 can reduce the time complexity to 1/4 through the four pipeline configurations. This allows the stereo matching device 100 to optimize memory size and access.

19 is a diagram illustrating a hardware parallel processing structure according to an embodiment of the present invention. As shown in FIG. 19, the stereo matching apparatus 100 can reduce the time complexity to 1/4 through four parallel processes when performing the second-order interpolation.

In this manner, the stereo matching apparatus 100 can reduce the time complexity and improve the memory usage through parallel processing.

It should be noted that the stereo matching apparatus 100 according to the present embodiment may be applied to an ADAS (Advanced Driver Assistance System) that recognizes objects around the vehicle.

Needless to say, the technical idea of the present invention can also be applied to a computer-readable recording medium in which a function of the stereo matching apparatus 100 according to the present embodiment and a computer program for performing a stereo matching method are recorded. In addition, the technical idea according to various embodiments of the present invention may be realized in the form of a computer-readable programming language code recorded on a computer-readable recording medium. The computer-readable recording medium is any data storage device that can be read by a computer and can store data. For example, the computer-readable recording medium may be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, a flash memory, a solid state disk (SSD), or the like. In addition, the computer readable code or program stored in the computer readable recording medium may be transmitted through a network connected between the computers.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present invention.

Claims

Receiving a stereo image including a first image and a second image simultaneously captured by two cameras;

Extracting a first horizontal edge image and a first vertical edge image for the first image, and extracting a second horizontal edge image and a second vertical edge image for the second image;

Matching a first edge pixel included in the first image with a second edge pixel included in the second image with respect to some or all of the edge pixels included in the first image; And

Calculating a disparity value that is a distance difference between a first edge pixel and a second edge pixel matched with respect to a part or all of the edge pixels included in the first image.
The method according to claim 1,

In the matching step,

The first horizontal edge image and the pixel values in the first vertical edge image for a plurality of first peripheral pixels arranged in the periphery of the first edge pixel are divided into pixel values of the second horizontal edge image and the second vertical edge image, And a second edge pixel which is matched with the first edge pixel by comparing the first edge pixel and the second edge pixel.
The method according to claim 1,

In the matching step,

The first horizontal edge image and the pixel values of the plurality of first peripheral pixels in the first vertical edge image and the second horizontal edge image and the second vertical edge image in the pixels of the second image in the row corresponding to the first edge pixel, Wherein a pixel having a minimum sum of difference values of pixel values of second surrounding pixels in an edge image is determined as a second edge pixel to be matched with a first edge pixel.
The method of claim 3,

The first peripheral pixels,

Wherein more of the first vertical edge image is contained in the first vertical edge image than that contained in the first horizontal edge image.
The method of claim 3,

The first peripheral pixels,

Wherein four regions arranged around the first edge pixel are divided, and neighboring pixels arranged in the same shape for each region are selected.
The method according to claim 1,

Designating matched pixels as support points to generate a support point map; And

And removing unnecessary support points from the support point map.
The method of claim 6,

In the removing step,

And removing unnecessary support points by removing all the support points in the area when the support point contains a certain number or less in the area of the specific size.
The method of claim 6,

And generating a first disparity map by performing a first-order interpolation on the disparity value of the support point map,

The step of generating the primary disparity map comprises:

Reducing the support point map at a constant rate;

Interpolating a disparity value by assigning pixels having no disparity value assigned in the reduced support point map to a smaller value among disparity values of pixels located on the left or right side; And

Expanding the reduced support point map to its original size if a disparity value is assigned to all pixels of the reduced support point map through interpolation.
The method of claim 8,

And generating a second disparity map by performing a second interpolation on the first disparity map,

The step of generating the second disparity map comprises:

For a pixel to which a disparity value is not assigned, the disparity values of the pixels having the four nearby disparity values and the distance from the pixels to the disparity value of the pixel to which the disparity value is not allocated are used Wherein the second disparity map is generated by interpolating the disparity values of the pixels to which all disparity values are not allocated.
Receiving a stereo image including a first image and a second image simultaneously captured by two cameras;

Extracting a first horizontal edge image and a first vertical edge image for the first image, and extracting a second horizontal edge image and a second vertical edge image for the second image;

Matching a first edge pixel included in the first image with a second edge pixel included in the second image with respect to some or all of the edge pixels included in the first image;

Calculating a disparity value that is a distance difference between a first edge pixel and a second edge pixel matched with respect to a part or all of the edge pixels included in the first image; A computer-readable recording medium containing a computer program.
A camera unit for photographing a stereo image including a first image and a second image simultaneously photographed by two cameras;

An edge extraction unit for extracting a first horizontal edge image and a first vertical edge image for the first image and extracting a second horizontal edge image and a second vertical edge image for the second image; And

The first edge pixels included in the first image are matched with the second edge pixels included in the second image with respect to a part or all of the edge pixels included in the first image and a part of the edge pixels included in the first image And a disparity value that is a difference in distance between the first edge pixel and the second edge pixel matched with respect to all of the first edge pixel and the second edge pixel.
An intelligent driver assistance system for recognizing objects around a vehicle using a stereo matching device,

The stereo matching apparatus comprises:

A camera unit for photographing a stereo image including a first image and a second image simultaneously photographed by two cameras;

An edge extraction unit for extracting a first horizontal edge image and a first vertical edge image for the first image and extracting a second horizontal edge image and a second vertical edge image for the second image; And

The first edge pixels included in the first image are matched with the second edge pixels included in the second image with respect to a part or all of the edge pixels included in the first image and a part of the edge pixels included in the first image And a disparity value which is a difference in distance between the first edge pixel and the second edge pixel which are matched with respect to all of the first edge pixel and the second edge pixel.