JP2008216126A - Distance image generating device, distance image generation method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a distance image generating device, a distance image generation method, and a program capable of precisely generating a distance image, at a high speed. <P>SOLUTION: The distance image generation device includes imaging devices 2a, 2b for acquiring a standard image and a reference image; multiple resolution image generating part 13 generating a plurality of hierarchical images with different resolution about the standard image and the reference image; a corresponding point search part 15, which includes an SAD calculation part 6 performing corresponding point search for at least the hierarchical image having the lowest resolution and a POC calculation part 7 for performing, corresponding point search for at least the hierarchical image having the highest resolution, which starts the corresponding point search from the hierarchical image having the low resolution, sequentially performs the corresponding point search for the hierarchical image having the high resolution, and sequentially repeats the corresponding point search, until the hierarchical image having the highest resolution is reached; a search area setting part 14 for setting a search area of the hierarchical image of the next hierarchy, on the basis of the search result about the immediately preceding hierarchy; and a distance image generation part 16 generating the distance image on the basis of the search result. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a distance image generation device, a distance image generation method, and a program, and more particularly, to a distance image generation device, a distance image generation method, and a program that generate a distance image by capturing a plurality of images of the same object.

  Conventionally, the same object (subject) is imaged from different positions by a plurality of imaging means to obtain a plurality of pieces of image information, and correlation by an SAD (Sum of Absolute Difference) calculation method, an SSD (Sum of Squared Difference) calculation method, or the like. An apparatus for calculating a correlation degree of the image information by performing an operation, obtaining a parallax value for the same object based on the degree of correlation, obtaining a position (distance value) of the object from the parallax value, and generating a distance image Are known.

  Further, in such an apparatus, a plurality of images having different resolutions are generated, and a hierarchical structure is formed for each resolution. Correlation by a sequential SAD calculation method, SSD calculation method, or the like from a lower resolution upper layer to a higher resolution lower layer A technique for generating a distance image by repeating the calculation is known (for example, see Patent Document 1).

  In the technique disclosed in Patent Document 1, an SAD calculation method, an SSD calculation method, or the like that can be processed at high speed is used as a calculation method, and the calculation is repeated from a lower resolution upper layer to a higher resolution lower layer. At this time, since the search area in the next layer is set based on the parallax value obtained in the lower resolution upper layer, the time for performing the arithmetic processing up to the lowest layer (the highest resolution layer) can be reduced.

  In recent years, as a method for obtaining three-dimensional position information with higher accuracy, a corresponding point between a reference image and a reference image is searched using a POC (Phase-Only Correlation) calculation method. It has been proposed to generate a distance image of an object based on the amount of positional deviation (see, for example, Non-Patent Document 1).

  In the POC calculation method, the original image data and the image data to be collated are mathematically processed by Fourier transformation to be decomposed into amplitude (grayscale data) and phase (image contour data). This is an algorithm for obtaining the correlation between both images using only the phase information. Unlike conventional correlation calculation methods using amplitude information, such as SAD calculation method and SSD calculation method, the algorithm is resistant to disturbance and has high accuracy. There is a feature that the calculation result can be obtained.

Furthermore, in the technique disclosed in Non-Patent Document 1, the POC operation is sequentially performed on the image formed in the hierarchical structure according to the resolution from the lower resolution upper layer to the higher resolution lower layer. It is possible to search for corresponding points with high accuracy and to generate an accurate distance image.
JP 2001-319229 A A Sub-Pixel Correspondence Search Technique for Computer Vision Applications (Tohoku University)

  However, as disclosed in Patent Document 1, when the SAD calculation method and the SSD calculation method are used, high-speed processing is possible, but it is difficult to calculate the parallax value with high accuracy. There was a problem with not being able to get.

  In addition, the POC calculation method disclosed in Non-Patent Document 1 can obtain a high-precision calculation result, but requires a long time for calculation processing. For this reason, for example, as in the technique disclosed in Non-Patent Document 1, if processing is performed for all hierarchies in order from the lowest resolution, there is a problem that the processing efficiency is reduced and the work efficiency is lowered. .

  Therefore, the present invention has been made to solve the above-described problems, and provides a distance image generation device, a distance image generation method, and a program that can generate a distance image at high speed and with high accuracy. It is the purpose.

In order to solve the above-described problem, a distance image generation device according to claim 1 is provided.
Image acquisition means for acquiring first image information and second image information to be compared;
Multi-resolution image generation means for generating a multi-resolution image in which a plurality of hierarchical images having different resolutions form a hierarchical structure in order from low resolution to high resolution for the first image information and the second image information acquired by the image acquisition means When,
The first image information and the second image information are collated to search for a corresponding point in a predetermined search area, and at least a first resolution layer image of the lowest resolution among the multi-resolution images A first calculation means for searching for corresponding points by a calculation method; and a second calculation for searching for corresponding points by a second calculation method that is more accurate than the first calculation method for at least the highest resolution hierarchical image. And starting the corresponding point search from a low-resolution hierarchical image among the multi-resolution images, sequentially performing the corresponding point search for a high-resolution hierarchical image, and sequentially reaching the highest-resolution hierarchical image. Corresponding point search means for repeating the corresponding point search;
Search area setting means for setting a search area for a next hierarchical image based on a search result for the hierarchical image when the search for one hierarchical image is completed by the corresponding point search means;
A distance image generating means for generating a distance image based on a search result by the corresponding point searching means;
It is characterized by having.

The invention according to claim 2 is the distance image generation device according to claim 1,
After the corresponding point search by the second calculation means is performed, the corresponding point search by the first calculation means is not performed for the subsequent hierarchical images.

The invention according to claim 3 is the distance image generation device according to claim 1 or 2, wherein
The first calculation method performed by the first calculation means is any one of an SAD calculation method, an SSD calculation method, and an NCC calculation method.

The invention according to claim 4 is the distance image generation device according to any one of claims 1 to 3,
The second calculation method performed by the second calculation means is a POC calculation method.

The distance image generation method according to claim 5,
An image acquisition step of acquiring first image information and second image information to be compared;
A multi-resolution image generation step of generating a multi-resolution image in which a plurality of hierarchical images having different resolutions form a hierarchical structure in order from a low resolution to a high resolution with respect to the first image information and the second image information acquired in the image acquisition step When,
The first image information and the second image information are collated to search for a corresponding point in a predetermined search area, and at least a first resolution layer image of the lowest resolution among the multi-resolution images A first calculation step of searching for corresponding points by a calculation method, and a second calculation of searching for corresponding points by a second calculation method having higher accuracy than the first calculation method for at least the highest resolution hierarchical image And starting the corresponding point search from a low-resolution hierarchical image among the multi-resolution images, sequentially performing the corresponding point search for a high-resolution hierarchical image, and sequentially reaching the highest-resolution hierarchical image. A corresponding point search step for repeating the corresponding point search;
A search region setting step of setting a search region for a hierarchical image of the next layer based on a search result for the hierarchical image when the search for one hierarchical image is completed in the corresponding point search step;
A distance image generation step of generating a distance image based on a search result in the corresponding point search step;
It is characterized by having.

Invention of Claim 6 is the distance image generation method of Claim 5, Comprising:
After the corresponding point search by the second calculation step is performed, the corresponding point search by the first calculation step is not performed for the subsequent hierarchical images.

Invention of Claim 7 is the distance image generation method of Claim 5 or Claim 6, Comprising:
The first calculation method performed in the first calculation step is any one of an SAD calculation method, an SSD calculation method, and an NCC calculation method.

Invention of Claim 8 is the distance image generation method as described in any one of Claim 5-7, Comprising:
The second calculation method performed in the second calculation step is a POC calculation method.

The invention according to claim 9 is a computer-readable program,
A multi-resolution image generation function for generating a multi-resolution image in which a plurality of hierarchical images having different resolutions form a hierarchical structure in order from a low resolution to a high resolution for the first image information and the second image information to be compared;
The first image information and the second image information are collated to search for a corresponding point in a predetermined search area, and at least a first resolution layer image of the lowest resolution among the multi-resolution images A first calculation function for searching for corresponding points by a calculation method, and a second calculation for searching for corresponding points by a second calculation method that is more accurate than the first calculation method for at least the highest resolution hierarchical image Among the multi-resolution images, the corresponding point search is started from a low-resolution hierarchical image, and the corresponding point search is sequentially performed for a hierarchical image with a high resolution, and the hierarchical image with the highest resolution is sequentially reached. A corresponding point search function for repeating the corresponding point search;
A search area setting function for setting a search area for the next hierarchical image based on a search result for the hierarchical image when the search for the hierarchical image is completed by the corresponding point search function;
A distance image generation function for generating a distance image based on a search result by the corresponding point search function;
Is realized by a computer.

Invention of Claim 10 is a program of Claim 9, Comprising:
After the corresponding point search by the second calculation method is performed, the corresponding point search by the first calculation method is not performed for the subsequent hierarchical images.

Invention of Claim 11 is a program of Claim 9 or Claim 10, Comprising:
The first calculation method is any one of an SAD calculation method, an SSD calculation method, and an NCC calculation method.

The invention according to claim 12 is the program according to any one of claims 9 to 11,
The second calculation method is a POC calculation method.

  According to the first, fifth, or ninth aspect of the invention, a multi-resolution image composed of a plurality of hierarchical images having different resolutions is generated, and the low-resolution hierarchical image is directed to the high-resolution hierarchical image. The corresponding point search process is repeated sequentially, and the search area of the next hierarchical image is set based on the search result obtained for the low resolution hierarchical image, enabling high-speed and high-accuracy corresponding point search. Thus, it is possible to generate an accurate distance image of the object based on the result.

  The corresponding point search means includes a first calculation means for searching corresponding points for at least the lowest resolution hierarchical image by the first calculation technique, and a second calculation technique for at least the highest resolution hierarchy image. And a second computing means for searching for corresponding points. For example, a computing means that performs computing processing at high speed is used for a low-resolution hierarchical image, and a high-precision hierarchical image is used for a high-resolution hierarchical image. By using the calculation means for performing the calculation process, it is possible to efficiently perform the corresponding point search, reduce the total processing time of the correlation calculation, and perform high-precision processing.

  According to the invention described in claim 2, claim 6 or claim 10, the calculation by the second calculation means for performing the corresponding point search once by the second calculation method having higher accuracy than the first calculation method. Since the first calculation means does not perform the calculation after it has been performed, the high-resolution hierarchical image can be subjected to high-precision calculation processing, and a highly reliable calculation result can be obtained. The effect of.

  According to the invention of claim 3, claim 7 or claim 11, a low-resolution hierarchical image can be processed by any one of the SAD operation method, SSD operation method, and NCC operation method capable of performing high-speed processing. Since the corresponding point search is performed, the corresponding point search at a low resolution can be efficiently performed, and the total processing time of the correlation calculation can be shortened.

  According to the invention of claim 4, claim 8 or claim 12, the corresponding point search for the high-resolution hierarchical image is performed by the POC calculation method which is a high-accuracy, robust and robust calculation method. In the corresponding point search at high resolution, there is an effect that a highly reliable calculation result with high reliability can be obtained.

  Hereinafter, an embodiment of a distance image generating apparatus according to the present invention will be described with reference to FIGS. 1 to 10. However, the scope of the invention is not limited to the illustrated examples.

  The distance image generation apparatus 1 according to the present embodiment calculates a positional deviation amount by collating two images, obtains distance information to an object based on the calculated positional deviation amount, and generates a distance image. is there.

As shown in FIG. 1, the distance image generating device 1 includes two imaging devices 2a and 2b that capture an image of a subject (target object).
The imaging devices 2a and 2b each include an imaging element (photoelectric conversion element) 3 and a lens 4 that forms a subject light image on an imaging surface (not shown) of the imaging element 3 as imaging means. The image pickup device 3 is an image sensor such as a charge coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS). The image pickup device 3 photoelectrically converts incident light transmitted through the lens 4 into an electric signal and takes it in. Thus, the subject light image is converted into an analog image signal. The distance image generation device 1 includes an A / D conversion unit 5 that converts an analog image signal obtained by the image sensor 3 into a digital image signal.

  The two imaging devices 2a and 2b capture the same object from different viewpoints and acquire image data as image information, one of which is reference image data (first image information) based on a reference image. , And the other acquires reference image data (second image information) based on an image to be compared (hereinafter referred to as “reference image”). Thus, the two imaging devices 2a and 2b function as an image acquisition unit that acquires image data of the standard image and the reference image.

  In addition, the distance image generation device 1 includes an image processing unit 10 that performs image processing on the image data acquired by each of the imaging devices 2a and 2b, calculates a parallax value, and generates a distance image.

  The image processing unit 10 includes a processing device such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a DSP (Digital Signal Processor), a system program, and a distance image generation processing program for performing distance image generation processing described later. Is a computer composed of a ROM (Read Only Memory) for storing various control programs and the like, a RAM (Random Access Memory) for temporarily storing various data (all not shown), etc. The lens distortion correction unit 11 that corrects the lens distortion, the image parallelization processing unit 12 that performs parallelization processing of the images acquired by the imaging devices 2a and 2b, and the image data acquired by the imaging devices 2a and 2b have different resolutions. A multi-resolution image generation unit 13 that generates a plurality of hierarchical images (described later), and a search region setting unit 1 that sets a search region for searching for corresponding points , And a corresponding point search section 15, distance image generation unit 16 generates a distance image or the like for the corresponding point search process.

  The lens distortion correction unit 11 corrects distortion (lens distortion) that occurs when an image is formed using the lens 4. Since the lens 4 is distorted, an image cannot be generated correctly, and the image is distorted particularly at the end portion. The lens distortion correction unit 11 performs such lens distortion correction. For example, a lens distortion correction parameter is created and stored in advance, and an image is corrected based on the parameter. The correction method by the lens distortion correction unit 11 is not limited to the one exemplified here, and various known means can be used.

  The image collimation processing unit 12 converts the two images acquired by the respective imaging devices 2a and 2b as if they were taken from a camera moved in parallel so that the epipolar lines become parallel. A parallelization process is performed. Since it is difficult to accurately install the two imaging devices 2a and 2b, the image parallelization process is performed in order to correct image distortion caused by an installation position shift of the two imaging devices 2a and 2b. Is called.

  By performing the parallelization processing on the images, the two images are not shifted in the vertical direction (Y-axis direction in FIG. 3), and when performing correlation calculation processing described later, the horizontal direction (X-axis in FIG. 3). It is sufficient to detect only the deviation in the direction. Note that the method of image collimation processing by the image collimation processing unit 12 is not limited to the one exemplified here, and various known means can be used.

  The multi-resolution image generation unit 13 has a plurality of images with different resolutions of the reference image data (first image information) and the reference image data (second image information) acquired by the imaging devices 2a and 2b. A multi-resolution image having a hierarchical structure is generated in order from high to low resolution. An image in each layer is hereinafter referred to as a “layer image”.

  FIG. 2 schematically shows an example of the image data of the multi-resolution image generated by the multi-resolution image generation unit 13. In FIG. 2, multiplexing is performed when the resolution (maximum resolution) of the original image data is 1280 × 960 and the minimum resolution is 40 × 30, which is 1/32 of the resolution of the original image data. 2 shows a hierarchical structure of image data of a resolution image. In this case, the 0th layer which is the highest resolution layer is a 1280 × 960 resolution layered image, the nth layer which is the lowest resolution layer is a 40 × 30 resolution layered image, from the 0th layer. Up to the nth layer, a plurality of hierarchical images having different resolutions are generated, and a multi-resolution image having many layers is generated.

Note that the number of layers of the multi-resolution image generated by the multi-resolution image generation unit 13 and the resolution of the minimum resolution are not particularly limited. For example, if the original image data is a low-resolution image of about 80 × 60, the 0th layer is a hierarchical image having a resolution of 80 × 60, and the first layer is a hierarchical image having a resolution of 40 × 30. In addition, the hierarchical structure is as few as two to three layers.
The structure of the multi-resolution image generated by the multi-resolution image generation unit 13 may be set in advance as a default according to the resolution of the original image data or the user can arbitrarily set the structure. Also good.

The search area setting unit 14 sets an area in which the corresponding point search unit 15 performs the corresponding point search, sets a distance measuring point in the reference image, and searches for a predetermined area centered on the distance measuring point. Set as area.
In the present embodiment, as described later, when the search for one hierarchical image is completed by the corresponding point search unit 15, the search result for the hierarchical image is output to the search region setting unit 14. In the second and subsequent corresponding point search processing, the search area setting unit 14 sets a search area for the next hierarchical image for the reference image based on the search result.

  That is, in the second and subsequent corresponding point search processing, the search region setting unit 14 uses the candidate coordinates (x0, y0) obtained in the immediately preceding corresponding point search processing and the surrounding region as the next hierarchy. It is set as a search area for the corresponding point search process in the image. In the present embodiment, for example, the search area setting unit 14 extracts an area corresponding to 5 pixels × 5 pixels around (x0, y0) and sets it as a search area. Note that the range of the search area set by the search area setting unit 14 is not limited to that exemplified here.

Corresponding point search unit 15 searches for corresponding points of both image data in the search area extracted and set from each of the base image data and the reference image data, and calculates the amount of displacement and obtains a correlation value. It is.
The corresponding point search unit 15 includes a SAD calculation unit 6 as a first calculation unit that searches for a corresponding point by the SAD calculation method (first calculation method), and more accurate than the SAD calculation method (first calculation method). And a POC calculation unit 7 as second calculation means for searching for corresponding points by a high calculation method POC calculation method (second calculation method). The corresponding point search unit 15 starts the corresponding point search from the low-resolution hierarchical image among the multi-resolution images, and sequentially reflects the search result in the previous hierarchy on the hierarchical image with the higher resolution sequentially. Corresponding point search is performed by the SAD computing unit 6 or the POC computing unit 7 until the hierarchical image of the original image resolution (maximum resolution) (in this embodiment, the 0th hierarchical image shown in FIG. 2) is reached. The corresponding point search is repeated in order.

  Specifically, the corresponding point search unit 15 first starts the corresponding point search from the lowest resolution hierarchical image by the SAD calculation unit 6, and when the search for the hierarchical image is finished, the corresponding point search processing for the hierarchical image is performed. The vertex of the obtained correlation value is calculated as a candidate coordinate, and is output to the search area setting unit 14 as a search result. Then, when the search area setting unit 14 sets the search area of the hierarchical image of the next hierarchy based on the search result for the hierarchical image, the corresponding point search unit 15 sets the search area within the set search area for the next hierarchical image. Search for corresponding points.

  The corresponding point search unit 15 determines whether the corresponding point search is performed by the SAD calculation unit 6 or the POC calculation unit 7. That is, the corresponding point search unit 15 searches the corresponding point for at least the lowest resolution hierarchical image among the multi-resolution images generated by the multi-resolution image generation unit 13, and searches for the corresponding point at least with the highest resolution. Corresponding point search is performed on the hierarchical image by the POC calculation unit 7, and the intermediate point hierarchical image is searched by the corresponding point search unit 15 by either the SAD calculation unit 6 or the POC calculation unit 7. It is determined whether a corresponding point search is performed. And corresponding point search is performed by the calculation method according to the said hierarchy image.

In the present embodiment, a case where the corresponding point search is performed by the POC calculation unit 7 from a hierarchical image having a resolution half that of the original image will be described below as an example.
That is, when the original image data has a resolution of 1280 × 960 and the minimum resolution is 40 × 30, as shown in FIG. 2, the SAD calculation unit starts from the nth layer having the resolution of 40 × 30. 6 is started, and the corresponding point search is performed by the POC calculation unit 7 from the resolution layer of 640 × 480, which is 1/2 resolution of 1280 × 960, up to the 0th layer of the highest resolution. Is called.

Note that the number of hierarchies from which the calculation by the POC calculation unit 7 is determined is not particularly limited, and is preset by default according to the resolution of the original image data, the number of hierarchies of the multi-resolution image, and the like. Alternatively, it may be arbitrarily set by the user.
However, once the corresponding point search by the POC calculation unit 7 is performed, the corresponding point search by the SAD calculation unit 6 is not performed on the subsequent hierarchical images, and the corresponding points by the POC calculation unit 7 until the highest resolution hierarchical image is reached. Search is to be performed.

  Hereinafter, the corresponding point search by the SAD calculation unit 6 and the POC calculation unit 7 will be specifically described.

  First, when one of the image data acquired by the image acquisition unit 2 is a standard image and the other is a reference image, the SAD calculation unit 6 calculates the correlation between the standard image and the reference image as SAD ( Sum of Absolute Differences (corresponding to the sum of absolute errors). Corresponding points are searched by searching for the corresponding points, and in the reference image, the difference between the reference image and the reference image in the reference image search region and the region having the highest degree of correlation. Corresponding point search processing for obtaining (parallax value) is performed.

  The correlation calculation between the reference image and the reference image will be described with reference to FIG. FIG. 3A represents a standard image, and FIG. 3B represents a reference image. In FIG. 3, each square represents one pixel, and the entire screen of the base image and the reference image is composed of 26 pixels in the X-axis direction (horizontal direction) and 20 pixels in the Y-axis direction (vertical direction). Take the case as an example.

  When performing the correlation calculation, the SAD calculation unit 6 divides the reference image into predetermined unit areas, and sets a certain area in the reference image as a position (search area) for calculating the parallax value. For setting the search area, for example, areas in a predetermined range are sequentially set with the upper left (X = 1, Y = 20) in FIG.

That is, a certain region (for example, 25 pixels of X = 1 to 5, Y = 16 to 20) is set as the search region, and when the correlation calculation for the search region is completed, for example, in the X-axis direction (the horizontal direction of the image) The next search region (for example, 25 pixels of X = 2 to 6, Y = 16 to 20) is set by sequentially shifting one pixel at a time, and correlation calculation is performed. When all the correlation calculations are completed for all the pixels in the X-axis direction (26 pixels in FIG. 3A), the next search area (for example, X = 1 to 5, Y = 15 to 19) is shifted by one pixel in the Y-axis direction. 25 pixels) is set, and correlation calculation is performed. When the correlation calculation is completed for the search region, the search region is sequentially shifted by one pixel in the X-axis direction to set the next search region (for example, 25 pixels of X = 2 to 6, Y = 15 to 19), Correlation is performed.
In this way, the search areas are sequentially set by shifting one pixel each in the X-axis direction and the Y-axis direction from the upper left to the lower right of the screen, and until the correlation calculation is completed for all the pixels on the entire screen, The correlation operation is repeated. Note that the order in which the correlation calculation is performed (how to set the search area) is not limited to the example illustrated here.

Next, a specific method of correlation calculation will be described.
For example, as shown in FIG. 3A, a range surrounded by a broken line in the reference image (25 pixels of X = 9 to 13, Y = 9 to 13) is set as a search region, and correlation calculation is performed. The SAD calculation unit 6 sequentially sets a comparison reference region in the X-axis direction (horizontal direction of the image) for the same Y-axis range (Y = 9 to 13) as the search region in the reference image, and sequentially performs correlation calculation. Thus, the degree of correlation between the search area and each comparison area is calculated. As a result, an area and a corresponding point having the highest correlation with the search area are searched for in the reference image.

That is, the SAD calculation unit 6 firstly has an image of each pixel constituting the search area (25 pixels in total in FIG. 3A, X = 9, Y = 13, X = 10, Y = 13...). Calculate the data value. In addition, each pixel (for example, a range surrounded by a broken line in FIG. 3B (X = 9, Y = 13, X = 10) in the comparison reference region set to be compared with the search region in the reference image. , Y = 13... For all 25 pixels))). Then, the absolute value is calculated by subtracting the image data value of each pixel constituting the comparison area from the image data value of each pixel constituting the search area.
For example, the image data value of a certain pixel in the search region (X = 9, Y = 10 in FIG. 3A) is 100, and the corresponding pixel in the comparison region in FIG. When the image data value of X = 9 and Y = 10 is 50, 100−50 = 50 and the absolute value is 50.
Further, for example, the image data value of a certain pixel in the search area (X = 13, Y = 10 in FIG. 3A) is 50, and the corresponding pixel in the comparison area is shown in FIG. In the case where the image data value of X = 13 and Y = 10 is 90, 50−90 = −40 and the absolute value is 40.
Such subtraction of the image data value and calculation of the absolute value are performed for all 25 pixels constituting the search area and the comparison area. This calculation may be performed sequentially for each pixel, or may be performed simultaneously for all the pixels in the region.

  Further, the SAD computing unit 6 adds all the calculated results (absolute values). The smaller the value after addition is, the higher the degree of correlation is. When the images are the same, the value approaches 0 as much as possible. For example, in the case of FIG. 3, a range surrounded by a broken line in FIG. 3A (25 pixels of X = 9 to 13 and Y = 9 to 13) and a range surrounded by an alternate long and short dash line in FIG. (25 pixels of X = 12 to 16, Y = 9 to 13) is the same image, and the image data value of the corresponding one of the pixels constituting the latter from the image data value of each pixel constituting the former, respectively. When the absolute value is obtained by subtraction and the calculation results are added, the value approaches 0 as much as possible.

In this way, when the reference image is searched for the region having the highest correlation with the search region of the reference image (the region where the extreme value of the correlation is obtained by the correlation calculation), the SAD calculation unit 6 A deviation amount (parallax value) indicating how much the highest area is deviated from the search area of the reference image is calculated, and as shown in FIG. 4, which parallax value corresponds to the area with the highest degree of correlation. .
That is, according to the example of FIG. 3, the search region (25 pixels of X = 9 to 13 and Y = 9 to 13 in FIG. 3A) and the region having the highest degree of correlation are illustrated in FIG. The range surrounded by the one-dot chain line (25 pixels of X = 12 to 16, Y = 9 to 13) is shifted by 3 pixels in the X-axis direction. Therefore, the parallax value in this case is 3.

  Note that the relationship between the parallax value and the distance value is distance value = constant / parallax value, and the distance value is proportional to the reciprocal of the parallax value. In other words, the distance becomes longer as the parallax value becomes smaller, and the distance becomes closer as the parallax value becomes larger. For this reason, the SAD calculating part 6 can acquire a distance value by calculating | requiring a parallax value by the said correlation calculation, and calculating | requiring the reciprocal number of a parallax value.

  In addition, the POC calculation unit 7 searches for corresponding points of both image data in a search area extracted and set from each of the standard image data and the reference image data by a POC (Phase-Only Correlation) calculation method. The correlation value is obtained by calculating the amount of positional deviation.

  As shown in FIG. 5, the POC calculation unit 7 includes window function units 71a and 71b, FFT (Fast Fourier Transform) units 72a and 72b, phase information extraction units 73a and 73b, a synthesis unit 74, IFFT ( An Inverse Fast Fourier Transform (Inverse Fast Fourier Transform) unit 75 and a correlation value calculation unit 76 are provided.

  The window function units 71a and 71b apply a window function to signals corresponding to the search areas for the standard image data and the reference image data. The type of window function is not particularly limited and can be changed as appropriate. Here, a signal obtained by applying a window function to the signal based on the standard image data is denoted by f1, and a signal obtained by applying the window function to the signal based on the reference image data is denoted by f2. The window function units 71a and 71b output the signals f1 and f2 multiplied by the window function to the FFT units 72a and 72b, respectively.

  The FFT units 72a and 72b perform a Fourier transform process on the signals f1 and f2 and output them to the phase information extraction units 73a and 73b. Specifically, the FFT unit 72a performs a two-dimensional discrete Fourier transform on the signal f1 to obtain Fourier image data F1 based on the reference image. Further, the FFT unit 72b performs a two-dimensional discrete Fourier transform on the signal f2 to obtain Fourier image data F2 based on the reference image. In addition, there is no restriction | limiting in particular in a Fourier-transform process, For the two-dimensional discrete Fourier transform, the "Introduction to computer image processing, the Japan Industrial Technology Center edition, issue of Soken publication, P44-45" was referred.

The phase information extraction units 73a and 73b are for extracting phase information by removing amplitude components from the signals output from the FFT units 72a and 72b. That is, the phase information extraction units 73a and 73b perform the phase limiting process on the Fourier-transformed signals F1 and F2 to obtain Fourier image data F3 and F4. The phase information extraction units 73a and 73b output the obtained Fourier image data F3 and F4 to the synthesis unit 74.
Note that the method by which the phase information extraction units 73a and 73b extract only the phase information is not particularly limited, and the phase information extraction unit 73a and 73b may extract only the phase by setting the amplitude to 1, or the amplitude component may be extracted by log processing or √ processing. It is good also as removing.

The synthesizer 74 synthesizes Fourier image data F3 and F4 obtained by extracting only phase information from the Fourier-transformed signals F1 and F2, and synthesizes Fourier image data F5 (u, v) = F3 ^* (u, v) F4 (u, v) is obtained. In the formula, “*” represents a complex conjugate, and (u, v) represents a coordinate in Fourier space. The synthesizer 74 outputs the obtained synthesized Fourier image data F5 to the IFFT unit 75.

  The IFFT unit 75 performs inverse Fourier transform on the synthesized Fourier image data F5 obtained by the synthesizing unit 74 to obtain synthesized inverse Fourier image data f5. The IFFT unit 75 outputs the synthesized inverse Fourier image data f5 to the correlation value calculation unit 76.

The correlation value calculation unit 76 performs a correlation calculation between the images from the synthesized inverse Fourier image data f5 to obtain a correlation value (POC value).
FIG. 6 shows an example of a result obtained by performing correlation calculation between each image by the POC calculation method. FIG. 6 shows an example in which a search area and a comparison target area of N ₁ × N ₂ pixels are set on the standard image and the reference image, and the correlation of a portion having a high correlation in the N ₁ × N ₂ pixel area. The value (POC value) is large. The position in the comparison target region on the reference image corresponding to this POC value peak (Jc in FIG. 6) corresponds to the corresponding point (candidate coordinates) on the reference image corresponding to the center point in the search region on the standard image. Will be.

  According to the calculation processing using the POC calculation method as described above, the correlation calculation is performed using only the phase component of the image by removing the amplitude component of the image. Corresponding points can be searched with high accuracy.

  After calculating the candidate coordinates, the correlation value calculating unit 76 outputs the candidate coordinates to the search region setting unit 14. If the hierarchical image on which the corresponding point search is performed is not the highest resolution image, the search region setting unit 14 determines a search region in the corresponding point search of the next layer based on the candidate coordinates. Further, when the hierarchical image subjected to the corresponding point search is the highest resolution image, the calculation result is output from the correlation value calculation unit 76 to the distance image generation unit 16.

  When the POC calculation unit 7 finishes the corresponding point search for the highest resolution hierarchical image, the distance image generation unit 16 determines the distance based on the calculation result of the positional deviation amount between the reference image data and the reference image data in the corresponding point search. Generate image data.

  In addition, the distance image generation device 1 is provided with an output unit 20 that outputs a distance image generated by the distance image generation unit 16. The output unit 20 is a monitor (display means) such as a CRT (Cathode Ray Tube) or LCD (Liquid Crystal Display), for example, and displays a distance image. Note that the output unit 20 is not limited to a monitor, and is, for example, a communication unit for connecting to an external device (output unit) such as a printer, and configured to transmit the generated distance image to the external device and output it. May be.

  Next, a distance image generation method performed by the distance image generation apparatus 1 according to this embodiment will be described with reference to FIGS. The distance image generation process is realized by the cooperation of the image processing unit 10 which is a computer and the distance image generation process program.

  As shown in FIG. 7, when performing correlation calculation for obtaining three-dimensional position information by the distance image generating device 1, first, the imaging devices 2 a and 2 b capture an object using two imaging units 2 a and 2 b. Then, the image data of the standard image and the reference image is acquired (step S1).

  Image data of the standard image and the reference image acquired by the imaging units 2 a and 2 b are A / D converted by the A / D conversion unit 5 and sent to the image processing unit 10. When the image data of the reference image and the reference image is sent to the image processing unit 10, distortion correction by the lens distortion correction unit 11 and image parallelization processing by the image parallelization processing unit 12 are performed on each image data (step S2). ).

  Further, in the multi-resolution image generation unit 13, for the image data of the standard image and the reference image, a multi-resolution in which a plurality of hierarchical images from the lowest resolution image to the highest resolution image having the same resolution as the original image have a hierarchical structure in order. An image is generated (step S3). The generated multi-resolution image is sent to the corresponding point search unit 15 to perform corresponding point search processing for searching for a corresponding point between the base image data and the reference image data (step S4).

  Here, the corresponding point search process (step S4 in FIG. 7) is specifically described in the corresponding point search unit 15 with reference to FIGS. In the following, an example in which the entire screen of an image is a distance measurement target (that is, all pixels are set as distance measurement points) and corresponding point search is performed for all pixels in the X-axis direction for one search region. Will be described.

  As shown in FIG. 8, when the multi-resolution image is sent to the corresponding point search unit 15, first, the SAD calculation unit 6 performs the SAD calculation to perform the first corresponding point search process (the nth layer hierarchical image). (Corresponding point search process) is performed (step S11).

FIG. 9 is a flowchart showing the first corresponding point search process. As shown in FIG. 9, the corresponding point search unit 15 sets the lowest resolution hierarchical image (the nth layer hierarchical image in FIG. 2) as a calculation target, and sets a distance measuring point in the reference image. (Step S21), a predetermined search area centered on the distance measuring point is set (Step S22).
When the search region is set, the corresponding point search unit 15 sets a comparison target region having the same size as the search region in the reference image (step S23), and performs the corresponding point search by the SAD calculation method by the SAD calculation unit 6. Thus, the correlation degree between the search area and the comparison target area is calculated (step S24).

  The SAD computing unit 6 determines whether or not the corresponding point search processing has been completed for all pixels in the X-axis direction of the comparison target image for the search region (step S25). If not completed (step S25; NO), the comparison target area in the reference image is shifted by one pixel in the X-axis direction (step S26), the process returns to step 24, and the corresponding point search process by the SAD computing unit 6 is performed. repeat. When the corresponding point search process has been completed for all the pixels in the X-axis direction (step S25; YES), the pixel having the highest degree of correlation is searched for in the corresponding point search process, and the parallax value is calculated (step S27). ).

When the SAD calculation unit 6 calculates the parallax value for the search region, it further determines whether the corresponding point search processing for all the distance measurement points (all pixels to be subjected to distance measurement) in the reference image has been completed (step). S28). In the present embodiment, since the corresponding point search process is performed for all the pixels of the image, it is determined whether the process has been completed for all the pixels.
If there are unprocessed pixels that should be measured in the image (step S28; NO), the unprocessed pixels are set as new ranging points (step S29). A search area centered on the distance measuring point is set (step S22), and the processing from step S23 to step S27 is repeated. On the other hand, if there is no unprocessed distance measuring point in the image (step S28; YES), the first corresponding point search process (corresponding point search process for the nth layer image) is terminated.

Returning to FIG. 8, when the first corresponding point search process is completed, the corresponding point search unit 15 calculates candidate coordinates based on the correlation value obtained in the corresponding point search process, and searches this as a search result. It outputs to the area | region setting part 14 (step S12). In addition, the corresponding point search unit 15 sets a hierarchical image of the next layer as a calculation target (step S13).
When the search result is sent from the corresponding point search unit 15, the search region setting unit 14 sets a search region for the next layer based on the search result (step S14). Furthermore, the corresponding point search unit 15 determines whether or not the next layer is a layer that is a target of calculation by the SAD calculation unit 6 (step S15), and if it is a target of the SAD calculation unit 6 (step S15; YES). In step S11, the corresponding point search process is performed by the SAD calculation unit 6, and the processes from step S12 to step S14 are repeated.

  On the other hand, when the next hierarchy is not a calculation target hierarchy by the SAD calculation unit 6 (step S15; NO), corresponding point search processing by the POC calculation unit 7 is performed for the hierarchy and the subsequent layers (step S16). . In the present embodiment, the calculation by the POC calculation unit 7 is performed on hierarchical images after the half of the resolution of the original image.

FIG. 10 is a flowchart showing the corresponding point search processing by the POC calculation unit 7.
In the POC calculation processing, first, the window function units 71a and 71b apply a window function to the signal based on the image data, and the FFT units 72a and 72b are two-dimensionally having a block size of 32 × 32 pixels centered on the central pixel. Perform Fourier transform processing.
That is, the window function unit 71a creates a signal f1 based on the reference image, and 72a performs a Fourier transform process on the signal f1 to convert the signal f1 into the signal F1 (step S31). Further, the window function unit 71b creates a signal f2 based on the reference image, and 72b performs a Fourier transform process on the signal f2 to convert the signal f2 into the signal F2. (Step S32).
The two-dimensional Fourier transform processing is centered on a block centered on a distance measuring point set in the reference image and a corresponding point (candidate coordinates) in a reference image corresponding to the parallax obtained by the arithmetic processing in the immediately preceding hierarchy. To the block.

Thereafter, the phase information extraction units 73a and 73b remove the amplitude components based on the signals F1 and F2 and extract the phase information F3 from the signal F1 and the phase information F4 from the signal F2 (step S33). Further, the signals F3 and F4 having only the phase information are combined by the combining unit 74 to generate a combined signal F5 (step S34). The synthesized signal F5 is output from the synthesizing unit 74 to the IFFT unit 75, and the IFFT unit 75 performs inverse Fourier transform processing on the synthesized signal F5 to convert it to f5 (step S35). Thereafter, the correlation value calculator 76 calculates the correlation value max _{x, y} f5 (u, v) based on f5 (step S36), and further calculates the corresponding coordinates of the corresponding point (step S37).

Returning to FIG. 8, when the POC calculation is completed for one layer, the corresponding point search unit 15 calculates candidate coordinates based on the correlation value obtained in the corresponding point search processing, and uses this as a search result as a search region. It outputs to the setting part 14 (step S17). Further, the corresponding point search unit 15 determines whether or not the POC calculation process has been completed up to the highest resolution image (whether the hierarchical image is the highest resolution image) (step S18). When the processing has been completed up to the highest resolution image (an image having the same resolution as the original image) (step S18; YES), the POC calculation processing by the POC calculation unit 7 is ended.
On the other hand, if the processing has not been completed up to the highest resolution hierarchical image (if there is a higher resolution next layer, step S18; NO), the corresponding point search unit 15 uses the next layer hierarchical image as a calculation target. Set (step S19). The search area setting unit 14 sets a search area based on the search result in the immediately preceding hierarchy (step S20). Then, returning to step S16, the corresponding point search processing is performed by the POC calculation unit 7 for the next layer, and the processing of the multi-resolution image is completed until the processing is completed for the highest-resolution layer image (image having the same resolution as the original image). The processing from step S17 to step S20 is repeated for each hierarchical image.

  Next, returning to FIG. 7, the SAD calculation is performed from the lowest resolution hierarchical image (the nth layer in FIG. 2) to the highest resolution image (the image having the same resolution as the original image, the 0th layer in FIG. 2). When the corresponding point search processing by the unit 6 and the POC calculation unit 7 is completed (step S18 in FIG. 8; YES), a distance image is generated by the distance image generation unit 16 (step S5), and output from the output unit 20 as necessary. Is done.

  As described above, according to the present embodiment, a multi-resolution image composed of a plurality of hierarchical images having different resolutions is generated, and corresponding point search processing is sequentially performed from the low-resolution hierarchical image to the high-resolution hierarchical image. In addition, since the search area setting unit 14 sets the search area of the next hierarchical image based on the search result obtained for the low resolution hierarchical image, the corresponding point search can be performed at high speed and the target object can be searched. It is possible to generate an accurate distance image.

  The corresponding point search unit 15 includes a SAD calculation unit 6 and a POC calculation unit 7, and performs a corresponding point search process by the SAD calculation unit 6 that can perform calculation processing at high speed for a low-resolution hierarchical image. For a hierarchical image having a resolution of 1/2 or more of the original image, the corresponding point search process is performed by the POC calculation unit 7 capable of performing a high-precision calculation process. It is possible to achieve both the reduction of the total processing time of calculation and the generation of a highly accurate distance image.

  Further, once the corresponding point search by the high-precision POC calculation unit 7 is performed, the corresponding point search by the SAD calculation unit 6 is not performed. Therefore, a high-precision calculation process is performed on a high-resolution hierarchical image. It is possible to obtain an operation result with high reliability.

  In the present embodiment, two image pickup means 2a and 2b are provided, the same object is picked up by the image pickup means 2a and 2b, and standard image data (first image information) and reference image data (second image information). However, the configuration of the imaging devices 2a and 2b is not limited to this. The imaging devices 2a and 2b need only be configured to include at least one imaging unit, and may acquire the standard image data and the reference image data by imaging the same object at different timings by one imaging unit. In addition, two or more imaging units may be provided, and the same object may be captured by each imaging unit to acquire the standard image data and the reference image data.

  Further, in the present embodiment, the case where the imaging devices 2a and 2b are provided as the constituent elements of the distance image generating device 1 has been described. However, the image acquiring unit is provided outside the distance image generating device 1, and the distance image generating device 1 is provided. Alternatively, reference image data (first image information) and reference image data (second image information) may be input from the outside, and processing such as correlation calculation may be performed based on the image data.

  In the present embodiment, the SAD calculation unit 6 and the POC calculation unit 7 calculate candidate coordinates based on the correlation value, and output the calculated candidate coordinates to the search region setting unit 14 as a search result. However, the search result output to the search area setting unit 14 is not limited to this. For example, the search region setting unit 14 outputs a calculation result indicating a correlation value from the SAD calculation unit 6 and the POC calculation unit 7 as a search result, and the search region setting unit 14 determines the highest degree of correlation based on the calculation result. It is also possible to calculate a point (candidate coordinate) having a high value and set the periphery of the candidate coordinate as a search area.

  Further, in the present embodiment, the case has been described as an example in which the entire screen of the image is the object of distance measurement (that is, all the pixels are the distance measurement points). If you want to perform distance measurement only on the part, you can limit the range of the corresponding point search process, such as setting the distance measurement point only to the target part you want to measure and performing the corresponding point search process. Good.

Further, in the present embodiment, a case where corresponding point search is performed for all pixels in the X-axis direction for one search region has been described as an example, but only a range of a certain distance value that is limited depending on the user's use or the like. In some cases, the distance may be obtained. In such a case, the range of parallax values corresponding to the distance range according to the application may be used as the calculation range, and the corresponding point search process may be performed by moving the comparison target region within this range.
For example, when it is sufficient to obtain only a distance that is at a distance of 100 m or more, the comparison target area is moved within this range within a range of parallax values corresponding to a distance value of 100 m or more. Thereby, in the corresponding point search process, the processing time of the correlation calculation can be shortened.

Further, in this embodiment, the case where the reference image is divided into unit areas of 25 pixels and a search region is set and correlation calculation is performed for each search region has been described as an example. It is not limited to 25 pixels each, and can be set to an arbitrary range.
In this embodiment, the search area is a predetermined area of 25 pixels, and the correlation calculation is performed for each area. For example, the correlation between the reference image and the reference image is calculated for each pixel. It may be.

  In the present embodiment, the SAD computing unit 6 that obtains the distance value by the SAD computing method is provided as the first computing means. However, the first computing means can perform processing for obtaining the distance value at high speed. It may be anything, and is not limited to this. For example, a calculation unit that obtains a distance value by an SSD (Sum of Squared Differences) calculation method or an NCC (Normalized Cross-Correlation) calculation method may be provided as the first calculation means.

  In the present embodiment, the POC calculation unit 7 for searching for corresponding points by the POC calculation method is provided as the second calculation means. However, the second calculation means can search for the corresponding points with high accuracy. It can be anything that can be done, and is not limited to the one based on the POC calculation method.

Further, in the present embodiment, the case where the image processing unit 10 is a computer including a CPU and the like and a memory such as a ROM and a RAM has been described as an example. You may comprise by wear.
That is, for example, the POC calculation unit 7 is preferably configured by dedicated hardware that performs phase-only correlation processing. As dedicated hardware, for example, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or the like is applicable.

  In addition, it is needless to say that the present invention is not limited to the above embodiment and can be modified as appropriate.

It is a block diagram which shows the whole structure of the correlation calculating apparatus which concerns on this embodiment. It is explanatory drawing which showed typically the structure of the multi-resolution image in this embodiment. FIG. 3A is a diagram illustrating an example of a reference image in the present embodiment, and FIG. 3B is a diagram illustrating an example of a reference image in the present embodiment. It is explanatory drawing which shows the relationship between a correlation degree and a parallax value. It is a block diagram which shows the structure of the POC calculating part shown in FIG. It is explanatory drawing which shows the example of the result obtained by POC arithmetic processing. It is a flowchart showing the distance image generation process in this embodiment. It is a flowchart showing the corresponding point search process shown in FIG. It is a flowchart showing the 1st SAD calculation process shown in FIG. It is a flowchart showing the POC calculation process shown in FIG.

Explanation of symbols

1 Distance Image Generating Device 2a, 2b Imaging Device (Image Acquisition Unit)
6 SAD computing unit (first computing means)
7 POC calculation section (first calculation means)
DESCRIPTION OF SYMBOLS 10 Image processing part 11 Lens distortion correction part 12 Image parallelization processing part 13 Multi-resolution image generation part (Multi-resolution image generation means)
14 Search area setting unit (search area setting means)
15 Distance image generation unit (distance image generation means)
71a, 71b Window function units 72a, 72b FFT units 73a, 73b Phase information extraction unit 74 Synthesis unit 75 IFFT unit 76 Correlation calculation unit

Claims (12)

Image acquisition means for acquiring first image information and second image information to be compared;
Multi-resolution image generation means for generating a multi-resolution image in which a plurality of hierarchical images having different resolutions form a hierarchical structure in order from low resolution to high resolution for the first image information and the second image information acquired by the image acquisition means When,
The first image information and the second image information are collated to search for a corresponding point in a predetermined search area, and at least a first resolution layer image of the lowest resolution among the multi-resolution images A first calculation means for searching for corresponding points by a calculation method; and a second calculation for searching for corresponding points by a second calculation method that is more accurate than the first calculation method for at least the highest resolution hierarchical image. And starting the corresponding point search from a low-resolution hierarchical image among the multi-resolution images, sequentially performing the corresponding point search for a high-resolution hierarchical image, and sequentially reaching the highest-resolution hierarchical image. Corresponding point search means for repeating the corresponding point search;
Search area setting means for setting a search area for a next hierarchical image based on a search result for the hierarchical image when the search for one hierarchical image is completed by the corresponding point search means;
A distance image generating means for generating a distance image based on a search result by the corresponding point searching means;
A distance image generating apparatus comprising:   2. The distance image generation apparatus according to claim 1, wherein after the corresponding point search by the second calculation unit is performed, the corresponding point search by the first calculation unit is not performed for the subsequent hierarchical images. .   The distance according to claim 1 or 2, wherein the first calculation method performed by the first calculation means is any one of an SAD calculation method, an SSD calculation method, and an NCC calculation method. Image generation device.   4. The distance image generating apparatus according to claim 1, wherein the second calculation method performed by the second calculation unit is a POC calculation method. 5. An image acquisition step of acquiring first image information and second image information to be compared;
A multi-resolution image generation step of generating a multi-resolution image in which a plurality of hierarchical images having different resolutions form a hierarchical structure in order from a low resolution to a high resolution with respect to the first image information and the second image information acquired in the image acquisition step When,
The first image information and the second image information are collated to search for a corresponding point in a predetermined search area, and at least a first resolution layer image of the lowest resolution among the multi-resolution images A first calculation step of searching for corresponding points by a calculation method, and a second calculation of searching for corresponding points by a second calculation method having higher accuracy than the first calculation method for at least the highest resolution hierarchical image And starting the corresponding point search from a low-resolution hierarchical image among the multi-resolution images, sequentially performing the corresponding point search for a high-resolution hierarchical image, and sequentially reaching the highest-resolution hierarchical image. A corresponding point search step for repeating the corresponding point search;
A search region setting step of setting a search region for a hierarchical image of the next layer based on a search result for the hierarchical image when the search for one hierarchical image is completed in the corresponding point search step;
A distance image generation step of generating a distance image based on a search result in the corresponding point search step;
A distance image generation method characterized by comprising:   6. The distance image generation method according to claim 5, wherein after the corresponding point search by the second calculation step is performed, the corresponding point search by the first calculation step is not performed for the subsequent hierarchical images. .   The distance according to claim 5 or 6, wherein the first calculation method performed in the first calculation step is any one of an SAD calculation method, an SSD calculation method, and an NCC calculation method. Image generation method.   The distance image generation method according to any one of claims 5 to 7, wherein the second calculation method performed in the second calculation step is a POC calculation method. A multi-resolution image generation function for generating a multi-resolution image in which a plurality of hierarchical images having different resolutions form a hierarchical structure in order from a low resolution to a high resolution for the first image information and the second image information to be compared;
The first image information and the second image information are collated to search for a corresponding point in a predetermined search area, and at least a first resolution layer image of the lowest resolution among the multi-resolution images A first calculation function for searching for corresponding points by a calculation method, and a second calculation for searching for corresponding points by a second calculation method that is more accurate than the first calculation method for at least the highest resolution hierarchical image Among the multi-resolution images, the corresponding point search is started from a low-resolution hierarchical image, and the corresponding point search is sequentially performed for a hierarchical image with a high resolution, and the hierarchical image with the highest resolution is sequentially reached. A corresponding point search function for repeating the corresponding point search;
A search area setting function for setting a search area for the next hierarchical image based on a search result for the hierarchical image when the search for the hierarchical image is completed by the corresponding point search function;
A distance image generation function for generating a distance image based on a search result by the corresponding point search function;
A computer-readable program characterized by causing a computer to realize the above.   10. The computer-readable computer program product according to claim 9, wherein after the corresponding point search by the second calculation method is performed, the corresponding point search by the first calculation method is not performed for the subsequent hierarchical images. program.   The computer-readable program according to claim 9 or 10, wherein the first calculation method is any one of an SAD calculation method, an SSD calculation method, and an NCC calculation method.   The computer-readable program according to any one of claims 9 to 11, wherein the second calculation method is a POC calculation method.

Images