JP5163164B2 - 3D measuring device - Google Patents

Description

  The present invention relates to a three-dimensional measurement device that is mounted on a moving body such as a vehicle and is realized as a device that measures a distance from a captured image of a so-called stereo camera.

  As efforts to increase vehicle safety, there are devices that detect obstacles ahead and warn of danger, and measure the distance from the preceding vehicle to keep the distance between vehicles constant. In such an apparatus, if the reflection of electromagnetic waves such as ultrasonic waves and laser light is used to measure the distance, the distance can be measured relatively accurately, but an erroneous measurement target may be captured. For this reason, there has been proposed an apparatus that performs distance measurement by recognizing an object from the captured image using the stereo camera (stereoscopic method).

  In Patent Document 1, a distance to a vertical edge portion of a front object is obtained, and if there is an approximately equidistant edge within a predetermined range in the horizontal direction (width assuming a vehicle), a so-called edge is recognized as an object between the edges. The distance to the front object is measured by the base stereo method. The edge-based stereo method measures three-dimensionally by associating edges with each other and is said to be easy to obtain accuracy because the association is clear. However, since only the edge portion is used. There is a problem that it is easily affected by noise such as image noise and an accurate distance cannot be obtained. In addition, the distance between the equidistant edges is not necessarily a vehicle. For example, when two vehicles run in parallel, two vehicles are arranged side by side and erroneously determined as one vehicle. There is also a problem that a determination means is required.

  On the other hand, in measuring the distance, a corresponding point search of two images is performed. For example, an in-vehicle distance measuring device is required to be easily installed by downsizing as well as needs to measure a distance to a distant preceding vehicle with high accuracy. As a method for increasing the accuracy, a method of increasing the focal length or increasing the base length is conceivable. However, the former has a drawback that the visual field range is narrowed and the latter is increased in size.

Therefore, as a method for improving the accuracy without such a defect, there is a subpixel matching. By performing the stereo image mapping operation with a resolution of a pixel unit or less, the resolution of stereo three-dimensional measurement can be improved. And in the stereo image processing apparatus of patent document 2, the example of the sub-pixel process of a stereo image is shown.
JP 7-225127 A JP 2005-250994 A

  There is a POC (phase-only correlation method) as an effective association method for the subpixelization of the association. However, there is a problem that the processing cost is high and the hardware scale increases because real-time processing is performed. It was.

  An object of the present invention is to provide a three-dimensional measuring apparatus capable of performing a distance measurement that is robust and accurate with simple processing.

The three-dimensional measuring apparatus according to the present invention is a three-dimensional measuring apparatus that acquires images at different viewpoint positions as a reference image and a reference image by an imaging unit and measures the three-dimensional coordinates of each part of an object from these captured images. Edge extraction means for extracting edges on the standard image and the reference image; and grouping similar edges among the edges on the standard image and the reference image extracted by the edge extraction means ; Edge set setting means for setting a region including a candidate area to which corresponding point search is to be applied ; edge array information extraction means for extracting edge array information of edge groups in the candidate area set by the edge set setting means; and It makes a comparison between the edge sequence information between the reference image and the reference image extracted by the edge sequence information extracting means calculates a similarity, resulting et al Based on the similarity, an edge group having a high similarity to the edge group in one image of the reference image or the reference image should be selected from the other image, and corresponding point search should be applied to a pair of candidate regions including those edge groups. a similarity computing means for determining a region pair, to the region pair including a high edge group similarity which is determined by the similarity calculation means, and the corresponding position calculating means for calculating a sub-pixel corresponding positions, the corresponding And a three-dimensional calculation means for calculating a three-dimensional coordinate of the object at the position of the target point from the subpixel corresponding position obtained by the position calculation means.

  According to the above-described configuration, an edge stereo or the like that acquires images at different viewpoint positions as a standard image and a reference image by an imaging unit and measures the three-dimensional coordinates of each part of the object from these captured images is called. In the three-dimensional measuring apparatus, first, edges on the standard image and the reference image are extracted by the edge extracting means, and the edges in the extracted standard image and reference image are equidistant by the edge set setting means. A plurality of similar edges are collected (grouped) to define an edge set. Then, the edge arrangement information in each edge set (group) is extracted by the edge arrangement information extraction unit, and the similarity calculation unit compares the edge arrangement information between the extracted reference image and the reference image. For the edge group pair having a high degree of similarity, the corresponding position calculation means obtains a high-accuracy corresponding position in units of subpixels, and based on the result, the object at the position of the target point by the three-dimensional calculation means Calculate the three-dimensional coordinates of an object.

  Therefore, by evaluating the positional deviation between the edge sets of the left and right images together with the similarity of the edge arrangement, it is possible to obtain the positional deviation robustly and accurately while judging the correspondence between the edge sets. In particular, a more robust distance measurement can be performed by calculating positional deviation and similarity based on phase information. In addition, although the positional deviation can be obtained with high accuracy in sub-pixel units, the location where the corresponding point search process using the phase information complicated in the calculation process is performed is limited to a portion where the process is good, and the process is performed. Can be minimized.

  In the three-dimensional measurement apparatus of the present invention, the edge set setting means performs the grouping based on edge lengths and end point coordinates.

  According to the above configuration, in the case of a vehicle as the object, since the object has a substantially rectangular shape, the edges derived from the same vehicle can be effectively obtained by grouping the edges having the same length and end points. Can be grouped.

  Furthermore, in the three-dimensional measuring apparatus according to the present invention, the edge set setting means performs grouping based on a spatial distance to the edge.

  According to said structure, the edge of the same object can be effectively grouped using the fact that the edge derived from the same object exists in substantially the same distance.

Also, three-dimensional measurement equipment of the present invention, an image of mutually different viewpoint positions acquired as the reference image and the reference image by the imaging means, the three-dimensional measuring three-dimensional coordinates of each part of the object from those of the captured image In the measurement apparatus, an edge extracting unit that extracts edges on the standard image and the reference image, and an edge set that groups similar edges among the edges on the standard image and the reference image extracted by the edge extracting unit Edge arrangement information extracting means for extracting edge arrangement information of edge groups grouped by the edge set setting means, and edge arrangement information between the reference image and the reference image extracted by the edge arrangement information extraction means A similarity calculation means for comparing each other and calculating a similarity, and an edge group having a high similarity obtained by the similarity calculation means. Corresponding position calculating means for calculating a subpixel corresponding position for the pair, and three-dimensional coordinates for calculating the three-dimensional coordinates of the object at the position of the target point from the subpixel corresponding position obtained by the corresponding position calculating means. The edge arrangement information is information on the number, interval, and length of edges in the window defined for the standard image and the reference image, and the similarity calculation means is based on the edge information. The similarity is calculated.

  According to said structure, edge arrangement | sequence information can be defined clearly.

Furthermore, 3-dimensional measurement equipment of the present invention, an image of mutually different viewpoint positions acquired as the reference image and the reference image by the imaging means, for measuring the three-dimensional coordinates of each part of the object from those of the captured image 3 In the dimension measurement apparatus, an edge extracting unit that extracts edges on the standard image and the reference image, and an edge that groups similar edges among the edges on the standard image and the reference image extracted by the edge extracting unit Set setting means, edge arrangement information extraction means for extracting edge arrangement information of edge groups grouped by the edge set setting means, and edge arrangement between the reference image and the reference image extracted by the edge arrangement information extraction means Similarity calculation means for comparing information and calculating similarity, and an edge with high similarity obtained by the similarity calculation means Corresponding position calculation means for calculating a subpixel corresponding position for the loop pair, and a three-dimensional calculation for calculating the three-dimensional coordinates of the object at the position of the target point from the subpixel corresponding position obtained by the corresponding position calculation means. And the edge array information is a signal obtained by frequency-resolving a pattern in a window defined in the base image and the reference image and suppressing an amplitude component.

  According to said structure, arrangement | sequence information can be obtained simply using conventional frequency analysis methods, such as FFT calculation.

Also, three-dimensional measurement equipment of the present invention, an image of mutually different viewpoint positions acquired as the reference image and the reference image by the imaging means, the three-dimensional measuring three-dimensional coordinates of each part of the object from those of the captured image In the measurement apparatus, an edge extracting unit that extracts edges on the standard image and the reference image, and an edge set that groups similar edges among the edges on the standard image and the reference image extracted by the edge extracting unit Edge arrangement information extracting means for extracting edge arrangement information of edge groups grouped by the edge set setting means, and edge arrangement information between the reference image and the reference image extracted by the edge arrangement information extraction means A similarity calculation means for comparing each other and calculating a similarity, and an edge group having a high similarity obtained by the similarity calculation means. Corresponding position calculating means for calculating a subpixel corresponding position for the pair, and three-dimensional coordinates for calculating the three-dimensional coordinates of the object at the position of the target point from the subpixel corresponding position obtained by the corresponding position calculating means. The corresponding position calculating means calculates the corresponding position based on the pattern similarity around the corresponding position candidate.

  According to said structure, the corresponding position in the said sub pixel precision can be calculated | required by performing an interpolation calculation using the pattern similarity around a corresponding position candidate.

  Furthermore, in the three-dimensional measuring apparatus according to the present invention, the corresponding position calculation means calculates a subpixel corresponding position by POC (phase-only correlation method).

  According to the above configuration, by comparing only the phase component with the amplitude component suppressed, it is possible to calculate the pattern similarity and the positional deviation with high accuracy without depending on the image quality.

  In the three-dimensional measuring apparatus according to the present invention, the imaging unit is a binocular parallel stereo camera, and the edge extracted by the edge extracting unit is only an edge perpendicular to the base line.

  According to the above configuration, when the imaging unit is a twin-lens parallel stereo camera, the edge perpendicular to the base line greatly contributes to the position shift calculation. Therefore, by paying attention to the edge perpendicular to the base line, the corresponding position calculation is performed. The accuracy can be improved when the high-accuracy positional shift amount in units of subpixels is obtained by the means.

  As described above, the three-dimensional measurement apparatus according to the present invention extracts, in the three-dimensional measurement apparatus called edge stereo, the edge on the reference image and the reference image acquired by the image pickup means, and the edge extraction means. The edge sets are defined by collecting (grouping) a plurality of similar edges, such as those at equal distances, by the edge set setting means, and then using the edge array information in each edge set (group) as an edge Extracted by the sequence information extraction means, the similarity calculation means compares the edge sequence information between the extracted standard image and reference image, calculates the similarity, and for the edge group pair having a high similarity Then, the corresponding position calculation means obtains a high-accuracy corresponding position in units of subpixels, and from the result, the three-dimensional calculation means calculates the three-dimensional coordinates of the object at the position of the target point. That.

  Therefore, by evaluating the positional deviation between the edge sets of the left and right images together with the similarity of the edge arrangement, it is possible to obtain the positional deviation robustly and accurately while judging the correspondence between the edge sets. In particular, a more robust distance measurement can be performed by calculating positional deviation and similarity based on phase information. In addition, although the positional deviation can be obtained with high accuracy in sub-pixel units, the location where the corresponding point search process using the phase information complicated in the calculation process is performed is limited to a portion where the process is good, and the process is performed. Can be minimized.

[Embodiment 1]
FIG. 1 is a diagram showing a schematic configuration of a distance measuring device 1 which is a three-dimensional measuring device according to the first embodiment of the present invention. This distance measuring device 1 is mounted on a vehicle and has stereo cameras 3 and 4 for obtaining a two-dimensional input image of (preceding vehicle, road obstacle, etc.), and each part from the output images of those stereo cameras 3 and 4 An arithmetic processing unit 5 that calculates a distance, and a warning display unit 6 that calculates the degree of danger from the calculation processing result and performs an approach warning to a preceding vehicle, automatic (preventive safety) braking, or the like. Composed. The warning display unit 6 can notify the driver by outputting sound from the audio unit, displaying on a car navigation screen, displaying on a meter panel, or the like.

  The stereo cameras 3 and 4 that are imaging means are two-lens parallel stereo cameras that are provided on the left and right sides of the vehicle body and output a pair of left and right images (a base image and a reference image) taken at the same timing. In this embodiment, for the sake of simplicity of explanation, it is assumed that the aberrations of the stereo cameras 3 and 4 are well corrected and installed parallel to each other. Further, even if the actual hardware is not under such conditions, it is possible to convert it to an equivalent image by image processing.

FIG. 2 is a diagram for explaining a method of three-dimensional calculation (distance calculation) in the calculation processing device 5 for the output images of the stereo cameras 3 and 4. As the stereo cameras 3 and 4, at least the focal length (f), the number of pixels of the imaging surfaces (CCD) 3 b and 4 b, and the size (μ) of the pixels are equal to each other, and a predetermined baseline (baseline) is used. ) When the object 2 is photographed by separating the optical axis 3a, 4a parallel to each other with the length (B) spaced apart from each other, the parallax (the number of shifted pixels) on the imaging surfaces 3b, 4b is Δd (= d1 + d2), the distance (D) to the object 2 is
D = f · B / Δd (1)
Can be obtained.

  In addition, the three-dimensional position (X, Y, Z) of each part of the object 2 is calculated as follows, where x and y are positions on the pixel.

X = x · D / f (2)
Y = y · D / f (3)
Z = D (4)
Here, for example, an in-vehicle stereo camera is required to be easy to install due to downsizing as well as needs to measure the distance to a distant preceding vehicle with high accuracy as described above.
The depth direction resolution ΔZ of the stereo camera is
ΔZ = (D ² / B) · (1 / f) · Δd (5)
Therefore, as a method for improving accuracy, a method of increasing the focal length f and a method of increasing the baseline length B can be considered. However, as described above, the former has a disadvantage that the visual field range is narrow, and the latter is large in size. As a method for improving the accuracy without the above drawbacks, there is subpixelization of correspondence. This is because by performing the associating operation with a resolution equal to or less than the pixel unit, the resolution of the parallax Δd can be reduced and the resolution of the stereo three-dimensional measurement can be made fine.

  FIG. 3 is a block diagram of the arithmetic processing unit 5 that calculates the three-dimensional position (X, Y, Z). Images captured by the stereo cameras 3 and 4 (standard image and reference image) are input to the edge extraction unit 11 and edges are extracted as described later. The obtained edge information is an edge set setting unit, and is input to the edge grouping unit 12 which is an edge arrangement information extraction unit, and among the edges on the reference image and the reference image extracted by the edge extraction unit 11, As will be described later, similar edges are grouped and a region to be processed is set.

  The edge group information grouped by the edge grouping unit 12 is input to the corresponding region setting unit 13, and the corresponding region setting unit 13 determines whether the edge groups of the left and right images are in accordance with the edge group information as described later. Set the corresponding pair candidate. The set corresponding pair candidates are input to the corresponding area correlation unit 14 to which the captured images from the stereo cameras 3 and 4 are input, and correlation calculation between the edge array information of the corresponding group pairs is performed as described later. Then, the degree of similarity and the amount of mutual positional deviation are obtained, and the subpixel corresponding position is further calculated for the obtained edge group pair having a high degree of similarity. From the sub-pixel corresponding position thus obtained, the distance calculation unit 15 calculates the three-dimensional coordinates of each part of the target object 2 by the stereo method as described later.

  The edge extraction unit 11 performs filter processing on the images captured by the stereo cameras 3 and 4 to detect edges. There are many known techniques for the filter processing, such as a prewitt filter, a Sobel filter, and a Laplacian filter. FIG. 4 shows a Laplacian filter kernel. FIG. 4A shows an example in which both vertical and horizontal edges are detected, and FIG. 4B shows an example in which only vertical edges are detected. In the present embodiment, as described above, the binocular parallel stereo cameras 3 and 4 are used. In this case, since the edge perpendicular to the base line greatly contributes to the positional deviation calculation, the edge extraction unit 11 extracts the edge. Edges are only edges in the direction perpendicular to the baseline, and only horizontal edges are not detected. Thus, the accuracy can be improved when the corresponding region correlation unit 14 obtains a highly accurate positional shift amount in sub-pixel units. FIGS. 5A and 5B are examples of edge-extracted images by the filters in FIGS. 4A and 4B, respectively.

  The edge grouping unit 12 sets a region to which the corresponding point search is applied based on the edge information. When an object such as a vehicle is present on the image, an edge derived from the shielding contour or the internal structure appears in the image. Therefore, since it can be assumed that the edge information derived from the same object is similar to each other, the edge grouping unit 12 groups the edge information that is close to each other to thereby obtain the edge information derived from the same object. Gather. In the following, two methods will be described, but each method may be applied singly or in combination.

  First, in the first method shown in FIG. 6, the regions sandwiched between edge pairs having the same length in the same direction and the same vertical and horizontal end positions are grouped. In other words, this is a selection method assuming that, if the vehicle is a preceding vehicle on the same road surface, an equal-length edge pair is projected symmetrically on the captured image, and FIG. An example using a filter for detecting a vertical edge as shown in FIGS. 4B and 5B will be described. First, as shown in FIG. 6 (a), when vertical edge detection processing is performed on an image in which one truck is traveling ahead, the result is as shown in FIG. 6 (b). When the edges with the same (start position) are viewed in order, it is detected that the points a, b, and c in the figure are equal in height, and the points d, e, f, and g are also equal in height. If there are two or more edges having such heights (starting positions), it is checked whether the ends are aligned. For example, for points a, b, and c, a and b, b and c, and a and c. Each combination is checked. If the ends are aligned, the same group is obtained as shown in FIG. 6C, and two regions in total are candidates in the figure. Specifically, since edge pairs with equal end portions and equal lengths are used, all the regions are rectangular. When the small area included in the large area is included in the large area, as shown in FIG. 6C, there are two candidate areas for the final corresponding area search. In this way, by grouping based on the edge length and the coordinates of the end points, in the case of a vehicle as the object, since it has a substantially rectangular shape, the edge derived from the same vehicle is effectively Can be grouped.

  Next, in the second candidate region extraction method, a general edge-based stereo method is used to obtain a spatial distance to an edge portion, and edges that are equidistant are grouped. That is, this is a selection method using the fact that edges from the same vehicle are at the same distance. When the edge extraction unit 11 uses a filter that detects vertical edges as shown in FIGS. 4B and 5B, the diagram of the filter processing is the same as that in FIG. First, as shown in FIG. 6 (a), when vertical edge detection processing is performed on an image in which one truck is traveling forward, the result is as shown in FIG. 6 (b). When the edges with the same distance in the same direction are viewed in order, it is detected that the points a, b, and c in the figure have the same distance, and the points d, e, f, and g have the same distance. If there are two or more edges with such distances, it is checked whether the ends are aligned. For example, for points a, b, and c, each combination of a and b, b and c, and a and c is checked. Is done. If the ends are aligned, the same group is obtained as shown in FIG. 6C, and two regions in total are candidates in the figure. Even when grouping is performed based on the spatial distance to the edge as described above, the edges derived from the same object can be effectively grouped by utilizing the fact that they are at substantially the same distance.

  FIG. 7 is a diagram for explaining the setting operation of the corresponding area in the standard image and the reference image in the corresponding area setting unit 13. When grouping (setting of candidate areas) is performed by the edge grouping unit 12, for example, the standard image becomes as shown in FIG. 7A and the reference image becomes as shown in FIG. 7B. Therefore, the corresponding area setting unit 13 sets a pair of edge group areas on the standard image and the reference image. In FIG. 7, since two edge group regions are set for both the base image and the reference image, if they are denoted by reference numerals A1, B1; A2, B2, the pair includes (A1-A2), (A1 -B2), (B1-A2), and (B1-B2) are selected. Alternatively, the number of pairs may be narrowed down by comparing the areas of the pairs. In the example of FIG. 7, there are two final pairs (A1-A2) and (B1-B2).

  FIG. 8 is a functional block diagram of the corresponding region correlation unit 14. With respect to the pair of corresponding regions obtained by the corresponding region setting unit 13, the corresponding region correlation unit 14 performs a correlation calculation between the edge group regions. In doing so, it is effective to take a correlation using the edge arrangement information included in the area because of the property that the area pattern is a set of edges. As shown in FIG. Each of the edge array information is extracted, and the similarity evaluation unit 22 performs a correlation calculation between the edge array information of the corresponding group pairs, and determines whether or not the correct region pair is a correct region pair based on the similarity. In such a case, the phase comparison unit 23 obtains the positional deviation amount between the region pairs.

  The edge array information extraction unit 21 extracts the number of edges, the distance between edges, and the length of corresponding edges as the edge array information. For example, each edge group in FIG. 7 has the following array information. In this way, the edge arrangement information can be clearly defined by calculating the similarity based on the number, interval, and length information of the edges in the window.

  And the said similarity evaluation part 22 performs the comparison of the similarity of area | region pairs based on the flow of FIG. In step S1, it is determined whether or not the number of edges is equal to each other. If not, it is determined that they are not similar, and if so, the process proceeds to step S2. In step S2, the edge lengths are compared to determine the difference ΔL. In step S3, it is determined that the difference ΔL is not more than a predetermined threshold th1, and if it is less than the threshold th1, the process proceeds to step S4. Move. In step S4, the edge interval is compared to determine the difference ΔD. In step S5, it is determined that the difference ΔL is not greater than a predetermined threshold th2, and is similar if it is less than the threshold th. It is determined. Thus, in the case of FIG. 7, the result that (A1-A2) and (B1-B2) are a pair with high similarity is obtained.

FIG. 10 is a diagram for explaining the operation of the phase comparison unit 23. The corresponding region correlation unit 14 measures a positional shift with respect to a region pair having a high degree of similarity. For this purpose, the phase comparator 23 first searches for a peak of similarity. In searching for where the image on the standard image 3c is on the reference image 4c, a window having a size corresponding to W pixels in the vertical and horizontal directions is set on the standard image 3c. A window having the same size is also set on 4c, and distances between the patterns are obtained on the reference image 4c at three positions -d and + d with the same position d as the window on the standard image 3c as the center. . The distance between the patterns is obtained by calculating the correlation value, the SAD value, and the SSD value at each position as follows. For these calculations, the brightness I ₁ and I ₂ of the same corresponding pixel in the window is used.

First, the calculation of the correlation value is as follows.

Next, the calculation of the SAD value is as follows.

Subsequently, the calculation of the SSD value is as follows.

  Then, since the similarity is higher as the distance between patterns is closer, the conversion from the inter-pattern distance to the similarity value is performed by the following equation so that the similarity value is 1 when the inter-pattern distance is 0. .

S (d) = 1 / (D (d) +1) (9)
The distance calculation unit 15 calculates a subpixel corresponding position from the corresponding point candidate positions around the similarity peak position and the similarities already calculated by the phase comparison unit 23 when calculating the similarity peak. A parabolic fitting method will be described as a specific method. FIG. 11 explains the parabola fitting. In the parabola fitting, the pixel dmax (integer value) of the maximum value S (dmax) of the similarity S (d) obtained by the equation 9 and one pixel dmax before and after the pixel dmax. −1, dmax + 1 similarity S (dmax−1), S (dmax + 1) is used, and in the vicinity of the maximum value S (dmax), the similarity S (d) can be approximated by a parabola Sp (d). The coefficient is obtained from the similarity S (dmax-1), S (dmax), S (dmax + 1). Specifically, as shown in FIG. 11, a parabola Sp (d) passing through these three points S (dmax-1) and S (dmax + 1) is drawn, and the pixel position d at the subpixel level of the peak position is immediately determined. It is obtained and used as an estimated value of the subpixel corresponding position dsub.

  With this configuration, images at different viewpoint positions are acquired as the standard image 3c and the reference image 4c by the stereo cameras 3 and 4, and the distance measurement for obtaining the distance from the captured images 3c and 4c to the object. In the apparatus 1, first, the edges on the standard image 3 c and the reference image 4 c are extracted by the edge extraction unit 11, and the edges on the extracted standard image 3 c and reference image 4 c are equidistant by the edge grouping unit 12. A plurality of similar edges are collected (grouped) to define an edge set, and edge arrangement information in each edge set (group) is extracted by the edge arrangement information extraction unit 21, and the similarity evaluation unit 22 is extracted. Then, the edge arrangement information between the extracted standard image 3c and reference image 4c is compared, the similarity is calculated, and the high similarity is obtained. The phase comparison unit 23 obtains a highly accurate corresponding position in units of sub-pixels for the digroup pair, and the distance calculation unit 15 calculates the distance to the target object at the position of the target point from the result. By evaluating the positional deviation between the edge sets of the image together with the similarity of the edge arrangement, it is possible to obtain the positional deviation robustly and accurately while judging the correspondence of the edge sets.

[Embodiment 2]
FIG. 12 is a functional block diagram of the corresponding area correlation unit 14 ′ in the distance measuring apparatus according to the second embodiment of the present invention. In the present embodiment, the configuration shown in FIGS. 1 and 3 can be used except that the corresponding region correlation unit 14 ′ is used instead of the corresponding region correlation unit 14 shown in FIG. The corresponding region correlation unit 14 ′ also performs correlation calculation between edge group regions for the pair of corresponding regions obtained by the corresponding region setting unit 13, and is similar to the edge array information extraction unit 21 ′. A degree evaluation unit 22 ′ and a phase comparison unit 23 ′ are provided.

  In the present embodiment, a method will be described in which edge arrangement information obtained by the edge arrangement information extraction unit 21 'is expressed by phase information for each frequency by frequency-decomposing the corresponding region. Known techniques for calculating the frequency-resolved signal of the pattern include Fourier transform, discrete cosine (sine) transform, wavelet transform, Hadamard transform, and the like. By performing the above conversion on the region pattern, the phase relationship for each arrangement period of the edge pattern can be calculated. For example, DFT (discrete Fourier transform) processing can be expressed as follows, where f is the pattern of the edge group region A1 and g is the pattern of A2.

_Two images having an image size of N ₁ × N ₂ pixels are defined as f (n ₁ , n ₂ ) and g (n ₁ , n ₂ ), and the index of the discrete space is set to n ₁ = −M ₁ ,. When M ₁ , n ₂ = −M ₂ ,... M ₂ and the image sizes are N ₁ = 2M ₁ +1 pixel and N ₂ = 2M ₂ +1 pixel, two-dimensional Fourier transform (2D) of these images DFT) is given by the following equation.
_{Here, k 1 = -M 1, ···} M 1, k 2 = -M 2, ··· M 2
And Σ _n1n2 is
It is. A _F (k ₁ , k ₂ ) and A _G (k ₁ , k ₂ ) are amplitude components, and e ^{jθF (k1, k2)} and e ^{jθG (k1, k2)} are phase components.

  The respective Fourier transform results are shown in FIG. Phase information for each frequency can be calculated.

  As the similarity evaluation unit 22 ′, a correlation method that suppresses an amplitude component, which is known as a robust pattern similarity calculation method, is used. In such a correlation method, similarity calculation is performed using only a phase component signal in which the amplitude component is suppressed from the frequency resolution signal of the pattern. Therefore, the robust correlation calculation can be realized. Further, unlike the conventional two-dimensional correlation method and feature extraction method using grayscale data, it has a feature that it is resistant to disturbances and can calculate with high accuracy even for images with low brightness and contrast. One example of the phase-only correlation method (POC) uses a Fourier transform for the conversion, and performs a correlation operation only on the phase component in which the amplitude component of the Fourier series is suppressed. Details will be described below by taking the phase-only correlation method (POC) as an example.

First, the combined phase spectrum R (k ₁ , k ₂ ) of the patterns f and g is defined as follows.
Here, the complex conjugate of G (k ₁ , k ₂ ) is shown with an overline. Further, θ (k ₁ , k ₂ ) = θ _F (k ₁ , k ₂ ) −θ _G (k ₁ , k ₂ ). FIG. 14 shows the phase and amplitude component of the combined phase spectrum.

Correlation calculation can be performed by performing inverse Fourier transform on the combined phase spectrum ^ R (k ₁ , k ₂ ). That is, as described above, θ (k ₁ , k ₂ ) = θ _F (k ₁ , k ₂ ) −θ _G (k ₁ , k ₂ ), and the POC function ^ r (n ₁ , n ₂ ) is R (k ₁ , k ₂ ) is a two-dimensional inverse discrete Fourier transform (2D IDFT) and is defined by the following equation.
Here, Σ _n1n2 is
It is.

  As shown in FIG. 15, the POC value obtained by the processing of the above equation 13 is known to have a steep similarity peak in the coordinates of the movement amount between images (reference window and reference window). Robustness is high. The height of the peak of the POC indicates the pattern similarity. Then, the phase comparison unit 23 ′ estimates the position shift amount (= parallax dsub) by estimating the peak position of the POC. At this time, since the POC is obtained discretely, high resolution corresponding region coordinates can be obtained by interpolating and estimating the peak position with subpixels. The peak position interpolation estimation method can be performed by fitting a function such as a parabola as shown in FIG. The positional deviation amount d between the candidate areas is an amount obtained by adding the positional deviation amount dsub of the sub-pixel obtained by the POC method to the positional deviation amount dpixel at the pixel level between the candidate areas.

  Even with this configuration, the positional deviation between the edge sets of the left and right images can be evaluated together with the similarity of the edge arrangement, thereby determining the positional deviation robustly and accurately while judging the correspondence between the edge sets. In addition, it is possible to limit the portion where the corresponding point search processing using the phase information complicated in the arithmetic processing is performed to a portion where the processing is good, thereby minimizing the processing.

  In addition, the array information extraction unit 21 ′ performs frequency decomposition on the patterns in the window defined in the standard image 3c and the reference image 4c as edge array information, and creates a signal in which the amplitude component is suppressed. Using conventional frequency analysis techniques such as computation, sequence information can be easily obtained.

  Furthermore, since the similarity evaluation unit 22 ′ calculates the position corresponding to the sub-pixel by POC (phase-only correlation method), by comparing only the phase component in which the amplitude component is suppressed, it is not dependent on the image quality. Pattern similarity and positional deviation can be calculated with high accuracy. In particular, a more robust distance measurement can be performed by calculating positional deviation and similarity based on phase information. In addition, although the positional deviation can be obtained with high accuracy in sub-pixel units, the location where the corresponding point search process using the phase information complicated in the calculation process is performed is limited to a portion where the process is good, and the process is performed. Can be minimized.

  Note that the imaging device in this case refers to a technique for restoring a shape from images taken from a plurality of viewpoints, and takes an image with a camera having three or more cameras as well as two eyes, or by moving a monocular camera. Including cases.

It is a figure which shows schematic structure of the ranging apparatus which is a three-dimensional measuring apparatus which concerns on the 1st Embodiment of this invention. It is a figure for demonstrating the method of the three-dimensional distance calculation in the arithmetic processing apparatus with respect to the output image of a stereo camera. It is a block diagram of the arithmetic processing unit in the distance measuring device. It is a figure which shows the kernel of the Laplacian filter used for edge detection. It is a figure which shows the example of an image of the edge extraction by the filter shown in FIG. It is a figure for demonstrating the extraction method of the candidate area | region where an object may exist. It is a figure for demonstrating the matching operation | movement of the said candidate area | region in a standard image and a reference image. It is a functional block diagram which shows an example of the corresponding area correlation part in the said arithmetic processing unit. It is a flowchart for demonstrating the comparison operation of similarity. It is a figure for demonstrating a phase comparison operation | movement. It is a graph for demonstrating the parabola fitting method. It is a functional block diagram which shows an example of the corresponding area correlation part in the arithmetic processing unit of the ranging apparatus which concerns on the 2nd Embodiment of this invention. It is a graph which shows the Fourier-transform result of a reference | standard image and a reference image. It is a graph which shows the synthetic | combination phase spectrum of a reference | standard image and a reference image. It is a figure which shows an example of a POC value.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Distance measuring device 2 Object 3, 4 Stereo camera 5 Arithmetic processing device 6 Warning display part 3b, 4b Imaging surface 11 Edge extraction part 12 Edge grouping part 13 Corresponding area setting part 14, 14 'Corresponding area correlation part 15 Distance calculation part 21, 21 ′ edge array information extraction unit 22, 22 ′ similarity evaluation unit 23, 23 ′ phase comparison unit

Claims (8)

In a three-dimensional measurement apparatus that acquires images at different viewpoint positions as a standard image and a reference image by an imaging unit, and measures the three-dimensional coordinates of each part of the target object from these captured images.
Edge extraction means for extracting edges on the reference image and the reference image;
Edge set setting for grouping similar edges among edges on the standard image and the reference image extracted by the edge extracting means, and setting a region including the edges as a candidate region to which the corresponding point search is to be applied Means,
Edge arrangement information extraction means for extracting edge arrangement information of edge groups in the candidate area set by the edge set setting means;
The edge arrangement information between the reference image and the reference image extracted by the edge arrangement information extraction means is compared, the similarity is calculated, and the edge in one image of the reference image or the reference image is calculated from the obtained similarity select high edge group similarity with a group from the other image, and similarity calculation means for determining those regions pair to apply a corresponding point search pairs candidate region including the edge group,
Corresponding position calculating means for calculating a subpixel corresponding position for a region pair including an edge group having a high similarity determined by the similarity calculating means;
A three-dimensional measuring apparatus comprising: a three-dimensional calculation unit that calculates a three-dimensional coordinate of an object at a position of a target point from a subpixel corresponding position obtained by the corresponding position calculation unit.   2. The three-dimensional measurement apparatus according to claim 1, wherein the edge set setting means performs the grouping based on edge lengths and end point coordinates.   The three-dimensional measurement apparatus according to claim 1, wherein the edge set setting unit performs grouping based on a spatial distance to the edge. In a three-dimensional measurement apparatus that acquires images at different viewpoint positions as a standard image and a reference image by an imaging unit, and measures the three-dimensional coordinates of each part of the target object from these captured images.
Edge extraction means for extracting edges on the reference image and the reference image;
Edge set setting means for grouping similar edges among the edges on the base image and the reference image extracted by the edge extraction means;
Edge arrangement information extraction means for extracting edge arrangement information of edge groups grouped by the edge set setting means;
Similarity calculation means for comparing edge arrangement information between the reference image and the reference image extracted by the edge arrangement information extraction means, and calculating similarity,
Corresponding position calculation means for calculating a subpixel corresponding position for an edge group pair having a high similarity determined by the similarity calculation means;
Three-dimensional calculation means for calculating the three-dimensional coordinates of the object at the position of the target point from the subpixel corresponding position obtained by the corresponding position calculation means,
The edge arrangement information is information on the number, interval, and length of edges in the window defined for the standard image and the reference image, and the similarity calculation means calculates the similarity based on the edge information. three-dimensional measuring device characterized by. In a three-dimensional measurement apparatus that acquires images at different viewpoint positions as a standard image and a reference image by an imaging unit, and measures the three-dimensional coordinates of each part of the target object from these captured images.
Edge extraction means for extracting edges on the reference image and the reference image;
Edge set setting means for grouping similar edges among the edges on the base image and the reference image extracted by the edge extraction means;
Edge arrangement information extraction means for extracting edge arrangement information of edge groups grouped by the edge set setting means;
Similarity calculation means for comparing edge arrangement information between the reference image and the reference image extracted by the edge arrangement information extraction means, and calculating similarity,
Corresponding position calculation means for calculating a subpixel corresponding position for an edge group pair having a high similarity determined by the similarity calculation means;
Three-dimensional calculation means for calculating the three-dimensional coordinates of the object at the position of the target point from the subpixel corresponding position obtained by the corresponding position calculation means,
The edge sequence information, the frequency decomposition pattern of the reference image and the window defined in the reference image, three-dimensional measuring device you being a signal obtained by suppressing the amplitude component. In a three-dimensional measurement apparatus that acquires images at different viewpoint positions as a standard image and a reference image by an imaging unit, and measures the three-dimensional coordinates of each part of the target object from these captured images.
Edge extraction means for extracting edges on the reference image and the reference image;
Edge set setting means for grouping similar edges among the edges on the base image and the reference image extracted by the edge extraction means;
Edge arrangement information extraction means for extracting edge arrangement information of edge groups grouped by the edge set setting means;
Similarity calculation means for comparing edge arrangement information between the reference image and the reference image extracted by the edge arrangement information extraction means, and calculating similarity,
Corresponding position calculation means for calculating a subpixel corresponding position for an edge group pair having a high similarity determined by the similarity calculation means;
Three-dimensional calculation means for calculating the three-dimensional coordinates of the object at the position of the target point from the subpixel corresponding position obtained by the corresponding position calculation means,
It said corresponding position calculating means, three-dimensional you and calculates the corresponding position based on the pattern similarity near the corresponding position candidate measurement device.   The three-dimensional measuring apparatus according to claim 1, wherein the corresponding position calculation unit calculates a subpixel corresponding position by POC (phase-only correlation method).   8. The apparatus according to claim 1, wherein the imaging unit is a binocular parallel stereo camera, and the edge extracted by the edge extraction unit is only an edge in a direction perpendicular to a baseline. Three-dimensional measuring device.