JP4946898B2 - Distance measuring device and distance measuring method - Google Patents

Description

  The present invention relates to a technique for measuring a distance to an object using images photographed using a plurality of photographing means.

  In recent years, a pre-crash safety technique has been adopted as one of safety measures when a car is running. This technology measures the distance to the preceding vehicle to determine whether the host vehicle maintains a safe inter-vehicle distance with respect to the preceding vehicle. A warning that the vehicle is approaching is issued to prevent accidents. One of the problems in this pre-crash safety is that the actual distance between the host vehicle and the preceding vehicle is measured in spite of being small but is often large, and false alarms are frequently generated.

  Therefore, what is desired for the distance measuring device applied to this pre-crash safety is that when there is no change in the actual distance to the preceding vehicle, the measured distance to the preceding vehicle will not change as much as possible. The variation of the distance to the preceding vehicle is made as small as possible.

  By the way, as a method for measuring the distance to the preceding vehicle, a method using a stereo camera is known. In this method, coordinate points of two images taken from different viewpoints are associated with each other, and a distance is calculated based on the principle of triangulation.

  Here, as a method for associating the coordinates of two images with each other, a phase-only correlation method capable of making a robust and highly accurate association has recently attracted attention (Non-Patent Document 1). This technique has a feature that it is not easily affected by luminance fluctuations and noise because the correlation calculation is performed using only the phase component by removing the amplitude component of the image. However, this method has a problem that the calculation load is high and processing time is required.

Therefore, in Patent Document 1, a vehicle contour is detected, a corresponding point search is performed only on the vehicle contour to calculate a parallax, the calculated parallax is histogrammed, and the maximum parallax whose frequency exceeds a threshold value. A method for calculating the distance to the vehicle using the above parallax as the representative parallax of the vehicle is disclosed.
K.Takita, T.Aoki, Y.Sasaki, T.Higuchi, and K.Kobayashi, "High-Accuracy Subpixel Image Registration Based on Phase-Only Correlation", IEICE Transactions. Fundamentals, E86-A, no8, pp. 1925 -1934, Aug. 2003. JP 7-334800 A

  However, in the method of Patent Document 1, the size of the vehicle is not taken into account at all, and when the vehicle becomes smaller, the number of pixels on the vehicle contour that is the object of the parallax calculation also decreases. And the accuracy of the representative parallax varies depending on the size of the vehicle, and the distance to the vehicle cannot be calculated stably.

  An object of the present invention is to provide a distance measuring device and a distance measuring method capable of stably calculating a distance to an object regardless of the size of the object to be measured for distance.

  A distance measuring device according to the present invention is a distance measuring device that measures a distance to an object to be measured using images photographed using a plurality of photographing means, and is photographed at the same timing by the photographing means. Extraction means for extracting one of the plurality of input images as a standard image and the other input image as a reference image, and extracting an object region in which the object is photographed from the standard image; Area calculating means for calculating the area of the object region extracted by the means, and search means for sequentially setting the attention points in the object region extracted by the extraction means and searching for corresponding points of each attention point from each reference image And a distance calculation unit that calculates a distance to the object based on a plurality of parallaxes between the attention point and the corresponding point, and the search unit calculates the object area calculated by the area calculation unit. surface Depending on the parallax resolution is a parameter for determining the measurement accuracy of the distance and changes the search process to be changed.

  The distance measuring method according to the present invention is a distance measuring device that measures a distance to an object to be measured using images photographed using a plurality of photographing means at the same timing by the photographing means. An extraction step of extracting any one of the plurality of input images taken as a standard image and another input image as a reference image and extracting an object region from which the target object was shot from the standard image; An area calculation step for calculating the area of the object region extracted by the extraction step, and attention points are sequentially set in the object region extracted by the extraction step, and a corresponding point of each attention point is searched from each reference image. A search step; and a distance calculation step for calculating a distance to the object based on a plurality of parallaxes between the attention point and the corresponding point, wherein the search step includes the surface integration Depending on the area of the calculated object region by means disparity resolution is a parameter for determining the measurement accuracy of the distance and changes the search process to be changed.

  According to these configurations, out of a plurality of input images captured at the same timing by the imaging unit, any one input image is set as a reference image, and the other input images are set as reference images. An object region that captures the object is extracted, attention points are sequentially set in the extracted object region, corresponding points of each attention point are searched from each reference image, and based on the parallax between the attention point and the corresponding point, The distance to the object is calculated. Here, the search process is changed so that the parallax resolution, which is a parameter for determining the search accuracy when searching for the corresponding points, is changed according to the area of the object region. Therefore, by adopting a search process that increases the parallax resolution as the area of the object area decreases, it is possible to compensate for a decrease in measurement accuracy due to a decrease in the number of processing points due to a decrease in the area of the object area. The variation in measurement accuracy due to the size of the area of the object can be reduced, and the distance to the object can be calculated stably.

  Further, it is preferable that the search unit changes the search process so that the parallax resolution increases as the area of the object region calculated by the area calculation unit decreases.

  According to this configuration, as the area of the object region becomes smaller, the search process is changed so that the parallax resolution becomes higher. Therefore, variation in measurement accuracy due to the size of the area of the object region is reduced, and the distance to the target object is reduced. Can be calculated stably.

  Further, the search means sets a reference window centered on the attention point in the reference image, sets a reference window in the reference image, and shifts the reference window with the reference image, and images in both windows The similarity calculation process for calculating the similarity is repeatedly performed to search for corresponding points. As the area of the object region calculated by the area calculation unit decreases, the sizes of the reference window and the reference window are reduced. It is preferable to enlarge it.

  According to this configuration, the size of the standard window and the reference window is increased as the area of the object region is reduced, so that it is possible to compensate for a decrease in measurement accuracy due to a decrease in the number of processing points due to a decrease in the area of the object region. And the distance to the object can be calculated stably.

  Further, when the area of the object region calculated by the area calculation unit is smaller than a predetermined value, the search unit sets the parallax resolution to a sub-pixel level having a higher resolution than the pixel level of the reference image, When the area of the object region is larger than a predetermined value, the parallax resolution is preferably set to the pixel level.

  According to this configuration, when the area of the object region is smaller than the predetermined value, the parallax resolution is set to the sub-pixel level of the reference image, and when the area of the object region is larger than the predetermined value, the parallax resolution is Since the pixel level is set, a decrease in measurement accuracy due to a decrease in the number of processing points accompanying a decrease in the area of the object region can be compensated, and a distance to the object can be stably calculated.

  The search process preferably includes a plurality of types of search processes with different parallax resolutions, and the search means selects a search process with a high parallax resolution as the area of the object region decreases.

  According to this configuration, a search process with high parallax resolution is selected as the area of the object region decreases, so that it is possible to compensate for a decrease in measurement accuracy due to a decrease in the number of processing points accompanying a decrease in the area of the object region. The distance to the object can be calculated stably.

  In the search process, a reference window is set around the target point in the reference image, a reference window is set in the reference image, and the images in both windows are shifted while the reference window is shifted with the reference image. The corresponding point is searched by repeatedly executing the similarity calculation process for calculating the similarity between the two windows based on the phase components obtained by frequency decomposition of the images in the reference window and the reference window. It is preferable to include a first search process that employs a similarity calculation process for obtaining the similarity of the images and a second search process that has a lower parallax resolution than the first search process.

  According to this configuration, the first search process for searching for corresponding points by obtaining the similarity between the images in both windows based on the phase components obtained by frequency decomposition of the images in the reference window and the reference window, If the search process includes a second search process having a lower parallax resolution than that of the first search process, and the area of the object region is small, the first search process having a higher parallax resolution is selected, and the area of the object region is If it is large, the second search process with a low parallax resolution is selected, so that the distance to the object can be calculated stably only by switching between the two types of search processes.

  The similarity calculation process in the first search process is preferably frequency-resolved using any of fast Fourier transform, discrete Fourier transform, discrete cosine transform, discrete sine transform, wavelet transform, and Hadamard transform. .

  According to this configuration, since frequency decomposition is performed using any of fast Fourier transform, discrete Fourier transform, discrete cosine transform, discrete sine transform, wavelet transform, and Hadamard transform, it is possible to perform frequency decomposition with high accuracy. it can.

  The similarity calculation process in the first search process is preferably a phase only correlation method.

  According to this configuration, since the degree of similarity is calculated using the phase-only correlation method, it is possible to search for corresponding points with robustness and high accuracy.

  The similarity calculation process in the second search process is preferably SAD.

  According to this configuration, in the similarity calculation process, since the similarity is calculated using SAD (Sum of Absolute Difference), it is possible to search for corresponding points at high speed.

  In addition, the search means sets a target point in a non-object region other than the object region for a period from the end of the search for the corresponding point of each target point set in the object region to the start of processing for the next frame. The corresponding points of each attention point are sequentially set and searched from each reference image, and the distance calculation means calculates the distance of the attention point from the attention point set in the non-object region and the corresponding point of the attention point. It is preferable.

  According to this configuration, it becomes possible to calculate the distance of the non-object region by using the free time of the calculation time in each frame, for example, information useful for grasping the surrounding environment such as a guardrail and a road shoulder. Thus, it is possible to notify the user of the approach of an obstacle other than the target object that is dangerous.

  Moreover, it is preferable that the search means sequentially sets a point of interest from the lower side to the upper side of the non-object region. According to this configuration, since the distance is calculated with priority from the attention point below the non-object region where there is a high possibility that an obstacle close to the user is present, the approach of the obstacle with a high degree of danger is performed by the user. It becomes possible to notify reliably.

  The object preferably includes at least one of an automobile, a motorcycle, a bicycle, and a person. According to this configuration, the distance between a car, a motorcycle, a bicycle, and a person can be calculated. In particular, by calculating the distance of a person riding a motorcycle and a bicycle and the distance of a person not riding a car and a bicycle, the distance between the two can be separated, and the movement of the person can be grasped.

  According to the present invention, it is possible to stably calculate the distance to the object regardless of the size of the object to be measured for distance.

(Embodiment 1)
Hereinafter, a distance measuring apparatus according to Embodiment 1 of the present invention will be described with reference to the drawings. FIG. 1 shows a block diagram of a distance measuring apparatus 1 according to Embodiment 1 of the present invention. The distance measuring device 1 is mounted on the body of a vehicle such as an automobile, and measures the distance to an object to be measured using images photographed using a plurality of photographing means. For example, in the present embodiment, two cameras 11 and 12 (an example of a photographing unit), an arithmetic processing device 13 and a display device 14 are provided.

  The cameras 11 and 12 are arranged at the same height so as to be symmetrical with respect to the traveling direction with a predetermined interval in the left and right direction that is orthogonal to the traveling direction of the vehicle body and parallel to the horizontal direction. Then, the front of the vehicle body is photographed at a predetermined frame rate. Here, the cameras 11 and 12 are synchronized in photographing timing so that frames are photographed at the same time, and are objects to be measured for distance. Take a picture. Here, the front vehicle body includes an automobile, a motorcycle, and a bicycle.

  The arithmetic processing device 13 performs image processing, which will be described later, on the two images taken by the cameras 11 and 12, and calculates the distance to the front vehicle body. The display device 14 includes a display device such as a liquid crystal display or an organic EL display, and displays information such as the distance to the front vehicle body calculated by the arithmetic processing device 13. Here, when the vehicle body is provided with a car navigation system, the display device 14 may be configured by a display device of the car navigation system or may be configured by a display device different from the car navigation system. Good.

  FIG. 2 is a block diagram showing a detailed configuration of the distance measuring apparatus 1. As shown in FIG. 2, the distance measuring apparatus 1 includes a reference camera 110, a reference camera 120, an extraction unit 20 (an example of an extraction unit), an area calculation unit 30 (an example of an area calculation unit), and a processing change unit 40 (of a search unit). An example), a corresponding point search unit 50 (an example of a search unit), a distance calculation unit 60 (an example of a distance calculation unit), and a display unit 70.

  The reference camera 110 is composed of, for example, the camera 11 which is one of the camera 11 and the camera 12 shown in FIG. 1, and images the front of the vehicle body at a predetermined frame rate of, for example, 30 frames per second. An input image including the front vehicle body is acquired. The reference camera 120 includes, for example, the camera 12 which is one of the camera 11 and the camera 12 illustrated in FIG. 1, and photographs the front of the vehicle body at the same frame rate as that of the reference camera 110 and is synchronized with the reference camera 110. An input image including the vehicle body is acquired. Here, as the input image, for example, digital image data in which a plurality of pixels having 256 gradation values from 0 (black) to 255 (white) are arranged in a matrix can be employed.

  The extraction unit 20 uses an input image sequentially input from the reference camera 110 at a predetermined frame rate as a reference image, and an input image sequentially input from the reference camera 120 at a predetermined frame rate as a reference image. An object region that is a region where is photographed is extracted. Here, for example, the extraction unit 20 may adopt a method of extracting a front vehicle body using an edge distribution and left-right symmetry in an image described in JP-A-7-334800, or an SVM (Support Vector). Machine), adaBoost, etc. may be used to learn vehicle body / non-vehicle body data, and a method of extracting the front vehicle body using pattern recognition may be employed. A technique for extracting In addition, when the extraction unit 20 extracts a plurality of regions indicating the front vehicle body, the extraction unit 20 extracts one or a plurality of regions indicating the front vehicle body that is predicted to be close to the own vehicle body as an object region. Also good. Note that, for example, a region having an area of a certain size or more may be employed as the region indicating the front vehicle body that is predicted to be close to the vehicle body.

  The area calculation unit 30 calculates the area of the object region extracted by the extraction unit 20. Here, for example, the area calculation unit 30 may count the number of pixels in the object region, and may employ the counted number of pixels as the area of the object region, or multiply the area per pixel by the counted number of pixels. The value may be adopted as the area of the object region.

  The process changing unit 40 changes the search process so that the parallax resolution, which is a parameter for determining the distance measurement accuracy, is changed according to the area of the object region calculated by the area calculating unit 30. Specifically, the process changing unit 40 changes the search process so that the parallax resolution increases as the area of the object region decreases. The distance measurement accuracy (α) is determined by the following equation (A).

α = β / γ ² (A)
Here, β indicates the number of processing points, and γ indicates the parallax resolution. Here, the number of processing points indicates the number of points of interest that are to be measured for the distance set in the object region. Further, the parallax resolution has a value corresponding to the resolution of the input images taken by the cameras 11 and 12, and the value decreases as the parallax resolution increases and increases as the parallax resolution decreases.

  If the area of the object region is small, the number of pixels in the object region is small, so that many attention points cannot be set, and the measurement accuracy of the distance to the front vehicle body represented by the object region is low. On the other hand, if the area of the object region is large, the number of pixels in the object region increases, so that many points of interest can be set, and the measurement accuracy of the distance to the front vehicle body represented by the object region increases. Therefore, as the area decreases, the number of processing points (β) decreases and the distance measurement accuracy decreases.

  Therefore, in the distance measuring apparatus 1, in order to compensate for the decrease in the number of processing points (β) due to the reduction in the area of the object region, the window size of the windows (reference window and reference window) used in the similarity calculation process described later is set. The value is increased to reduce the parallax resolution (γ), thereby increasing the distance measurement accuracy. Note that the point of interest is a point that is sequentially set in the object region by the corresponding point search unit 50 and is a point to be measured for distance.

  FIG. 3 shows two images from which the object region has been extracted by the extraction unit 20, (a) shows a case where the area of the object region D1 is large, and (b) shows a case where the area of the object region D1 is small. ing. 4 shows an image in which the window W is set, FIG. 4 (a) shows an image when the window W is set with respect to FIG. 3 (a), and FIG. 4 (b) shows FIG. 3 (b). An image when the window W is set for is shown. In FIG. 3 and FIG. 4, for convenience of explanation, the object region is shown as a rectangle, but actually has a more complicated shape.

  The object area D1 shown in FIG. 3B has a smaller area than the object area D1 shown in FIG. 3A, and a large number of attention points cannot be set. Measurement accuracy is lower than in the case of). Therefore, if the parallax resolution is fixed, a difference occurs in the distance measurement accuracy in FIGS. 3A and 3B, and the measurement accuracy is not stable.

  Therefore, the process changing unit 40 reduces the window size when the area of the object region D1 is large as shown in FIG. 4A, and the area of the object region D1 is reduced as shown in FIG. In the case of being small, by increasing the window size, the parallax resolution is increased and the parallax resolution value is decreased to compensate for the decrease in the measurement accuracy due to the decrease in the number of processing points, thereby stabilizing the measurement accuracy.

  Here, the similarity calculation process is a process of calculating the similarity by setting a window W for both the standard image and the reference image and obtaining the correlation between the images in both windows W, as will be described later. Therefore, as the size of the window W increases, it becomes possible to calculate the degree of similarity using more information, and as a result of increasing the degree of similarity calculation, the corresponding point search accuracy is increased and the resolution of the input image is increased. As with the case, the measurement accuracy can be increased. Therefore, when the window size W is increased, the parallax resolution is increased and the value of the parallax resolution is decreased, α in Expression (A) is increased, and the measurement accuracy is increased.

  Here, the processing point (β) of the large object region D1 shown in FIG. 4A is 100, the processing point (β) of the small object region D1 shown in FIG. 4B is 25, and FIG. Assume that the parallax resolution (γ) of the small window is 0.3, the parallax resolution (γ) of the large window shown in FIG. 4B is 0.15, and the threshold value of the measurement accuracy (α) is 1000. Consider the case.

  Case 1: A small window is applied to the large object region D1 shown in FIG. At this time, α = 100 / (0.3 × 0.3) = 3333.3 is obtained, α is larger than the threshold value (1000), and measurement accuracy equal to or higher than the threshold value can be obtained.

  Case 2: A small window is applied to the small object region D1 shown in FIG. At this time, α = 25 / (0.3 × 0.3) = 277.7 is obtained, α is smaller than the threshold value (1000), and measurement accuracy equal to or higher than the threshold value cannot be obtained.

  Case 3: A large window is applied to the small object region D1 shown in FIG. At this time, α = 25 / (0.15 × 0.15) = 1111.1 is obtained, α is larger than the threshold (1000), and measurement accuracy equal to or higher than the threshold can be obtained.

  As can be seen from the above three cases, when the object region D1 is small, by applying a large window, as in the case where the object region D1 is large, the measurement accuracy can be made equal to or greater than the threshold, and the size of the object region D1 Regardless, the measurement accuracy can be stabilized.

  Returning to FIG. 2, the corresponding point search unit 50 sequentially sets the attention points in the object region extracted by the extraction unit 20, and uses the search processing adopted by the processing change unit 40 to correspond to each corresponding point. Are searched from the reference image acquired at the same time as the base image in which the attention point is set.

  Specifically, the corresponding point search unit 50 hierarchically generates a plurality of standard images and reference images having different resolutions so that the resolution becomes higher from the lower hierarchy toward the upper hierarchy. Based on the corresponding point search result, a search range smaller than the search range of the hierarchy is set as a reference image one level higher, and search processing for searching for a corresponding point within the search range is performed in the upper hierarchy. The corresponding points are searched by executing in a hierarchical manner.

  Here, as the search process, a reference window is set around the target point in the reference image, a reference window having the same size as the reference window is set as the reference image, and the reference window is set within the search range set in the reference image. While shifting, it is possible to employ a process of searching for corresponding points by repeatedly executing a similarity calculation process for obtaining a similarity between the image in the reference window and the image in the reference window.

  As the similarity calculation process, a process of calculating similarity based on a phase component obtained by frequency-decomposing the image in the standard window and the image in the reference window may be adopted, or frequency decomposition may be performed. Alternatively, a process for calculating the similarity may be employed.

  Here, as frequency decomposition, methods such as fast Fourier transform, discrete Fourier transform, discrete cosine transform, discrete sine transform, wavelet transform, and Hadamard transform can be adopted. From the viewpoint of speeding up the processing, fast Fourier transform It is preferable to employ conversion.

  As processing for calculating the similarity based on the phase component, a phase only correlation (POC) method or a DCT code only correlation method can be adopted. However, since the POC is more robust, this In the embodiment, POC is adopted. Further, as a method for calculating the similarity without using the phase component, SAD (Sum of Absolute Difference), SSD (Sum of the squares of the density difference), NCC (Normalized Cross Correlation), or the like can be adopted. In the present embodiment, SAD is employed from the viewpoint of speeding up processing and increasing accuracy.

  Examples of the DCT code limiting method include, for example, “Fusion of image signal processing and image pattern recognition—DCT code limited correlation and its application, Hitoshi Kiya, Tokyo Metropolitan University, Faculty of System Design, Dynamic Image Processing Realization Workshop 2007 ′. The method described in 2007.7.7.3.8-9 ″ can be adopted.

  The POC is less susceptible to differences in imaging conditions of the cameras 11 and 12 and noise than the SAD and is robust. However, the POC is costly because of the frequency decomposition. Therefore, it is preferable to adopt POC when importance is attached to robustness, and it is preferable to adopt SAD when speeding up the processing.

  Further, when the extraction unit 20 extracts a plurality of object regions, POC is adopted for the object region indicating the front vehicle body with high importance, and SAD is adopted for the object region indicating the front vehicle body with low importance. Also good. Note that the front vehicle body that is important in avoiding danger may be adopted as the front vehicle body having higher importance. For example, the front vehicle body having a larger object area may have higher importance.

  Next, details of the POC will be described. FIG. 5 is a flowchart showing the flow of POC processing. In addition, in FIG. 5, the case where a Fourier transform is employ | adopted as frequency decomposition is illustrated. First, in steps S1 and S2, Fourier transformation is performed on the image in the standard window, and Fourier transformation is performed on the image in the reference window. Here, assuming that the image in the standard window is represented by equation (1) and the image in the reference window is represented by equation (2), the image in the standard window is Fourier transformed by equation (3), The image in the reference window is Fourier-transformed by equation (4).

  Here, n1 and n2 indicate vertical and horizontal coordinates, N1 and N2 indicate vertical and horizontal sizes of the reference window and reference window, and M1 indicates the vertical direction from the center point of the reference window and reference window. The distance from the center point of the reference window and the reference window to the edge in the horizontal direction is indicated by M2.

  Next, in steps S3 and S4, both images Fourier-transformed in steps S1 and S2 are normalized using equation (5). Next, in step S5, both images normalized in steps S3 and S4 are synthesized. Here, both normalized images are synthesized by multiplying F ′ (k1, k2) and the complex conjugate of G ′ (k1, k2) as shown in Expression (6). Next, in step S6, the image synthesized in step S5 is subjected to inverse Fourier transform using equation (7) to calculate a POC value (r (k1, k2)).

  FIG. 6 is a graph showing the POC value. In this graph, the x-axis and the y-axis represent coordinates in the standard window and the reference window, and the z-axis represents the POC value. As shown in FIG. 6, the POC value has a steep correlation peak, and is known to have high robustness and estimation accuracy in image matching. The height of the correlation peak increases as the correlation between the standard window and the reference window increases. Therefore, by using, for example, the height of the correlation peak as the similarity between the standard window and the reference window, it is possible to indicate how close the images in both windows are. In addition, the position of the correlation peak indicates the amount of deviation between the standard window and the reference window.

  Then, corresponding points are searched for while shifting the reference window. In this case, since the position of the correlation peak indicates the amount of deviation between the two windows, it is not necessary to shift the reference window one pixel at a time. For example, half the window size may be set as the amount of shift.

  Finally, the position of the corresponding point is obtained from the position of the reference window and the position of the correlation peak of the POC value. Although the position of the reference window is a pixel level value, the position of the correlation peak indicates the amount of deviation between the windows and does not necessarily exist at the pixel level position. Therefore, the amount of deviation can be obtained at the sub-pixel level by interpolating and estimating the position of the correlation peak at the sub-pixel level. Here, as the interpolation estimation method of the position of the correlation peak, a method of fitting a function such as a parabola can be employed. Then, by adding the position of the correlation peak interpolated and estimated at the subpixel level to the position of the reference window, the corresponding point can be obtained at the subpixel level.

Next, details of SAD will be described. FIG. 7 is a schematic diagram for explaining SAD. As shown in FIG. 7, the attention point CP is set in the object region D1 set in the reference image, and the reference window W1 is set around the attention point CP. Here, the attention point CP is sequentially set so as to perform raster scanning in the object region D1. Next, a predetermined pixel within the search range set in the reference image is set as the center point O1 of the reference window W2, and the reference window W2 is set around the center point O1. Here, the center point O1 has the same height in the vertical direction as the point of interest CP, and is set on a straight line of the reference image parallel to the horizontal direction. Next, a correlation value COR _p between the image Img1 in the standard window W1 and the image Img2 in the reference window W2 is calculated using equations (8) and (9).

  Where W is the window size of the standard window W1 and the reference window W2, i is the vertical coordinate, j is the horizontal coordinate, p is a variable that satisfies 0 ≦ p ≦ max_disp, and max_disp is Indicates the final search point.

When the correlation value COR _p is obtained, p is increased by 1, the reference window W2 is shifted in the horizontal direction, and the next correlation value COR _p is obtained. In SAD, since the reference window W2 is shifted by controlling p, the vertical position is the same as the point of interest, and the center point O1 is positioned on a straight line parallel to the horizontal direction. To be displaced.

Then, the correlation value COR _p is adopted as the similarity of the center point O1 of the image in the reference window W2, the position where the similarity is the highest in the reference image is searched, and the point of interest set in the reference image as the searched position It is specified as a corresponding point of CP. As shown in the graph of FIG. 7, the correlation value COR _p has a higher correlation between the image Img1 and the image Img2, and the higher the similarity, the higher the correlation value COR _p .

  Here, since the reference window is discretely shifted in units of pixels, the similarity is calculated with pixel level resolution. However, similar to POC, the similarity may be calculated at the sub-pixel level. Good.

  Returning to FIG. 2, the distance calculation unit 60 obtains the distance to the front vehicle body and the three-dimensional position indicated by the object region D <b> 1 using a stereo distance measurement method based on the attention point and the corresponding point. FIG. 8 is a schematic diagram for explaining a distance measuring method by the stereo method.

  In FIG. 8, f indicates the focal length of the cameras 11 and 12, and the number of pixels on the imaging surface (CCD) of the cameras 11 and 12 and the size (μ) of one pixel are the same. Further, the cameras 11 and 12 are arranged so as to be separated from each other in the left-right direction by a predetermined base line length L and to have parallel optical axes. In this case, if the parallax (the number of displaced pixels) on the imaging surface is d (= d1 + d2), the distance (Z) to the front vehicle body can be obtained by Z = (L × f) / (μ × d). . As the three-dimensional position, a relative position that is the position of the front vehicle body when a three-dimensional coordinate space is set with respect to the own vehicle body may be adopted, or if the vehicle body is equipped with GPS, GPS The absolute position of the front vehicle body may be calculated from the absolute position of the vehicle body obtained by the above, and this absolute position may be adopted.

  Here, for example, the distance calculation unit 60 obtains the representative parallax from the parallax of each pair of the target point set in the object region and the corresponding point of each target point, and the front vehicle body that represents the distance of the representative parallax is represented by the object region Or the distance of each pair of the target point and the corresponding point may be obtained, and the representative distance may be obtained from the obtained distance. As a method for obtaining the representative parallax or the representative distance, a method using a histogram described in Japanese Patent Laid-Open No. 7-334800 may be employed, and the distance or parallax average value, median value, A method of calculating an average value of remaining values obtained by removing the maximum value and the minimum value from the pair distance or parallax as the representative distance or the representative parallax may be adopted.

  Returning to FIG. 2, the display unit 70 includes the display device 14 illustrated in FIG. 1, and displays the distance to the front vehicle body calculated by the distance calculation unit 60, the three-dimensional position, and the like. Here, when the distance calculation unit 60 calculates the distances of the plurality of front vehicle bodies, the display unit 70 may notify each distance to the driver.

  Next, the operation of the distance measuring device 1 will be described. FIG. 9 is a flowchart showing the operation of the distance measuring apparatus 1. In the following flowchart, for convenience of explanation, the number of object regions D1 included in the base image and the reference image is one.

  First, in step S <b> 11, the extraction unit 20 acquires an input image input from the reference camera 110 as a reference image, and acquires an input image input from the reference camera 120 as a reference image.

  In step S12, the extraction unit 20 extracts the object region D1 from the reference image acquired in step S11 as shown in FIG. In step S13, the area calculation unit 30 calculates the area of the object region D1 extracted in step S12.

  In step S14, the process changing unit 40 sets the window size so that the window sizes of the standard window and the reference window increase as the area of the object region D1 decreases as shown in FIG. Here, the process changing unit 40 predetermines a preferable window size in each stage when the area of the object area is divided in stages, and sets a window size corresponding to the stage to which the area of the object area to be processed belongs. You only have to set it.

  In step S15, the corresponding point search unit 50 performs a search process. FIG. 10 is a flowchart showing search processing. First, in step S41, the corresponding point search unit 50 executes a process of reducing the resolution of the standard image and the reference image at a resolution determined in advance for each layer, and hierarchically selects a plurality of standard images and reference images having different resolutions. To generate. Here, the number of layers and the resolution in each layer are determined in advance, and the resolution increases from the lower layer to the upper layer, and the base image and the reference image that have not undergone resolution conversion are the highest layer. Is done.

  In step S42, the corresponding point search unit 50 sets a point of interest in the reference image of the highest hierarchy.

  In step S43, the corresponding point search unit 50 sets a search range for the reference image of the lowest hierarchy. FIG. 12 is a diagram showing a standard window and a reference window set for the standard image and the reference image. As shown in FIG. 12, the search range DR is set to the entire area on the straight line of the reference image parallel to the horizontal direction and having the same height in the vertical direction as the target point CP set in the standard image in step S41. Has been.

  In step S44, the corresponding point search unit 50 executes similarity calculation processing. FIG. 11 is a flowchart showing similarity calculation processing. First, in step S61, the corresponding point search unit 50 sets the reference window W1 to the reference image in the same hierarchy as the reference image for which the search range DR is set in step S43. Here, the point on the reference image of the hierarchy whose processing target corresponds to the position of the attention point set in the reference image of the highest hierarchy in step S42 is set as the attention point CP of the reference image of the hierarchy. Is done. Then, as shown in FIG. 12, the reference window W1 is set so that the center is located at the attention point CP.

  In step S62, the reference window W2 is set to the reference image of the hierarchy to be processed. Here, the reference window W2 is set so that, for example, the left end position of the search range DR shown in FIG. 12 is the initial position of the center point O1.

  In step S63, the corresponding point search unit 50 calculates the similarity between the image in the standard window W1 and the image in the reference window W2 using the above-described POC or SAD.

  In step S64, when the center point O1 of the reference window W2 has not reached the final position of the search range DR (NO in step S64), the corresponding point search unit 50 shifts the reference window W2 (step S65) and performs processing. Returning to step S63, if the center point of the reference window W2 has reached the final position of the search range DR (YES in step S64), the similarity calculation process is terminated, and the process proceeds to step S45 shown in FIG. Here, as the final position, for example, the right end of the search range DR can be employed. Also, as shown in FIG. 12, in the reference window W2, the center point O1 is sequentially set from the left end point to the right end point of the search range DR, and is shifted from the left side to the right side along the search range DR. To go. Then, the corresponding point search unit 50 calculates the similarity every time the reference window W2 is shifted, and sets the calculated similarity as the similarity at the center point O1. As a result, a distribution of similarity on the search range DR is obtained.

  Returning to FIG. 10, in step S <b> 45, the corresponding point search unit 50 refers to the reference image of the hierarchy whose processing target is the point having the maximum similarity among the similarities on the search range DR obtained by the similarity calculation processing. It is obtained as a corresponding point.

  In step S46, when the corresponding point search unit 50 does not set the highest hierarchy as the processing target (NO in step S46), the corresponding hierarchy search unit 50 sets the upper hierarchy as the processing target and searches for a reference image in the hierarchy. The range DR is set (step S48), the process returns to step S44, and if the highest hierarchy has already been set as the process target (YES in step S46), the process proceeds to step S47.

  FIG. 13 is a diagram for explaining the search range DR, where (a) shows a standard image and a reference image in a certain hierarchy, and (b) is a standard image in a hierarchy one level higher than the hierarchy shown in (a). And a reference image. The corresponding point search unit 50 includes a corresponding point TP obtained in the reference image shown in FIG. 13A, and searches for a reference image that is one level higher than the search range DR shown in FIG. The range DR is set (FIG. 13B). That is, the corresponding point search unit 50 sets the search range DR on any straight line that has the same vertical height as the point of interest CP and is parallel to the horizontal direction. The range is narrowed toward the higher levels. Here, the resolution of the standard image and the reference image becomes higher as the hierarchy becomes higher. Therefore, the search range DR is set in a wide range with the low-resolution reference image to search the approximate position of the corresponding point TP, and the search result DR of the high-resolution reference image is narrowed using the search result. This makes it possible to narrow down and search for corresponding points at high speed and with high accuracy. Note that the search range DR set in each layer may be a range having a certain width predetermined for each layer, with the corresponding point TP obtained in the lower layer as the center.

  Returning to FIG. 10, if the target point has not reached the final position in step S47 (NO in step S47), the corresponding point search unit 50 performs the process to set the next target point in the reference image in step S42. If the target point is set at the final position (YES in step S47), the search process is terminated and the process proceeds to step S16 shown in FIG.

  Returning to FIG. 9, in step S <b> 16, the distance calculation unit 60 obtains the distance to the front vehicle body shown in the object region D <b> 1, and in step S <b> 17, the display unit 70 displays the distance calculated by the distance calculation unit 60. In step S18, for example, when an operation command for ending the process is input by the user, the process is terminated (YES in step S18), the process is terminated, and the process is not terminated (in step S18). NO), the extraction unit 20 acquires the input image of the next frame acquired by the standard camera 110 as a standard image, and acquires the input image of the next frame acquired by the reference camera 120 as a reference image (step S19). The process returns to step S12.

  As described above, according to the distance measuring device 1, the input images captured by the cameras 11 and 12 at the same timing are acquired as the standard image and the reference image, and the object region D1 that captures the front vehicle body is extracted from the standard image and extracted. Attention points are sequentially set in the object region, the corresponding points of each attention point are searched from each reference image, and the distance to the object is calculated based on the parallax between the attention point and the corresponding point. Here, the window size of the standard window and the reference window is increased as the area of the object region is reduced, so that the parallax resolution is increased, and the reduction in measurement accuracy due to the reduction in the number of processing points can be compensated. Regardless of the size, the distance measurement accuracy can be kept constant, and the distance measurement accuracy can be stabilized.

(Embodiment 2)
Next, a distance measuring device 1a according to Embodiment 2 of the present invention will be described. In the present embodiment, since the overall configuration and detailed configuration are the same as those in Embodiment 1, FIGS. 1 and 2 are used. In the present embodiment, the same elements as those in the first embodiment are not described, and only differences are described.

  When the area of the object region is smaller than a predetermined value, the process changing unit 40 shown in FIG. 2 sets the parallax resolution to a sub-pixel level having a higher resolution than the pixel level of the reference image, and the area of the object region is a predetermined value. If larger, the search process is changed by setting the parallax resolution to the pixel level. The predetermined value may be a value that does not cause a large difference in measurement accuracy variation from the resolution value at the sub-pixel level, and may be an experimentally obtained value.

  Here, when the corresponding point is searched for at the sub-pixel level of the reference image, the parallax resolution (γ) shown in the above formula (A) increases and the value of γ decreases, so that the measurement accuracy (α) can be increased. it can. Therefore, when the area is smaller than the predetermined value, searching for corresponding points at the sub-pixel level of the reference image compensates for a decrease in measurement accuracy (α) due to a decrease in the number of processing points, and increases or decreases the area of the object region D1. Regardless, the measurement accuracy can be stabilized.

  Here, the processing point (β) of the large object region D1 shown in FIG. 3A is 100, the processing point (β) of the small object region D1 shown in FIG. 3B is 25, and the parallax resolution (γ ) Is 1, the sub-pixel level parallax resolution (γ) is estimated as 0.3, and the threshold of measurement accuracy (α) is 90, and the following three cases are considered.

  Case 1: A corresponding point is searched for at a pixel level in the large object region D1 shown in FIG. At this time, α = 100 / (1.0 × 1.0) = 100 is obtained, α is larger than the threshold value (90), and measurement accuracy equal to or higher than the threshold value can be obtained.

  Case 2: A corresponding point is searched for at a pixel level with respect to the small object region D1 shown in FIG. At this time, α = 25 / (1.0 × 1.0) = 25 is obtained, α is smaller than the threshold value (90), and measurement accuracy equal to or higher than the threshold value cannot be obtained.

  Case 3: A corresponding point is searched at the sub-pixel level for the small object region D1 shown in FIG. At this time, α = 25 / (0.3 × 0.3) = 277.7 is obtained, α is larger than the threshold value (90), and measurement accuracy equal to or higher than the threshold value can be obtained.

  As can be seen from the above three cases, when the object region D1 is small, searching for corresponding points at the sub-pixel level can make the measurement accuracy equal to or higher than the threshold, as in the case where the object region D1 is large. Measurement accuracy can be stabilized regardless of the size of the region D1.

  Next, the operation of the distance measuring device 1a will be described. FIG. 14 is a flowchart showing the operation of the distance measuring device 1a. The processes in steps S81 to S83 and steps S85 to S89 are the same as the processes in steps S11 to S13 and steps S15 to S19 shown in FIG.

  In step S84, when the area of the object region is smaller than a predetermined value, the process changing unit 40 sets the parallax resolution to a sub-pixel level having a higher resolution than the pixel level of the reference image, and the area of the object region is a predetermined value. If larger, the search process is changed by setting the parallax resolution to the pixel level. Here, when searching for corresponding points at the sub-pixel level, the process changing unit 40 may increase the parallax resolution as the area of the object region decreases. Thereby, variation in measurement accuracy due to the size of the area of the object region can be further reduced. In this case, the process changing unit 40 predetermines a preferable parallax resolution at each stage when the area of the object region is divided in stages, and sets the parallax resolution corresponding to the stage to which the area of the object area to be processed belongs. You only have to set it.

  In the distance measuring device 1a, the process of step S63 shown in FIG. 11 is a process of calculating the similarity according to the parallax resolution set in step S84.

  Thus, according to the distance measuring device 1a, when the area of the object region is smaller than the predetermined value, the parallax resolution is set to a sub-pixel level higher than the pixel level of the reference image, and the area of the object region is a predetermined value. If it is larger, the search processing is changed by setting the parallax resolution to the pixel level, so it can compensate for the decrease in measurement accuracy due to the decrease in the number of processing points, and the distance measurement accuracy regardless of the size of the object area Can be kept constant, and the distance measurement accuracy can be stabilized.

(Embodiment 3)
Next, a distance measuring device 1b according to Embodiment 3 of the present invention will be described. In the present embodiment, since the overall configuration and detailed configuration are the same as those in Embodiment 1, FIGS. 1 and 2 are used. In the present embodiment, the same elements as those in the first and second embodiments are omitted, and only differences are described.

  The process changing unit 40 illustrated in FIG. 2 selects a search process with a higher parallax resolution from a plurality of types of search processes with different parallax resolutions as the area of the object region decreases. Here, the search process includes a first search process with a high parallax resolution and a second search process with a lower parallax resolution than the first search process.

  In both the first and second search processes, a reference window is set around the target point in the reference image, a reference window is set in the reference image, and the images in both windows are shifted while the reference window is shifted from the reference image. The corresponding point is searched by repeatedly executing the similarity calculation process for calculating the similarity of the first component. The first search process is a phase component obtained by frequency-resolving the images in the reference window and the reference window. The similarity calculation process for obtaining the similarity of the images in both windows based on the above is employed, and the second search process employs a similarity calculation process for obtaining the similarity without performing frequency decomposition. .

  Here, the similarity calculation processing in the first search processing is frequency-resolved using any of fast Fourier transform, discrete Fourier transform, discrete cosine transform, discrete sine transform, wavelet transform, and Hadamard transform. In the embodiment, fast Fourier transform is adopted from the viewpoint of speeding up the processing.

  In addition, as the similarity calculation process in the first search process, a POC or DCT code limited correlation method can be adopted. However, since the POC is more robust, this embodiment employs the POC.

  In addition, as the similarity calculation process in the second search process, SAD (Sum of Absolute Difference), SSD (Sum of squares of density difference), NCC (Normalized Cross Correlation), or the like can be adopted. In this embodiment, SAD is adopted from the viewpoint of speeding up and accuracy.

  Then, the process changing unit 40 selects the first search process with high parallax resolution when the area of the object region is smaller than a predetermined value, and selects the first search process with low parallax resolution when the area of the object region is larger than the predetermined value. 2 search processing is selected. Since the POC can search for corresponding points more accurately than the SAD, the measurement accuracy can be increased as in the case where the resolution of the reference image is increased. Therefore, when the POC is employed, the parallax resolution is increased and the value of the parallax resolution is decreased, α in the above formula (A) is increased, and the measurement accuracy is increased.

  As the predetermined value, an experimentally obtained value that does not cause a large difference in variation in measurement accuracy from the parallax resolution value estimated by POC and SAD can be used.

  Here, the processing point (β) of the large object region D1 shown in FIG. 3A is 100, the processing point (β) of the small object region D1 shown in FIG. 3B is 25, and the parallax resolution (γ) in SAD. Is estimated as 0.6, the parallax resolution (γ) in POC is estimated as 0.3, and the threshold of measurement accuracy (α) is 270, and the following three cases are considered.

  Case 1: A corresponding point is searched by SAD for the large object region D1 shown in FIG. At this time, α = 100 / (0.6 × 0.6) = 277.7 is obtained, α is larger than the threshold value (270), and measurement accuracy equal to or higher than the threshold value can be obtained.

  Case 2: A corresponding point is searched by SAD for the small object region D1 shown in FIG. At this time, α = 25 / (0.6 × 0.6) = 69.4 is obtained, α is smaller than the threshold value (270), and measurement accuracy higher than the threshold value cannot be obtained.

  Case 3: A corresponding point is searched by POC for the small object region D1 shown in FIG. At this time, α = 25 / (0.3 × 0.3) = 277.7 is obtained, α is larger than the threshold value (270), and measurement accuracy equal to or higher than the threshold value can be obtained.

  As can be seen from the above three cases, when the object area D1 is small, searching for corresponding points by POC enables the measurement accuracy to be equal to or greater than the threshold, as in the case where the object area D1 is large. Regardless of the size, the measurement accuracy can be stabilized.

  Next, the operation of the distance measuring device 1b will be described. FIG. 15 is a flowchart showing the operation of the distance measuring device 1b. The processing in steps S101 to S103 and steps S105 to S109 is the same as the processing in steps S11 to S13 and steps S15 to S19 shown in FIG.

  In step S104, the process changing unit 40 selects the first search process when the area of the object region is smaller than the predetermined value, and selects the second search process when the area of the object region is larger than the predetermined value. To do. In the distance measuring device 1b, in the process of step S63 shown in FIG. 11, when POC is selected in step S104, the similarity is calculated using the POC, and when SAD is selected in step S104, This is a process for calculating the similarity using SAD.

  As described above, according to the distance measuring device 1b, when the area of the object region is smaller than the predetermined value, POC is selected, and when the area of the object region is larger than the predetermined value, SAD is selected. It is possible to compensate for the decrease in measurement accuracy due to the decrease in the distance, and to keep the distance measurement accuracy constant regardless of the size of the area of the object region, and to stabilize the distance measurement accuracy.

(Embodiment 4)
Next, a distance measuring device 1c according to Embodiment 4 of the present invention will be described. In the present embodiment, since the overall configuration and detailed configuration are the same as those in Embodiment 1, FIGS. 1 and 2 are used. In the present embodiment, the same elements as those in the first to third embodiments are not described, and only differences are described.

  The extraction unit 20 illustrated in FIG. 2 extracts candidate areas that are candidate object areas from the reference image, and extracts object areas from the extracted candidate areas. Here, the extraction unit 20 extracts candidate regions using the same method as the method for extracting the object region according to the first embodiment, and selects one or more candidate regions having high importance from the extracted candidate regions. What is necessary is just to extract as an area | region. Here, as the importance, for example, a value that is higher for a candidate area with a larger area and higher for a candidate area located below the reference image may be employed. Then, the extraction unit 20 may extract candidate areas having importance values greater than a predetermined value as object areas, or extract a predetermined number of candidate areas in descending order of importance as object areas. May be. Here, the reason why the importance level is set higher for a candidate area located on the lower side and having a larger area is that the object indicated by such a candidate area is more likely to be closer to the vehicle body. .

  The correspondence point search unit 50 performs a period on the reference image other than the object region extracted by the extraction unit 20 until the processing for the next frame is started after the search for the corresponding point of each target point set in the object region is completed. A third search process is executed in which attention points are sequentially set in a non-object region, which is a region, and corresponding points of each attention point are searched from each reference image.

  In other words, the third search process uses the free time until the input image of the next frame is acquired after the search for the corresponding point of each target point set in the object area in the current frame. In this process, attention points are sequentially set in the non-object region to search for corresponding points, and when the input image of the next frame is acquired, the corresponding point search process in the non-object region is terminated.

  Here, the third search process may be performed by searching for corresponding points using the above-described SAD or POC. In executing the third search process, the corresponding point search unit 50 sequentially sets a point of interest from, for example, the leftmost pixel to the rightmost pixel of the first line from the bottom of the non-object region. When all the processes are completed, the attention points may be set sequentially from the lower side to the upper side of the non-object region so that the attention points are sequentially set for the line immediately above. At this time, the corresponding point search unit 50 may set all the pixels in the non-object region as a point of interest, or set a pixel obtained by thinning the non-object region in the horizontal and vertical directions with a certain thinning width as the point of interest. May be. Further, the corresponding point search unit 50 may cause the area calculation unit 30 to calculate the area of the non-object region, and may set the attention point so that the thinning width increases as the area of the non-object region increases.

  The distance calculation unit 60 calculates the distance of the front vehicle body indicated by the object area in the same manner as in the first embodiment, and also uses the stereo method described above from the attention point and the corresponding point in the non-object area. Is used to find the distance between each point of interest. Here, the distance calculation unit 60 displays the display unit when the target point set in the non-object region includes a target point with a shorter distance than a predetermined distance assumed to be close to the vehicle body. Information indicating that an obstacle is approaching may be displayed on 70 to notify the user of the danger.

  Next, the operation of the distance measuring device 1c will be described. FIG. 16 is a flowchart showing the operation of the distance measuring device 1c. Step S121 is the same as step S11 shown in FIG. In step S122, the extraction unit 20 extracts candidate areas from the reference image, obtains importance levels of the candidate areas, and extracts candidate areas having importance levels greater than a predetermined value as object areas. Here, the extraction unit 20 obtains, for example, the center of gravity of each candidate region, assigns a higher importance to a candidate region whose center of gravity is located below the reference image, and multiplies the assigned importance by the area of each candidate region. A value may be set as the importance of each candidate area. In the following description, it is assumed that one object region is extracted for convenience of description.

  Since the process of step S123-S125 is the same as the process of step S13-S15 shown in FIG. 9, description is abbreviate | omitted.

  In step S126, the corresponding point search unit 50 executes a third search process. FIG. 17 is a flowchart showing details of the third search process. First, in step S131, the corresponding point search unit 50 determines whether there is a free time for executing the third search process until the next frame is acquired. In step S131, a non-object area is specified from the reference image (step S132). On the other hand, if it is determined in step S131 that there is no free time (NO in step S131), the third search process is terminated and the process returns to step S127 shown in FIG.

  Next, the corresponding point search unit 50 sets one attention point in the non-object region (step S133), searches for a corresponding point for the set attention point (step S134), and returns the process to step S131. Here, the corresponding point search unit 50 sequentially sets the attention points according to the setting pattern 1 for setting each pixel of the non-object region as the attention point or the setting pattern 2 for setting the pixels obtained by thinning out the non-object region as the attention point. .

  FIG. 18 is a diagram showing the points of interest set in the non-object region over time using the setting pattern 1, and the time elapses toward (a) to (c). In FIG. 18, a triangular mark indicates a target point set in the object region D1, a white circle indicates a pixel in the non-object region D2 that is not set as the target point, and a black circle indicates a non-set point that has been set as the target point. The pixel of the object area is shown. Further, the non-object region D2 is a region obtained by removing the object region D1 from the reference image.

  As shown in FIG. 18A, attention points are sequentially set from the leftmost pixel to the rightmost pixel of the first line from the bottom of the non-object region D2, and corresponding points are sequentially searched, and a search for the rightmost pixel is performed. When the processing is completed, as shown in FIG. 18B, attention points are sequentially set from the leftmost pixel to the rightmost pixel of the second line from the bottom, and corresponding points are sequentially searched, and FIG. As shown in FIG. 5, when the pixel at the left end of the seventh line from the bottom is set as a point of interest and the search process is completed, the search process is terminated because there is no free time.

  FIG. 19 is a diagram showing the points of interest set in the non-object region using the setting pattern 2 over time, and the time elapses toward (a) to (b).

  In setting pattern 2, since the thinning width is set by skipping one pixel in the horizontal and vertical directions, as shown in FIG. 19A, from the leftmost pixel to the rightmost pixel in the first line from the bottom. It can be seen that the points of interest are sequentially set with the skipping width of one pixel skip. Then, when the search process for the first line from the bottom is completed, the attention points are sequentially set in such a manner that the attention points are sequentially set by skipping one pixel in the next two lines.

  Also, in the setting pattern 2, when the search process in the first line from the top is completed, that is, when the first cycle search process is completed, if there is a free time remaining, the thinning is performed as a point of interest. Pixels that have not been set are sequentially set as attention points, and search processing in the second cycle is started.

  That is, as shown in FIG. 19B, when the search process in the first cycle is completed, pixels that are not set as attention points from the lower line toward the upper line are sequentially set as attention points. The search process of the second cycle is executed. Here, in the search process in the second cycle, pixels that are not set as the attention point in the same line as the line in which the attention point was set in the first cycle may be sequentially set as the attention point by skipping one pixel. The attention point may be sequentially set by skipping one pixel in a line different from the line where the attention point is set in the first cycle. Note that in the third search process, the thinning width is not limited to skipping one pixel, and may be greater than or equal to skipping two pixels.

  Returning to FIG. 16, in step S <b> 127, the distance calculation unit 60 calculates the distance at each point of interest in the non-object region in addition to the object region, and when the calculated distance is shorter than a predetermined value, the obstacle approaches. Is displayed on the display unit 70 (step S128). The processing in steps S129 to S130 is the same as steps S18 to S19 shown in FIG.

  As described above, according to the distance measuring device 1c, it is possible to calculate the distance of the non-object region by using the free time of the calculation time in each frame, and for example, grasp the surrounding environment such as a guardrail and a road shoulder. The above useful information can be obtained, and it is possible to notify the user of the approach of an obstacle other than the target object that is dangerous.

  In the fourth embodiment, the point of interest is set in the non-object region. Instead, the point of interest may be set in a non-candidate region other than the candidate region.

  In the first to fourth embodiments, the number of object regions is one, but the present invention is not limited to this and may be two or more. In this case, the processing of FIGS. 9 to 11 in the first embodiment, the processing of FIG. 14 in the second embodiment, the processing of FIG. 15 in the third embodiment, and the processing in FIG. What is necessary is just to perform the process of FIGS. And the display part 70 should just display the distance for every object area | region D1.

  Moreover, in the said Embodiment 1-4, although the front vehicle body was employ | adopted as a target object, it is not limited to this, A person may be made into a target object, and animals, such as dogs and cats other than a person, may be made into a target object. Good. In this case, for example, the extraction unit 20 may prepare a template for each target object that is a distance measurement target, and extract various target objects by template matching.

  In the first to fourth embodiments, considering that the heights of the cameras 11 and 12 are the same, the search range has the same vertical height as the point of interest and is parallel to the horizontal direction. Although it is set on a straight line, the present invention is not limited to this, and a quadrangular region having a certain width in the vertical direction around the straight line may be set as the search range.

  In the first to fourth embodiments, the number of cameras is two. However, the number of cameras is not limited to this and may be three or more. In this case, the image captured by any one of the cameras is used as a standard image, and the reference image captured by the other camera is used as the reference image to cause the extraction unit 20 to the distance calculation unit 60 to perform the above-described processing, Find the distance.

  Furthermore, the types of search processing shown in the third embodiment are not limited to the first and second search processing, and three or more types of search processing may be employed. For example, search processing for searching corresponding points at the pixel level by POC (POC-pixel level), search processing for searching corresponding points at the sub-pixel level by POC (POC-subpixel), and searching for corresponding points at the pixel level by SAD You may employ | adopt three types of search processes, such as the search process (SAD-pixel level). In this case, since the parallax resolution decreases in the order of SAD-pixel level, POC-pixel level, and POC-subpixel level, the object region is divided into three stages of large, medium, and small according to the area, and the area is large. The SAD-pixel level may be adopted for the object region, the POC-pixel level may be adopted for the object region having a medium area, and the POC-subpixel level may be adopted for the object region having a small area. In this case, a SAD-subpixel level may be added. Furthermore, the SAD-subpixel level and the POC-subpixel level having different subpixel level resolutions may be employed as the search processing. By so doing, variations in measurement accuracy due to the size of the area of the object region can be further suppressed, and the measurement accuracy can be further stabilized.

1 shows a block diagram of a distance measuring device according to an embodiment of the present invention. FIG. It is a block diagram which shows the detailed structure of the distance measuring device shown in FIG. 3 shows two images from which an object region has been extracted by the extraction unit shown in FIG. The window shows the set image. It is the flowchart which showed the flow of the process of POC. It is the graph which showed the POC value. It is a schematic diagram explaining SAD. It is a schematic diagram explaining the ranging method by a stereo method. 3 is a flowchart showing the operation of the distance measuring apparatus according to the first embodiment. It is a flowchart which shows a search process. It is a flowchart which shows a similarity calculation process. It is the figure which showed the reference | standard window and reference window which are set to a reference | standard image and a reference image. It is a figure explaining a search range. 6 is a flowchart showing an operation of the distance measuring apparatus according to the second embodiment. 10 is a flowchart showing the operation of the distance measuring apparatus according to the third embodiment. 10 is a flowchart showing the operation of the distance measuring apparatus according to the fourth embodiment. It is a flowchart which shows the detail of the 3rd search process shown in FIG. It is the figure which showed the attention point set to a non-object area | region using the setting pattern 1 with time. It is the figure which showed the attention point set to a non-object area | region using the setting pattern 2 with time.

Explanation of symbols

1 1a 1b 1c Distance measurement device 13 Arithmetic processing device 14 Display device 20 Extraction unit 30 Area calculation unit 40 Processing change unit 50 Corresponding point search unit 60 Distance calculation unit 70 Display unit 110 Reference camera 120 Reference camera CP Attention point D1 Object region DR Search range O1 Center point TP Corresponding point W1 Base window W2 Reference window

Claims (9)

A distance measuring device that measures a distance to an object to be measured using images photographed using a plurality of photographing means,
An object region in which any one of the plurality of input images photographed at the same timing by the photographing unit is used as a standard image and another input image is used as a reference image, and the target object is photographed from the standard image Extracting means for extracting
Area calculating means for calculating the area of the object region extracted by the extracting means;
Search means for sequentially setting the attention points in the object region extracted by the extraction means, and searching for the corresponding points of each attention point from each reference image;
A distance calculating means for calculating a distance to the object based on a plurality of parallaxes between the attention point and the corresponding point;
The search means, wherein as the area of the area calculation object area calculated by means decreases, the disparity resolution is a parameter for determining the distance measurement accuracy of changes the search process to be higher,
The search means sets a reference window centered on the point of interest in the reference image, sets a reference window in the reference image, and shifts the reference window with the reference image, and resembles the images in both windows. The similarity calculation process for calculating the degree is repeatedly executed to search for corresponding points. As the area of the object region calculated by the area calculation unit decreases, the sizes of the reference window and the reference window are increased. A distance measuring device characterized by: A distance measuring device that measures a distance to an object to be measured using images photographed using a plurality of photographing means,
An object region in which any one of the plurality of input images photographed at the same timing by the photographing unit is used as a standard image and another input image is used as a reference image, and the target object is photographed from the standard image Extracting means for extracting
Area calculating means for calculating the area of the object region extracted by the extracting means;
Search means for sequentially setting the attention points in the object region extracted by the extraction means, and searching for the corresponding points of each attention point from each reference image;
A distance calculating means for calculating a distance to the object based on a plurality of parallaxes between the attention point and the corresponding point;
The search means, wherein as the area of the area calculation object area calculated by means decreases, the disparity resolution is a parameter for determining the distance measurement accuracy of changes the search process to be higher,
The search process includes a plurality of types of search processes with different parallax resolutions,
The search means selects a search process with a high parallax resolution as the area of the object region decreases,
In the search process, a reference window is set around the target point in the reference image, a reference window is set in the reference image, and the reference window is shifted with the reference image, and the images in both windows are similar. The corresponding points are searched by repeatedly executing the similarity calculation process for calculating the degree, and the images in both windows are obtained based on the phase components obtained by frequency decomposition of the images in the reference window and the reference window. A first search process that employs a similarity calculation process for determining the similarity of the second search process, and a second search process that has a lower parallax resolution than the first search process,
The distance measurement apparatus according to claim 2, wherein the similarity calculation process in the second search process is SAD . The similarity calculation process in the first search process is characterized in that frequency decomposition is performed using any of fast Fourier transform, discrete Fourier transform, discrete cosine transform, discrete sine transform, wavelet transform, and Hadamard transform. The distance measuring device according to claim 2 . 4. The distance measuring apparatus according to claim 3, wherein the similarity calculation process in the first search process is a phase only correlation method. The search means sequentially sets attention points in a non-object region other than the object region for a period from the end of the search for the corresponding point of each attention point set in the object region to the start of processing for the next frame. And search corresponding points of each attention point from each reference image,
Said distance calculating means, distance measurement according to any one of claims 1 to 4, characterized in that to calculate the distance of the point of interest from the the corresponding point of the target point and the target point set in the non-object region apparatus. The distance measuring apparatus according to claim 5 , wherein the searching unit sequentially sets a point of interest from the lower side to the upper side of the non-object region. A distance measuring device that measures a distance to an object to be measured using images photographed using a plurality of photographing means,
An object region in which any one of the plurality of input images photographed at the same timing by the photographing unit is used as a standard image and another input image is used as a reference image, and the target object is photographed from the standard image Extracting means for extracting
Area calculating means for calculating the area of the object region extracted by the extracting means;
Search means for sequentially setting the attention points in the object region extracted by the extraction means, and searching for the corresponding points of each attention point from each reference image;
A distance calculating means for calculating a distance to the object based on a plurality of parallaxes between the attention point and the corresponding point;
The search means, wherein as the area of the area calculation object area calculated by means decreases, the disparity resolution is a parameter for determining the distance measurement accuracy of changes the search process to be higher,
When the area of the object region calculated by the area calculation unit is smaller than a predetermined value, the search unit sets the parallax resolution to a sub-pixel level having a resolution higher than the pixel level of the reference image, and the object region If the area is larger than a predetermined value, the parallax resolution is set to the pixel level,
The search means sequentially sets attention points in a non-object region other than the object region for a period from the end of the search for the corresponding point of each attention point set in the object region to the start of processing for the next frame. And search corresponding points of each attention point from each reference image,
The distance calculation means calculates the distance of the target point from the target point set in the non-object region and the corresponding point of the target point,
The search means, the distance measuring apparatus characterized that you set the sequential target point from the lower side toward the upper side of the non-object region. The object is an automobile, motorcycles, bicycles, and among the human, the distance measuring apparatus according to any one of claims 1 to 7, characterized in that it comprises at least any one. A distance measurement method for measuring a distance to an object to be measured using images photographed using a plurality of photographing means,
An object region in which any one of the plurality of input images photographed at the same timing by the photographing unit is used as a standard image and another input image is used as a reference image, and the target object is photographed from the standard image An extraction step to extract
An area calculating step for calculating an area of the object region extracted by the extracting step;
A search step for sequentially setting a point of interest in the object region extracted by the extraction step, and searching for a corresponding point of each point of interest from each reference image;
A distance calculating step of calculating a distance to the object based on a plurality of parallaxes between the attention point and the corresponding point;
The search step, the as the area of the area calculation object area calculated by means decreases, the disparity resolution is a parameter for determining the distance measurement accuracy of changes the search process to be higher,
The searching step sets a reference window centered on the point of interest in the reference image, sets a reference window in the reference image, and shifts the reference window with the reference image, and resembles the images in both windows. The corresponding point is searched by repeatedly executing the similarity calculation process for calculating the degree, and the size of the reference window and the reference window is increased as the area of the object region calculated by the area calculation step decreases. A distance measurement method characterized by: