JP2006322853A - Distance measuring device, distance measuring method and distance measuring program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To acquire highly-reliable distance information. <P>SOLUTION: This distance measuring device 1 is equipped with an operation part 21 for operating the distance to an object positioned in an imaging visual field based on an image signal group outputted from an imaging part 10, a reliability acquisition part 22 for acquiring reliability of an operated value by the operation part 21, an interpolation part 31 for replacing an operated value whose reliability does not satisfy an evaluation criteria with a detected value from a radar 60, a detection range setting part 32 for setting a range corresponding to the operated value whose reliability does not satisfy the evaluation criteria as a detection range of the radar 60, and the radar 60 for detecting the distance to the object positioned in the detection range set by the detection range setting part 32. In the distance measuring device 1, since distance data acquired by replacing the operated value whose reliability does not satisfy the evaluation criteria with the highly-accurate detected value from the radar 60 are outputted, the highly-reliable distance information can be acquired. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a distance measuring device, a distance measuring method, and a distance measuring program for measuring a distance to an object located within a predetermined imaging field of view.

  In recent years, with the popularization of vehicles, various devices mounted on vehicles have been put into practical use. As such a vehicle-mounted device, there is a distance measuring device that measures an inter-vehicle distance between the host vehicle and a preceding vehicle and performs various processes such as alarm output based on the measured inter-vehicle distance.

  Conventionally, an optical distance measuring device having an image sensor has been proposed as such a distance measuring device (see Patent Document 1). The distance measuring device includes an imaging unit having two left and right lenses and an image sensor corresponding to each of the left and right lenses. Then, among each image output from the image sensor, an image signal that matches an arbitrary image signal in the other image is detected from one image, and based on the amount of movement in the detected image signal, Calculate the distance to the object using the principle of triangulation.

Japanese Examined Patent Publication No. 63-46363

  However, in the conventional distance measuring device, the reliability of the calculated distance value depends on the degree of matching between an arbitrary image signal and the detected image signal. That is, the distance value calculated based on the image signal having a low degree of matching with an arbitrary image signal has to be low in reliability. For this reason, when distance information including an operation value with low reliability is output from a conventional distance measuring device, there is a problem in that various processes such as alarm output cannot be performed accurately using such distance information. It was.

  The present invention has been made in view of the above-described drawbacks of the prior art, and an object thereof is to provide a distance measuring device capable of acquiring highly reliable distance information in an optical distance measuring device. .

  In order to solve the above-described problems and achieve the object, a distance measuring device according to the present invention has a predetermined imaging field of view, an imaging unit that generates an image signal group corresponding to the imaging field of view, and the image signal An arithmetic means for calculating a distance to an object located in the imaging field based on a group, and a detectable range including at least the imaging field, and detecting a distance to the object located in the detectable range A detection means; a reliability acquisition means for acquiring the reliability of the calculation value obtained by the calculation means; and the calculation value for which the reliability acquired by the reliability acquisition means does not satisfy an evaluation criterion. And interpolating means for outputting distance data in place of corresponding detection values of the detecting means.

  Further, in the distance measuring apparatus according to the present invention, the interpolation unit sets a range corresponding to the calculated value whose reliability does not satisfy the evaluation standard as a detection range of the detection unit, and the detection unit includes: A distance to an object located within the detection range is detected.

  Further, in the distance measuring device according to the present invention, the imaging means has the first image signal group imaged through the first optical path and the second image signal group imaged through the second optical path. And the arithmetic means detects an image signal that matches an arbitrary image signal of the first image signal group from the second image signal group, and the arbitrary image in the detected image signal A distance to an object located in the imaging field of view is calculated based on a movement amount from the signal.

  The distance measuring apparatus according to the present invention is characterized in that the reliability obtaining means obtains a degree of matching between the arbitrary image signal and the detected image signal as the reliability.

  The distance measuring device according to the present invention is characterized in that the imaging means includes a pair of optical systems and a pair of imaging elements that convert optical signals output from the pair of optical systems into electrical signals. To do.

  In the distance measuring device according to the present invention, the imaging means has a pair of light guide optical systems and an imaging region corresponding to each light guide optical system, and takes each optical signal guided by each light guide optical system. And an image pickup device for converting into an electric signal in the region.

  The distance measuring device according to the present invention is characterized in that the distance measuring device is mounted on a vehicle.

  The distance measuring method according to the present invention is a distance measuring method for calculating a distance to an object located within a predetermined imaging field, an imaging step for generating an image signal group corresponding to the imaging field, and the image signal A calculation step for calculating a distance to an object located in the imaging field based on a group, a reliability acquisition step for acquiring a reliability of a calculation value in the calculation step, and at least within a detectable range including the imaging field A detection step of detecting a distance to an object located at a position, and the calculated value in which the reliability in the reliability acquisition step does not satisfy an evaluation criterion is replaced with a detection value in the detection step corresponding to the calculated value And an interpolation step for generating distance data.

  The distance measurement program according to the present invention is a distance measurement program for calculating a distance to an object located in a predetermined imaging field of view, an imaging procedure for generating an image signal group corresponding to the imaging field of view, and the image signal A calculation procedure for calculating a distance to an object located in the imaging field based on a group, a reliability acquisition procedure for acquiring a reliability of a calculation value in the calculation procedure, and within a detectable range including at least the imaging field A detection procedure for detecting a distance to an object located at a position, and the calculated value in which the reliability in the reliability acquisition procedure does not satisfy an evaluation criterion is replaced with a detection value in the detection procedure corresponding to the calculated value And an interpolation procedure for generating distance data.

  According to the distance measuring device of the present invention, the reliability acquisition unit that acquires the reliability of the calculation value calculated by the calculation unit, and the calculation value that the reliability acquired by the reliability acquisition unit does not satisfy the evaluation criterion is detected. By providing interpolation means that replaces the detected value in the means, highly reliable distance information can be obtained, and safe driving support processing such as alarm output in various processing is accurately performed based on this distance information. Is possible. In addition, by using the distance measurement method and the distance measurement program according to the present invention, highly reliable distance information can be acquired.

  Hereinafter, a distance measuring device according to an embodiment of the present invention will be described with reference to the drawings, taking a distance measuring device mounted on a vehicle as an example. Various safe driving support processes are performed by other devices based on the distance information output by the distance measuring device. In addition, this invention is not limited by this embodiment. In the description of the drawings, the same parts are denoted by the same reference numerals.

  First, the distance measuring device according to the embodiment will be described. The distance measuring device according to the present embodiment obtains the reliability of a distance calculation value calculated using a predetermined image signal group, and calculates a distance calculation value whose reliability does not satisfy a predetermined evaluation criterion as a radar detection value. Replace with. FIG. 1 is a block diagram showing a schematic configuration of the distance information device according to the present embodiment.

  As shown in FIG. 1, the distance measuring device 1 according to the present embodiment has a predetermined imaging field of view, an imaging unit 10 that generates an image signal group corresponding to the imaging field of view, and an imaging unit 10 generates A distance calculation unit 20 that calculates the distance to an object located in the imaging field based on the image signal group, a control unit 30 that controls each process and each operation of each component constituting the distance measuring device, and distance information Output unit 40 that outputs various information including a storage unit 50 that stores various information including distance information, and a detectable range that includes at least an imaging field of view in imaging unit 10, and is located within the detectable range And a radar 60 for detecting the distance to the object. The imaging unit 10, the distance calculation unit 20, the output unit 40, the storage unit 50, and the radar 60 are electrically connected to the control unit 30. The distance calculation unit 20 includes a calculation unit 21, a reliability acquisition unit 22, and a memory 23. In addition, the control unit 30 includes an interpolation unit 31 having a detection range setting unit 32.

  The imaging unit 10 includes a right camera 11a and a left camera 11b. The right camera 11a and the left camera 11b output image signal groups corresponding to respective imaging fields. The right camera 11a and the left camera 11b include lenses 12a and 12b, image sensors 13a and 13b, analog / digital (A / D) converters 14a and 14b, and frame memories 15a and 15b, respectively. The lenses 12a and 12b collect light incident from a predetermined viewing angle. The image sensors 13a and 13b arranged corresponding to the lenses 12a and 12b are realized by a CCD or a CMOS, and detect the light transmitted through the lenses 12a and 12b and convert it into an analog image signal. The A / D converters 14a and 14b convert analog image signals output from the image sensors 13a and 13b into digital image signals. The frame memories 15a and 15b store digital image signals output from the A / D conversion units 14a and 14b, and a digital image signal group corresponding to one captured image is used as an image signal group corresponding to the imaging field of view as needed. Output.

  The distance calculation unit 20 processes the image signal group output from the imaging unit 10 to calculate the distance to the object located in the imaging field of view, and obtains the reliability of the calculation value by the calculation unit 21 And a memory 23 for storing the image signal group output from the imaging unit 10, the calculated value calculated by the calculation unit 21, the reliability acquired by the reliability acquisition unit 22, and the like.

The computing unit 21 computes the distance to the object located in the imaging field based on the image signal group output from the imaging unit 10 using the stereo method. The computing unit 21 detects an image signal that matches an arbitrary image signal in the left image signal group output from the left camera 11b from the right image signal group output from the right camera 11a, and detects the detected image signal. The distance is calculated by the principle of triangulation based on the amount of movement I from an arbitrary image signal. The movement amount described here indicates a parallax amount generally referred to. The calculating part 21 calculates | requires the distance R from the imaging part 10 to the vehicle C which is a target object using the following (1) Formula. In the equation (1), f is the focal length of the lenses 12a and 12b, and L is the width between the optical axes of the lenses 12a and 12b. Further, the movement amount I may be obtained based on the number of moved pixels and the pixel pitch.
R = f · L / I (1)
The calculation unit 21 calculates a distance R corresponding to each image signal, and the distance calculation unit 20 outputs calculation data 51 in which the calculation value and position information in the imaging field of view are associated with each other to the control unit 30. For simplicity, the parallel stereo is described here, but the optical axes intersect with an angle, the focal length is different, the positional relationship between the image sensor and the lens is calibrated, and so on. It is also possible to realize parallel stereo by arithmetic processing.

  Here, the calculation unit 21 calculates each image signal of the right image signal group located on the same straight line as an arbitrary straight line that passes through the image signal that is the calculation target of the left image signal group and the image signal that is the calculation target. By sequentially comparing, an image signal that most closely matches the image signal to be calculated is detected from the right image signal group. Specifically, a local region centering on the image signal to be calculated is provided in the left image signal group, and a region similar to this local region is provided in the right image signal group. Then, the local area having the highest degree of matching with the local area in the left image signal group is searched while scanning the local area in the right image signal group on the straight line described above. As a result of this search, the image signal located at the center of the local region having the highest degree of matching can be detected as the image signal most matching with the image signal to be calculated. The calculation unit 21 calculates an SSD (Sum of Squared Difference), which is a sum of squares of differences between pixel signals in the local region, as the degree of matching. The calculation unit 21 calculates the SSD every time the local area is searched, and detects the image signal positioned at the center of the local area having the SSD that is the minimum value as the image signal that most closely matches the image signal to be calculated. .

  The reliability acquisition unit 22 acquires the degree of matching calculated by the calculation unit 21 for each image signal as the reliability, and the distance calculation unit 20 associates the acquired reliability with position information in the imaging field of view. The degree data 52 is output to the control unit 30.

  The control unit 30 is realized by a CPU or the like that executes a processing program stored in the storage unit 50, and controls each process or operation of the imaging unit 10, the distance calculation unit 20, the output unit 40, the storage unit 50, and the radar 60. To do. The control unit 30 performs predetermined input / output control on information input / output to / from each of these components and performs predetermined information processing on the information.

  Based on the calculation data 51 and the reliability data 52 output from the distance calculation unit 20, the interpolation unit 31 calculates an operation value whose reliability does not satisfy a predetermined evaluation criterion in the calculation data 51. It replaces with the detection value of the radar 60 corresponding to the value and outputs it as distance data 54. Based on the calculation data 51 and the reliability data 52, the detection range setting unit 32 obtains a range corresponding to a calculation value whose reliability does not satisfy the evaluation criteria, and sets this range as the detection range of the radar 60. . The control unit 30 instructs the radar 60 to perform a detection process for detecting the distance of an object located within the detection range set by the detection range setting unit 32. As a result, the detection data 53 output from the radar 60 becomes the detection result of the detection range set by the detection range setting unit 32. Using the detection data 53 output from the radar 60, the interpolation unit 31 uses the detection value of the radar 60 corresponding to the calculation value to calculate a calculation value whose reliability does not satisfy the evaluation standard in the calculation data 51. Replaced. Since the detection value of the radar 60 is a highly accurate value, the distance data 54 in which the calculated value whose reliability does not satisfy the evaluation standard is replaced with the detection value of the radar 60 has the required reliability. It is thought to be a thing. The predetermined evaluation standard is determined based on the reliability required for the distance data 54 output from the distance measuring device 1.

  The output unit 40 is realized by a liquid crystal display, an organic electroluminescence display, or the like, and displays and outputs various display information such as an image captured by the imaging unit 10 in addition to the distance information. Moreover, the output part 40 is further provided with a speaker, and outputs various audio | voice information, such as a warning audio | voice which alert | reports that it approached the preceding vehicle C other than distance information.

  The storage unit 50 includes a ROM in which various types of information such as processing programs are stored in advance, and a RAM in which calculation parameters for each processing, various types of information output from various components, writing information, audio information, and the like are stored. . The storage unit 50 stores various types of information such as calculation data 51 and reliability data 52 output from the distance calculation unit 20, detection data 53 output from the radar 60, and distance data 54 output from the interpolation unit 31.

  The radar 60 detects the distance to an object located within the detection range set by the detection range setting unit 32 in the detectable range under the control of the control unit 30. The radar 60 transmits a predetermined transmitted wave, receives a reflected wave reflected by the surface of the object, and a distance from the radar 60 to the object that reflects the transmitted wave based on the transmitted state and the received state. And the direction in which the object is located. The radar 60 determines the distance based on the transmission angle of the transmitted wave, the incident angle of the reflected wave, the reception intensity of the reflected wave, the time from transmission of the transmitted wave to reception of the reflected wave, frequency change of the reflected wave, and the like. The distance from the measuring device 1 to the object reflecting the transmitted wave is detected. The radar 60 outputs detection data 53 in which a detection distance value to an object located within the detection range is associated with position information within the detection range to the control unit 30. The radar 60 transmits laser light, infrared rays, or millimeter waves as a transmission wave.

  Next, of the processing operations performed by the distance measuring device 1, processing operations until the distance calculation unit 20 outputs the distance data 54 will be described. FIG. 2 is a flowchart showing a processing procedure until the distance calculation unit 20 completes outputting the distance data 54 in the distance measuring apparatus 1.

  As illustrated in FIG. 2, first, the control unit 30 instructs the imaging unit 10 to perform an imaging process, and the imaging unit 10 performs an imaging process for imaging a predetermined imaging field of view under the control of the control unit 30. In step S102, the right camera 11a and the left camera 11b each output an image signal group.

  The control unit 30 instructs the distance calculation unit 20 to process the image signal group output from the imaging unit 10 and calculate a distance calculation process for calculating the distance to an object located in the imaging field of view. The distance calculation unit 20 receives an instruction from the control unit 30, and the calculation unit 21 sets a distance value corresponding to the image signal for each image signal of the image signal group output from the right camera 11a and the left camera 11b. A distance calculation process is performed (step S104). Further, the reliability obtaining unit 22 obtains the minimum value of the SSD value obtained when detecting as the image signal that most closely matches the image signal to be computed in the distance computing process in the computing unit 21 as the reliability. The degree acquisition process is performed (step S106).

  Thereafter, the calculation unit 21 determines whether or not the distance calculation process has been completed for all the image signals of the image signal group output from the imaging unit 10 (step S108). If the calculation unit 21 determines that the distance calculation process has not been completed for all image signals (step S108: No), the calculation unit 21 proceeds to step S104, and next performs the distance calculation process on the image signal to be calculated. To do. Further, when the calculation unit 21 determines that the distance calculation processing has been completed for all image signals (step S108: Yes), the distance calculation unit 20 sends the calculation data 51 and the reliability data 52 to the control unit 30. Are output (step S110).

  Next, in the control unit 30, the interpolation unit 31 refers to the reliability data 52, compares each reliability with the evaluation criterion, and determines whether there is a reliability that does not satisfy the evaluation criterion. (Step S112).

  When the interpolation unit 31 determines that there is a reliability that does not satisfy the evaluation criterion (step S112: Yes), the detection range setting unit 32 is based on position information corresponding to the reliability that does not satisfy the evaluation criterion. Then, a range in which the reliability that does not satisfy the evaluation standard is distributed is obtained, and this range is set as the detection range of the radar 60 (step S114). In the calculation data 51, each calculation value is associated with position information in the imaging field, and in the reliability data 52, the reliability in each calculation value and the imaging field corresponding to the calculation value are stored. Is associated with position information. For this reason, the range in which the reliability that does not satisfy the evaluation standard is distributed is a range corresponding to the calculated value having the reliability that does not satisfy the evaluation standard. Therefore, the detection range setting unit 32 sets the range corresponding to the calculated value having the reliability that does not satisfy the evaluation standard as the detection range of the radar 60.

  Thereafter, under the control of the control unit 30, the radar 60 performs detection processing for detecting the distance to the object located within the detection range set by the detection range setting unit 32 (step S116), and outputs detection data 53. To do. The interpolation unit 31 replaces the calculated value of the calculated data 51 whose reliability does not satisfy the evaluation standard with the detected value of the radar 60 in the detected data 53 (step S118), and replaces the replaced data with the distance data. It outputs as 54 (step S120).

  When the interpolation unit 31 determines that there is no reliability that does not satisfy the evaluation standard as a result of referring to the reliability data 52 (step S112: No), each calculation value of the calculation data 51 is necessary. Therefore, the calculation data 51 output from the calculation unit 21 is output as the distance data 54 (step S120).

  Next, each processing procedure shown in FIG. 2 will be specifically described. First, the distance calculation process (step S104) in the calculation unit 21 will be described. FIG. 3 is a diagram illustrating the distance calculation process, and is a diagram schematically illustrating the right image signal group output from the right camera 11a and the left image signal group output from the left camera 11b. As shown in FIG. 3, first, the computing unit 21 provides a local region B0 centered on the image signal 25b to be computed in the left image signal group 16b. In addition, in the right image signal group 16a, the arithmetic unit 21 also provides a local region A0 having the same range as the local region B0 centered on the reference signal 250 at the same position as the image signal 25b. Thereafter, the calculation unit 21 scans the local area on the same straight line as an arbitrary straight line that passes through the image signal 25b while sequentially calculating the SSD that is the matching degree. As a result, the SSD value change with respect to the movement amount change of the local region as shown by the curve l1 in FIG. 4 is obtained. For example, the image signal 25a located at the center of the local region A1 having the smallest SSD value is the image signal 25b. Is detected as the image signal that most closely matches. Thereafter, the calculation unit 21 calculates a movement amount I1 with reference to the reference signal 250, and calculates a distance value corresponding to the image signal 25b using the equation (1). Further, the reliability acquisition unit 22 acquires, as the reliability, the SSD that is the minimum value among the SSD values calculated in the distance calculation process for the image signal 25b (step S106). As a result of the distance calculation processing performed on all the image signals, as shown in FIG. 5, calculation data 51a in which each calculation value is associated with the position information of each image signal that was the calculation target, and FIG. As shown in FIG. 6, the distance calculation unit 20 outputs reliability data 52a in which the reliability in each calculation value is associated with the position information of each image signal that was the calculation target (step S110).

  The interpolation unit 31 refers to the reliability data 52a and determines whether each reliability of the reliability data 52a satisfies the evaluation criteria. In this case, since the SSD value is acquired as the reliability, it is considered that the lower the SSD value, the higher the matching degree and the higher the reliability. For this reason, the interpolation unit 31 determines whether each reliability exceeds a predetermined evaluation criterion S. For example, as shown in FIG. 4, the reliability S1 of the calculated value corresponding to the curve l1 is lower than the evaluation criterion S. For this reason, since the reliability S1 has the required reliability, it is not necessary to replace the calculated value corresponding to the reliability S1 with the detection value of the radar 60, and detection by the radar 60 is not required. On the other hand, the reliability S2 of the calculated value corresponding to the curve l2 shown in FIG. 4 exceeds the evaluation criterion S and does not have the required reliability. For this reason, since it is necessary to replace the calculated value corresponding to the reliability S2 with the detection value of the radar 60, it is necessary to detect the range in which the reliability S2 is distributed by the radar 60.

  In this way, the interpolation unit 31 determines whether or not each reliability satisfies the evaluation criterion, and obtains a low reliability region 52b in which the reliability that does not satisfy the evaluation criterion is distributed as shown in FIG. . Next, as shown in FIG. 7, the detection range setting unit 32 sets, as the detection range 63, a region corresponding to the low reliability region 52b obtained by the interpolation unit 31 in the detectable range 62 (step S114). ). Thereafter, the radar 60 performs a detection process for detecting a distance to an object located within the detection range 63 set by the detection range setting unit 32 (step S116). As a result, for example, as shown in FIG. 8, detection data 53 a including only the detection value of the detection range corresponding to the low reliability region is output from the radar 60. Thereafter, as illustrated in FIG. 8, the interpolation unit 31 calculates the calculation value of the low reliability region 51b in which the calculation values having the reliability that does not satisfy the evaluation criterion are distributed among the calculation values of the calculation data 51a. It replaces with the detected value of 53a (step S118), creates the distance data 54a and outputs it (step S120).

  As described above, in the distance measuring apparatus 1 according to the present embodiment, the reliability for each calculation value of the calculation data is acquired, and the calculation value whose reliability does not satisfy the evaluation criterion is replaced with the detection value of the radar 60. The distance information is output. The detection value of the radar 60 is higher in reliability than the calculation value calculated by the distance calculation unit 20. For this reason, according to the distance measuring device 1 according to the present embodiment, it is possible to acquire highly reliable distance information that satisfies a predetermined reliability. Therefore, various safe driving support processes based on the distance information are accurately performed. Can be performed.

  Further, in the present embodiment, the detection range setting unit 32 detects only the range in which the calculated values whose reliability does not satisfy the evaluation criteria from the calculation data output from the distance calculation unit 20 is distributed. It is set as. For this reason, in the present embodiment, compared to the case where the radar 60 detects the distance value for all the detectable ranges, the time required for the detection process can be shortened, and the highly reliable distance information. Can be acquired quickly.

  In the present embodiment, the SSD value is calculated as the degree of matching between the image signals and the SSD value is obtained as the degree of reliability. However, the present invention is not limited to this. Other values indicating the degree of matching may be calculated and acquired as the reliability. For example, SAD (Sum of Absolute Difference), which is the sum of absolute values of differences between image signals between local regions, or NCC (Normalized Cross Correlation), which is a normalized cross-correlation between image signals within the local region, is trusted. You may obtain it as a degree. When calculating the SAD value, the calculation unit 21 detects the image signal having the smallest SAD value as the most consistent image signal, and the reliability acquisition unit 22 acquires the SAD value corresponding to the image signal as the reliability. To do. The interpolation unit 31 determines that the SAD value, which is the reliability, falls below a predetermined evaluation criterion, and that the SAD value satisfies the evaluation criterion. If the SAD value exceeds the predetermined evaluation criterion, the SAD value is Judge that it does not meet. In addition, when calculating the NCC value, the calculation unit 21 detects the image signal having the maximum NCC value as the most consistent image signal, and the reliability acquisition unit 22 determines the NCC value corresponding to the image signal as the reliability. Get as. The interpolating unit 31 determines that this NCC value satisfies the evaluation criterion when the NCC value that is the reliability exceeds a predetermined evaluation criterion, and if the NCC value falls below the predetermined evaluation criterion, the NCC value is determined as the evaluation criterion. Judge that it does not meet.

  In the present embodiment, the case where the SSD value itself is compared with the evaluation criterion has been described. However, the present invention is not limited to this, and the Q value of the curve indicating the SSD value change with respect to the movement amount change of the local region is used as the reliability. It may be determined and compared with the evaluation standard based on whether or not the Q value exceeds a predetermined value as the evaluation standard. For example, when the Q value is Q3 equal to or less than the evaluation criterion Q0 as in the curve l3 shown in FIG. 9, the image signal detected in the distance calculation process is compared with the adjacent image signal, The degree of matching with a certain image signal is remarkably high. For this reason, the calculated value calculated based on the image signal detected in this case is considered to satisfy the required reliability. Therefore, the calculated value corresponding to the curve l3 has the required reliability, and it is not necessary to replace the calculated value with the detection value of the radar 60. On the other hand, when the Q value is Q4 exceeding the evaluation criterion Q0 as shown by the curve l4 shown in FIG. 9, the image signal detected in the distance calculation process is compared with the adjacent image signal, There is little difference in the degree of matching with a certain image signal. Here, when the Q value exceeds the evaluation standard Q0, the difference between the degree of matching of the detected image signal and the degree of matching of the adjacent image signal becomes minute when the accuracy of the processing condition in each process is considered. The detected image signal does not always have the highest degree of matching. For this reason, the calculation point based on the image signal corresponding to the Q value which is Q4 is likely to be low in reliability. Therefore, the detection range setting unit 32 determines that the calculation value corresponding to the curve 14 needs to be replaced with the detection value of the radar 60, and sets the detection range corresponding to this calculation value.

  Moreover, although the reliability acquisition part 22 demonstrated the case where the reliability degree was acquired as a reliability degree in this Embodiment, the reliability degree acquisition part 22 uses the color information contained in the image signal which is a calculation object as a reliability degree. You may get as In this case, the interpolation unit 31 determines that a region in which the color information of the image signals located in the vicinity is substantially the same as the low reliability region, and the detection range setting unit 32 determines the low reliability determined by the interpolation unit 31. The area is set as the detection range of the radar 60. This is because, as shown in FIG. 10, it is difficult for the computing unit 21 to detect from the other image signal group an image signal that matches the image signal included in the black region 19a having substantially the same color information. . That is, also in the other image signal group, the image signals included in the area corresponding to the black road area 19a have the same color information. Therefore, the image signal distributed in this area. The matching levels are almost the same, and there is no difference in the matching levels. As a result, the calculation unit 21 cannot detect an image signal that matches the image signal included in the black region 19a from the other image signal group. For this reason, in the present embodiment, based on the color information acquired as the reliability, an area where the color information of the image signals located in the vicinity is substantially the same, for example, the black area 19a and the white area 19b shown in FIG. May be obtained as a low reliability region. In this case, the detection range setting unit 32 sets the detection range of the radar 60 corresponding to the black region 19a and the white region 19b, and the interpolation unit 31 detects the calculation values in the black region 19a and the white region 19b. Replace with value. As a result, the distance measuring device 1 can output highly reliable distance data.

  In the present embodiment, the matching between the positional relationship in the image information group captured by the imaging unit 10 and the positional relationship in the detection range in the radar 60 is obtained in advance as follows, and each process is performed. For example, the distance measuring apparatus 1 performs an imaging process in the imaging unit 10 and a detection process in the radar 60 on an object whose shape is known, and the position of the known object in the imaging unit 10 and the position of the known object in the radar 60. Ask for. Thereafter, the distance measuring apparatus 1 obtains a relationship between the position of the known object in the imaging unit 10 and the position of the known object in the radar 60 using a least square method or the like, and in the image information group captured by the imaging unit 10 The positional relationship and the positional relationship in the detection range of the radar 60 are matched.

  Further, in the distance measuring device 1, even when the imaging origin of the imaging unit 10 and the detection origin of the radar 60 are misaligned, the distance from the imaging point and the detection point to the distance measuring device 1 is sufficiently large. In this case, it can be considered that the imaging origin and the detection origin almost overlap each other. Further, when the positional relationship in the image information group captured by the imaging unit 10 and the positional relationship in the detection range in the radar 60 are accurately matched, the deviation between the imaging origin and the detection origin is performed by geometric transformation. It is also possible to correct.

  Further, in the distance measurement device 1 according to the embodiment, the case where each radar detection point is located at a predetermined interval in the pixel row where each image signal is located has been described. However, each image signal output from the imaging unit 10 is not necessarily required. Each radar detection point does not necessarily exist in the pixel row where the position is located. In this case, the calculation control unit 21 uses the primary interpolation method or the like based on a plurality of laser detection points located in the vicinity of each image signal to be judged on the evaluation criterion, and each image to be judged and corrected. A laser interpolation value in the same pixel row as the signal may be obtained, and determination processing and correction processing may be performed using this interpolation value.

  Moreover, although the imaging part 10 provided with a pair of imaging element 13a, 13b corresponding to each of a pair of lenses 12a, 12b was demonstrated as an imaging part in this Embodiment, it is not restricted to this, As shown in FIG. In addition, an image pickup unit including a pair of light guide optical systems and an image pickup device having an image pickup region corresponding to each light guide optical system and converting an optical signal guided by each light guide optical system into an electric signal in each image pickup region 110 (for example, see JP-A-8-171151 by the present applicant). As illustrated in FIG. 11, the imaging unit 110 converts the light collected by the pair of mirrors 111a and 111b, the mirrors 112a and 112b corresponding to the mirrors 111a and 111b, the lens 112c, and the lens 112c into an analog image. An image sensor 113 that converts signals, an A / D converter 114 that changes an analog image signal output from the image sensor 113 into a digital image signal, and a frame memory 115 that stores the digital signal are provided. The mirrors 111a and 111b receive light from a subject such as the vehicle C, and the mirrors 112a and 112b reflect the light received by the mirrors 111a and 111b to the lens 112c. Therefore, an image corresponding to each optical system is formed on the image sensor 113. Therefore, the imaging unit 110 outputs an image 116 including an image 116a and an image 116b as shown in FIG. Based on such images 116a and 116b, the distance calculation unit 20 can calculate the distance value corresponding to each image signal.

  Further, in the present embodiment, the distance measuring device 1 including a plurality of cameras has been described. However, the present invention is not limited to this, and the present invention may be applied to a distance measuring device including a single camera. In this case, the distance calculation unit 20 uses, for example, a shape from focus method, a shape from defocus method, a shape from motion method, or a shape from shading method based on the image signal group output from the imaging unit. Calculate the distance within. Even in a distance measuring apparatus equipped with a single camera, by performing a processing procedure similar to the above-described processing procedure, a radar detection range is set based on the reliability corresponding to each calculation value, and the reliability is A calculated value that does not satisfy the evaluation criteria may be replaced with a radar detection value, and distance data may be output. The shape from focus method is a method for obtaining a distance from a focus position when the focus is best achieved. The shape from defocus method is a method of obtaining a relative blur amount from a plurality of images having different in-focus distances and obtaining a distance based on a correlation between the blur amount and the distance. The shape from motion method is a method for obtaining a distance to an object on the basis of the movement trajectory of a predetermined feature point in a plurality of temporally continuous images. The shape-from-shading method is a method for obtaining a distance to an object based on shading in an image, reflection characteristics of a target object, and light source information.

  The imaging unit 10 may constitute a so-called trinocular stereo camera structure, or may constitute a so-called four-eye stereo camera structure. When the imaging unit has a three-eye stereo camera structure or a four-eye stereo camera structure, a distance measurement device that can obtain a reliable and stable distance calculation result by performing a three-dimensional reconstruction process or the like is realized. It becomes possible to do. In particular, when a plurality of cameras are arranged so as to have a baseline length in two directions, even when a plurality of objects are arranged in a complicated configuration, three-dimensional reconstruction processing is possible, and the distance calculation result is stable. Can be obtained. In this case, it is possible to employ a multi-baseline method in which a plurality of cameras are arranged in the base length direction in one direction, and highly accurate distance measurement can be realized.

  In the present embodiment, the distance measuring apparatus 1 including the radar 60 has been described. However, instead of the radar 60, a light source such as a semiconductor laser element, a light emitting diode, or a laser diode that transmits infrared light or visible light is used. It may be a distance measuring device equipped with a detector realized by a light receiving element such as a photosensor that receives reflected light from an object, or may be reflected using light waves, microwaves, millimeter waves, or ultrasonic waves. A distance measuring device including a radar-type detector that measures a distance from a wave delay may be used.

  Further, as the present embodiment, the distance measuring device 1 mounted on the vehicle has been described as an example. However, the present invention is not limited to this, and may be applied to a distance measuring device mounted on another moving body. Further, the present invention is not limited to the distance measuring device mounted on the moving body, and may be applied to a distance measuring device that measures the distance within the detection range in a state of being fixed at a predetermined position, for example.

It is a block diagram showing a schematic structure of a distance measuring device concerning an embodiment. It is a flowchart which shows the process sequence until the output of distance data is completed in the distance measuring device shown in FIG. It is a figure explaining the distance calculation process shown in FIG. It is a figure explaining the reliability acquisition process shown in FIG. It is a figure which shows an example of the calculation data shown in FIG. It is a figure which shows an example of the reliability data shown in FIG. It is a figure explaining the detectable range of the radar shown in FIG. It is a figure explaining the process of the interpolation part shown in FIG. It is a figure explaining the reliability acquisition process shown in FIG. It is a figure which shows an example of the image signal group output from the imaging part 10 shown in FIG. It is a block diagram which shows the other example of schematic structure of the imaging part shown in FIG. It is a figure which shows an example of the image output from the imaging part shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Distance measuring device 10,110 Image pick-up part 11a Right camera 11b Left camera 12a, 12b, 112c Lens 13a, 13b, 113 Image pick-up element 14a, 14b, 114 A / D conversion part 15a, 15b, 115 Frame memory 16a Right image signal group 16b Left image signal group 18 Image signal group 19a Black region 19b White region 20 Distance calculation unit 21 Calculation unit 22 Reliability acquisition unit 23 Memory 25a, 25b Image signal 250 Reference signal 30 Control unit 31 Interpolation unit 32 Detection range setting unit 40 Output Unit 50 storage unit 51, 51a calculation data 51b, 52b low reliability region 52, 52a reliability data 53, 53a detection data 54, 54a distance data 60 radar 62 detectable range 63 detection range 111a, 111b, 112a, 112b mirror 116 , 11 a, 116b image C vehicle

Claims (9)

Imaging means having a predetermined imaging field of view and generating an image signal group corresponding to the imaging field of view;
A computing means for computing a distance to an object located in the imaging field based on the image signal group;
Detection means having a detectable range including at least the imaging field, and detecting a distance to an object located in the detectable range;
Reliability obtaining means for obtaining the reliability of the calculated value by the computing means;
Interpolating means for outputting distance data by replacing the calculated value that the reliability acquired by the reliability acquiring means does not satisfy the evaluation criteria with a detection value of the detecting means corresponding to the calculated value;
A distance measuring device comprising: The interpolating unit sets a range corresponding to the calculated value whose reliability does not satisfy the evaluation criterion as a detection range of the detecting unit,
The distance measuring apparatus according to claim 1, wherein the detecting unit detects a distance to an object located within the detection range. The imaging means generates a first group of image signals captured through a first optical path and a second group of image signals captured through a second optical path;
The calculation means detects an image signal that matches an arbitrary image signal of the first image signal group from the second image signal group, and a movement amount of the detected image signal from the arbitrary image signal The distance measuring device according to claim 1, wherein a distance to an object located in the imaging field of view is calculated based on   The distance measurement apparatus according to claim 3, wherein the reliability acquisition unit acquires a degree of matching between the arbitrary image signal and the detected image signal as the reliability. The imaging means includes
A pair of optical systems;
A pair of image sensors that convert optical signals output by the pair of optical systems into electrical signals;
The distance measuring device according to any one of claims 1 to 4, further comprising: The imaging means includes
A pair of light guiding optical systems;
An imaging device that has an imaging region corresponding to each light guide optical system and converts an optical signal guided by each light guide optical system into an electrical signal in each imaging region;
The distance measuring device according to any one of claims 1 to 4, further comprising:   The distance measuring apparatus according to claim 1, wherein the distance measuring apparatus is mounted on a vehicle. In a distance measurement method for calculating the distance to an object located within a predetermined imaging field of view,
An imaging step for generating an image signal group corresponding to the imaging field;
A calculation step of calculating a distance to an object located in the imaging field based on the image signal group;
A reliability acquisition step of acquiring the reliability of the calculated value in the calculation step;
A detection step of detecting a distance to an object located within a detectable range including at least the imaging field;
An interpolation step for generating distance data by replacing the calculated value in which the reliability in the reliability obtaining step does not satisfy the evaluation criteria with a detection value in the detection step corresponding to the calculated value;
A distance measurement method comprising: In a distance measurement program that calculates the distance to an object located within a predetermined imaging field of view,
An imaging procedure for generating an image signal group corresponding to the imaging field;
A calculation procedure for calculating a distance to an object located in the imaging field based on the image signal group,
A reliability acquisition procedure for acquiring the reliability of the calculated value in the calculation procedure;
A detection procedure for detecting a distance to an object located within a detectable range including at least the imaging field;
An interpolation procedure for generating distance data by replacing the calculated value in which the reliability in the reliability obtaining procedure does not satisfy the evaluation criterion with a detection value in the detection procedure corresponding to the calculated value;
The distance measurement program characterized by including.

Images