JP7152880B2 - Ranging device, ranging system and ranging method - Google Patents

Description

The present invention relates to a ranging device, a ranging system and a ranging method.

A distance measuring device that uses pixel cost and connection cost in stereo matching of images obtained by a stereo camera is known (for example, Patent Document 1). The pixel cost represents the degree of dissimilarity between a reference pixel included in one of the left and right images and a comparison pixel included in the other image. The connection cost represents the amount of positional deviation between the comparison pixels corresponding to adjacent reference pixels when one comparison pixel is selected for each reference pixel.

JP 2015-114269 A

Such a distance measuring device calculates a combination of corresponding points (hereinafter also referred to as a "path") so as to minimize the total cost, which is the sum of the pixel cost and the connection cost representing the amount of change in parallax. , the parallax may be determined. However, in the prior art, it cannot be said that sufficient studies have been made on how temporary noise on images due to vehicle wipers or bad weather such as snowfall affects parallax determination. We found that when there is temporal noise in the image, the prior art incorrectly computes the pixel cost, and this incorrect pixel cost is used in the path computation, resulting in an incorrect disparity determination. It has been found that there is a problem that

The present disclosure has been made to solve at least part of the above-described problems, and can be implemented as the following forms or application examples.
[Mode 1]
According to one aspect of the present disclosure, a ranging device (30) is provided. This rangefinder measures a pixel value of a reference pixel (La), which is one pixel extracted from a reference image (PL, PL2), which is one of a plurality of images, and the reference pixel value of the plurality of images. Dissimilarity between the reference pixel and the comparison pixel obtained by the reciprocal of the degree of similarity with the pixel value of the comparison pixel (Ra, Rb, Rc) extracted from the comparison image (PR, PR2), which is an image different from the image. of the reference pixel or the comparison pixel using the dissimilarity calculation unit (32) that calculates the degree of dissimilarity for each of the reference pixels, and the obtained dissimilarity of the plurality of comparison pixels to the reference pixel. A disturbance estimator (34) for calculating the probability (P ₀ ) that at least one of the pixels is a disturbance for each of the reference pixels; a weighting unit (36) for calculating each pixel; and a corresponding point setting unit (38) for setting corresponding points, which are the comparison pixels corresponding to each of the reference pixels, using the weighted dissimilarity. Prepare. A distance to an object included in the reference image is determined using the amount of deviation between the reference pixel and the corresponding point.

According to one aspect of the present disclosure, a ranging device (30) is provided. This distance measuring device includes: a reference pixel (La) that is one pixel extracted from a reference image (PL, PL2) that is one of a plurality of images, and the reference image among the plurality of images. A dissimilarity between the reference pixel and the comparison pixel is calculated from the pixel value of the comparison pixel (Ra, Rb, Rc) extracted from the comparison image (PR, PR2), which is another image, for each of the reference pixels. a dissimilarity calculator (32); using the obtained dissimilarity of the plurality of comparison pixels to the reference pixel, the probability that at least one of the reference pixel or the comparison pixel is a disturbance ( P ₀ ) for each of the reference pixels; and a weighting unit (36) for calculating the weighted dissimilarity, which is the dissimilarity weighted by the probability, for each of the reference pixels. ); and a corresponding point setting unit (38) that sets corresponding points, which are the comparison pixels corresponding to each of the reference pixels, using the weighted dissimilarity. A distance to an object included in the reference image may be determined using the amount of deviation between the reference pixel and the corresponding point.

According to the distance measuring device of the above aspect, the dissimilarity is calculated as a weighted dissimilarity considering the probability of the disturbance calculated by the disturbance estimating section. As a result, it is possible to reduce the occurrence of an erroneous parallax calculation under the influence of disturbance, and reduce the problem of correcting a correct parallax to an erroneous parallax.

All of the plurality of constituent elements of the above-described embodiments of the present invention are not essential, and in order to solve some or all of the above problems, or some or all of the effects described in this specification In order to achieve the above, it is possible to appropriately change, delete, replace with new other components, and partially delete the limited content for some of the plurality of components. In addition, in order to solve part or all of the above problems or achieve part or all of the effects described in this specification, the technical features included in one aspect of the present invention described above It is also possible to combine some or all of the technical features included in the other forms of the present invention described above to form an independent form of the present invention.

FIG. 1 is a block diagram showing a schematic configuration of an automatic driving vehicle equipped with a ranging system; FIG. 4 is a flowchart showing distance image generation processing executed by the distance measurement system; Schematic diagrams of a left image and a right image captured by a ranging system. FIG. 10 is an explanatory diagram showing a result of mapping disparity and dissimilarity by a dissimilarity calculation unit; Explanatory drawing showing the result which the disturbance estimation part calculated|required the probability that it is a disturbance for every pixel. FIG. 4 is an explanatory diagram showing dissimilarities weighted by a weighting unit; FIG. 4 is an explanatory diagram schematically showing a method of setting corresponding points by a corresponding point setting unit; FIG. 4 is an explanatory diagram of processing for generating a parallax image from acquired left and right images;

A. First embodiment:
An automatically driven vehicle 100 shown in FIG. 1 is a vehicle that automatically executes all of drive unit control, brake control, and steering angle control. The automatic driving vehicle 100 includes a ranging system 20 and a control section 50 . The distance measuring system 20 has a function of capturing an image of the front of the vehicle and calculating the distance to the object shown in the captured image. The ranging system 20 includes an image acquisition section 22 and a ranging device 30 . In this specification, "automatic driving" means driving in which drive unit control, brake control, and steering angle control are all automatically performed without the driver performing a driving operation of the vehicle.

The image acquisition unit 22 is a camera capable of acquiring a plurality of images with an imaging range in front of the automatic driving vehicle 100 . In this embodiment, a stereo camera that acquires a pair of captured images is used as the image acquisition unit 22 . The image acquisition units 22 are arranged at the same height with respect to the direction of gravity of the automatically driven vehicle 100 and at positions separated to the left and right of the traveling direction of the automatically driven vehicle 100 . The image acquisition unit 22 is preferably a color camera capable of distinguishing the color of the object in order to distinguish white lane markings and yellow lane markings, for example.

The distance measuring device 30 is a camera ECU, and is a circuit including a CPU and a memory. By executing a computer program stored in a non-volatile storage medium, the distance measuring device 30 calculates a dissimilarity calculation unit 32, a disturbance estimation unit 34, a weighting unit 36, a corresponding point setting unit 38, and a distance image. The functions of the generating unit 39 and the smoothing processing unit 40 are realized respectively. The distance measuring device 30 uses the left and right images captured by the image acquisition unit 22 as a reference image and a comparison image to generate parallax images. A parallax image is also called a "distance image" because it is an image representing the distance to an object existing in the reference image. A part of the functions of the distance measuring device 30 may be realized by a hardware circuit.

The control unit 50 is composed of a plurality of ECUs. The control unit 50 acquires the result of image recognition processing from the distance measuring system 20, and executes drive unit control, brake control, and steering angle control for avoiding a collision.

As shown in FIG. 2, each part of the distance measuring system 20 of this embodiment executes each process of the distance image generation process. In step S10, the image acquisition unit 22 acquires a reference image and a comparison image using the left and right cameras. Control from step S<b>20 to step S<b>70 is executed by control of each part of distance measuring device 30 of distance measuring system 20 .

In step S20, the dissimilarity calculator 32 calculates the dissimilarity from the pixel values of the reference pixel and the comparison pixel. More specifically, in step S22, the dissimilarity calculator 32 selects one reference pixel from the acquired reference image. In step S24, the dissimilarity calculator 32 selects one comparison pixel having the same height in the Y direction as the selected reference pixel. In step S25, the dissimilarity calculator 32 calculates the parallax of the comparison pixel with respect to the reference pixel. In step S26, the dissimilarity calculator 32 calculates the dissimilarity between the reference pixel and the comparison pixel. In step S28, the dissimilarity calculator 32 confirms whether or not all comparison pixels in the comparison image have been selected for one reference pixel. If the dissimilarity calculation unit 32 does not determine that all comparison pixels have been selected (S28: NO), the process returns to step S24. When determining that all the comparison pixels have been selected (S28: YES), the dissimilarity calculation unit 32 confirms whether or not all the reference pixels in the reference image have been selected in step S29. If it is not determined that all the reference pixels have been selected (S29: NO), the process of the dissimilarity calculator 32 returns to step S22. When all the reference pixels have been selected (S29: YES), that is, when the dissimilarity calculation for all the reference pixels is completed, the processing by the dissimilarity calculation unit 32 is terminated, and the processing of step S30 is executed. be done.

In step S30, the disturbance estimating unit 34 uses the lowest dissimilarity and the second lowest dissimilarity among the dissimilarities calculated in step S20 in equations (1) to (5) described later, A probability P ₀ that the reference pixel is a disturbance (hereinafter also simply referred to as “probability P ₀ ”) is calculated for each reference pixel. In step S40, the weighting unit 36 calculates a weighted dissimilarity by multiplying the dissimilarity calculated in step S20 by the probability _P0 .

In step S50, the corresponding point setting unit 38 sets corresponding points for each reference pixel using the weighted dissimilarity calculated in step S40 and the connection cost. In step S60, the distance image generator 39 creates a parallax image as a distance image from the reference image and the comparison image. In step S<b>70 , the smoothing processing unit 40 performs processing for smoothing the parallax using a median filter to obtain a parallax image in which the parallax is corrected. With the above, the distance image generation processing by the distance measuring system 20 of the present embodiment ends.

A function of calculating a parallax value by the dissimilarity calculation unit 32 will be described with reference to FIG. FIG. 3 shows an example in which the left image PL and the right image PR captured by the image acquisition unit 22 are sampled. A left image PL schematically shows a dot-matrix image captured by the left camera of the image acquisition unit 22 . Similarly, the right image PR schematically shows a dot-matrix image captured by the right camera of the image acquisition section 22 . The resolution of these images is expressed in pixels. The left image PL and the right image PR in FIG. 3 are composed of 14×8 pixels for easy understanding of the technology, but are not limited to this, and images composed of pixels larger than 14×8 pixels. may be Note that the X direction in the drawing represents the horizontal direction of the image, and the Y direction represents the gravity direction of the image.

The left and right cameras of the image acquisition unit 22 are arranged at the same height in the direction of gravity as described above. Therefore, the left image PL and the right image PR are the same height in the gravitational direction, and are taken from respective positions separated from each other in the horizontal direction to the left and right, respectively, in front of the automatic driving vehicle 100 in the same direction. Become. Therefore, the heights of the line numbers of the reference image and the line numbers of the comparison image match each other. Here, in the right image PR, the wiper Dp mounted on the automatic driving vehicle 100 of this embodiment is imaged. This wiper Dp is a so-called disturbance imaged only in the right image PR. This wiper Dp is hereinafter also referred to as "disturbance Dp". In this specification, a disturbance means an object imaged by only one of the left and right cameras of the image acquisition unit 22 .

The dissimilarity calculator 32 has a function of extracting a reference pixel from a reference image, extracting a comparison pixel from a comparison image, and calculating a dissimilarity from the pixel values of the reference pixel and the comparison pixel. The dissimilarity calculator 32 of this embodiment uses the left image PL as a reference image. Hereinafter, the left image PL will also be referred to as a reference image PL. The dissimilarity calculator 32 selects one pixel as a reference pixel from among the pixels forming the reference image PL, and repeats this in order to select all the pixels of the reference image PL. The order of selection of reference pixels may be any order. In the present embodiment, the dissimilarity calculation unit 32 first selects the upper left pixel as the reference pixel, and sequentially shifts it to the right by one pixel. After selecting the reference pixels up to the right end, the dissimilarity calculation unit 32 shifts one pixel downward in the Y direction and continues in the same manner from the left end. In this way, all pixels are selected as reference pixels. In this embodiment, when a reference pixel is selected, eight surrounding pixels surrounding the reference pixel are also selected. A rectangular area surrounding the reference pixel and surrounding pixels is called a window. In this embodiment, the window is a square with 3 pixels on each side. One side of the window is not limited to 3 pixels, and may be 2 pixels or 4 pixels or more.

The dissimilarity calculator 32 of this embodiment uses the right image PR as a comparison image. Hereinafter, the right image PR will also be referred to as a comparison image PR. Note that the distance measurement system 20 may be configured such that the right image PR is the reference image and the left image PL is the comparison image. Peripheral pixels included in the window are also selected when the comparison pixels are selected. Pixels to be selected as comparison pixels are pixels in the comparison image PR whose position in the Y direction, ie, height, matches that of the reference pixel. Template matching (also called “block matching”) is used to extract comparison pixels from the comparison image PR. In this embodiment, NCC (Normalized Cross Correlation) is used to calculate the similarity between windows. Note that SAD (Sum of Absolute Difference) or SSD (Sum of Squared Difference) may be used to calculate the degree of similarity. The degree of similarity is a quantified degree of similarity between pixel values of windows containing pixels to be compared. In this embodiment, the degree of similarity is expressed in a range from 0 to 1, and the closer to 1, the more similar. In this specification, a pixel value is a density value (also called a "luminance value") that corresponds to the brightness of a pixel. RGB density values are used as pixel values in this embodiment. When the captured image is a color image, the value of luminance Y represented by the following equation (a), in which each density value of RGB is expressed as R, G, and B, respectively, may be used.
Y=0.299R+0.587G+0.144B (a)

The dissimilarity calculation unit 32 determines a parallax value, which is the amount of deviation between the reference pixel and the comparison pixel, along with the selection of the comparison pixel. A parallax value is expressed in units of pixels. In the reference image PL of FIG. 3, an example is shown in which a pixel positioned 6 pixels from the left in the X direction and 4 pixels from the top in the Y direction is selected as the reference pixel La. Hereinafter, X=n denotes that the position in the X direction is the position of the n-th pixel from the left. The comparative image PR in FIG. 3 shows an example in which the comparative pixel Ra positioned at X=2, the comparative pixel Rb positioned at X=6, and the comparative pixel Rc positioned at X=10 are selected. . For the comparison pixel Rb, the parallax value with respect to the reference pixel La is zero. On the other hand, the comparison pixel Rc has a parallax value of −4 pixels with respect to the reference pixel La, and the comparison pixel Ra has a parallax value of 4 pixels with respect to the reference pixel La. In this way, the dissimilarity calculation unit 32 determines the parallax value between the target reference pixel and each comparison pixel having the same position in the Y direction for each reference pixel.

The function of calculating the dissimilarity by the dissimilarity calculator 32 will be described with reference to FIG. The dissimilarity calculator 32 maps the result of the dissimilarity between the reference pixel and the comparison pixel as nodes in a grid pattern. The horizontal axis of FIG. 4 represents the position of the reference pixel for each pixel in the X direction of the reference image PL. The vertical axis in FIG. 4 represents the parallax value of the comparison pixel corresponding to the reference pixel. Although the number of pixels in the X direction and the number corresponding to the parallax value are mapped to the nodes, only some of the nodes are shown in FIG. In the present embodiment, the dissimilarity is a quantified degree of dissimilarity between pixel values of a window containing one reference pixel and a window containing one comparison pixel, and is also called pixel cost. The degree of dissimilarity is not limited to the degree of dissimilarity between windows, and may be a numerical value of the degree of dissimilarity between pixels. The dissimilarity is expressed as 1/(similarity) by a formula that is the reciprocal of the similarity. A higher dissimilarity value indicates a higher degree of dissimilarity.

The dissimilarity calculator 32 maps the dissimilarity between each reference pixel in the reference image PL and each comparison pixel for each parallax. In the example of FIG. 4, a reference pixel having a height of four pixels from the top in the Y direction is selected. The dissimilarity mapping as exemplified in FIG. 4 is created for eight groups corresponding to the height of the reference pixel, that is, eight rows in the example of the reference image PL.

The function of the disturbance estimator 34 included in the distance measuring device 30 of this embodiment will be described. The disturbance estimation unit 34 uses the dissimilarity calculated by the dissimilarity calculation unit 32 to calculate the probability that at least one of the reference pixel and the comparison pixel is the disturbance of the reference pixel in the reference image PL. Compute for each. First, the disturbance estimating unit 34 uses the following equations (1) and (2) to calculate the reliability C _MSM by the matching score measure (MSM) method and the reliability C by the PKRN (peak-ratio naive) method _{Calculate PKRN} .

c1 represents the lowest dissimilarity value, _c2 represents the _second smallest dissimilarity value, and σ represents the standard deviation of the calculated dissimilarity. This reduces the impact of statistical outliers (also called outliers). Next, the disturbance estimation unit 34 calculates probabilities using the following equations (3) and (4).

_N1 and _N2 are fixed values that are determined according to characteristics such as the camera and usage conditions. In this embodiment, values of N ₁ =10 and N ₂ =1 are used. Finally, the disturbance estimator 34 calculates the probability P _o that at least one of the reference pixel and the comparison pixel is the disturbance using the following equation (5) using P _MSM and P _PKRN . The probability _Po is calculated for each reference pixel.

FIG. 5 illustrates part of the _probability P 0 calculated for each reference pixel in a reference pixel row having a height of 4 pixels from the top in the Y direction of the reference image PL. In this way, the probability that at least one of the reference pixel and the comparison pixel is the disturbance is calculated using the reliability that reduces the influence of the statistical outlier. By introducing this probability of being a disturbance and calculating the dissimilarity, a parallax in which the influence of the disturbance is reduced can be obtained.

The function of the weighting unit 36 included in the distance measuring device 30 of this embodiment will be described with reference to FIG. Although the number of nodes corresponding to the number of pixels in the X direction and the number of parallax values are mapped, only some of the nodes are shown in FIG. The weighting unit 36 has a function of calculating a weighted dissimilarity. The weighted dissimilarity is a dissimilarity obtained by weighting the dissimilarity calculated by the dissimilarity calculator 32 by the probability P ₀ calculated by the disturbance estimator 34 . The weighted dissimilarity is obtained by multiplying the dissimilarity of each mapped reference pixel (see FIG. 4) by the probability P ₀ (see FIG. 5) for each column in the X direction. For example, all dissimilarities in column X1 of FIG. 6 are multiplied by the probability P ₀ =1 for X=1 in FIG. 5 are multiplied by the probability P ₀ =0.1 of X=2. That is, between disparities of mapped nodes, the multiplied probabilities P ₀ are the same. Each node in FIG. 6 shows the weighted dissimilarity in the state in which the dissimilarity is multiplied by the probability _P0 , that is, the state in the process of calculation, in order to facilitate understanding of the technology. Also, FIG. 6 shows an example in which a reference pixel having a height of 4 pixels from the top in the Y direction is selected. The weighting unit 36 calculates the weighted dissimilarity with respect to the reference pixels of all heights. That is, the calculation of the weighted dissimilarity as illustrated in FIG. 6 is performed for the height of the reference pixel, that is, for eight rows corresponding to eight groups in the example of the reference image PL.

Functions of the corresponding point setting unit 38 and the distance image generation unit 39 included in the distance measuring device 30 of the present embodiment will be described with reference to FIG. In FIG. 7, as in FIG. 6, illustration of some nodes is omitted to facilitate understanding of the technology. The corresponding point setting unit 38 uses the Viterbi algorithm to set corresponding points, which are comparison pixels corresponding to each of the reference pixels, from the weighted dissimilarity calculated by the weighting unit 36 . The Viterbi algorithm is a type of dynamic programming algorithm that searches for plausible paths. The corresponding point setting unit 38 sets corresponding points for all heights of the reference image and sets corresponding points for all reference pixels.

The Viterbi algorithm of this embodiment employs a multi-path Viterbi Algorithm. The Viterbi algorithm of this embodiment uses the above-described weighted dissimilarity and a parallax change amount (hereinafter also referred to as “connection cost”) that is a positional deviation amount between nodes corresponding to adjacent reference pixels in the X direction. be done. The number of connection cost candidates to be calculated by the Viterbi algorithm is the number of nodes of pixels adjacent to one node in the X direction. The corresponding point setting unit 38 calculates the sum of the weighted dissimilarity and the connection cost. The corresponding point setting unit 38 sets the combination of nodes that minimizes the sum as the combination of corresponding points at one height of the reference image. The corresponding point setting unit 38 repeats this for all heights of the reference image, and sets corresponding points for all reference pixels in the reference image.

FIG. 7 illustrates node points NP1 to NP5 and four arrows D1 to D4 schematically representing connections from one node to adjacent nodes. Numerical values representing connection costs corresponding to the respective arrows are schematically represented near the respective arrows. The connection cost is set lower as the amount of change in parallax is smaller. In this embodiment, the connection cost between adjacent nodes with the same disparity value is zero, and the connection cost increases by one for each pixel disparity in the disparity values of the connected nodes. In the example of FIG. 7, the connection cost corresponding to the arrow D1 is the connection cost between the node NP1 and the node NP2 with the same parallax value, which is zero. The connection cost of arrow D2 between node NP2 and node NP3 is also zero. Therefore, the total cost, which is the sum of the weighted dissimilarity and the connection cost from the node NP1 to the node NP3 via the node NP2, is 2.5, except for the portion omitted from the drawing. On the other hand, in the arrow D3 between the node NP1 and the node NP4, the connection cost is 2 because the parallax value is shifted by 2 pixels between the node NP1 and the node NP4. Therefore, the total cost, which is the sum of the weighted dissimilarity and the connection cost from the node NP1 to the node NP5 via the node NP4, except for the portion omitted from the drawing, is 4.6.

The corresponding point setting unit 38 thus calculates the sum of the weighted dissimilarity and the connection cost for all the combinations of the nodes, and selects the combination that minimizes the sum of the weighted dissimilarity and the connection cost as the corresponding point. set. In FIG. 7, the combination from node NP1 to node NP3 via node NP2 has the smallest sum of weighted dissimilarity and connection cost among all combinations. At this time, the corresponding point setting unit 38 sets a combination of pixels corresponding to the nodes NP1, NP2, and NP3 as corresponding points. The corresponding point setting unit 38 sets corresponding points for each height of the reference image, and sets corresponding points for all reference pixels. As described above, according to the ranging system 20 of the present embodiment, the multipath Viterbi algorithm performs optimization by combining corresponding points that minimize the sum of the weighted dissimilarity and the parallax change amount. The larger the value of the parallax change amount, the higher the possibility of extracting an erroneous parallax. The ranging system 20 of the present embodiment uses the parallax change amounts indicated by D1 to D4 in the figure for setting corresponding points. Therefore, a combination of corresponding points can be obtained while reducing the possibility of extracting an erroneous parallax. Also, by applying the existing multi-pass Viterbi algorithm, it is possible to set the weighted dissimilarity of the most probable sequence by a simpler method. In addition, compared to the aspect of setting corresponding points only by the weighted dissimilarity without using the parallax change amount, there is a problem that an erroneous parallax is determined by selecting an erroneous weighted dissimilarity due to the influence of disturbance. The occurrence can be suppressed.

The distance image generator 39 creates a parallax image as a distance image from the parallax values corresponding to the set corresponding points. The smoothing processing unit 40 uses a median filter to correct the parallax images by smoothing the parallax values of the parallax images. The smoothing unit 40 of this embodiment utilizes the constant time weighted median filter. A weighted median filter or an unweighted median filter may be used as the median filter. By introducing the median filter, it is possible to reduce the noise included in the parallax values on the parallax image and more clearly extract the edge portion, which is the boundary of the object.

An image generated by distance image generation processing by the distance measurement system 20 will be described with reference to FIG. At the top of FIG. 8, the reference image PL2 and the comparison image PR2 obtained by the image obtaining section 22 are shown. The comparative image PR2 shows a wiper mounted on the automatic driving vehicle 100 (hereinafter also referred to as "disturbance Dp2"). The dissimilarity calculator 32 calculates the dissimilarity from the reference image PL2 and the comparison image PR2. The disturbance estimator 34 calculates a probability _P0 of disturbance. The image represented by the calculated disturbance probability _P0 is the image PC1 in the middle of FIG. In the image PC1, the higher the probability _P0 of the disturbance, the closer the image is to black. The lower right of the image PC1 is the periphery of the position corresponding to the disturbance Dp in the right image PR2, and the area DR with a high probability of being the disturbance _P0 is shown.

The weighting unit 36 calculates a weighted _{dissimilarity} by weighting the dissimilarity with the probability P0 of disturbance. A corresponding point setting unit 38 sets corresponding points, and a distance image generating unit 39 generates a parallax image. The parallax image is smoothed by a smoothing processor 40 and generated as a distance image DI. In the vicinity of the lower right of the distance image DI, the portion affected by the disturbance corresponding to the area DR is removed. Thus, according to the ranging system 20 of the present embodiment, the dissimilarity is calculated as a weighted dissimilarity considering the probability _P0 of the disturbance calculated by the disturbance estimator 34 . This makes it possible to obtain a parallax image in which the influence of disturbance is reduced. Therefore, it is possible to reduce the occurrence of the problem of calculating an incorrect parallax under the influence of disturbance, and reduce the problem of correcting the correct parallax to an incorrect parallax.

B. Other embodiments:
B1. Alternative Embodiment 1:
In the distance measuring system 20 of the above-described embodiment, the function of each unit is shown as an example in which it operates without being restricted by the environment in which it is used. Alternatively, the processes of the disturbance estimating section 34, the weighting section 36, and the corresponding point setting section 38 may be executed according to the environment in which the ranging system is used. As described above, the disturbance means an object imaged by only one of the left and right cameras of the image acquisition unit. For example, the following cases are assumed as environments in which the influence of disturbance is considered to be large in vehicle travel. This includes using the distance measuring system at night, activating the wipers provided in the autonomous vehicle, turning on the lights provided in the autonomous vehicle, and heavy snowfall. When the ranging system corresponds to at least one of these environments, the disturbance estimator, the weighting unit, and the corresponding point setting unit perform each process. Each process may not be executed when it does not apply. In such an aspect, the processing load can be reduced in an environment where the influence of disturbance is small.

B2. Alternative Embodiment 2:
In the ranging system 20 of the above-described embodiment, the disturbance estimator 34 uses the minimum dissimilarity value c1 and the _second smallest dissimilarity value c2 to calculate the reliability _{C MSM} and the reliability The probability P _o of being a disturbance was obtained using the degree C _PKRN . On the other hand, the calculation of reliability by the ranging system, for example, uses only the lowest dissimilarity, or uses the lowest dissimilarity and the second lowest dissimilarity, and the third lowest dissimilarity It is also possible to obtain the probability P _o of disturbance by various combinations that reduce the influence of outliers, such as a mode using degrees.

B3. Alternative Embodiment 3:
In the distance measuring system 20 of the above-described embodiment, the corresponding point setting unit 38 determines a combination of corresponding points using the parallax change amount, which is the amount of positional deviation between nodes corresponding to reference pixels adjacent in the X direction. On the other hand, it is also possible to set corresponding points based only on the weighted dissimilarity without considering the parallax change amount. In such an aspect, it is preferred that a smaller weighted dissimilarity is selected.

B4. Alternative Embodiment 4:
In the ranging system 20 of the above-described embodiment, the corresponding point setting unit 38 sets the combination that minimizes the sum of the weighted dissimilarity and the connection cost as corresponding points. On the other hand, the ranging system further weights either one of the weighted dissimilarity and the connection cost according to the environment in which the ranging system is used, for example. may be set as the corresponding points, or a combination that minimizes the sum of only the weighted dissimilarities may be set as the corresponding points.

B5. Alternative Embodiment 5:
In the distance measuring system 20 of the above-described embodiment, the smoothing processing unit 40 uses a median filter for the parallax images created by the distance image generation unit 39, and smoothes the parallax values of the parallax images to smooth the parallax images. to correct. On the other hand, the distance image generation unit may generate the parallax images after smoothing is performed by the smoothing processing unit before the generation of the parallax images. Further, the distance measuring system 20 may be configured without a smoothing processor and without using a median filter. In such a mode, it is preferable to reduce the influence of noise on the density value of each pixel of the image to be generated, such as smoothing with various filters such as a weighted average filter and a moving average filter.

B6. Alternative Embodiment 6:
In the above-described embodiment, a mode in which the distance measuring system 20 is provided in the automatically driving vehicle 100 has been described as an example. On the other hand, the ranging system may be provided in a manually operated vehicle. "Manual operation" includes operations for controlling the drive unit (depressing the accelerator pedal), operations for controlling the brakes (depressing the brake pedal), and operations for controlling the steering angle (rotation of the steering wheel). ) means the driving performed by the driver.

The present disclosure is not limited to the embodiments described above, and can be implemented in various configurations without departing from the scope of the present disclosure. For example, the technical features in the embodiments corresponding to the technical features in the respective modes described in the Summary of the Invention column may be used to solve some or all of the above problems, or Substitutions and combinations may be made as appropriate to achieve part or all. Also, if the technical features are not described as essential in this specification, they can be deleted as appropriate.

30 ranging device, 32 dissimilarity calculating unit, 34 disturbance estimating unit, 36 weighting unit, 38 corresponding point setting unit, 39 range image generating unit, 40 smoothing processing unit, La reference pixel, P ₀ probability of being disturbance, PL , PL2 reference image, PR, PR2 comparison image, Ra, Rb, Rc comparison pixel

Claims (8)

A ranging device (30),
A pixel value of a reference pixel (La), which is one pixel extracted from a reference image (PL, PL2), which is one of a plurality of images, and an image other than the reference image among the plurality of images The dissimilarity between the reference pixel and the comparison pixel obtained by the reciprocal of the similarity with the pixel value of the comparison pixel (Ra, Rb, Rc) extracted from the comparison image (PR, PR2) is a dissimilarity calculator (32) that calculates each of
Using the obtained dissimilarity of the plurality of comparison pixels to the reference pixel, the probability (P ₀ ) that at least one of the reference pixel and the comparison pixel is a disturbance is calculated for each of the reference pixels. a disturbance estimator (34) that calculates
a weighting unit (36) for calculating a weighted dissimilarity, which is the dissimilarity weighted by the probability, for each of the reference pixels;
a corresponding point setting unit (38) that sets corresponding points, which are the comparison pixels corresponding to each of the reference pixels, using the weighted dissimilarity;
Determining a distance to an object included in the reference image using the amount of deviation between the reference pixel and the corresponding point;
rangefinder. The distance measuring device according to claim 1,
The disturbance estimating unit obtains the probability using the lowest dissimilarity (c ₁ ) and the second lowest dissimilarity (c ₂ ) among the plurality of dissimilarities. distance device. The distance measuring device according to claim 1 or claim 2,
The corresponding point setting unit selects the comparison pixels one by one for each of the reference pixels, and sets the parallax value of the one comparison pixel selected with respect to the one reference pixel to the one reference pixel. determining a combination of the corresponding points corresponding to each of the reference pixels using a parallax change amount obtained by a difference from a parallax value of another comparison pixel selected with respect to another adjacent reference pixel ; distance device. The distance measuring device according to claim 3,
The distance measuring device, wherein the corresponding point setting unit determines the combination so that the sum of the weighted dissimilarity of the selected comparison pixel and the parallax change amount is minimized. The rangefinder according to any one of claims 1 to 4,
The distance measuring device further includes a smoothing processing unit that corrects the deviation amount by smoothing the deviation amount for each of the reference pixels. The rangefinder according to any one of claims 1 to 5,
The rangefinder is mounted on a vehicle (100),
At least one of a first case in which the range finder is used at night, a second case in which the wipers provided in the vehicle are operated, and a third case in which the lights provided in the vehicle are turned on. the disturbance estimating section, the weighting section, and the corresponding point setting section are executed when any of the first case, the second case, and the third case The distance measuring device does not execute each of the processes of the disturbance estimating section, the weighting section, and the corresponding point setting section when the case of (1) does not apply. an image acquisition unit (22) for acquiring a plurality of images;
and the ranging device according to any one of claims 1 to 6,
A ranging system (20). A distance measurement method for a distance measurement device including an image acquisition unit that acquires a plurality of images,
A pixel value of a reference pixel that is one pixel extracted from a reference image that is one of the plurality of images, and a comparison image that is an image different from the reference image among the plurality of images calculating, for each of the reference pixels, a degree of dissimilarity between the reference pixel and the comparison pixel, which is obtained from the reciprocal of the degree of similarity with the pixel value of the comparison pixel;
calculating the probability that at least one of the reference pixel and the comparison pixel is a disturbance for each of the reference pixels, using the obtained dissimilarity of the plurality of comparison pixels to the reference pixel; ,
calculating a weighted dissimilarity, which is the dissimilarity weighted by the probability, for each of the reference pixels;
and setting a corresponding point, which is the comparison pixel corresponding to each of the reference pixels, using the weighted dissimilarity.

Images