JP2001067484A - Stereo monitor device for vehicle's outside - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the reliability of vehicle's outside monitoring more by stably outputting a proper number of distance data irrelevantly to monitoring environment. SOLUTION: A stereo matching circuit 5 of the stereo monitor device which monitors the external state of a vehicle specifies a pixel block, having correlation with luminance characteristics for every pixel block in one of a couple of picked-up images of the scene outside the vehicle, in the other image to calculate the parallax of the pixel block. A decision circuit 6 compares the quantity of variation in luminance between adjacent pixels in the pixel block having its parallax calculated with a luminance decision threshold to specify a pixel block having a large quantity of variation in luminance and also calculates the parallax regarding the specified pixel block as distance data. A microcomputer 9 adjusts the luminance decision threshold according to the number of pieces of distance data calculated by the decision circuit 6.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a stereo outside monitoring apparatus for recognizing a situation outside a vehicle based on distance data calculated from a stereo image pair.

[0002]

2. Description of the Related Art In recent years, attention has been paid to a stereo exterior monitoring device using a pair of in-vehicle cameras (stereo cameras) incorporating a solid-state imaging device such as a CCD. In this monitoring device, a pixel block having a correlation with a pixel block in one image is specified in the other image (stereo matching), and parallax, which is a relative shift amount between the two, is calculated. From the calculated parallax for each pixel block, distance data is calculated using the principle of triangulation, and an object (for example, a preceding vehicle) in the captured image is recognized using an image recognition technique. Then, according to the recognition result, vehicle control such as alerting the driver and downshifting is performed.

In such a distance measurement technique, it is important to verify the edge state of a pixel block that has undergone stereo matching in order to improve the reliability of the parallax calculated for each pixel block. As one of the verification methods, there is a method of evaluating a luminance change amount of a pixel block in a horizontal direction (horizontal direction). For example, for a certain matched pixel block, the luminance change amount of a pair of pixels adjacent in the horizontal direction is compared with a predetermined luminance determination threshold value, and the number of pixel pairs having a change amount equal to or larger than this threshold value is determined. Count. Then, the disparity relating to the pixel blocks having the count number equal to or greater than the predetermined number is output as the distance data, and the disparity relating to the pixel blocks smaller than that is excluded as having low reliability. Since the outside monitoring is performed based on the distance data output in this way, if a certain number of distance data are not output, the necessary information amount is insufficient, so that it is difficult to perform appropriate monitoring.

[0004]

In a conventional vehicle exterior monitoring apparatus, a fixed brightness determination threshold value is set at an initial adjustment stage, and is uniformly applied regardless of an actual monitoring environment. . Therefore, the value may not always be appropriate depending on the monitoring environment. As a result, there is a problem that the distance data required for monitoring outside the vehicle is not output depending on the monitoring environment or, conversely, the number of outputs becomes excessive and noise-like distance data increases.

Accordingly, an object of the present invention is to automatically adjust a luminance determination threshold value according to a monitoring environment, and to stably output an appropriate number of distance data regardless of the monitoring environment, thereby enabling monitoring outside the vehicle. Is to further improve the reliability.

[0006]

In order to solve this problem, the present invention calculates distance data for each pixel block having a plurality of pixels based on a pair of images obtained by capturing a scene outside the vehicle. In a stereo outside monitoring device that monitors the situation outside the vehicle, for each pixel block in one image, by specifying a pixel block having a correlation with the luminance characteristic of the pixel block from the other image,
Calculation means for calculating the parallax of the pixel block, and for the pixel block for which the parallax has been calculated by the calculation means, by comparing the amount of change in luminance between adjacent pixels in the pixel block with a luminance determination threshold value, A specifying unit that specifies a pixel block having a large amount of change in luminance and calculates parallax of the specified pixel block as distance data; and a luminance determination threshold value according to the number of distance data calculated by the specifying unit. And an adjusting means for adjusting the distance.

In the above configuration, the adjusting means includes a luminance determination threshold value such that the number of distance data calculated by the specifying means approaches a target value defining the number of distance data to be calculated by the specifying means. Is preferably adjusted.

Further, it is desirable to further provide a filter means for judging the reliability of the distance data calculated by the specifying means. In this case, the filter unit removes the distance data determined to be low in reliability and outputs the distance data determined to be high in reliability as effective distance data used for monitoring a situation outside the vehicle. Then, the adjusting means adjusts the luminance determination threshold value according to the number of distance data removed by the filter means.

The adjusting means adjusts the luminance determination threshold value so that the number of distance data removed by the filter means approaches a target value which defines the number of distance data to be removed by the filter means. You may do so.

Further, the adjusting means includes, as parameters, a target value defining the number of distance data to be specified by the specifying means and a target value defining the number of distance data to be removed by the filter means, and Based on an evaluation formula including, as variables, the number of distance data calculated by the specifying means and the number of distance data removed by the filter means, the appropriateness of the luminance determination threshold is calculated, and the calculated appropriateness is calculated. It is preferable to adjust the luminance determination threshold based on

On the other hand, the above filter means may include a group filter. This group filter groups adjacent pixel blocks having distance data of close values, calculates the area of the grouped pixel blocks, and calculates the distance data according to the size of the calculated area. This is a filter that outputs or removes.

On the other hand, the above-mentioned filter means may include a pairing filter. This pairing filter outputs distance data relating to any of the pixel blocks when a plurality of pixel blocks in one image are correlated with the same pixel block in the other image, This is a filter for removing distance data relating to the remaining pixel blocks.

It is preferable that the specifying means specifies the distance data based on a change in luminance between pixels adjacent in the horizontal direction of the image.

In this case, the specifying means may specify, as distance data, a parallax relating to a pixel block having a predetermined number or more of adjacent pixel pairs having a change in luminance equal to or greater than the luminance determination threshold.

[0015]

FIG. 1 is a block diagram of a vehicle exterior monitoring apparatus according to the present embodiment. FIG. 2 is a diagram for explaining the data processing method according to the present embodiment. A stereo camera that captures a scene in front of the vehicle is composed of a pair of cameras 1 and 2 having a built-in image sensor such as a CCD. These cameras 1 and 2 are mounted near the rearview mirror at predetermined intervals in the vehicle width direction of a vehicle such as an automobile. The main camera 1 is mounted on the right side in the traveling direction of the vehicle, and outputs a reference image. On the other hand, the sub camera 2 is attached on the left side in the traveling direction, and outputs a comparison image. In a state where the camera pairs 1 and 2 are synchronized, the analog image output from each of the cameras 1 and 2 is converted into a predetermined luminance gradation (for example, 256 gray scales) by the A / D converter 3. Is converted to a digital image. These digitized images (stereo images) are subjected to luminance correction, geometric conversion of the images, and the like in the correction circuit 4. Usually, since the mounting positions of the stereo cameras 1 and 2 have an error to some extent, a shift caused by the error exists in the left and right images. Therefore, geometric transformation such as rotation or translation of the image is performed using affine transformation or the like. This guarantees that the horizontal line of the reference image matches the horizontal line of the comparison image, which is a premise in stereo matching described later. After the image processing as described above, the horizontal direction is calculated from the output signal of the main camera 1 by 512.
Reference image data having 200 pixels in the vertical direction is generated. Also, from the output signal of the sub-camera 2, comparative image data having the same vertical length as the reference image and a longer horizontal length than the reference image is generated (for example, 640 pixels in the horizontal direction and 200 pixels in the vertical direction). Pixel). The reference image data and the comparison image data are stored in the image data memory 7.

Next, the stereo matching circuit 5 and the determination circuit 6 calculate distance data based on the reference image data and the comparison image data. This distance data is calculated in pixel block units composed of 4 × 4 pixels. First, when one pixel block (for example, the pixel block PB1 shown in FIG. 2) in the reference image data is to be investigated, the stereo matching circuit 5 compares the pixel blocks having a correlation with the luminance characteristics of the target pixel block. Search and specify image data. As is well known, the distance between the objects displayed in the stereo image appears as a parallax in the stereo image, that is, a horizontal shift amount with respect to the positions of the reference image and the comparison image. Therefore, when trying to specify a pixel block having a correlation with the target pixel block PB1 (stereo matching), a comparative image on a horizontal line (epipolar line) having the same value as the j coordinate value of the target pixel block PB1 may be searched. The stereo matching circuit 5 sets 1 on this epipolar line.
By shifting each pixel, the correlation of all the comparison pixel blocks within a predetermined pixel range on the epipolar line is evaluated.

The stereo matching circuit 5 calculates a city block distance for each pixel block existing on the epipolar line. By calculating the city block distance, it is possible to accurately detect the correlation between the pixel blocks without increasing the amount of calculation. FIG. 3 is a diagram for explaining a method of calculating a city block distance. “A _n” and “b _n” shown in the figure indicate the luminance values of the pixels constituting the pixel block. The city block distance CB between the reference pixel block PB1 and one pixel block PBX on the epipolar line can be calculated according to the following equation.

[0018]

(Equation 1)

As can be seen from the above equation, the more similar the luminance characteristics of two pixel blocks (ie, the greater the correlation), the smaller the city block distance CB becomes. If both are exactly the same, this value CB becomes It becomes 0. In this way, the city block distances CB for a plurality of pixel blocks on the epipolar line are obtained, and basically, the pixel block PB3 having the minimum value is determined to be a pixel block having a correlation with the reference pixel block PB1. to decide. Then, the parallax between the reference pixel block PB1 and the specified pixel block PB3 is obtained (the parallax is 8 pixels in the example of FIG. 3). With the above method, the parallax for all the pixel blocks in the reference image data is calculated. The details of the stereo matching including the hardware configuration for calculating the city block distance are disclosed in Japanese Patent Application Laid-Open No. Hei 5-11409, and should be referred to if necessary.

The determination circuit 6 evaluates the horizontal luminance edge (luminance change amount) of the pixel block with reference to the reference image data, thereby obtaining the stereo matching circuit 5.
Performs a filtering process on the parallax calculated in.
FIG. 4 is a diagram for explaining a method of evaluating a luminance edge in the horizontal direction for a pixel block. In a pixel block, which is a unit for calculating distance data, the luminance change amount (absolute value) ΔP _n (1 ≦ n ≦) of a pair of pixels adjacent in the horizontal direction.
16) is calculated. However, the left-most pixel column (P _{1 1,}
For P ₁₂ , P ₁₃ , and P ₁₄ ), the luminance change amount ΔP is calculated from the rightmost pixel column of the pixel block adjacent to the left. Next, out of these 16 luminance change amounts ΔP _n , the number of those whose luminance judgment threshold value is DCDXth or more is counted. The brightness determination threshold value DCDXth is appropriately set in the adjustment unit 17 by feedback control. Then, parallax for a pixel block in which the number of luminance change amounts ΔP equal to or larger than the DCDX threshold value is less than four is excluded, and only parallax for a pixel block larger than that is output as distance data. The amount of change in luminance in the horizontal direction is small,
Even if a pixel block having little luminance characteristic is matched in the stereo matching circuit 5, the reliability of the parallax is often low. Therefore, the reliability of the distance data is improved by removing the parallax calculated for such a pixel block in the determination circuit 6. The distance data output from the determination circuit 6 through such processing is stored in the distance data memory 8. The determination circuit 6 counts the output number N of the distance data, and notifies the microcomputer 9 of the number.

The recognition unit 10 which is a functional block in the microcomputer 9 is used to determine the road shape (straight line or curve curvature).
And three-dimensional objects (running vehicles) in front of the vehicle. This recognition is based on the image data stored in the image data memory 7,
This is performed based on the data that has passed through the filter processing unit 13 out of the distance data stored in the distance data memory 8. Further, sensor information from a vehicle speed sensor and a steering angle sensor (not shown), navigation information, and the like are also referred to. Then, when it is determined that an alarm is necessary based on these recognition results, the alarm device 11 such as a monitor or a speaker is operated to call the driver's attention. Further, by controlling the control device 12 as necessary, vehicle control such as downshifting of an automatic transmission (AT), suppression of engine output, or operation of a brake is executed.

The filter processing unit 13 verifies the reliability of the distance data of the pixel block that has been subjected to the stereo matching in the stereo matching circuit 5 based on a predetermined condition, and outputs only the distance data meeting the condition. I do. Thereby, since noise-like distance data can be efficiently removed, recognition accuracy in the recognition unit 10 at the subsequent stage can be improved. The filter processing unit 13 in the present embodiment includes a pairing filter 14 and a group filter 15.

The pairing filter 14 detects a contradiction in the positional relationship between the pixel blocks determined to be correlated in the stereo matching circuit 5, and excludes the distance data relating to the contradiction. Specifically, when a plurality of pixel blocks in the reference image data are correlated with the same pixel block in the comparison image, the distance data relating to any of the pixel blocks is output, and the remaining data is output. The distance data related to the pixel block is removed. From the principle of the stereo method, one pixel block in the comparison image data is correlated (matched) with one pixel block in the reference image data. However, in a situation where a plurality of similar objects such as a pylon row of roads, a plurality of lighting lamps or a plurality of utility poles are projected, as shown in FIG. 2, the plurality of pixel blocks PB1 and PB2 are replaced with the same pixel block PB.
It may match B3. Thereby, an incorrect parallax is calculated in the stereo matching circuit 5.

Therefore, the correlation destination of the pixel block PB3 is
When overlapping with a plurality of pixel blocks PB1 and PB2, only one of the distance data is valid. From a statistical point of view, those with large distance data (parallax) have a high probability of being false information. Therefore, the smallest distance data value (parallax 3 of the pixel block PB2) is output as effective distance data, and the other data (parallax 8 of the pixel block PB1) is removed as invalid distance data. However, if both of the distance data are equal to or smaller than a certain value, they are output as effective distance data. The fact that the number of distance data removed by the pairing filter 14 is large indicates that there is much mismatching. Note that a specific processing procedure in the pairing filter is described in JP-A-7-15.
Since it is disclosed in JP-A-2914, refer to it if necessary.

The pairing filter 14 outputs only effective distance data to the group filter 15 from a large number of distance data relating to a certain stereo image stored in the distance data memory 8 by the above-described procedure. At the same time, the number of distance data invalidated for the image is counted, and the total number is notified to the calculation unit 16 as the invalid distance data number Pn.

The distance data output from the pairing filter 14 is further filtered in the group filter 15. The group filter 14 groups adjacent pixel blocks having distance data of close values, calculates the area of the grouped pixel blocks, and outputs or removes the distance data according to the size of the area. It is a filter to be performed. Distance data generated due to mismatch due to noise or the like has a characteristic that it is significantly different from surrounding distance data, and the area of a group having a value similar to that of the distance data is relatively small. Therefore, the group filter 15 detects the area of the distance data group having the same value in the distance data area, and excludes the distance data having a small area. As described above, since the distance data is calculated in units of 4 × 4 pixel blocks, the distance data area is 128 × 50 (the unit is a pixel block). FIG. 5 is a diagram showing an output state of distance data in a part of such a distance data area. As filter processing in the group filter 15, first, for distance data adjacent in the vertical and horizontal directions, those whose change amounts are within a predetermined threshold are grouped (for example, the groups A and B shown in FIG. 5). ). Note that this threshold is changed according to the size of the distance data. Next, the size of the area of the group is detected, and a group having an area larger than a predetermined size (for example, a two-pixel block) is set as an effective group. The group filter 15 outputs distance data belonging to the effective group as effective distance data. On the other hand, distance data (isolated distance data) belonging to a group having an area equal to or smaller than a predetermined size is determined to have low reliability and is removed. In the example of FIG. 5, two distance data (the values of which are 5,
30) is removed as invalid distance data. Since a specific processing procedure in the group filter is disclosed in Japanese Patent Application Laid-Open No. 10-285558,
Please refer to it if necessary.

The group filter 15 recognizes only the effective distance data among the many distance data output from the pairing filter 14 by the above procedure.
Output to At the same time, the number of distance data invalidated by the filter 15 is counted, and the total number is calculated as the invalid distance data number Gn by the arithmetic unit 16.
Notify.

The calculating section 16 calculates the distance data output number N calculated by the determination circuit 6, the invalid distance data number Pn calculated by the pairing filter 14, and the invalid distance data number Gn calculated by the group filter 15. , The suitability E of the luminance determination threshold value DCDXth is evaluated. In the present embodiment, the suitability E is calculated by the following evaluation formula.

[0029]

E = (N / TGT1-1) + (Yn / TGT2-1) * 2 + (N
/ Ng / TGT3-1) Yn = N-Pn-Gn Ng = N-Pn

In the above equation, Yn is the number of effective distance data, that is, the number of distance data output from the group filter 15, in other words, the number of distance data finally used by the recognition unit 10. Ng is the pairing filter 14
Is the number of outputs from. Further, TGT1, TGT2, and TGT3 are target values. That is, the first target value TGT1 is
The second target value TGT2 is a value that defines the target of the number of distance data to be output from the group filter 15. Further, the third target value TGT3 is determined by the group filter 15
Is a value indicating the ratio of the distance data to be removed in. The values of the target values TGT1, TGT2, and TGT3 need to be appropriately set based on the output characteristics of distance data through experiments, simulations, and the like. Note that these target values are changed according to the brightness of the imaging environment (day and night). As an example, in the daytime, the first target value TGT1 is 3000, the second target value TGT2 is 1600, and the third target value TGT3 is 33%.
Is set to At night, the first target value
TGT1 is set to 2000, the second target value TGT2 is set to 1000, and the third target value TGT3 is set to 33%. Number of distance data N, P
If n and Gn correspond to the target values TGT1, TGT2, and TGT3, the suitability is determined to be 0, that is, the output state is determined to be the most appropriate. Since the outside recognition is performed based on the effective distance data, the output number Yn of the effective distance data is the most important value. Therefore, the term relating to the number of effective distance data outputs Yn is multiplied by the weight coefficient 2.

The adjusting unit 17 calculates a luminance determination threshold value DCDXth based on the suitability E calculated by the calculating unit 16.
Is adjusted within the range of 3 to 6. Theoretically, if the suitability E is 0
If not (that is, if the number of output distance data is not equivalent to the target value), the brightness determination threshold DCDXth may be changed. However, considering the stability of the control, if the suitability E is in the range of -0.3 to 0.3, the luminance determination threshold DC
Do not change DXth. On the other hand, when the suitability E is 0.3 or more, the luminance determination threshold value DCDXth is updated to a value larger than the current value by adding a constant value a to the current value. However, the upper limit of the luminance determination threshold DCDXth is 6. When the suitability E is −0.3 or less, the luminance determination threshold value DCDXth is updated to a value smaller than the current value by subtracting the constant value a from the current value. However, the brightness determination threshold DCDXth has a lower limit of 3, and thus the brightness determination threshold D
CDXth is fed back to the decision circuit 6.

Through such feedback control,
The output number of distance data and the number of removals thereof approach each other so as to become a target value corresponding thereto. For example, if attention is paid only to the distance data output number N (assuming that the values of other evaluation terms are unchanged), if the distance data output number N is equivalent to the first target value TGT1 (ie, the distance The appropriateness ratio N / TGT1 of the data output is 1), and the appropriateness E calculated by Expression 2 is 0. Therefore, the distance data output number N
Is determined to be appropriate, and the luminance determination threshold value DCDXth
Is not changed. On the other hand, if the distance data output number N is larger than the target value TGT1, the above-described appropriate rate N / TG
Since T1 is larger than 1, the suitability E is positive. Therefore, the luminance determination threshold DCDXth is changed to a value larger than the current value (when E> 0.3). As a result, since the evaluation criterion of the luminance change amount for the pixel block in the determination unit 6 becomes strict, the number N of distance data outputs decreases (approaches the first target value TGT1). On the other hand, when the distance data output number N is smaller than the target value TGT1, the suitability E becomes negative, and the luminance determination threshold DCDXth is changed to a value smaller than the current value (when E <−0.3). As a result, the evaluation criterion of the luminance change amount for the pixel block in the determination unit 6 is relaxed, so that the number N of distance data outputs increases (approaches the first target value TGT1).

The same applies to the effective distance data output number Yn.
(And its appropriate rate Yn / TGT2), and the effective distance data output number Yn through feedback control.
Approaches the target value TGT2. Since the information input to the recognition unit 10 for monitoring outside the vehicle is effective distance data, the number Yn of the distance data is equal to the number N of other distance data,
It is more important than Pn and Gn. From such a viewpoint, as described above, the evaluation term relating to the number of effective distance data Yn in Expression 2 is multiplied by the weight coefficient 2.

The third target value TGT3 defines the target value of the distance data removal rate in the group filter 15. When attention is paid only to the rightmost evaluation term in Equation 2, the removal rate of the group filter 15 (Ng / Gn)
Is equal to the third target value TGT3, the suitability E becomes zero. Therefore, in this case, it is determined that the removal rate is appropriate, and the luminance determination threshold DCDXth is not changed.

As described above, in the feedback control of the luminance determination threshold value DCDXth according to the present embodiment, the appropriateness of the luminance determination threshold value DCDXth is determined by substituting the number of outputs and the number of removals related to the distance data into the evaluation formula. E is being evaluated. Then, the luminance determination threshold value DCDXth is increased or decreased so that these aptitudes converge in the range of -0.3 to 0.3. This makes it possible to stably obtain effective distance data that is relatively reliable for monitoring outside the vehicle without depending on the monitoring environment. Then, such effective distance data can be obtained by the number corresponding to the target value. As a result, the accuracy and stability of monitoring outside the vehicle can be further improved. The reason will be described assuming the following two situations.

(1) A situation in which the brightness of a captured image is different (for example, during the daytime and at night) In a monitoring environment where a brightness edge clearly appears in an image such as during the daytime, a relatively large brightness determination threshold value is set. DCD
Even if Xth is set, it is possible to obtain as many distance data as necessary for monitoring outside the vehicle. However, if it is used at night, a sufficient number of distance data cannot be obtained, and it becomes difficult to perform appropriate outside monitoring (lack of information).
Conversely, a relatively small luminance determination threshold D assuming nighttime
If CDXth is used during the day, excessive distance data will be output (excessive information amount). In the present embodiment, based on the evaluation formula focusing on the suitability Yn / TGT1 of the number of outputs of the effective distance data (that is, multiplying by the weighting factor 2),
The suitability E of DXth is evaluated. Therefore, an appropriate number of effective distance data can be obtained without depending on the brightness of the imaging environment.

However, when the suitability of the luminance determination threshold value DCDXth is evaluated using only the target value TGT1 (or TGT2) that defines the target value of the number of outputs of the effective distance data (or distance data), the mismatch of the pixel block may be reduced. Due to the increase, only noise-like distance data (effective distance data) may increase. Therefore, the filter processing unit 1
The suitability E of the luminance determination threshold DCDXth is evaluated based on an evaluation formula that also takes into account the target value of the number of distance data to be removed in Step 3. The target values that are the basis for this adjustment are:
It is set appropriately for each monitoring situation based on the output characteristics of distance data obtained by experiments, simulations, and the like. Therefore, even during nighttime when a luminance edge is unlikely to occur, a luminance determination threshold DC that increases the number of effective distance data while removing noise-like distance data due to mismatching.
DXth can be set automatically.

(2) A situation where many luminance edges exist Even if the brightness of the monitoring environment is the same, the distance data may be excessively calculated depending on the scene. For example, in a situation where a forest or a hedge is present around the traveling path, a large number of distance data due to the luminance edge tends to be calculated. The suitability E calculated in such a situation tends to increase because more distance data than the number specified by the target value is output (or removed). Therefore,
Through feedback control, the luminance determination threshold value DXD
It is adjusted so that Xth increases. As a result, the evaluation criterion for the luminance edge becomes strict, and the number of output distance data and the like are suppressed. At that time, the change amount of the luminance edge caused by the forest or hedge is often smaller than that of the white line or the preceding vehicle. Therefore, even if the evaluation criteria for the luminance determination are made strict, the distance data relating to the white line and the preceding vehicle (distance data important for performing the outside monitoring) is not easily excluded, so that the reliability of the outside monitoring is not impaired.

For the above reasons, in the present embodiment, it is possible to obtain the necessary number of distance data for performing the monitoring control without increasing the noise-like distance data irrespective of the monitoring environment. it can. If monitoring outside the vehicle is performed based on the distance data calculated through such processing, highly reliable monitoring control can be performed without being greatly affected by the monitoring environment.

The present invention is not limited to the above-described embodiments, but can be widely applied to, for example, the following methods.

(1) In the above-described embodiment, the amount of change in the luminance in the horizontal direction in the pixel block is determined by the
Compared with Xth, the parallax regarding the pixel block with a large change in luminance is specified as distance data. This is based on the knowledge that if there is no feature in the luminance change pattern of a pixel block, the reliability of parallax related thereto is low. Therefore, the present invention can be widely applied to a method of evaluating a luminance characteristic in a pixel block based on a luminance determination threshold. This includes, for example, a method of evaluating the amount of change in luminance in the vertical direction or a method of evaluating the amount of change in luminance in both the vertical direction and the horizontal direction.

(2) Although the filter processing section in this embodiment is composed of a pairing filter and a group filter, either one of the filters may be used. Further, a low-pass filter, a high-pass filter, or the like may be used. For example, when the distance to be monitored is relatively close only, large distance data becomes noise. Therefore,
A low-pass filter that removes distance data having a large value may be used. In this case, the number of removals in the low-pass filter may be counted, and the suitability may be evaluated based on the counted number. The same applies to a high-pass filter used when the distance to be monitored is relatively long.

(3) Further, in this embodiment, the number of removals of each filter (specifically, the removal numbers Pn and Gn) is counted, but the number of outputs of each filter (specifically, pairing) The number of outputs of the filter and the number of outputs of the group filter may be counted. This is because, when the number of inputs to the filter is specified, if one of the removal number or output number is specified, the other is uniquely specified. In that sense, counting the number of filters removed is equivalent to counting the number of outputs.

As described above, according to the present invention, the luminance judgment threshold value is automatically adjusted according to the monitoring environment, and an appropriate number of highly reliable distance data can be stabilized regardless of the monitoring environment. Can be output. Therefore, by performing outside monitoring based on such distance data, the stability and reliability of monitoring control can be further improved.

[Brief description of the drawings]

FIG. 1 is a block diagram of a vehicle exterior monitoring device according to an embodiment;

FIG. 2 is a diagram for explaining a data processing method according to the embodiment;

FIG. 3 is a diagram for explaining a method of calculating a city block distance.

FIG. 4 is a diagram for explaining a method of evaluating a luminance edge in a horizontal direction with respect to a pixel block;

FIG. 5 is a diagram for explaining filter processing in a group filter.

[Explanation of symbols]

1 main camera, 2 sub camera, 3
A / D converter, 4 correction circuit, 5 stereo matching circuit, 6 judgment circuit, 7 image data memory, 8 distance data memory, 9 microcomputer, 10 recognition unit, 11 alarm device,
12 control device, 13 filter processing unit,
14 Pairing filter, 15 Group filter, 16 Operation unit, 17 Adjustment unit

Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat II (reference) G08G 1/16 B60R 21/00 624C 5L096 // G05D 1/02 G06F 15/62 380 9A001 F term (reference) 2F065 AA04 AA06 BB05 CC11 EE04 FF05 JJ03 JJ05 JJ26 KK02 MM06 QQ03 QQ08 QQ32 QQ33 QQ36 RR02 RR06 SS09 UU05 2F112 AC06 BA06 CA05 FA03 FA07 FA08 FA09 FA21 FA32 FA36 FA45 FA50 GA10 5B057 AA16 BA02 DC30 DC02 DC30 LL08 LL09 5H301 AA01 BB20 CC03 CC06 DD05 DD16 EE02 EE08 EE12 FF06 FF13 GG03 GG11 GG16 HH03 LL01 LL06 LL07 LL11 5L096 AA06 BA04 CA05 CA14 DA02 FA34 FA52 FA66 GA19 GA41 GA51 GA55 HA02 9A001 BB06

Claims (9)

[Claims] 1. A stereo outside monitoring apparatus for monitoring the outside of a vehicle by calculating distance data for each pixel block having a plurality of pixels based on a pair of images of a scene outside the vehicle. A calculating unit that calculates a parallax of the pixel block by specifying, from the other image, a pixel block having a correlation with the luminance characteristic of the pixel block for each pixel block in the image; and calculating the parallax in the calculating unit. By comparing the amount of change in luminance between adjacent pixels in the pixel block with the luminance determination threshold value for the pixel block,
Along with specifying a pixel block having a large change in luminance,
A specifying unit that calculates the parallax regarding the specified pixel block as distance data; and an adjusting unit that adjusts the luminance determination threshold according to the number of distance data calculated by the specifying unit. Stereo exterior monitoring device. 2. The brightness determination threshold value so that the number of distance data calculated by the specifying means approaches a target value defining the number of distance data to be calculated by the specifying means. The stereo type outside monitoring apparatus according to claim 1, wherein the value is adjusted. 3. The apparatus according to claim 2, further comprising: a filter for judging the reliability of the distance data calculated by said specifying means, wherein said filter removes the distance data judged to be low in reliability and has a high reliability. The distance data determined to be high is output as effective distance data used to monitor a situation outside the vehicle, and the adjusting means determines the luminance determination threshold value according to the number of distance data removed by the filter means. The stereo-type external monitoring apparatus according to claim 2, wherein the control is performed. 4. The brightness determination threshold value so that the number of distance data removed by the filter means approaches a target value that defines the number of distance data to be removed by the filter means. The stereo type outside monitoring apparatus according to claim 3, wherein the value is adjusted. 5. The adjusting means includes, as parameters, a target value defining the number of distance data to be specified by the specifying means and a target value defining the number of distance data to be removed by the filter means. And, based on an evaluation formula including, as variables, the number of distance data calculated by the specifying unit and the number of distance data removed by the filter unit, calculate an appropriateness of the luminance determination threshold, The stereo exterior monitoring device according to claim 4, wherein the brightness determination threshold value is adjusted based on the calculated appropriateness. 6. The filter means includes a group filter, wherein the group filter groups adjacent pixel blocks having distance data of close values, calculates an area of the grouped pixel blocks, and 6. The output or removal of the distance data according to the calculated area size.
A stereo exterior monitoring device as described in. 7. The filter means includes a pairing filter, wherein the pairing filter is provided when a plurality of pixel blocks in one image are correlated with the same pixel block in the other image. The method according to claim 3, further comprising: outputting distance data relating to any of the pixel blocks, and removing distance data relating to the remaining pixel blocks.
A stereo exterior monitoring device according to 4, 5, or 6. 8. The apparatus according to claim 1, wherein said specifying means specifies the distance data based on an amount of change in luminance between pixels adjacent in the horizontal direction of the image. Stereo type exterior monitoring device. 9. The apparatus according to claim 1, wherein said specifying means specifies, as distance data, a parallax of a pixel block having a predetermined number or more of adjacent pixel pairs having a change in luminance equal to or greater than a luminance determination threshold. 8. The stereo exterior monitoring device according to 8.