JP6453571B2 - 3D object recognition device - Google Patents

Description

  The present invention relates to a three-dimensional object recognition device.

  Conventionally, a three-dimensional object recognition device that recognizes a three-dimensional object such as a vehicle using a pair of left and right cameras (stereo camera) is known. This three-dimensional object recognition device recognizes a three-dimensional object based on the occurrence of parallax between the position of the three-dimensional object in the image acquired by the left camera and the position of the same three-dimensional object in the image acquired by the right camera. (See Patent Document 1).

JP 2013-104740 A

  In a conventional three-dimensional object recognition device, it may be difficult to accurately recognize a three-dimensional object due to the characteristics of the three-dimensional object. The present invention has been made in view of these problems, and an object thereof is to provide a three-dimensional object recognition device that can solve the above-described problems.

  The three-dimensional object recognition device of the present invention includes an image acquisition unit that acquires a first image captured by a first image sensor and a second image captured by a second image sensor, a first image, and a second image. A three-dimensional object candidate extraction unit that extracts a three-dimensional object candidate based on the parallax generated between the three-dimensional image, a certainty factor calculation unit that calculates a certainty factor of the three-dimensional object candidate, and a three-dimensional object having a certainty factor equal to or greater than a predetermined threshold. A three-dimensional object recognition unit for recognizing a candidate as a three-dimensional object.

  Furthermore, in the three-dimensional object recognition device of the present invention, the certainty factor calculation unit calculates the certainty factor so as to satisfy one or more conditions selected from the following (a) to (o). .

(A) The smaller the variation in the parallax within the three-dimensional object candidate, the higher the certainty factor.
(B) The greater the parallax, the higher the certainty factor.
(C) The degree of certainty is higher as the color variation in the three-dimensional object candidate is smaller.

(D) The higher the solid object candidate is, the higher the certainty factor is.
(E) The degree of certainty is higher as the angle between the opposing surface and the vertical surface in the three-dimensional object candidate is smaller.

(F) The degree of certainty is higher as the color of the three-dimensional object candidate is closer to black or white in the color space.
(G) The more certain the roundness of the shape of the outer periphery of the three-dimensional object candidate is, the higher the certainty factor is.

(H) When the weather is rainy, the certainty level is lower than when the weather is not rainy.
(I) The reliability is higher in the daytime than in the nighttime.
(J) The higher the illuminance, the higher the certainty factor.

(K) The higher the shutter speed of the first image sensor and the second image sensor, the higher the certainty factor.
(L) The certainty factor is higher as the steering angle of the host vehicle is smaller.

(M) When the speed of the host vehicle is equal to or higher than a predetermined threshold and the three-dimensional object candidate exists on the road side, the certainty factor is lower than in other cases.
(N) The certainty factor is higher as the temperatures of the first image sensor and the second image sensor are lower.

(O) The lower the signal amplification factor of the first image sensor and the second image sensor, the higher the certainty factor.
According to the three-dimensional object recognition device of the present invention, a three-dimensional object can be accurately recognized.

2 is a block diagram illustrating a configuration of a three-dimensional object recognition device 1. FIG. It is a flowchart showing the whole process which the solid-object recognition apparatus 1 performs. It is a flowchart showing the solid object candidate extraction process which the solid object recognition apparatus 1 performs. It is a flowchart showing the certainty factor calculation process which the solid-object recognition apparatus 1 performs. It is explanatory drawing showing the extraction method of a solid object candidate. It is explanatory drawing showing the method of calculating parallax. 7A and 7B are graphs showing the relationship between σB and the certainty factor Ka. 8A and 8B are graphs showing the relationship between ave ΔB and certainty factor Kb. It is explanatory drawing showing the height H of the solid thing candidate 217. FIG. It is explanatory drawing showing angle (theta) of the opposing surface 219 and the vertical surface 221. FIG. FIG. 11A is a plan view showing roundness in the outer peripheral portion 223 of the three-dimensional object candidate 217, and FIG. 11B is an explanatory diagram showing the outer peripheral portion 223 in the right image 201 and the left image 203. It is a flowchart showing the certainty factor calculation process which the solid-object recognition apparatus 1 performs.

Embodiments to which the present invention is applied will be described below with reference to the drawings.
<First Embodiment>
1. Configuration of the three-dimensional object recognition device 1 The configuration of the three-dimensional object recognition device 1 will be described with reference to FIG. The three-dimensional object recognition device 1 is a device mounted on a vehicle. Hereinafter, the vehicle on which the three-dimensional object recognition device 1 is mounted is referred to as the own vehicle 101.

  The three-dimensional object recognition device 1 is obtained by installing a program for executing processing to be described later on a known computer. The three-dimensional object recognition device 1 functionally includes an image acquisition unit 3, a three-dimensional object candidate extraction unit 5, a certainty factor calculation unit 7, and a three-dimensional object recognition unit 9. The processing executed by each unit will be described later.

  In addition to the three-dimensional object recognition apparatus 1, the host vehicle 101 includes a right camera 103, a left camera 105, a clock 107, an illuminometer 109, a rain gauge 110, a steering angle sensor 111, a speed sensor 113, a thermometer 115, and an alarm / vehicle. A control unit 117 is provided.

  The right camera 103 and the left camera 105 capture the front of the host vehicle 101 and create an image representing the front of the host vehicle 101. The right camera 103 is provided near the right end of the host vehicle 101, and the left camera 105 is provided near the left end of the host vehicle 101. The height from the road surface of the right camera 103 and the left camera 105 is the same. The right camera 103 and the left camera 105 constitute a stereo camera and perform imaging simultaneously. In the right camera 103 and the left camera 105, settings of image brightness, shutter speed, signal amplification factor, and the like are the same.

  The clock 107 acquires time information. The illuminance meter 109 measures the illuminance outside the vehicle 101. The rain gauge 110 detects rainfall. The steering angle sensor 111 detects the magnitude of the steering angle in the host vehicle 101. The speed sensor 113 detects the speed of the host vehicle 101. The thermometer 115 measures the temperature of the right camera 103 and the left camera 105.

  The alarm / vehicle control unit 117 performs well-known alarm processing and vehicle control processing on the three-dimensional object recognized by the three-dimensional object recognition device 1. For example, when the distance between the three-dimensional object and the host vehicle 101 becomes small, an alarm process is performed. In addition, processing for avoiding a collision with a three-dimensional object (for example, automatic braking, automatic steering, etc.) is performed. Further, the alarm / vehicle control unit 117 performs a process in which the host vehicle 101 follows a preceding vehicle that is an example of a three-dimensional object.

The right camera 103 and the left camera 105 are examples of a first image sensor and a second image sensor.
2. The process which the solid object recognition apparatus 1 performs The process which the solid object recognition apparatus 1 performs is demonstrated based on FIGS. In step 1 of FIG. 2, the image acquisition unit 3 captures the front of the host vehicle 101 using the right camera 103 and the left camera 105 and acquires an image. Hereinafter, the image of the right camera 103 is referred to as a right image 201, and the image of the left camera 105 is referred to as a left image 203. The right image 201 and the left image 203 are an example of a first image and a second image.

  In step 2, the three-dimensional object candidate extraction unit 5 executes a process of extracting a three-dimensional object candidate from the right image 201 and the left image 203 acquired in step 1. The three-dimensional object candidate means one that may be a three-dimensional object among those displayed in the right image 201 and the left image 203.

  Examples of solid objects include other vehicles (preceding vehicles, oncoming vehicles, parallel vehicles, stopped vehicles), solid objects on the road or around it (side walls, median strips, road shoulders, street lights, etc.), walking Or the like. A specific method for extracting a three-dimensional object candidate will be described later.

  In step 3, the three-dimensional object candidate extraction unit 5 determines whether or not the three-dimensional object candidate has been extracted in step 2. If the three-dimensional object candidate can be extracted, the process proceeds to step 4; otherwise, the process ends.

  In step 4, the certainty factor calculation unit 7 calculates the certainty factor of the three-dimensional object candidate extracted in step 2. The certainty factor is an index representing the certainty that the three-dimensional object candidate is a three-dimensional object. When a plurality of three-dimensional object candidates are extracted in step 2, the certainty factor is calculated for each three-dimensional object candidate. A specific method for calculating the certainty factor will be described later.

  In step 5, the three-dimensional object recognition unit 9 determines whether or not the certainty factor calculated in step 4 exceeds a predetermined threshold value. If it exceeds the threshold value, the process proceeds to step 6; otherwise, the process ends.

In step 6, the three-dimensional object recognition unit 9 recognizes the three-dimensional object candidate determined in step 5 as having a certainty degree exceeding a predetermined value as a three-dimensional object.
In step 7, information on the three-dimensional object recognized in step 6 (presence of three-dimensional object, position of three-dimensional object (distance from own vehicle 101 to three-dimensional object, direction of three-dimensional object with reference to own vehicle 101)) is alarmed. / Output to vehicle control unit 117.

Next, the process in which the three-dimensional object candidate extraction unit 5 extracts a three-dimensional object candidate in step 2 will be described in detail with reference to FIG.
In step 11, the left image 203 is searched for a region having high similarity to the model image representing the three-dimensional object (hereinafter referred to as a left similar region). As the model image, for example, an image obtained by actually capturing the corresponding three-dimensional object can be used. The model image has a certain size and is smaller than the left image 203. The similarity with the model image can be determined by a known method such as a neural network or a support vector machine.

  In step 12, it is determined whether or not a left similar region has been found in the processing of step 11. If the left similar region can be found, the process proceeds to step 13, and if it cannot be found, the process ends.

  In step 13, an area displaying the same display as the left similar area found in the process of step 11 is searched in the right image 201. The search can be performed as follows, for example.

  First, as shown in FIG. 5, it is assumed that the left similar region 205 (part of the left image 203) found in the process of step 11 is overlapped with part of the right image 201. At this time, let P be any position in the left similar region 205. A difference ΔA between the luminance of the pixel of the left image 203 at the position P and the luminance of the pixel of the right image 201 is obtained. This ΔA is obtained for all the pixels in the left similar region 205, and an average value aveΔA thereof is calculated.

  Then, the average value aveΔA is calculated at each position while moving the left similar region 205 little by little in the right image 201. In the right image 201, an area overlapping the left similar area 205 when the average value aveΔA is minimum is an area displaying the same as the left similar area. Hereinafter, the region found by this search is referred to as a right similarity region.

Returning to FIG. 3, in step 14, the parallax between the left similar region and the right similar region is calculated. Specifically, it is performed as follows. First, as shown in FIG. 6, an arbitrary point in the left similar region 205 is set to P _L, and a distance from the right boundary line 209 of the left image 203 to the point P _L is set to x _L. Further, the lower boundary line 211 in the left image 203, the distance to the point _{P L} and y _L.

Further, in the right similar region 207, a point corresponding to the point _{P L} and _{P R.} That is, the left similar regions 205 and the right similar region 207, assuming that the stacked as four sides match, point coincident with the point P _L is a point P _R.

From the right border 213 of the right image 201, the distance to the point _{P R} and _{x R.} Further, the lower boundary line 215 of the right image 201, the distance to the point _{P R} and _{y R.}
In this _case, a parallax ΔB at _(x R -x _L) is the point _{P L,} the point _{P R.} The parallax ΔB is calculated as described above for all the pixels in the left similar region 205 and the right similar region 207, and an average value aveΔB thereof is calculated.

  Returning to FIG. 3, in step 15, it is determined whether or not the parallax calculated in step 14 satisfies the three-dimensional object candidate condition. The three-dimensional object candidate condition is to satisfy both of the following (A) and (B).

(A) aveΔB is equal to or greater than a predetermined threshold value.
(B) The variation (for example, standard deviation) σB of the parallax ΔB calculated for each pixel in the left similar region 205 and the right similar region 207 is equal to or less than a predetermined threshold.

If the three-dimensional object candidate condition is satisfied, the process proceeds to step 16; otherwise, the process ends.
In step 16, the display contents in the left similar region found in step 11 and the right similar region found in step 13 are extracted as solid object candidates.

As described above, the three-dimensional object candidate extraction unit 5 extracts a three-dimensional object candidate based on the parallax generated between the right image and the left image.
Next, the process in which the certainty factor calculation unit 7 calculates the certainty factor in Step 4 will be specifically described with reference to FIG.

  In step 21, a certainty factor Ka regarding the parallax variation σB in the three-dimensional object candidate is calculated. The certainty factor calculation unit 7 includes a map that preliminarily defines the relationship between σB and the certainty factor Ka, and the certainty factor Ka can be calculated by inputting σB to this map.

  As shown in FIG. 7A, the relationship between σB and the certainty factor Ka defined in the above map is such that the smaller the σB, the larger the certainty factor Ka. Therefore, the certainty factor Ka increases as σB decreases. The relationship between σB and the certainty factor Ka defined by the map may be a relationship in which the certainty factor Ka increases stepwise as σB decreases, as shown in FIG. 7B.

  In step 22, a certainty factor Kb regarding the magnitude of parallax (aveΔB) is calculated. The certainty factor calculation unit 7 includes a map that prescribes the relationship between the ave ΔB and the certainty factor Kb in advance, and the certainty factor Kb can be calculated by inputting the ave ΔB to this map.

  As shown in FIG. 8A, the relationship between aveΔB and the certainty factor Kb defined by the above map is a relationship in which the certainty factor Kb increases as the aveΔB increases. Therefore, the certainty factor Kb increases as aveΔB increases. Note that the relationship between the ave ΔB and the certainty factor Kb defined by the map may be a relationship in which the certainty factor Kb increases stepwise as the ave ΔB increases, as shown in FIG. 8B.

  In step 23, a certainty factor Kc regarding the color variation in the three-dimensional object candidate is calculated. The certainty calculation unit 7 first represents the color of each pixel in the three-dimensional object candidate in the right image 201 and the left image 203 by coordinates in the color space. Next, the variation in coordinates in the color space is digitized. Hereinafter, the digitized value is referred to as a color variation value.

  The certainty factor calculation unit 7 includes a map that preliminarily defines the relationship between the color variation value and the certainty factor Kc, and calculates the certainty factor Kc by inputting the color variation value to this map. . The relationship between the color variation value and the certainty factor Kc defined by the map is such that the smaller the color variation value, the higher the certainty factor Kc. Therefore, the certainty factor Kc increases as the color variation value decreases.

  In step 24, a certainty factor Kd regarding the height of the three-dimensional object candidate is calculated. First, the certainty factor calculation unit 7 obtains the height H of the solid object candidate 217 in the right image 201 and the left image 203 as shown in FIG. The certainty factor calculation unit 7 has a map that prescribes the relationship between the height H and the certainty factor Kd in advance, and the certainty factor Kd can be calculated by inputting the height H into this map. .

  The relationship between the height H and the certainty factor Kd defined by the above map is that the certainty factor Kd increases as the height H increases. Therefore, the certainty factor Kd increases as the height H increases.

  In step 25, the certainty factor Ke regarding the angle θ between the opposing surface and the vertical surface in the three-dimensional object candidate is calculated. Here, as illustrated in FIG. 10, the facing surface 219 is a surface of the three-dimensional object candidate 217 that faces the host vehicle 101. Further, the angle θ is an angle between the facing surface 219 and the vertical surface 221.

The certainty factor calculation unit 7 includes a map that preliminarily defines the relationship between the angle θ and the certainty factor Ke, and the certainty factor Ke can be calculated by inputting the angle θ into this map.
The relationship between the angle θ and the certainty factor Ke defined by the above map is that the certainty factor Ke increases as the angle θ decreases. Therefore, the certainty factor Ke increases as the angle θ decreases. The angle θ can be calculated from the amount of change in parallax with respect to the vertical direction.

In step 26, a certainty factor Kf for a specific color is calculated. That is, as the color of the three-dimensional object candidate is closer to black or white in the color space, the certainty factor Kf is set higher.
In step 27, the certainty factor Kg regarding the roundness of the shape at the outer periphery of the three-dimensional object candidate is calculated. As shown in FIGS. 11A and 11B, the outer peripheral portion 223 of the three-dimensional object candidate 217 is a portion located on the outer periphery of the three-dimensional object candidate 217 when viewed from the host vehicle 101. The certainty factor calculation unit 7 first obtains the curvature at the outer peripheral portion 223. Specifically, this curvature can be obtained from the amount of parallax change in the vicinity of the outer periphery or the value of the approximate curve of the parallax change.

  The certainty factor calculation unit 7 is previously provided with a map that outputs a higher certainty factor Kg as the curvature at the outer peripheral portion of the three-dimensional object candidate is smaller (as the roundness is more significant). The certainty factor Kg can be calculated by inputting the curvature obtained as described above into this map. The certainty factor Kg increases as the roundness of the outer periphery of the three-dimensional object candidate increases.

  In step 28, a certainty factor Kh regarding the weather is calculated. That is, the certainty calculation unit 7 determines whether or not the weather is rainy based on the output signal of the rain gauge 110, and if it is raining, makes the certainty Kh lower than in other cases.

  In step 29, a certainty factor Ki for the time zone is calculated. The certainty factor calculation unit 7 acquires time information using the clock 107 and determines whether the time is daytime or nighttime. In the daytime, the certainty factor Ki is set higher than in the nighttime.

  In step 30, a certainty factor Kj relating to illuminance is calculated. The certainty factor calculation unit 7 acquires illuminance using the illuminance meter 109. The certainty factor calculation unit 7 is previously provided with a map that outputs a higher certainty factor Kj as the illuminance is higher. The confidence Kj is calculated by inputting the illuminance acquired as described above to this map. The certainty factor Kj increases as the illuminance increases.

  In step 31, the certainty factor Kk regarding the shutter speed of the right camera 103 and the left camera 105 is calculated. The certainty factor calculation unit 7 acquires shutter speed information from the right camera 103 and the left camera 105. In addition, the certainty factor calculation unit 7 includes a map that outputs a higher certainty factor Kk as the shutter speed is faster. The confidence Kk is calculated by inputting the shutter speed acquired as described above to this map. The certainty factor Kk increases as the shutter speed increases.

  In step 32, a certainty factor KL relating to the steering angle is calculated. The certainty factor calculation unit 7 acquires the steering angle of the host vehicle 101 from the steering angle sensor 111. The certainty factor calculation unit 7 is previously provided with a map that outputs a higher certainty factor KL as the steering angle is smaller. The confidence factor KL is calculated by inputting the steering angle acquired as described above to this map. The certainty factor KL increases as the steering angle decreases.

  In step 33, the certainty factor Km regarding the speed of the own vehicle and the position of the three-dimensional object candidate is calculated. The certainty factor calculation unit 7 acquires the speed of the host vehicle 101 from the speed sensor 113. In addition, the certainty factor calculation unit 7 acquires the positions of the three-dimensional object candidates in the right and left directions in the right image 201 and the left image 203. The speed of the host vehicle 101 is equal to or higher than a predetermined threshold, and the positions of the three-dimensional object candidates in the right image 201 and the left image 203 in the left-right direction are rightward or leftward (that is, the three-dimensional object candidates exist on the road side. ), The certainty factor Km is made lower than in other cases.

  In step 34, the certainty factor Kn regarding the temperature of the right camera 103 and the left camera 105 is calculated. The certainty factor calculation unit 7 acquires the temperatures of the right camera 103 and the left camera 105 from the thermometer 115. The certainty factor calculation unit 7 includes a map that outputs a higher certainty factor Kn as the temperature of the right camera 103 and the left camera 105 is lower. The confidence level Kn is calculated by inputting the temperatures of the right camera 103 and the left camera 105 acquired as described above to this map. The certainty factor Kn increases as the temperature of the right camera 103 and the left camera 105 decreases.

  In step 35, the certainty factor Ko regarding the signal amplification factors (gains) of the right camera 103 and the left camera 105 is calculated. The certainty factor calculation unit 7 acquires the signal amplification factor from the right camera 103 and the left camera 105. The certainty factor calculation unit 7 is previously provided with a map that outputs a higher certainty factor Ko as the signal amplification factor is lower. The confidence factor Ko is calculated by inputting the signal gain obtained as described above to this map. The certainty factor Ko increases as the signal amplification factor decreases.

  In step 36, a certainty factor Kp regarding the repetitive shape is calculated. That is, the certainty factor calculation unit 7 determines whether or not the three-dimensional object candidate is an array of a plurality of objects having the same shape or similar shape, and in such a case, the solid object candidate is more than in other cases. The reliability Kp is lowered.

  In step 37, the certainty factors Ka, Kb, Kc, Kd, Ke, Kf, Kg, Kh, Ki, Kj, Kk, KL, Km, Kn, Ko, and Kp calculated in steps 21 to 36 are all multiplied. The final certainty factor K is calculated. The certainty factor K calculated in this way satisfies the following conditions.

(A) The certainty factor K is higher as the parallax variation (σB) is smaller in the three-dimensional object candidate.
(B) The greater the parallax (aveΔB), the higher the certainty factor K.
(C) The confidence K is higher as the color variation in the three-dimensional object candidate is smaller.

(D) The greater the height H of the three-dimensional object candidate, the higher the certainty factor K.
(E) The certainty factor K is higher as the angle θ between the opposing surface and the vertical surface in the three-dimensional object candidate is smaller.
(F) The reliability K is higher as the color of the three-dimensional object candidate is closer to black or white in the color space.

(G) The greater the roundness of the shape of the outer periphery of the three-dimensional object candidate, the higher the certainty factor K.
(H) When the weather is rainy, the certainty factor K is lower than when the weather is not rainy.
(I) The certainty factor K is higher in the daytime than in the nighttime.

(J) The higher the illuminance, the higher the certainty factor K.
(K) The higher the shutter speed of the right camera 103 and the left camera 105, the higher the certainty factor K.

(L) The confidence K is higher as the steering angle of the host vehicle 101 is smaller.
(M) When the speed of the host vehicle 101 is equal to or higher than a predetermined threshold and the three-dimensional object candidate exists on the road side, the certainty factor K is lower than in other cases.

(N) The lower the temperature of the right camera 103 and the left camera 105, the higher the certainty factor K.
(O) The lower the signal amplification factor of the right camera 103 and the left camera 105, the higher the certainty factor K.
(P) When the three-dimensional object candidate is an array of a plurality of objects having the same shape or similar shapes, the certainty factor K is lower than in other cases.

3. Effects produced by the three-dimensional object recognition device 1 (1A) The three-dimensional object recognition device 1 reduces the certainty of the three-dimensional object candidate when there is a high possibility that the three-dimensional object candidate is not actually a three-dimensional object. That is, when the three-dimensional object candidate is not actually a three-dimensional object, generally, the parallax variation (σB) increases, the parallax size (aveΔB) decreases, and the color variation within the three-dimensional object candidate varies. The height (H) of the three-dimensional object candidate becomes small and the angle θ between the opposing surface and the vertical surface in the three-dimensional object candidate becomes large. In these cases, the three-dimensional object recognition device 1 has certainty factors Ka and Kb. , Kc, Kd, Ke are reduced.

As a result, in the case of a three-dimensional object candidate that is not actually a three-dimensional object, the certainty factor K decreases, so that it is unlikely that a three-dimensional object candidate that is not actually a three-dimensional object will be erroneously recognized as a three-dimensional object.
(1B) Even if the three-dimensional object candidate is actually a three-dimensional object, the certainty factor is likely to be low in some cases. For example, when the color of the three-dimensional object candidate is close to black or white, or when the shape of the three-dimensional object candidate is markedly rounded, the certainty factor tends to be low. In those cases, the three-dimensional object recognition device 1 increases the certainty factors Kf and Kg. As a result, since the certainty factor K is increased, the three-dimensional object can be recognized more easily.

  (1C) Depending on the situation, it may be difficult to accurately determine whether the three-dimensional object candidate is a three-dimensional object. For example, in the case of rain, at night, when the illuminance is low, when the shutter speed of the right camera 103 and the left camera 105 is slow, when the steering angle of the host vehicle 101 is large, the speed of the host vehicle 101 is equal to or higher than a predetermined threshold. If there is a three-dimensional object candidate on the road side, if the temperature of the right camera 103 and the left camera 105 is high, or if the signal amplification factor of the right camera 103 and the left camera 105 is high, the right image 201 and the left image 203 are three-dimensional. It is difficult to determine whether an object candidate is unclear and is a three-dimensional object. In these cases, if it is determined whether the three-dimensional object candidate is a three-dimensional object without considering the situation, there is a possibility that an object that is not actually a three-dimensional object is erroneously determined to be a three-dimensional object.

  In the above case, the three-dimensional object recognition device 1 reduces the certainty factors Kh, Ki, Kj, Kk, KL, Km, Kn, and Ko. As a result, the certainty factor K is lowered, and the possibility that a thing that is not actually a three-dimensional object is erroneously determined to be a three-dimensional object can be reduced.

  (1D) As a three-dimensional object, there are street lamps arranged along the road, and illumination arranged on the ceiling surface of the tunnel along the traveling direction of the tunnel. These are not likely to collide with the host vehicle 101, and need not be recognized as compared with other three-dimensional objects.

The three-dimensional object recognition device 1 has a certainty factor Kp when the three-dimensional object candidate has characteristics peculiar to the above-mentioned street lamp or tunnel illumination (when a plurality of objects having the same shape or similar shapes are arranged). Lower. As a result, the certainty factor K is lowered, and it is possible to make it difficult to recognize the above-mentioned street lamps, tunnel lighting, and the like.
<Second Embodiment>
1. The configuration of the three-dimensional object recognition device 1 and the processing to be executed The configuration in the second embodiment and the processing to be executed are basically the same as those in the first embodiment. The difference will be mainly described.

In the present embodiment, the certainty factor K is calculated as follows. In step 41 of FIG. 12, by multiplying the confidence Kb and confidence Ka, it calculates the initial confidence K _0. The certainty factors Ka and Kb are the same as those in the first embodiment.

  In step 42, it is determined whether or not there is a feature whose reliability should be lowered in the three-dimensional object candidate. For example, there is one or more features selected from the following as features that should reduce the certainty factor. These features are likely to appear when the three-dimensional object candidate is not actually a three-dimensional object.

The parallax variation (σB) is greater than or equal to a predetermined threshold.
The magnitude of parallax (aveΔB) is less than or equal to a predetermined threshold.
・ Color variation within the three-dimensional object candidate is large.

The height (H) of the three-dimensional object candidate is below a predetermined threshold value.
-Angle (theta) of the opposing surface and vertical surface in a solid object candidate is more than a predetermined threshold value.
If there is the above feature, the process proceeds to step 43. If there is no such feature, the process proceeds to step 44.

In step 43, the certainty factor K is decreased by a predetermined amount.
In step 44, it is determined whether or not the solid object candidate has a feature indicating that the certainty factor K calculated in step 41 is too small. Such features include, for example, one or more selected from the following.

-Solid object candidate color is close to black or white.
-The roundness of the shape at the outer periphery of the three-dimensional object candidate is remarkable.
If there is the above feature, the process proceeds to step 45, and if there is no such feature, the process proceeds to step 46.

In step 45, the certainty factor K is increased by a predetermined amount.
In step 46, the host vehicle 101 determines whether or not it is difficult to accurately determine whether or not the three-dimensional object candidate is a three-dimensional object. Such a situation includes, for example, one or more selected from the following. In any of these situations, the three-dimensional object candidates are likely to become unclear in the right image 201 and the left image 203, and it is difficult to determine whether or not the object is a three-dimensional object.

・ The situation in rainy weather.
・ Night situation.
・ Low illuminance.

A situation where the shutter speed of the right camera 103 and the left camera 105 is slow.
A situation in which the steering angle of the host vehicle 101 is large.

A situation where the temperature of the right camera 103 and the left camera 105 is high.
A situation where the signal amplification factors of the right camera 103 and the left camera 105 are high.
If so, the process proceeds to step 47; otherwise, the process proceeds to step 48.

In step 47, the certainty factor K is decreased by a predetermined amount.
In step 48, it is determined whether or not the three-dimensional object candidate has features of a three-dimensional object that need not be recognized. Examples of the three-dimensional object that need not be recognized include a plurality of street lamps arranged along the road and a plurality of lights arranged along the traveling direction of the tunnel on the ceiling surface of the tunnel. These three-dimensional objects have a feature of a shape in which a plurality of objects having the same shape or similar shapes are arranged.

If the three-dimensional object candidate has the above-described feature, the process proceeds to step 49, and if it does not have such a feature, the process proceeds to step 50.
In step 49, the certainty factor K is decreased by a predetermined amount.

  In step 50, the final certainty factor K is determined. When the certainty factor is corrected in any of the steps 43, 45, 47, and 49, the corrected value is set as the final certainty factor K. On the other hand, if no correction is made in any of Steps 43, 45, 47, and 49, the value calculated in Step 41 is the final certainty factor K.

2. Effects exhibited by the three-dimensional object recognition device 1 The three-dimensional object recognition device 1 of the present embodiment has the same effects as the effects (1A) to (1D) of the first embodiment.
<Other embodiments>
As mentioned above, although embodiment of this invention was described, this invention can take a various form, without being limited to the said embodiment.

  (1) In the first embodiment, some of the certainty factors Ka to Kp may be omitted. In this case, the certainty factor K can be obtained by multiplying the certainty factors Ka to Kp which are not omitted.

  (2) In the first embodiment, the certainty factor K may be calculated by a method other than multiplying the certainty factors Ka to Kp. For example, the certainty factor K may be obtained by inputting the certainty factors Ka to Kp into known functions other than multiplication.

(3) In the second embodiment, steps 42 and 43 may be omitted. Steps 44 and 45 may be omitted. Further, steps 46 and 47 may be omitted. Steps 48 and 49 may be omitted.
(4) The functions of one component in the embodiment may be distributed as a plurality of components, or the functions of a plurality of components may be integrated into one component. In addition, at least a part of the configuration of each embodiment may be replaced with a known configuration having the same function. Moreover, you may abbreviate | omit a part of structure of each said embodiment. Further, at least a part of the configuration of the embodiment may be added to or replaced with the configuration of the other embodiments. In addition, all the aspects included in the technical idea specified only by the wording described in the claim are embodiment of this invention.
(5) In addition to the three-dimensional object recognition device, various systems such as a system including the three-dimensional object recognition device, a program for causing a computer to function as the three-dimensional object recognition device, a medium on which the program is recorded, a three-dimensional object recognition method, etc. The present invention can also be realized in the form of.

DESCRIPTION OF SYMBOLS 1 ... Solid object recognition apparatus, 3 ... Image acquisition unit, 5 ... Solid object candidate extraction unit, 7 ... Certainty factor calculation unit, 9 ... Solid object recognition unit, 101 ... Own vehicle, 103 ... Right camera, 105 ... Left camera, 111 ... steering angle sensor, 113 ... speed sensor, 115 ... thermometer, 117 ... alarm / vehicle control unit, 201 ... right image, 203 ... left image, 205 ... left similar region, 207 ... right similar region, 209 ... right boundary 211, lower boundary line, 213, right boundary line, 215, lower boundary line, 217, solid object candidate, 219, opposing surface, 221, vertical surface, 223, outer periphery

Claims (2)

An image acquisition unit (3) for acquiring a first image captured by the first image sensor (103) and a second image captured by the second image sensor (105);
A three-dimensional object candidate extraction unit (5) for extracting a three-dimensional object candidate based on a parallax generated between the first image and the second image;
A certainty factor calculation unit (7) for calculating the certainty factor of the three-dimensional object candidate;
A three-dimensional object recognition unit (9) for recognizing the three-dimensional object candidate having the certainty factor equal to or higher than a predetermined threshold as a three-dimensional object;
A three-dimensional object recognition device (1) comprising:
The certainty factor calculation unit calculates the certainty factor so as to satisfy one or more conditions selected from the following (a) to (o) and satisfy the following condition (p): A three-dimensional object recognition device.
(A) The smaller the variation in the parallax within the three-dimensional object candidate, the higher the certainty factor.
(B) The greater the parallax, the higher the certainty factor.
(C) The degree of certainty is higher as the color variation in the three-dimensional object candidate is smaller.
(D) The higher the solid object candidate is, the higher the certainty factor is.
(E) The degree of certainty is higher as the angle between the opposing surface and the vertical surface in the three-dimensional object candidate is smaller.
( H) The reliability is lower when the weather is rainy than when it is not rainy.
(I) The reliability is higher in the daytime than in the nighttime.
(J) The higher the illuminance, the higher the certainty factor.
(K) The higher the shutter speed of the first image sensor and the second image sensor, the higher the certainty factor.
(L) The certainty factor is higher as the steering angle of the host vehicle is smaller.
(M) When the speed of the host vehicle is equal to or higher than a predetermined threshold and the three-dimensional object candidate exists on the road side, the certainty factor is lower than in other cases.
(N) The certainty factor is higher as the temperatures of the first image sensor and the second image sensor are lower.
(O) The lower the signal amplification factor of the first image sensor and the second image sensor, the higher the certainty factor.
(P) When the solid object candidate is an array of a plurality of objects having the same shape or similar shapes, the certainty factor is lower than in other cases. The three-dimensional object recognition device according to claim 1,
The three-dimensional object recognition apparatus, wherein the three-dimensional object is a vehicle.

Images