JP7196781B2 - Information processing device and driving support system - Google Patents

Description

The present disclosure relates to technology for recognizing objects around a vehicle.

2. Description of the Related Art Conventionally, there has been known a technique for detecting the surrounding environment and the like based on an image captured by a camera mounted on a vehicle. For example, Patent Literature 1 below proposes a technique for detecting the presence of an entrance/exit of a tunnel based on a photographed image.

JP 2009-255722 A

A technique for detecting objects around the vehicle based on images captured by a camera mounted on the vehicle and assisting driving has also been proposed. However, as a result of a detailed study by the inventor, it was found that in a scene where the brightness changes suddenly, such as at the entrance of a tunnel, the object is not shown well in the captured image, making it difficult to continue driving support. rice field.

One aspect of the present disclosure provides a technology capable of extending the duration of driving assistance.

One aspect of the present disclosure is an information processing device (13) used in a vehicle including an imaging device (11) and a driving support device (12). The information processing device includes a first determination section (33), a second determination section (34), an estimation section (35), and an output section (36). The first determination unit is configured to determine that a tunnel entrance exists in front of the vehicle. The second determination unit is configured to determine that a preceding vehicle traveling in front of the vehicle passes through the entrance/exit of the tunnel. The estimating unit acquires estimation information, which is information indicating the estimated behavior of the preceding vehicle, based on peripheral images (55B, 56B, 57B) that are images acquired using an imaging device and that indicate the periphery of the vehicle. configured to The output unit is configured to output the estimation information acquired by the estimation unit to the driving support device when output conditions including at least the following requirements (i) and (ii) are satisfied. (i) The first determination unit determines that there is an entrance/exit in front of the vehicle. (ii) The second determination unit determines that the preceding vehicle passes through the entrance.

With such a configuration, it is possible to output estimated information when it is determined that a tunnel entrance exists in front of the vehicle and the preceding vehicle passes through the tunnel entrance, that is, in a situation where tracking of the preceding vehicle is likely to be difficult. Therefore, the duration of driving support can be extended.

Another aspect of the present disclosure is a driving assistance system (1) mounted on a vehicle (3) and used, comprising a photographing device (11), a driving assistance device (12), and the information processing device (13) described above. ) and According to such a configuration, it is possible to realize the effect of extending the duration of driving assistance by providing the information processing device of another aspect of the present disclosure described above.

1 is a block diagram showing the configuration of a driving assistance system; FIG. 1 is a functional block diagram of an information processing device according to an embodiment; FIG. FIG. 3A is a diagram showing an image before the preceding vehicle enters the tunnel, and FIG. 3B is a diagram showing an image after the preceding vehicle enters the tunnel. FIG. 4A is a diagram showing an image before the preceding vehicle exits the tunnel, and FIG. 4B is a diagram showing an image after the preceding vehicle has exited the tunnel. It is a flow chart of information output processing. 8 is a flowchart of environmental tunnel determination processing; It is a figure explaining the setting method of a determination area. 4 is a flowchart of object tunnel determination processing; FIG. 9A is an image showing a preceding vehicle immediately before entering a tunnel, and FIG. 9B is a parallax image from the same viewpoint as in FIG. 9A. FIG. 10A is an image showing a preceding vehicle entering a tunnel, and FIG. 10B is a parallax image from the same viewpoint as in FIG. 10A. FIG. 11A is an image showing the preceding vehicle after entering the tunnel, and FIG. 11B is a parallax image from the same viewpoint as in FIG. 11A. 7 is a graph showing an example of changes in the rate of change in average luminance and changes in the rate of change in the number of parallax points; It is a figure explaining the example of the timing of an extrapolation process. It is a figure explaining the example of the timing of an extrapolation process. It is a figure explaining the setting range of an object rectangular frame. It is a functional block diagram of an information processing device of a modification.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.
[1. embodiment]
[1-1. composition]
A driving assistance system 1 shown in FIG. 1 is a system mounted on a vehicle 3 and used, and includes a stereo camera 11 , a driving assistance device 12 , and an information processing device 13 .

The stereo camera 11 includes a right camera 11a and a left camera 11b. The right camera 11a and the left camera 11b are equipped with image sensors such as CCD sensors and CMOS sensors that are synchronized with each other. The right camera 11a and the left camera 11b are mounted, for example, inside the windshield of the vehicle 3 at the same height from the road surface with a predetermined gap in the vehicle width direction. The right camera 11a and the left camera 11b repeatedly photograph the surroundings of the vehicle 3 at predetermined intervals (for example, 0.1 second intervals). The stereo camera 11 corresponds to a photographing device.

The driving support device 12 is a device provided with a microcomputer including a CPU, ROM, RAM and I/O (not shown). The driving support device 12 is configured to perform so-called adaptive cruise control (hereinafter, ACC) that accelerates and decelerates so as to maintain a distance to the preceding vehicle. The driving assistance device 12 performs ACC using the object information and estimation information output by the information processing device 13 .

The information processing device 13 includes a microcomputer having a CPU 21 and a semiconductor memory such as RAM or ROM (hereinafter referred to as memory 22). Each function of the information processing device 13 is realized by executing a program stored in the memory 22 by the CPU 21 .

The information processing device 13 includes a parallax detection unit 31, a first determination unit 33, a second determination unit 34, an estimation unit 35, and an output unit 36, as shown in FIG. The information processing device 13 may also include an object recognition/tracking unit 32 .

The parallax detection unit 31 detects a parallax point, which is a characteristic point indicating a preceding vehicle running in front of the vehicle 3 in the parallax image acquired using the stereo camera 11 . A parallax image includes a large number of parallax points centering on an edge portion on the image. A parallax point can be obtained by a known technique. In the present embodiment, the parallax detection unit 31 converts the analog data of the left and right images acquired each time the right camera 11a and the left camera 11b take pictures into digital data with a predetermined luminance gradation for each pixel. do. Then, the parallax detection unit 31 performs stereo matching of the left and right digital images to calculate parallax and generate a parallax image. A parallax image corresponds to a peripheral image. Also, the parallax detection unit 31 corresponds to the detection unit.

The object recognition/tracking unit 32 detects an object from the parallax images, tracks the same object in the parallax images acquired at different timings, calculates the moving speed of the object and the distance from the vehicle 3, and further , to determine the type of the object. The object recognition/tracking unit 32 performs these processes using known methods.

The first determination unit 33 determines that a tunnel entrance exists in front of the vehicle 3 . The term "entrance" as used herein includes both an entrance when the vehicle 3 enters the tunnel and an exit when the vehicle 3 leaves the tunnel.

The first determination unit 33 determines whether the central area 61, which will be described later, is brighter or darker than the road surface area 62 and the peripheral area 63, which are located around the central area 61, and the tunnel is located in front of the vehicle 3. is determined to exist. A central area 61 is an area including a vanishing point in an image of the front of the vehicle 3 . The central area 61 corresponds to the first range, and the road surface area 62 and the peripheral edge area 63 correspond to the second range. The brightness of each area in the captured image can be determined, for example, based on luminance, but is not limited to this. parameters can be used.

The second determination unit 34 determines that a preceding vehicle traveling in front of the vehicle 3 passes through the entrance/exit of the tunnel. When the rate of change in brightness of a predetermined region in the parallax image where the preceding vehicle exists is greater than a predetermined threshold and the rate of change in the number of parallax points in the predetermined region is greater than a predetermined threshold, the second determination unit 34 It is determined that the preceding vehicle passes through the tunnel entrance.

The estimation unit 35 acquires object information and estimation information, which are information indicating the behavior of the preceding vehicle. These information include information about the position or movement of objects included in the parallax images. In the following description, an object refers to an object detected based on parallax images.

The object information includes object distance, longitudinal relative velocity, object lateral position, lateral relative velocity, object lateral width, object type, and the like. The object distance is the distance from the vehicle 3 to the detected object. The longitudinal relative velocity is the relative velocity of an object with respect to the vehicle 3 in the traveling direction of the vehicle 3 . The relative velocity can be calculated by using feature points related to a plurality of past parallax images. The object lateral position is the position of the object in the direction perpendicular to the traveling direction of the vehicle 3 with respect to the vehicle 3 . The lateral relative velocity is the relative velocity of an object with respect to the vehicle 3 in a direction perpendicular to the traveling direction of the vehicle 3 . The object width is the length of the object in the orthogonal directions mentioned above. The object type is the type of object estimated from the size, moving speed, shape, and the like of the object. Note that in this embodiment, when the object is the preceding vehicle, the object information and the estimated information are acquired by computation. Note that the above content of the object information is merely an example, and is not limited to the above content. For example, various information that can be used for driving assistance can be used as object information.

The estimation unit 35 also acquires object information based on the parallax points detected by the parallax detection unit 31 . The estimation unit 35 identifies the position of the object based on the position of the parallax point on the image and the distance indicated by the parallax point. The estimation unit 35 also obtains the velocity of the object using the parallax points of the preceding vehicle included in the current and past parallax images. The object information is information indicating the current behavior of the preceding vehicle, which is obtained using parallax points related to the preceding vehicle included in the latest parallax image.

The estimated information includes the same types of content as the object information, that is, object distance, vertical relative velocity, horizontal position of object, relative horizontal velocity, horizontal width of object, type of object, and the like. On the other hand, unlike the object information, the estimation information is information indicating the behavior estimated for the preceding vehicle. In other words, it is information that indicates the predicted behavior of the position and speed of the preceding vehicle at the timing after the timing at which a certain parallax image is acquired (hereinafter referred to as the reference timing). Each content of estimated information is obtained as follows. The object distance is calculated in consideration of the movement of the vehicle 3, assuming that the object has a constant velocity at the reference timing. As for the vertical relative velocity, the horizontal position of the object, the horizontal width of the object, and the type of the object, the values and contents at the reference timing are used as they are. The lateral relative velocity is set to "0". In other words, it is presumed that relative movement in the lateral direction has not occurred. The method of calculating and setting each content of the estimated information described above is merely an example, and is not limited to the content described above. For example, the lateral relative velocity may be a value at the reference timing.

The output unit 36 outputs the estimation information or the object information acquired by the estimation unit 35 to the driving assistance device 12 . The output of estimated information is also described as extrapolation processing. The estimated information is output when a predetermined output condition is satisfied. The output conditions are (i) that the first determination unit 33 determines that there is a tunnel entrance in front of the vehicle 3, and (ii) that the second determination unit 34 determines that the preceding vehicle will pass through the entrance. as a requirement.

The technical idea of the present application will be described with reference to FIGS. 3A to 4B showing examples of images of the front of the vehicle 3. FIG. As in the image 51 shown in FIG. 3A, the preceding vehicle 101 driving just before the area darkened by the tunnel is clearly shown in the image 51 . However, when the preceding vehicle 101 enters from the tunnel entrance, the preceding vehicle 101 becomes dark as shown in FIG. 3B. If the dynamic range of the camera is set based on the brightness of the outside of the tunnel, it becomes difficult to recognize the dark part inside the tunnel, and the preceding vehicle 101 cannot be recognized. As a result, object information cannot be acquired, and effects such as continuation of ACC control, for example, occur. Therefore, when the preceding vehicle 101 enters the tunnel, it is assumed that the preceding vehicle 101 exists for a certain period of time, and the estimation information is output to the driving support device 12 . It should be noted that the same is true at the exit of the tunnel, and as shown in FIG. disappears from the image when it exits the tunnel. In this case as well, the estimation information is output to the driving assistance device 12 on the assumption that the preceding vehicle 101 exists.

[1-2. process]
Next, information output processing executed by the information processing device 13 will be described using the flowchart of FIG. This process is performed periodically, for example, each time an image is acquired by the right camera 11a and the left camera 11b.

First, in S1, the first determination unit 33 of the information processing device 13 performs environment tunnel determination processing. Here, it is determined whether or not there is a tunnel entrance in front of the vehicle 3 based on an image of the front of the vehicle captured by the stereo camera 11 . The image to be subjected to the tunnel determination process may be a parallax image, or may be an image that is not a parallax image captured by either the right camera 11a or the left camera 11b, for example. In the following description, if a parallax image or another image that captures the same range is acceptable, it will simply be referred to as a captured image. Details of the environment tunnel determination process will be described later.

In S<b>2 , the information processing device 13 determines whether or not it is determined in S<b>1 that there is a tunnel entrance in front of the vehicle 3 . When the information processing device 13 determines in S1 that there is a tunnel entrance/exit, the information processing device 13 proceeds to S3. On the other hand, the information processing device 13 terminates the information output process of FIG. 5 when it is not determined in S1 that there is a tunnel entrance/exit.

In S3, the second determination unit 34 of the information processing device 13 performs an object tunnel determination process. Here, based on the acquired parallax images, the object detected in front of the vehicle (that is, the preceding vehicle) is either a "tunnel entrance object" located at the tunnel entrance or a "tunnel exit object" located at the tunnel exit. or neither. Details of the object tunnel determination process will be described later.

In S4, when the information processing device 13 determines in S3 that the object is a tunnel entrance object, the information processing device 13 proceeds to S5, sets the object tunnel state to "entrance", and then proceeds to S7. On the other hand, when the information processing device 13 determines in S3 that the object is the tunnel exit object, the information processing device 13 proceeds to S6, sets the object tunnel state to "Exit", and then proceeds to S7. When the information processing device 13 determines in S4 that the object is neither a tunnel entrance object nor a tunnel exit object, the information output processing of FIG. 5 is terminated after clearing the setting of the object tunnel state.

In S7, the information processing device 13 performs extrapolation processing of the estimated information. After that, the information output processing of FIG. 5 is ended. Note that extrapolation processing will be described later.
Next, the environment tunnel determination process executed in S1 of the information output process of FIG. 5 will be described using the flowchart of FIG.

First, in S21, the first determination unit 33 performs execution condition determination, and determines whether or not a predetermined implementation condition is satisfied. Specifically, the implementation conditions here are that the vehicle 3 is traveling at 50 km/h or more, that it is not raining, that it is not nighttime, and that the road on which it is traveling is a straight road. . When all of these requirements are met, it is determined that the implementation conditions have been met. The requirements of the implementation conditions are not limited to the contents described above. For example, other requirements than those mentioned above may be included. Moreover, it is not necessary to include all of the above requirements, and only a part of the requirements may be included.

When the first determination unit 33 determines in S21 that the execution condition is not satisfied, the process proceeds to S30. On the other hand, when the first determination unit 33 determines in S21 that the execution condition is satisfied, the process proceeds to S22.

In S22, the first determination unit 33 sets a determination area. Here, as shown in FIG. 7, a central area 61, a road surface area 62, and a peripheral edge area 63 are set from the photographed image 51. FIG. The central area 61 is a region including a vanishing point, and for example, the central position in the image 51 can be set as the central area 61 . The road surface area 62 is an area below the central area 61 . The peripheral edge area 63 is a target area around the central area 61, and is an area between an outer dashed line 63a positioned near the outer edge of the image 51 and an inner dashed line 63b positioned outside the central area 61. is.

In S<b>23 , the first determination unit 33 extracts features of the central area 61 . Here, the luminance difference between the central area 61 and the road surface area 62 is digitized. For example, the difference between the average luminance values of both areas may be obtained as the luminance difference.

In S<b>24 , the first determination unit 33 extracts features of the peripheral edge area 63 . Here, the luminance difference between the central area 61 and the peripheral edge area 63 is quantified. For example, the difference between the average luminance values of both areas may be obtained as the luminance difference.

In S25, the first determination unit 33 performs instantaneous environment determination to determine whether or not there is a tunnel. Here, it is determined whether or not there is a tunnel based on the luminance difference obtained in S23 and the luminance difference obtained in S24. For example, the first determination unit 33 may determine the entrance/exit of a tunnel when the difference in luminance is greater than or equal to a predetermined threshold value. Furthermore, when the brightness value of the central area 61 is larger than other brightness values, it may be determined as a tunnel exit, and when smaller, it may be determined as a tunnel entrance. The determination result obtained in S25 is hereinafter referred to as an instantaneous environment. The instantaneous environment is the result of determining whether or not a tunnel is included in one image to be processed.

When the first determination unit 33 determines that it is not a tunnel in S25, the process proceeds to S29. On the other hand, when the first determination unit 33 determines in S25 that it is a tunnel entrance, it proceeds to S26 and updates the instantaneous environment continuation number. The memory 22 stores the number of consecutive determinations that the instantaneous environment is the entrance/exit of a tunnel. Here, 1 is added to the number of times.

In S27, the first determination unit 33 determines whether or not the number of instantaneous environmental continuations is greater than or equal to a predetermined threshold. The threshold may be, for example, 5 times. The first determination unit 33 proceeds to S28 if the instantaneous environment continuation number is equal to or greater than the predetermined threshold. On the other hand, when the first determination unit 33 determines that the instantaneous environment continuation number is less than the predetermined threshold value, the process proceeds to S29.

In S28, the first determination unit 33 outputs "tunnel entrance" or "tunnel exit" as environment information (hereinafter, output environment) output by this environmental tunnel determination process. After that, the environmental tunnel determination process of FIG. 6 is terminated.

In S29, the first determination unit 33 determines whether or not it is within the output environment holding period. The output environment holding period is a period provided to ensure a state of outputting the output environment in S28 for a certain period of time after the momentary environment is no longer determined to be the tunnel entrance/exit. The output environment holding period starts counting from the time when the environment information indicating that it is the entrance/exit of the tunnel is finally output in S28. When the first determination unit 33 determines in S29 that it is within the output environment holding period, the process proceeds to S28. On the other hand, if the first determination unit 33 determines in S29 that it is not within the output environment holding period, it proceeds to S30, and after outputting "Others" meaning other than the tunnel entrance as the output environment, the output environment shown in FIG. Terminate the environmental tunnel determination process.

Next, the object tunnel determination process executed in S3 of the information output process of FIG. 5 will be described using the flowchart of FIG. The outline of this process is to determine an object for which the following two characteristics (a) and (b) are obtained as an object entering or exiting a tunnel.

(a) When an object detected from an image enters or exits a tunnel, blacks are crushed or highlights are blown out due to the large difference in brightness between inside and outside of the tunnel, resulting in a large change in brightness. As a result, the ratio of the number of parallax points is greatly reduced. An edge is a portion where the brightness of adjacent pixels in the horizontal direction or the vertical direction changes abruptly.

First, in S41, the second determination unit 34 performs lost determination. Here, if the object recognition/tracking unit 32 continues tracking the object based on the parallax image, it is determined that the object is not lost, and if tracking is no longer possible, it is determined that the object is lost.

When the second determination unit 34 determines that the game is not lost in S41, the process proceeds to S42. On the other hand, when the second determination unit 34 determines that the game is lost in S41, the process proceeds to S50.

In S42, the second determination unit 34 determines whether or not the execution condition is satisfied. The implementation conditions here are, for example, that the object to be detected is in the same lane as the vehicle 3, that the same object is continuously tracked for a certain period of time, that the object is a vehicle, that the object is moving The requirement includes a speed of 45 km/h or higher. The requirements of the implementation conditions are not limited to the contents described above. For example, other requirements than those mentioned above may be included. Moreover, it is not necessary to include all of the above requirements, and only a part of the requirements may be included.

When the second determination unit 34 determines in S42 that the execution condition is not satisfied, the process proceeds to S50. On the other hand, when the second determination unit 34 determines in S42 that the execution condition is satisfied, the process proceeds to S43.

In S43, the second determination unit 34 calculates the average luminance/parallax point number ratio. 9A is an image 55A showing the preceding vehicle 101 immediately before entering a tunnel, and FIG. 9B is a parallax image 55B obtained by superimposing parallax points 81 on FIG. 9A. The term "entering a tunnel" as used herein more strictly means entering a space where sunlight is blocked by a tunnel. The distance information of the parallax points can be indicated by colors, for example, but the display of the distance information is omitted for easy understanding of the drawing. 10A is an image 56A showing the preceding vehicle 101 entering a tunnel, and FIG. 10B is a parallax image 56B obtained by superimposing a parallax point 81 on FIG. 10A. 11A is an image 57A showing the preceding vehicle 101 after entering the tunnel, and FIG. 11B is a parallax image 57B obtained by superimposing the parallax points 81 on FIG. 11A.

In S43, the second determination unit 34 obtains the average luminance and parallax point number ratio of the object rectangular frame 71 representing the object. The object rectangular frame 71 is a frame that identifies a predetermined area in the image where the preceding vehicle 101 exists. The object rectangular frame 71 may be set by the object recognition/tracking unit 32 . The average brightness is the average brightness of each pixel included in the object rectangular frame 71 . The parallax point number ratio is a value obtained by dividing the number of parallax points by the area of the object rectangular frame 71 .

In S<b>44 , the second determination unit 34 stores the average luminance/number of parallax point ratio information acquired in S<b>43 in the memory 22 . These pieces of information may be erased when it is determined in S41 that they are lost, or when it is determined in S42 that the conditions are not met.

In S<b>45 , the second determination unit 34 determines the number of stored information of the average luminance/number of parallax points ratio information, and determines whether or not the number is six. The value used as this criterion is not limited to 6, and may be another number.

When the second determination unit 34 determines that the number of stored information is less than 6 in S45, the process proceeds to S50. On the other hand, when the second determination unit 34 determines that the number of stored information is 6 in S45, the process proceeds to S46.

In S46, the second determination unit 34 calculates the change rate of the object average brightness. As shown in FIG. 9A, the preceding vehicle 101 is brightly displayed just before entering the tunnel, but the bright range is reduced in FIG. 10A and is almost non-existent in FIG. 11A. The average brightness within the object rectangular frame 71 sharply decreases upon entering the tunnel. Therefore, as shown in FIG. 12, the rate of change in average brightness decreases as the preceding vehicle enters the tunnel. On the other hand, a reverse change occurs when escaping from a tunnel. Therefore, the rate of change of the average luminance serves as a criterion for determining whether the preceding vehicle is entering a tunnel. The method of obtaining the rate of change is not particularly limited. A rate of change may be determined for the average value.

In S47, the second determination unit 34 calculates the change rate of the object parallax point number ratio. As shown in FIG. 9B, the edge of the preceding vehicle 101 is clear just before entering the tunnel, but the range of the clear edge decreases in FIG. 10B, and the clear range becomes very small in FIG. 11B. The number of parallax points of the object rectangular frame 71 representing the object decreases sharply when entering the tunnel. Therefore, as shown in FIG. 12, the parallax point number ratio change rate decreases as the preceding vehicle enters the tunnel. On the other hand, a reverse change occurs when escaping from a tunnel. Therefore, the rate of change in the number of parallax points serves as a criterion for determining whether the preceding vehicle is entering a tunnel. The method of obtaining the rate of change is not particularly limited, but for example, of the accumulated six parallax point number ratio information, the average value of the three parallax point number ratios with the oldest acquisition time is used as a reference, and the three parallax point numbers with the latest acquisition time are used as a reference. Further, the rate of change of the average value of the ratios may be obtained.

In S48, the second determination unit 34 determines an instantaneous tunnel state. Here, if the rate of change obtained in S47 and S48 both exceeds a predetermined threshold value, it is determined as "tunnel entry" or "tunnel exit".

In S49, the second determination unit 34 updates the instantaneous tunnel state continuation number. The memory 22 stores the number of times the result of the momentary tunnel state is continuously determined as "entering the tunnel" or "exiting the tunnel". Here, 1 is added to the number of times.

In S50, the second determination unit 34 outputs the tunnel state. Here, if it continues for a predetermined number of times (for example, two times) or more in S49, "tunnel entrance object" or "tunnel exit object" is output. Otherwise, output "other". After that, the object tunnel determination process of FIG. 8 is terminated.

[1-3. Extrapolation]
The extrapolation process executed in S7 of FIG. 5 will be described. Extrapolation processing is processing for outputting estimated information to the driving support device 12 . The output unit 36 outputs the object information to the driving support device 12 when the extrapolation process is not performed. The extrapolation process may be performed when object information is not output to the driving assistance device 12 . The period for executing the extrapolation process can be set as follows.

The extrapolation process may be started at the timing when the object recognition/tracking unit 32 cannot normally track the object while the object tunnel state is determined to be “tunnel exit” or “tunnel entrance”.

The timing at which the object cannot be recognized normally may be the timing at which the object recognition/tracking unit 32 cannot track the object momentarily (that is, in one parallax image). Further, in a configuration in which driving support processing such as ACC is stopped when object tracking failure continues for a certain period of time (for example, three frames), the timing may be set after the above-described certain period of time has elapsed. In other words, as shown in FIG. 13, the first object recognition state "lost" is not extrapolated because the recognition is restored before the lapse of a certain period of time. Extrapolation is performed from that point.

The start of extrapolation processing is not limited to the timing described above, and can be set to start at various timings related to the entry or exit of an object into or out of a tunnel.
The extrapolation process may end after a predetermined stop time has elapsed. The stop time is the time set based on the arrival time, which is the time until the vehicle 3 reaches the position of the preceding vehicle 101 at the timing when the output of the estimated information to the driving support device 12 is started. As shown in FIG. 14, the arrival time can be obtained from the distance from the vehicle 3 to the preceding vehicle 101 and the running speed of the vehicle 3 . The stop time may be the same as the above arrival time, or may be set longer or shorter than the arrival time. When the stop time and the arrival time are the same, it can be said that the extrapolation process ends when the vehicle 3 reaches the position of the preceding vehicle 101 at the start timing of the extrapolation process. Further, as shown in FIG. 14, when the following distance is determined by ACC, the passage of time between vehicles by ACC may be used as the arrival time. Also, when setting the stop time, as shown in FIG. The stop time may be determined including the period of "lost". Note that the extrapolation process may end at a timing other than these.

[1-4. effect]
According to the embodiment detailed above, the following effects are obtained.
(1a) When the information processing device 13 determines that a tunnel entrance exists in front of the vehicle 3 and the preceding vehicle passes through the tunnel entrance, that is, the object recognition/tracking unit 32 becomes difficult to track the preceding vehicle. Output estimated information in situations where it is easy to Therefore, the duration of driving support can be extended.

(1b) The information processing device 13 determines whether or not there is a tunnel entrance in front of the vehicle 3 based on the brightness of the image showing the front of the vehicle 3 . Therefore, the tunnel entrance/exit can be determined with high accuracy.

(1c) The information processing device 13 determines that the preceding vehicle has entered or exited the tunnel based on both the luminance change rate of the area indicating the preceding vehicle in the image and the parallax point change rate in the parallax image. To detect. Therefore, it is possible to determine with high accuracy whether the preceding vehicle is entering or exiting a tunnel.

(1d) The information processing device 13 ends the output of the estimated information to the driving support device 12 after the stop time set based on the time required for the vehicle 3 to reach the position of the preceding vehicle at the timing when the output is started. do. Therefore, until the vehicle 3 enters the tunnel or exits the tunnel, the output of the estimated information can be maintained, and driving support such as ACC continues until the object recognition/tracking unit 32 recovers recognition of the preceding vehicle. make it easier to

(1e) The information processing device 13 outputs the estimated information to the driving support device 12 when the preceding vehicle 101 cannot be tracked normally by the object recognition/tracking unit 32 . Therefore, while the preceding vehicle 101 is being tracked normally, highly accurate driving assistance can be provided based on the object information rather than the estimated value.

(1f) According to the information processing device 13, by using the parallax points included in the parallax images acquired by the stereo camera 11, highly accurate object recognition is possible.
[2. Other embodiments]
Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments, and various modifications can be made.

(2a) In the above embodiment, the parallax point was exemplified as the feature point included in the peripheral image, but the present invention is not limited to this. For example, when detecting edge points from a captured image and using the edge points, the edge points may be used as feature points. If points other than parallax points such as edge points are used as feature points, a camera other than a stereo camera may be used. Note that an edge point is a portion where the brightness of pixels adjacent in the horizontal direction or the vertical direction changes abruptly.

(2b) In the above embodiment, the content of the driving assistance executed by the driving assistance device 12 is ACC. However, the driving assistance executed by the driving assistance device 12 may be driving assistance other than ACC. For example, the driving assistance device 12 may be configured to perform a lane departure prevention function or an automatic driving function.

(2c) In the above embodiment, the configuration in which the first determination unit 33 detects a tunnel based on the brightness of the central area 61, the road surface area 62, and the peripheral edge area 63 of the photographed image was exemplified, but the present invention is not limited to this. not something. For example, a tunnel may be detected by recognizing the shape of the tunnel through image processing, or may be detected by comparing the current position of the vehicle 3 with a map including position information of the tunnel.

(2d) In the above embodiment, the configuration in which the second determination unit 34 determines whether or not the preceding vehicle enters the tunnel based on the luminance of the preceding vehicle and the rate of change in the number of parallax points in the image was exemplified, but the configuration is limited to this. not a thing For example, for either or both of the luminance and the number of parallax points, it may be determined whether or not the requirements are satisfied based on whether or not the average luminance or the number of parallax points exceeds a predetermined threshold, instead of the rate of change. Alternatively, either one of the luminance and the number of parallax points may be used, or another parameter may be further considered to determine tunnel entry.

(2e) The object rectangular frame 71, i.e., the predetermined area, exists within the lane 91 on which the vehicle 3 is traveling and the lane 92 adjacent thereto, as shown in the image 58 of FIG. It may be set for the preceding vehicles 101a and 101b. Further, when a plurality of object rectangular frames 71a and 71b exist in the image 58, for each of the plurality of object rectangular frames, the determination by the second determination unit 34, the estimation information acquisition by the estimation unit 35, and the output unit 36 may be configured to be performed.

(2f) In the above embodiment, the output conditions for the extrapolation process are (i) that the first determination unit 33 determines that the entrance/exit of the tunnel exists in front of the vehicle 3, and (ii) the second determination unit. 34 has exemplified the configuration including as a requirement that the preceding vehicle is determined to pass through the entrance/exit. However, the output conditions are not limited to the above contents. For example, like the information processing device 13A shown in FIG. One of the requirements of the output condition may be that the unit 41 determines that the vehicle is in a straight running state. The determination by the road determination unit 41 may be performed based on the image captured by the imaging device, or may be performed by comparing the current position of the vehicle 3 with map information including information indicating whether the road is a straight road. good too.

Further, as in the information processing apparatus 13A shown in FIG. 16, a nighttime determination unit 42 is provided for determining whether or not it is nighttime. It may be configured not to output estimated information. With such a configuration, it is possible to reduce the risk of erroneous determination when there is a risk that the accuracy of tunnel detection will be degraded.

(2g) Information processors 13, 13A and techniques described in this disclosure are provided by configuring a processor and memory programmed to perform one or more functions embodied by a computer program. may be implemented by a dedicated computer designed for Alternatively, the information processing apparatus and techniques described in this disclosure may be implemented by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the information processing apparatus and method thereof described in the present disclosure are a combination of a processor and memory programmed to perform one or more functions and a processor configured by one or more hardware logic circuits. may be implemented by one or more dedicated computers configured by Computer programs may also be stored as computer-executable instructions on a computer-readable non-transitional tangible storage medium. The method of realizing the functions of each part included in the information processing apparatus does not necessarily include software, and all the functions may be realized using one or more pieces of hardware.

(2h) A plurality of functions possessed by one component in the above embodiment may be realized by a plurality of components, or a function possessed by one component may be realized by a plurality of components. . Also, a plurality of functions possessed by a plurality of components may be realized by a single component, or a function realized by a plurality of components may be realized by a single component. Also, part of the configuration of the above embodiment may be omitted. Moreover, at least part of the configuration of the above embodiment may be added or replaced with respect to the configuration of the other above embodiment.

(2i) In addition to the information processing device 13 described above, a system having the information processing device 13 as a component, a program for causing a computer to function as the information processing device 13, and a non-transitional device such as a semiconductor memory storing this program The present disclosure can also be implemented in various forms such as a physical recording medium and a raindrop determination method.

REFERENCE SIGNS LIST 1 driving support system 3 vehicle 11 stereo camera 12 driving support device 13, 13A information processing device 33 first determination unit 34 second determination unit 35 estimation unit 36 Output unit 55B, 56B, 57B... parallax image

Claims (9)

An information processing device (13, 13A) used in a vehicle (3) including an imaging device (11) and a driving support device (12),
a first determination unit (33) configured to determine that a tunnel entrance exists in front of the vehicle;
a second determination unit (34) configured to determine that a preceding vehicle traveling in front of the vehicle passes through the entrance;
Estimated information, which is information indicating the estimated behavior of the preceding vehicle, is acquired based on peripheral images (55B, 56B, 57B) that are images acquired using the photographing device and that indicate the periphery of the vehicle. an estimator (35) configured to:
an output unit (36) configured to output the estimated information acquired by the estimation unit to the driving support device when output conditions including at least the following requirements (i) and (ii) are satisfied ; ,
(i) the first determination unit determines that the doorway exists in front of the vehicle; (ii) the second determination unit determines that the preceding vehicle passes through the doorway;
a detection unit (31) configured to detect a feature point indicating the preceding vehicle in the peripheral image;
with
The second determination unit determines that a rate of change in brightness of a predetermined area in the surrounding image where the preceding vehicle exists is greater than a predetermined threshold, and a rate of change in the number of feature points in the predetermined area is greater than a predetermined threshold. The information processing device configured to determine that the preceding vehicle passes through the doorway when the vehicle ahead passes through the doorway . The information processing device according to claim 1 ,
The information processing device, wherein the predetermined area is set for an object existing in a range of a lane (91) on which the vehicle is traveling and a lane (92) adjacent to the lane. The information processing device according to claim 1 or claim 2 ,
If a first range including a vanishing point in an image taken in front of the vehicle is brighter or darker than a second range surrounding the first range, the first determination unit an information processing device configured to determine that the doorway exists in the . The information processing device according to any one of claims 1 to 3 ,
The output unit terminates the output of the estimated information to the driving support device after a stop time set based on the time required for the vehicle to reach the position of the preceding vehicle at the timing when the output is started. An information processing device configured to: The information processing device according to any one of claims 1 to 3 ,
Further comprising a tracking unit (32) for tracking the preceding vehicle based on the peripheral image,
The information processing device, wherein the output unit is configured to output the estimated information to the driving support device when the tracking unit cannot normally track the preceding vehicle. The information processing device according to any one of claims 1 to 5 ,
The imaging device is a stereo camera,
The peripheral image is a parallax image acquired using the stereo camera,
The information processing apparatus, wherein the feature point is a parallax point (81) shown in the parallax image. The information processing device according to any one of claims 1 to 6 ,
Furthermore, a road determination unit (41) configured to determine whether or not the vehicle is in a straight running state in which the vehicle is running on a straight road,
The information processing device, wherein the output condition includes, as one of requirements, that the road determination unit determines that the vehicle is in a straight running state. The information processing device according to any one of claims 1 to 7 ,
Furthermore, a night determination unit (42) that determines whether it is night or not,
The information processing apparatus, wherein the output unit is configured not to output the estimated information when the nighttime determination unit determines that it is night time. A driving support system (1) mounted on a vehicle (3) and used,
an imaging device (11);
a driving support device (12);
A driving support system comprising the information processing device (13) according to any one of claims 1 to 8 .

Images