JP7279390B2 - electronic controller - Google Patents

Description

The present disclosure relates to an electronic control device mounted on a vehicle.

2. Description of the Related Art Conventionally, a technique is known in which a captured image of the surroundings of a vehicle is acquired from a camera mounted on the vehicle, and the distance between the vehicle and objects around the vehicle is output based on the captured image.
For example, Patent Literature 1 below discloses a technique for allowing the driver to grasp the distance between an object and the vehicle by superimposing a distance display line indicating the distance from the rear of the vehicle on the captured image.

JP 2010-184604 A

Further, for example, as a technique for allowing the driver to grasp the distance between an object and the vehicle, there is known a technique for generating a bird's-eye view image based on a captured image of the surroundings of the vehicle and displaying the bird's-eye view image on a display device. .

However, in the bird's-eye view image, the three-dimensional object is enlarged toward the distance and displayed distorted. Moreover, the greater the height from the road surface, the greater the distortion. As a result, in a bird's-eye view image, a situation may arise in which the display of a three-dimensional object causes misrecognition regarding distance, shape, and the like.

One aspect of the present disclosure is to provide a technique capable of suppressing inconvenience due to misrecognition in a situation where display of a three-dimensional object in a bird's eye view image may cause misrecognition.

One aspect of the present disclosure is an electronic control device (30) mounted on a vehicle, comprising a captured image acquisition unit (35), a bird's eye view image generation unit (36, 61), a trajectory estimation unit (37), A situation determination unit (38, S150, S160) and an output unit (38, S210) are provided. The captured image acquisition unit repeatedly acquires a captured image of the surroundings of the vehicle including the ground from a camera mounted on the vehicle. The bird's eye view image generator generates a bird's eye view image based on the captured image. The trajectory estimator predicts the trajectory of the vehicle.

The situation determination unit determines whether or not the target object, which is an object existing on the predicted trajectory of the vehicle predicted by the trajectory estimation unit, is displayed in the bird's-eye view image in a caution situation. Configured. The output unit is configured to perform an output based on the determination result of the situation determination unit when the situation determination unit determines that the situation is a caution situation.

The attention situation refers to, for example, a situation in which misidentification of a three-dimensional object based on a bird's-eye view image may occur.
As a result, an output is made based on the result of determination as to whether or not it is a caution situation. Based on this output, for example, a misrecognition by the driver looking at the bird's-eye view image or an error in vehicle control based on the recognition of the bird's-eye view image is performed. Inconvenience due to misidentification in caution situations can be suppressed.

The block diagram which shows the structure of a driving assistance device. FIG. 4 is an explanatory diagram for explaining the position of the camera and the imaging range of the camera; Explanatory drawing explaining the function of ECU. Explanatory drawing explaining the feature point in a captured image. Explanatory drawing explaining the detection of the distance by a stereo camera. FIG. 4 is an explanatory diagram for explaining that the estimated distance and the detected distance match at the grounding point; Explanatory drawing explaining the object which exists on an orbit, and the object which exists above an orbit. 4 is a flowchart of output generation processing; Explanatory drawing explaining a non-attentive situation. Explanatory drawing explaining a caution situation. Explanatory drawing explaining a proximity non-grounding point.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.
[1. composition]
A driving assistance device 1 shown in FIG. 1 is installed in a vehicle such as a passenger car, and is a device for assisting the driver of the vehicle in driving. Hereinafter, the vehicle equipped with the driving assistance device 1 is also referred to as the own vehicle. The driving support device 1 includes an electronic control unit (hereinafter referred to as ECU) 30 . The driving assistance device 1 may include a camera unit 10, a sensor group 20, a notification control device 40, a vehicle control device 50, and a distance detection device 70.

The camera section 10 has at least one camera. In this embodiment, the camera section 10 includes a front camera 11 , a left side camera 12 , a right side camera 13 and a rear camera 14 . Each camera 11-14 is configured using a CCD image sensor or a CMOS image sensor.

As shown in FIG. 2 , the front camera 11 is installed, for example, on the bumper at the front end of the vehicle 9 so that the road surface in front of the vehicle becomes an imaging range 101 . The left side camera 12 is installed, for example, on the left side mirror so that the road surface on the left side of the vehicle becomes the shooting range 102 . The right side camera 13 is installed, for example, on the right side mirror so that the road surface on the right side of the vehicle becomes the shooting range 103 . The rear camera 14 is installed, for example, on the bumper at the rear end of the vehicle 9 so that the road surface behind the vehicle becomes the shooting range 104 .

Each of the cameras 11-14 repeatedly captures images at preset time intervals, and outputs captured images, which are captured images, to the ECU 30 via, for example, a dedicated line or an in-vehicle bus such as CAN. CAN is a registered trademark. The captured image is an image captured by the camera unit 10, that is, each camera 11 to 14, and is an image of the surroundings of the vehicle 9 including the road surface.

Returning to FIG. 1, the description continues. The sensor group 20 includes a plurality of sensors for detecting vehicle information. The vehicle information is information detected by the sensors included in the sensor group 20 and is information related to travel of the vehicle 9 . Vehicle information may include information such as vehicle speed, steering angle, direction of travel, and the like. A plurality of sensors output vehicle information, which is a detection result, to the ECU 30 via, for example, a dedicated line or an in-vehicle bus such as CAN.

The sensor group 20 can include sensors such as a vehicle speed sensor 21, a steering angle sensor 22, and a shift detection sensor 23, for example. A vehicle speed sensor 21 detects the speed of the vehicle 9 . A steering angle sensor 22 detects the steering angle of the vehicle 9 . The shift detection sensor 23 detects that the traveling direction of the vehicle 9 is backward when the shift lever is set to reverse.

The notification control device 40 is a device for notifying the driver. The notification control device 40 includes a microcomputer (hereinafter referred to as a microcomputer) 41 having a CPU, a ROM, a RAM, and a semiconductor memory such as a flash memory, and a notification device 42 that executes notification.

The notification device 42 includes a display device 43 for displaying images and a speaker 44 for outputting sounds. The display device 43 is arranged within the range of the driver's field of vision in the driver's seat in the vehicle interior. The display device 43 may include, for example, a liquid crystal display installed on a console panel or the like, a head-up display using a windshield, or the like.

The microcomputer 41 outputs image signals output from the ECU 30 to the display device 43 . Thereby, the image output from the ECU 30 is displayed on the display device 43 .
The speaker 44 is arranged inside the vehicle 9 . The microcomputer 41 outputs sound to the speaker 44 based on the sound output signal output from the ECU 30 . The audio output signal here is a signal (hereinafter referred to as a trigger signal) that causes the notification control device 40 to operate. The audio signal output from the speaker 44 may be an audio signal stored in the notification control device 40 in advance, or may be an audio signal transmitted from the ECU 30 .

The vehicle control device 50 is a device for performing various controls on the vehicle 9 . The vehicle control device 50 includes a microcomputer 51 and a control execution device 52 . The control execution device 52 is, for example, a device or an actuator for controlling the acceleration/deceleration of the vehicle 9, the steering angle of the vehicle 9, such as a brake, an accelerator, or a steering wheel. The microcomputer 51 controls the control execution device 52 based on the output signal from the ECU 30 . The output signal here is a trigger signal that activates the vehicle control device 50 .

The microcomputer 51, for example, generates a drive signal for operating a brake based on an output signal from the ECU 30 and moving the vehicle 9 according to a predicted trajectory which will be described later. The drive signal is a signal for driving the control execution device 52 such as brake, accelerator, steering, and the like. The microcomputer 51 outputs drive signals to the control execution device 52 .

In this manner, the microcomputer 51 drives the control execution device 52 so as to move the vehicle 9 based on the predicted trajectory, for example, when traveling. Further, the microcomputer 51 drives the control execution device 52 so as to move the vehicle 9 based on the predicted trajectory when the vehicle is parked in a parking space or the like.

The distance detection device 70 is a stereo camera in this embodiment. The stereo camera repeatedly images the front of the vehicle 9 including the road surface at a predetermined cycle, and outputs the captured images to the ECU 30 via, for example, a dedicated line or an in-vehicle bus such as CAN.

In this embodiment, the driving support device 1 includes a stereo camera as the distance detection device 70 as a separate configuration from the camera unit 10, but the present invention is not limited to this. A camera included in the camera unit 10 may be configured to also serve as a stereo camera as the distance detection device 70 . A stereo camera as the distance detection device 70 may use a plurality of cameras, or may use a single camera. When using one camera, the stereo camera may perform triangulation based on two captured images captured by the one camera as the vehicle moves. Moreover, the distance detection device 70 is not limited to a stereo camera. The distance detection device 70 may be any device capable of detecting the distance to an object, such as a sonar or radar device.

The ECU 30 includes a microcomputer having a CPU 31, a ROM 33, a RAM 34, and a semiconductor memory (hereinafter referred to as memory) 32 such as a flash memory. Each function realized by the ECU 30 is realized by the CPU executing a program stored in a non-transitional substantive recording medium. In this example, the memory 32 corresponds to a non-transitional substantive recording medium storing programs. Also, by executing this program, a method corresponding to the program is executed. The ECU 30 may have one microcomputer or may have a plurality of microcomputers.

The ECU 30 includes a signal input section 35, an image processing section 36, a trajectory estimation section 37, and an output generation section 38, as shown in FIG. The image processing unit 36 also includes a bird's-eye view conversion unit 61 , a distance estimation unit 62 , an object detection unit 63 , a grounding point estimation unit 64 , and a non-grounding part estimation unit 65 .

The method of realizing the functions of these units is not limited to software, and some or all of the functions may be realized using one or more pieces of hardware. For example, when the above functions are realized by an electronic circuit that is hardware, the electronic circuit may be realized by a digital circuit, an analog circuit, or a combination thereof.

The signal input unit 35 repeatedly acquires captured images captured by the cameras 11 to 14 and supplies them to the image processing unit 36 . In addition, the signal input unit 35 acquires vehicle information such as vehicle speed and steering angle detected by the sensor group 20 and the distance detection device 70, detection results, etc. It is supplied to the output generator 38 .

The bird's eye conversion unit 61 generates a bird's eye view image. The bird's-eye view image referred to here is an image obtained by converting an image captured by the camera unit 10 into an image from a viewpoint above the vehicle 9 . Specifically, the bird's-eye view conversion unit 61 converts the images captured by the four cameras 11 to 14 into a bird's-eye view viewed from a preset virtual viewpoint, synthesizes them, and generates a bird's-eye view image showing the surroundings of the vehicle 9. do. The bird's eye view conversion unit 61 stores the generated bird's eye view image in the memory 32 .

The distance estimating unit 62 estimates the distance from the vehicle 9 to the feature point of the object existing around the vehicle 9 based on the image captured by the camera unit 10 . The object here means a tangible object. The tangible objects may include, for example, pedestrians, other vehicles, various structures such as guardrails, fences, signboards, etc., and various structures such as buildings. The feature point here is at least one pixel that is determined as a part of the object in the captured image. The distance estimation unit 62 detects feature points from the captured image using techniques such as pattern matching and scene recognition.

For example, a feature point can be at least one pixel that is determined to be part of another vehicle. Other vehicle parts may include tires, bumpers, license plates, and the like. Also for example, the feature point may be at least one pixel determined to be part of the fence 200, as shown in FIG. 9 described below. A portion of the fence 200 may include legs 210, body 220, and the like.

The distance estimation unit 62 records in the memory 32 the actual distance (hereinafter referred to as the estimated distance) from the vehicle 9 to the feature point estimated from the captured image. A real distance is a distance in a three-dimensional space. Specifically, the distance estimating unit 62 uses a table that associates the image distance, which is the distance from the lower end D of the captured image 431, with the actual distance, as shown in FIG. (hereinafter, estimated distance). The table is stored in the memory 32 in advance.

In the table, based on the image distance Li to the point at infinity P and the image distance to the feature point of the object Q, it is assumed that each feature point exists on the road surface, and the actual distance to the point at infinity P corresponds to The actual distance of each feature point is associated as the estimated distance.

For example, for the feature point ga of the object Q, the estimated distance is calculated based on the image distance La assuming that the feature point ga exists on the road surface. Similarly, for other feature points gb of the object Q, the estimated distance is calculated based on the image distance Lb assuming that the feature point gb exists on the road surface.

Also, the distance estimation unit 62 estimates the direction of each feature point viewed from the vehicle 9 based on the captured image. The distance estimator 62 stores the result of estimating the direction of each feature point in the memory 32 together with the estimated distance.

In this embodiment, the distance estimating section 62 estimates the estimated distance and direction of the feature point of the object in front of the vehicle 9 based on the image captured by the front camera 11 . Further, the distance estimating section 62 estimates the estimated distance and direction of the feature point of the object behind the vehicle 9 based on the image captured by the rear camera 14 .

However, it is not limited to this. The distance estimating unit 62 estimates the distance and orientation of the feature points of the object in the vicinity of the vehicle 9 such as the front, rear, left, right, etc. of the vehicle 9 based on images captured by each camera included in the camera unit 10. can be configured to detect.

Returning to FIG. 3, the description continues. The object detection section 63 detects the actual distance from the vehicle 9 to the feature point based on the output of the distance detection device 70 . In the present embodiment, as shown in FIG. 5, the distance M to each feature point gc on the object Q is detected based on triangulation using images from a stereo camera as the distance detection device 70 .

However, it is not limited to this. The distance detection device 70 may be, for example, a device capable of detecting the actual distance from the vehicle 9 to the feature point, such as a sonar or radar device. That is, the object detection unit 63 can be configured to detect the actual distance from the vehicle 9 to the feature point based on the output from the distance detection device 70 such as a camera, sonar, radar device, or the like.

Hereinafter, the actual distance from the vehicle 9 to the feature point, which is detected based on the output of the distance detection device 70, will be referred to as the detected distance. Also, the object detection unit 63 detects the orientation of the feature point viewed from the vehicle 9 based on the output of the distance detection device 70 . The object detection unit 63 stores in the memory 32 the detection results of the detection distance and orientation for each feature point.

Returning to FIG. 3, the description continues. The contact point estimator 64 estimates contact points from among the feature points. A grounding point is a feature point that is grounded among the feature points. Being on the ground means being on the road surface. In other words, the contact point is the feature point that is in contact with the road surface.

As described above, the estimated distance and the detected distance are calculated for each feature point. The contact point estimating unit 64 associates the feature points detected by the distance estimating unit 62 with the feature points detected by the object detecting unit 63, and calculates the estimated distance and the detected distance for each of the associated feature points. Determine if they match.

Correlation of feature points can be performed based on detected orientations, images, and the like. Then, the contact point estimating section 64 estimates a feature point where the estimated distance and the detected distance match as the contact point. The contact point estimator 64 stores in the memory 32 information indicating that the feature point estimated to be the contact point among the feature points is the contact point.

The reason why the grounding point is estimated by such a method will be described with reference to FIG. In the object Q shown in FIG. 6, a feature point 301 is a grounded point and a feature point 302 is a non-grounded point. A non-grounded point is a feature point that is not grounded among the feature points.

Here, regarding the feature point 301, the distance in the horizontal direction from the observation point R of the camera unit 10 and the distance detection device 70 to the feature point (hereinafter referred to as the horizontal distance) is the result of detection by the camera unit 10 and the distance detection device 70. All of the detection results are Lc. That is, the estimated distance and the detected distance are the same for the feature point 301 as the ground point. It should be noted that the term “match” here is not limited to match in a strict sense, and does not have to be a strict match as long as the same effect can be achieved.

On the other hand, regarding the feature point 302 as the non-grounding point, in the image captured by the camera unit 10, the distance in the horizontal direction from the observation point R to the feature point is at a position 310 farther than the actual position of the feature point 302. is observed. Therefore, for the feature point 302, the estimated distance Ld is larger than the detected distance Lc. That is, the estimated distance and the detected distance do not match for the feature point 302 as the non-grounded point.

For this reason, the contact point estimating section 64 estimates a feature point where the estimated distance and the detected distance match as the contact point. For example, a characteristic point at the lower end of the tire of another vehicle, a characteristic point at the lower end of the leg 210 of the fence 200, or the like can be estimated as the contact point.

The non-grounded portion estimation unit 65 estimates non-grounded points from among the feature points. A non-grounded point is a feature point that is not grounded as described above, and is a feature point that is present above the road surface and is floating in the air.

The non-grounded portion estimating unit 65 associates the feature points detected by the distance estimating unit 62 with the feature points detected by the object detecting unit 63, and selects the estimated feature points from the associated feature points. A feature point where the distance and the detected distance do not match is estimated as a non-grounded point. The non-grounded portion estimation unit 65 stores in the memory 32 information indicating that the feature points estimated to be the non-grounded points are the non-grounded points.

For example, characteristic points on bumpers and license plates of other vehicles, characteristic points on main body 220 of fence 200, and the like can be estimated as non-grounding points. Also, the bumper, the license plate, the body portion 220 of the fence 200, and the like correspond to the non-grounding portion. A non-grounded portion is a portion of a three-dimensional object that does not include a grounded point and includes only a non-grounded point. The non-grounded part is not grounded and is floating in the air from the road surface.

Note that the method for estimating the grounding point and non-grounding point of the detected object is not limited to this, and various known methods can be used.
The trajectory estimation unit 37 estimates a predicted trajectory based on vehicle information output from the sensor group 20 . The predicted trajectory is a trajectory that the vehicle 9 is predicted to travel from now on. For example, the trajectory estimation unit 37 estimates the predicted trajectory based on the above-described vehicle information such as the vehicle speed, steering angle, whether the vehicle is moving in reverse, and the traveling direction. The trajectory estimator 37 estimates a predicted trajectory from the current time to the end of a predetermined prediction period. The prediction time can be set, for example, several seconds to several tens of seconds later.

The trajectory estimation unit 37 may estimate the predicted trajectory when the vehicle 9 is reversed, such as when parking in a parking space. Further, the trajectory estimation unit 37 may estimate the predicted trajectory when the vehicle 9 moves forward.

Specifically, as shown in FIG. 9 to be described later, the predicted trajectory 130 is the predicted trajectory of the vehicle 9 . In this embodiment, the predicted trajectory 130 is a band-like region of a predetermined width between the predicted left tire trajectory 110 and the predicted right tire trajectory 120 . The width of the predicted trajectory 130 can be set to the same width as the width between the left and right tires. However, it is not limited to this. The width of the predicted trajectory can be set to a width narrower than the width between the left and right tires, the same width as the vehicle width, or a width wider than the vehicle width by a predetermined width.

In this way, the trajectory estimator 37 calculates the predicted trajectory 130 of a predetermined width from the current time point to the end of the prediction period based on at least one of the predicted left tire trajectory 110 and the predicted right tire trajectory 120 . to estimate The trajectory estimator 37 expresses the position of the estimated predicted trajectory, for example, in rectangular coordinates using distance or polar coordinates using azimuth and distance, and stores it in the memory 32 as an estimation result.

Note that the method of estimating the predicted trajectory is not limited to this. Various known methods can be used to calculate the predicted trajectory.
The output generation unit 38 is configured to execute an output generation process, which will be described later, and to perform an output based on the determination result when it is determined that the attention state is present. The attention situation means, for example, a situation in which display of a three-dimensional object in a bird's-eye view image may cause misrecognition.

When it is determined that the above-mentioned caution state is present corresponds to when the target object is in the caution state. The target object here is an object represented by a plurality of feature points and existing on the predicted trajectory of the vehicle 9 . "Existing on the predicted trajectory" includes both existing adjacent to the predicted trajectory and existing above the predicted trajectory. In other words, the feature points contacting the predicted trajectory are included in the target object described above. In addition, feature points existing above the predicted trajectory are also included in the above-described target object.

Here, "existing in contact with the predicted trajectory" means "existing on the predicted trajectory", such as the object S1 shown in FIG. say. For example, the tires of other vehicles, the legs 210 of the fence 200, and the like are examples of objects that "exist in contact with the predicted trajectory."

On the other hand, "existing above the predicted trajectory" means, for example, the object S2 shown in FIG. say. For example, the bumpers and license plates of other vehicles, the main body 220 of the fence 200, and the like are examples of objects that "exist above the predicted trajectory."

That is, in the above example, each of tires, bumpers, license plates, and other vehicles including these correspond to target objects. Also, each of the leg portion 210 and the body portion 220, and the fence 200 including these correspond to the target object.

[2. process]
Next, output generation processing executed by the output generation unit 38 will be described using the flowchart of FIG. The output generation process is repeatedly executed at a predetermined cycle of, for example, several milliseconds to several tens of milliseconds.

First, in S100, the output generation unit 38 acquires an image captured by the camera unit 10. FIG.
Next, in S<b>110 , the output generation unit 38 acquires the bird's eye view image generated by the bird's eye conversion unit 61 and outputs the acquired bird's eye view image to the display device 43 . Thereby, the bird's-eye view image is displayed on the display device 43 .

Subsequently, in S120, the output generator 38 acquires vehicle information, which is various detection results by the sensor group 20. FIG.
Next, in S<b>130 , the output generator 38 acquires the position of the predicted trajectory, which is the result of estimation by the trajectory estimator 37 .

Subsequently, the output generation unit 38 acquires the processing result by the image processing unit 36 in S140. The processing results include the estimated distance and position of each feature point estimated by the distance estimating section 62, the detected distance and position of each feature point detected by the object detecting section 63, and the ground point among the feature points. Information, information indicating that feature points are non-grounded points, and the like may be included.

Next, in S150, the output generator 38 determines whether or not the user is in a non-attentive state. A non-attentive situation refers to a situation that is not an attentive situation. In a non-attentive situation, display of a three-dimensional object in a bird's-eye view image is less likely to cause misrecognition. Specifically, the output generation unit 38 determines that the inattentive state is present when at least one contact point is in contact with the predicted trajectory. In other words, it is determined that it is not in a caution state. The at least one ground point is a ground point included in the target object. If the touchdown point is tangent to the predicted trajectory, then the touchdown point is within the predicted trajectory represented on the road surface.

Here, the output generator 38 determines that the non-attentive state is present when the ground contact point estimated by the ground contact point estimating section 64 is in contact with the predicted trajectory. Also, the output generation unit 38 determines that the non-attentive state is not present when the ground contact point estimated by the ground contact point estimation unit 64 is not in contact with the predicted trajectory.
For example, in FIG. 9, at least one of the grounding points 211 included in the legs 210 of the fence 200 is in contact with the predicted trajectory 130, so it is determined that the situation is inattentive. As for the contact point, the distance from the vehicle 9 is accurately reflected when the image captured by the camera unit 10 is converted into the bird's-eye view image. Therefore, when the grounding point of the target object is in contact with the predicted trajectory 130, it is considered unlikely that the target object will be misidentified, and it is determined that the target object is in a non-attentive state.

Here, the output generation unit 38 shifts the process to S<b>200 when it is determined that the user is in a non-attentive state. On the other hand, the output generation unit 38 shifts the process to S160 when determining that the situation is not inattentive.
In S160, the output generation unit 38 determines whether or not it is a caution situation. Specifically, the output generation unit 38 determines that the caution situation is present when at least one ground contact point is not in contact with the predicted trajectory. The fact that the ground contact point is not in contact with the predicted trajectory means that the ground contact point exists outside the range of the predicted trajectory represented on the road surface.

In the present embodiment, the output generator 38 determines that the caution situation is present when the non-grounded point estimated by the non-grounded portion estimator 65 exists above the predicted trajectory. An object that includes a non-grounding point that exists above the predicted trajectory corresponds to the target object.

For example, in FIG. 10, in the fence 200, the grounding point 211 included in the leg 210 is not in contact with the predicted trajectory 130, and the non-grounding point 212 included in the main body 220 exists above the predicted trajectory 130. Therefore, it is judged to be in a caution situation.

Here, the output generation unit 38 shifts the process to S<b>170 if it determines that it is in a caution state, and ends this output generation processing if it determines that it is not in a caution state.
The output generator 38 estimates the height of the lower end of the target object in S170. The target object here is an object that includes a non-grounding point that exists above the predicted trajectory and that is floating in the air. Height is the height from the road surface. For example, in the situation of FIG. 10, the output generator 38 estimates the height of the lower end of the main body 200 with the main body 220 as the target object. The output generator 38 estimates the height from the road surface to the feature point at the lower end of the main body 200 as the height of the lower end of the main body 200 based on the detection result by the object detection unit 63 .

The ECU 30 may be configured to estimate the height of the lower end of the target object in a process different from the output generation process executed by the output generation section 38 . The output generator 38 may then be configured to acquire the estimated height in this step.

Next, in S180, the output generation unit 38 determines whether or not the difference between the height of the lower end of the target object and the height of the vehicle 9 (hereinafter referred to as vehicle height) is less than a predetermined allowable threshold. do. The target object here is an object including a non-grounding point above the predicted trajectory in S160. The vehicle height and allowable threshold are stored in the memory 32 in advance. The permissible threshold is set to a distance that allows the vehicle 9 not to come into contact with the lower end of the target object.

The output generation unit 38 shifts the process to S210 when the difference between the height of the lower end of the target object and the vehicle height is less than the allowable threshold, and shifts the process to S190 when the difference is equal to or greater than the allowable threshold. Let

In S190, the output generator 38 determines whether or not the speed of the vehicle 9 (hereinafter referred to as vehicle speed) is equal to or higher than a predetermined speed (hereinafter referred to as vehicle speed threshold). The vehicle speed is acquired at S120. The vehicle speed threshold is pre-stored in the memory 32 . The vehicle speed threshold can be set to a speed that allows the driver to stop the running vehicle 9 by a momentary braking operation. Specifically, the vehicle speed threshold can be set to a vehicle speed, such as several kilometers per hour or less. This vehicle speed is also suitable for operating the vehicle 9 when parked.

Here, the output generation unit 38 shifts the process to S210 when the vehicle speed is equal to or higher than the vehicle speed threshold, and ends the output generation process when the vehicle speed is less than the vehicle speed threshold.
The output generator 38 determines whether or not there is an adjacent ungrounded point in S200 to which it is determined that the vehicle is in a non-attentive state. The near non-grounding point referred to here is a non-grounding point that exists above the predicted trajectory 130, and the distance from the vehicle 9 is greater than the distance from the vehicle 9 to the grounding point that is in contact with the predicted trajectory 130 specified in S150. is a small ungrounded point. The object that includes the ground contact point on the predicted trajectory 130 identified in S150 corresponds to the target object.

For example, as shown in FIG. 11, the output generator 38 is included in the target object Q at a position closer to the vehicle 9 than the ground contact gd included in the target object Q in the horizontal direction by the non-grounded portion estimator 65. If the non-grounded point ge is detected, it is determined that there is an adjacent non-grounded point. Proximity ground points are included in the target object. The presence of a close non-grounding point means that the object exists at a position closer to the vehicle 9 than the grounding point of the target object.

Here, when the output generation unit 38 determines that there is a proximate ungrounded point, the process proceeds to S210. On the other hand, when the output generation unit 38 determines that there is no proximity non-grounded point, the output generation processing ends.

In S210, the output generation unit 38 performs an output based on the determination result of whether or not it is a caution situation. In the present embodiment, the output generator 38 causes the notification device 42 to notify the occupants of the vehicle 9 including the driver in the caution situation. The notification referred to here is notification that calls attention to the bird's-eye view image, and in particular, notification that calls attention to the distance from the vehicle 9 in the bird's-eye view image.

For example, the notification can be performed in a manner in which an image showing predetermined notification content is displayed on the display device 43 by being superimposed on the bird's-eye view image. Further, for example, the notification may be performed in such a manner that the speaker 44 is caused to output a sound indicating the predetermined content of the notification. The voice may be a message indicating the contents of the notification, or may be a warning sound such as a buzzer sound.

The content of the notification may include content such as "be careful as there is a possibility that an obstacle exists near the vehicle 9". In addition, the content of the notification may include content such as "although it may not be displayed in the bird's-eye view image, there is a possibility that an obstacle exists near the vehicle 9, so be careful." . In addition, the content of the above-mentioned notification may include content such as "be careful because the accuracy of the distance from the own vehicle 9 to the obstacle in the bird's-eye view image may not be good".

The aforementioned obstacles are target objects represented by the aforementioned ungrounded points. In other words, it is a floating object that exists above the predicted trajectory.
Note that, in the present embodiment, when the output generator 38 determines that there is a proximity non-grounding point, the output generation unit 38 generates a signal to warn the occupant of the vehicle 9 that there is a possibility of collision with the proximity non-grounding point. Notify.

The output generation unit 38 terminates the output generation process after making these notifications.
[3. effect]
According to the embodiment detailed above, the following effects can be obtained.

(3a) In the bird's-eye view image, when the captured image is converted to the bird's-eye view image, the three-dimensional object is displayed as if it were tilted toward the road surface, and the display of the three-dimensional object in the bird's eye view image causes misrecognition regarding the distance, shape, etc. Attentional situations can arise.

Therefore, the ECU 30 performs an output based on the determination result when it is determined in S210 that the caution situation is present. As a result, in this embodiment, based on the output, it is possible to suppress inconvenience caused by misrecognition in a caution situation. Inconvenience here includes, for example, erroneous recognition by the driver looking at the bird's-eye view image, erroneous vehicle control based on the bird's-eye view image, and the like.

For example, in S210, the ECU 30 outputs an output for causing the notification device 42, which performs notification, to perform notification based on the determination result that the caution situation is present. As a result, it is possible to call attention to the occupant of the vehicle 9 in the caution situation.

(3b) In S140, the ECU 30 acquires the estimation result from the contact point estimator 64 configured to estimate the contact point of the target object. In S160, the ECU 30 determines whether or not the grounding point of the target object is in contact with the predicted trajectory, and if the grounding point of the target object is not in contact with the predicted trajectory, the ECU 30 determines that the caution state is present. That is, when the target object exists above the predicted trajectory, the above-described misrecognition may occur with respect to the display of the three-dimensional object in the bird's-eye view, and such a situation is determined to be a warning situation.

As a result, a situation in which the target object exists above the predicted trajectory can be detected as a warning situation. Then, it is possible to make a notification in such a caution situation.
(3c) In S180, the ECU 30 acquires the height of the lower end of the target object when it is determined to be in a caution situation, and if the difference between the height of the lower end of the target object and the vehicle height is less than the allowable threshold, determine whether there is As a result, it can be determined whether or not there is a risk of collision between the target object and the vehicle 9 in the height direction. Then, it becomes possible to perform various outputs, controls, etc., using the determination result.

(3d) When it is determined in S210 that the caution situation is present and the difference between the height of the lower end of the target object and the vehicle height is less than the allowable threshold value, the ECU 30 outputs an output based on the determination result. I do. In the present embodiment, the ECU 30 issues a notification when it is determined that the caution situation is present and the height difference is less than the allowable threshold.

For example, in the attention situation shown in FIG. 10, the ECU 30 issues a notification when it is determined that the difference between the height h1 of the lower end of the main body 220, which is the target object, and the vehicle height h0 is less than the allowable threshold.

As a result, it is possible to warn the occupant of the vehicle 9 that there is a risk of collision between the target object and the vehicle 9 in the height direction in the warning situation. Then, collisions between the target object and the vehicle 9 can be suppressed.

(3e) In S190, the ECU 30 determines whether or not the speed of the vehicle 9 is greater than or equal to the vehicle speed threshold. At S210, the ECU 30 performs an output based on the determination result, at least when it is determined that the caution situation is present and the vehicle speed is equal to or higher than the vehicle speed threshold.

In this embodiment, the ECU 30 determines that it is in a caution situation, determines that the difference between the height of the lower end of the target object and the vehicle height is greater than or equal to the allowable threshold, and further determines that the vehicle speed is greater than or equal to the vehicle speed threshold. If it is determined that the above is the case, the notification is made based on the result of such determination.

For example, in the attention situation shown in FIG. 10, even if it is determined that the difference between the height h1 of the lower end of the main body 220, which is the target object, and the vehicle height h0 is equal to or greater than the allowable threshold, When the speed is equal to or higher than the vehicle speed threshold, the ECU 30 notifies. As a result, the driver of the vehicle 9 can be urged to drive safely so as not to drive faster than necessary.

(3f) In S160, the ECU 30 determines that the caution situation is not present when the ground contact point of the target object is in contact with the predicted trajectory. This is because, when the grounding point of the target object touches the predicted trajectory, misrecognition of the grounding point of the target object is less likely to occur in the bird's-eye view image. As a result, a situation in which the target object is in contact with the predicted trajectory can be detected as a warning situation.

(3g) In S200, when the ECU 30 determines that the caution state is not present, the target object is a non-grounding point and the distance from the vehicle 9 is smaller than the distance from the vehicle 9 to the grounding point of the target object. It is determined whether or not there is a nearby ungrounded point. The ECU 30 causes the notification device 42 to perform notification when it is determined that there is a close non-grounded portion.
As a result, for example, as shown in FIG. 11, when there is an object Q whose ground contact point is in contact with the predicted trajectory and which protrudes toward the vehicle 9, a warning is issued as a caution situation. Therefore, the occupants of the vehicle 9 can be alerted.

[4. Other embodiments]
Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments, and various modifications can be made.

(4a) The ECU 30 does not output based on the determination result when it is determined that the caution situation is present and the difference between the height of the lower end of the target object and the vehicle height is equal to or greater than the allowable threshold. It may be configured as Outputting here may mean performing notification. In other words, the ECU 30 may be configured not to perform notification when it is determined as described above.

In this modification, in the flowchart of FIG. 8, the ECU 30 is configured to end the output generation process in S180 when the difference between the height of the lower end of the target object and the vehicle height is less than the allowable threshold. . Also, S190 is deleted.

For example, in the caution situation shown in FIG. 10, when it is determined that the difference between the height h1 of the lower end of the main body 220, which is the target object, and the vehicle height h0 is greater than or equal to the allowable threshold, the ECU 30 does not issue a notification. . As a result, even in a caution situation, in a situation where there is little risk of collision between the target object and the vehicle 9 in the height direction, it is possible to prevent the occupants of the vehicle 9 from being annoyed by the notification.

(4b) The ECU 30 is configured to perform an output based on the determination result when it is determined that the caution situation is present, and is configured not to perform an output based on the determination result when it is determined that the caution situation is not present. may be Outputting here may mean performing notification.

In this modification, in the flowchart of FIG. 8, the ECU 30 is configured to end the output generation process when it is determined in S150 that the vehicle is in a non-attentive state. Further, the ECU 30 is configured to shift the process to S210 when it is determined in S160 that the situation is a caution situation, and to perform notification in S210. S200 is deleted. Also, S170-S190 are deleted.

As a result, it is possible to prompt the occupant of the vehicle 9 to notify the driver whenever there is a caution situation. Further, for example, the occupant of the vehicle 9 is bothered by an unnecessary notification rather than a comparison device configured to always perform some notification irrespective of whether or not there is a warning situation regarding the display of a three-dimensional object in a bird's-eye view image. can be suppressed.

(4c) In the above embodiment, the output generation unit 38 is configured to perform the above notification in S210, but the present invention is not limited to this. For example, the output generation unit 38 may be configured to output such as to stop the control of the vehicle 9 based on the bird's-eye view image when it is determined that the caution situation is present. The control of the vehicle 9 based on the bird's-eye view image referred to here includes running the vehicle 9 along the predicted trajectory 130 in the bird's-eye view image, or running the vehicle 9 so as to avoid collision with an object detected in the bird's-eye view image. , and the like.

As a result, errors in vehicle control based on the bird's-eye view image can be suppressed.
(4d) In the above embodiment, in the ECU 30, the output generator 38 determines in S150 whether or not the vehicle is in a caution situation based on whether or not the ground contact point is located on the predicted trajectory 130. Further, in S160, the output generation unit 38 determines that the caution situation is not present based on the fact that the non-grounding point is located above the predicted trajectory 130. FIG. However, it is not limited to this.

In S150, the output generator 38 may identify a group of feature points including a plurality of feature points as part of the object. Then, the output generation unit 38 may determine that the caution situation is not present when the feature point at the lower end of the cluster of feature points is in contact with the predicted trajectory 130 . Here, a part of the object represented by the group of feature points corresponds to the target object. In addition, a feature point at the lower end of the group of feature points and in contact with the predicted trajectory 130 corresponds to the grounding point.

In addition, in S160, the output generation unit 38 determines that the caution situation is present when the feature point at the lower end of the group of feature points described above is not in contact with the predicted trajectory 130 and exists above the predicted trajectory 130. You may Here, a part of the object represented by the group of feature points corresponds to the target object. Further, a feature point located at the lower end of the group of feature points and located above the predicted trajectory 130 corresponds to a non-grounding point.

(4e) ECU 30 and techniques described in this disclosure can be performed by a dedicated computer provided by configuring a processor and memory programmed to perform one or more functions embodied by a computer program. , may be implemented. Alternatively, the ECU 30 and techniques described in this disclosure may be implemented by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the ECU 30 and techniques described in this disclosure may comprise a combination of a processor and memory programmed to perform one or more functions and a processor configured by one or more hardware logic circuits. may be implemented by one or more dedicated computers configured as Computer programs may also be stored as computer-executable instructions on a computer-readable non-transitional tangible storage medium. The method of realizing the function of each part included in the ECU 30 does not necessarily include software, and all the functions may be realized using one or more pieces of hardware.

(4f) A plurality of functions possessed by one component in the above embodiment may be realized by a plurality of components, or a function possessed by one component may be realized by a plurality of components. . Also, a plurality of functions possessed by a plurality of components may be realized by a single component, or a function realized by a plurality of components may be realized by a single component. Also, part of the configuration of the above embodiment may be omitted. Moreover, at least part of the configuration of the above embodiment may be added or replaced with respect to the configuration of the other above embodiment.

(4g) In various forms, such as the above-described ECU 30, the driving support device 1 including the ECU 30, a program for functioning the ECU 30, a non-transitional substantive recording medium such as a semiconductor memory recording this program, and a driving support method The present disclosure can also be implemented.

In addition, in the above embodiment, the ECU 30 corresponds to the electronic control unit. Further, the signal input unit 35 corresponds to the captured image acquisition unit, the image processing unit 36 and the bird's eye view conversion unit 61 correspond to the bird's eye view image generation unit, and the output generation unit 38 corresponds to the situation determination unit, the output unit, the height determination unit, It corresponds to a vehicle speed determination section, a proximity determination section, and a proximity notification section. Further, S150 and S160 correspond to the processing of the situation determination unit, S210 corresponds to the processing of the output unit, S180 corresponds to the processing of the height determination unit, and S190 corresponds to the processing of the vehicle speed determination unit. S200 corresponds to the processing of the proximity determination unit, and S210 corresponds to the processing of the proximity notification unit.

30 ECU, 35 signal input unit, 36 image processing unit, 37 trajectory estimation unit, 38 output generation unit.

Claims (7)

An electronic control device (30) mounted on a vehicle,
a captured image acquisition unit (35) that repeatedly acquires captured images of the surroundings of the vehicle including the ground from a camera mounted on the vehicle;
a bird's-eye view image generator (36, 61) for generating a bird's-eye view image based on the captured image;
a trajectory estimation unit (37) for predicting the trajectory of the vehicle;
For a target object existing on the predicted trajectory which is the trajectory of the vehicle predicted by the trajectory estimation unit, it is determined whether or not the display of the target object in the bird's eye view image is in a caution situation. the situation judgment unit (38, S150, S160),
an output unit (38, S210) configured to perform an output based on the judgment result of the situation judgment unit when the situation judgment unit judges that it is the caution situation;
with
The situation determination unit (S160) determines whether or not a contact point of the target object, which is a contact point between the target object and the road surface, touches the predicted trajectory. An electronic control unit configured to determine that the caution situation exists when the predicted trajectory is not in contact with the trajectory. The electronic control device according to claim 1,
When it is determined that the caution situation is present, the height of the lower end of the target object is obtained, and whether the difference between the height of the lower end and the height of the vehicle is less than a predetermined allowable threshold. A height determination unit (38, S180) configured to determine whether
An electronic controller further comprising: The electronic control device according to claim 2,
The output unit determines that the situation determination unit determines that the caution situation exists and that a difference between the height of the lower end of the target object and the height of the vehicle is less than a predetermined allowable threshold. An electronic control unit configured to output based on the determination result when the determination result is determined. The electronic control device according to claim 2 or 3,
The output unit determines that the caution situation has been determined by the situation determination unit, and that a difference between the height of the lower end of the target object and the height of the vehicle is equal to or greater than a predetermined allowable threshold. An electronic control device configured not to output based on the determination result when the determination result is determined. The electronic control device according to any one of claims 1 to 4,
further comprising a vehicle speed determination unit (38, S190) configured to determine whether the speed of the vehicle is equal to or greater than a predetermined vehicle speed threshold;
The output unit outputs an output based on the determination result when the situation determination unit determines that the caution situation is present and determines that the speed of the vehicle is equal to or greater than the vehicle speed threshold. Configured electronic controller. The electronic control device according to any one of claims 2 to 5,
The situation determination unit (S150) is configured to determine that the caution situation is not present when the ground contact point of the target object is in contact with the predicted trajectory. The electronic control device according to claim 6,
When it is determined that the caution situation is not present, there is a non-grounding point on the target object, the proximity non-grounding point having a distance from the vehicle smaller than the distance from the vehicle to the grounding point of the target object. a proximity determination unit (38, S200) configured to determine whether or not there is;
a proximity notification unit (38, S210) configured to notify a notification device that performs notification when the proximity determination unit determines that there is the proximity non-grounding point;
An electronic controller further comprising:

Images