JP6832164B2 - Driving assistance device and driving assistance method - Google Patents

Description

The present invention relates to a driving assist device and a driving assist method for assisting a driver's driving operation.

Patent Document 1 has an object of providing a white line detection device capable of detecting a white line more effectively even if an intersection exists.

In order to solve this problem, the white line detecting device according to Patent Document 1 detects an imaging means for imaging the road surface in front of the vehicle, a detecting means for detecting an intersection based on the captured image, and a white line on the road surface. It includes a white line detecting means for detecting a white line and a setting means for setting at least one of a plurality of threshold values used for searching a linear approximation region for detecting a white line when an intersection exists.

Japanese Unexamined Patent Publication No. 2014-149636

In Patent Document 1, when the edge of the white line is detected and recognized as an intersection, the approximate straight line is extended far from the edge of the white line on the front side of the intersection to reduce the sensitivity of the threshold value, and the distant white line on the approximate straight line is reduced. I am trying to search the area (edge of the white line).

However, since the sensitivity of the threshold value is lowered to search for the edge of the white line, another object or dirt on the road can be detected as the edge of the white line far from the intersection, or the white line edge on the right side on the front side can be detected. There is a risk of connecting to the white line edge on the left side on the back side with an approximate straight line. At intersections and the like, the length of the lane mark candidates to be extracted is short in the first place, and it is difficult to correctly recognize the lane mark candidates on the front side.

The present invention has been made to solve the above-mentioned problems, and by interpolating between the detected front lane mark candidate and the back lane mark candidate, for example, a virtual lane mark can be accurately created at an intersection. It is an object of the present invention to provide a driving assistance device and a driving assistance method that can be set and enable suitable automatic driving according to a plan of a vehicle movement route.

[1] The first driving assisting device according to the present invention is a driving assisting device that assists a driver's driving operation, and is predetermined on an image and a lane mark detecting unit that detects a lane mark existing on a road. Interpolation processing that interpolates between the lane mark candidate on the back side and the lane mark candidate on the front side when a specific scene in which lane mark candidates exist on the back side and the front side is detected at intervals of a distance or more. It is characterized by having a part.

Conventionally, when the white line on the front side is detected and then the white line on the back side is detected on the approximate straight line, it is recognized as a lane mark. Therefore, another object or dirt on the road can be detected at a distance from the intersection. It may be detected as the edge of the white line, or the white line edge on the right side on the front side and the white line edge on the left side on the back side may be connected by an approximate straight line. At intersections and the like, the length of the lane mark candidates to be extracted is short in the first place, and it is difficult to correctly recognize the lane mark on the front side.

However, in the first invention, when a specific scene in which lane mark candidates exist on the back side and the front side at intervals of a predetermined distance or more is detected on the image, the lane mark candidates on the back side and the front side are detected. Since the interpolation process is performed between the lane mark candidates and the lane mark candidates, the virtual lane mark can be set with high accuracy. That is, it is possible to make the ECU or the like recognize the virtual lane mark with high accuracy and with a small load, and it is possible to perform suitable automatic driving according to the plan of the own vehicle movement route.

[2] In the first invention, in the interpolation processing unit, the interpolation processing unit is within a predetermined range in a direction in which the lane mark candidate on the back side and the lane mark candidate on the front side are orthogonal to the lane mark candidate on the front side. If this is the case, interpolation processing may be performed between the lane mark candidate on the back side and the lane mark candidate on the front side.

As a result, a pair of virtual lane marks can be set accurately at an intersection where there is no actual lane mark, and the ECU or the like can recognize the virtual lane mark accurately and with a small load. , It enables suitable automatic driving according to the plan of the own vehicle movement route.

[3] In the first invention, in the interpolation process, a virtual lane mark is set between the lane mark candidate on the back side and the lane mark candidate on the front side, and the lane mark candidate on the back side is set. And the lane mark candidate on the front side are connected.

[4] In the first invention, the interpolation processing unit excludes an area between the area where the lane mark candidate on the back side exists and the area where the lane mark candidate on the front side exists, and the back side. The side area and the front area may be connected.

As a result, it is possible to connect the lane mark candidate on the back side and the lane mark candidate on the front side, so that even if the sensor or the like is shaken, the lane mark candidate on the back side and the lane on the front side can be surely connected. Interpolation processing can be performed between the mark candidates.

Even if the lane mark candidate on the back side and the lane mark candidate on the front side are not connected, the distance between the lane mark candidate on the back side and the lane mark candidate on the front side becomes short, so that the sensor or the like is shaken. Even if it occurs, a virtual lane mark can be surely set between the lane mark candidate on the back side and the lane mark candidate on the front side.

[5] Further, the interpolation processing unit may set a plurality of three or more regions along the road and perform the interpolation processing when a lane mark candidate is not detected in the region between the regions.

Scenes in which lane marks exist on the back side and the front side include intersections and lanes with dashed lane marks. Since the lane on which the broken line lane mark is drawn is drawn as a broken line on the image, the ECU or the like can recognize it as a lane mark.

On the other hand, at intersections and the like, since the solid line lane mark and the broken line lane mark are not drawn, the ECU or the like cannot recognize the traveling lane. Therefore, when the candidate for the lane mark is not detected in the area between the three or more areas, that is, when the dashed lane mark or the like is not detected, the interpolated lane mark is virtually generated. By extending it to a long lane mark, it can be recognized as a virtual lane mark.

That is, in a situation where the interpolation process does not need to be performed, the waste of performing the interpolation process can be eliminated, the recognition load by the ECU or the like can be suppressed, and recognition without a time lag can be performed.

[6] In the first invention, the specific scene may be an intersection. At the intersection, the solid line lane mark and the broken line lane mark are not drawn, so that the ECU or the like cannot recognize the traveling lane. Therefore, by performing the above-mentioned interpolation processing, it can be recognized as a virtual lane mark at an intersection.

[7] The second driving assisting method according to the present invention is a driving assisting method for assisting a driver's driving operation, and lane marks are provided on the back side and the front side at intervals of a predetermined distance or more on the image. It is characterized by having a step of detecting a specific scene in which is present, and a step of interpolating between the lane mark candidate on the back side and the lane mark candidate on the front side based on the detection of the specific scene. ..

According to the driving assistance device and the driving assistance method according to the present invention, for example, a virtual lane mark can be set accurately at an intersection or the like, and suitable automatic driving according to the plan of the own vehicle movement route is possible. Can be done.

It is a block diagram which shows the structure of the vehicle which includes the traveling electronic control device as a driving assistance device which concerns on this embodiment. It is a flowchart which shows the overall flow of the automatic operation control. It is a functional block diagram concerning the configuration of the 1st peripheral monitoring unit, mainly the peripheral monitoring control related to the recognition of intersections and the interpolation of virtual lane marks to intersections and the like. FIG. 4A is an explanatory diagram showing an example of an image of a specific scene in a camera image, and FIG. 4B is an explanatory view showing an example of converting a specific scene into a bird's-eye view (BEV: bird's eye view) image (hereinafter referred to as a BEV image). It is a figure. FIG. 5A is an explanatory diagram showing the distance between the first lane mark candidate on the back side and the third lane mark candidate on the front side in the direction orthogonal to the third lane mark candidate on the front side, and FIG. 5B is the third lane mark candidate on the back side. It is explanatory drawing which shows the distance in the direction orthogonal to the 4th lane mark candidate of the front side in 2 lane mark candidate and the 4th lane mark candidate of the front side. In FIG. 6A, the virtual left lane mark is set so as to interpolate the first lane mark candidate and the third lane mark candidate, and the virtual right lane is interpolated between the second lane mark candidate and the fourth lane mark candidate. FIG. 6B is an explanatory diagram showing an example in which marks are set, and FIG. 6B is an explanatory diagram showing an example of an image in which two lane marks extending along the traveling direction of the own vehicle are drawn in the intersection. It is a flowchart which shows the 1st processing operation of the 1st interpolation processing part. It is a flowchart which shows the 2nd processing operation of the 1st interpolation processing part. It is a functional block diagram concerning the configuration of the 2nd peripheral monitoring unit, mainly the peripheral monitoring control related to the recognition of intersections and the interpolation of virtual lane marks to intersections and the like. FIG. 10A is an explanatory diagram showing a state in which a BEV image of a specific scene is separated into four regions, and FIG. 10B is an explanatory diagram showing a state in which a region on the back side and a region on the front side are connected. FIG. 11A is an explanatory diagram showing a state in which the first lane mark candidate and the third lane mark candidate are not connected, and the second lane mark candidate and the fourth lane mark candidate are not connected, and FIG. 11B is an explanatory diagram. A virtual left lane mark is set to interpolate the first lane mark candidate and the third lane mark candidate, and a virtual right lane mark is set to interpolate the second lane mark candidate and the fourth lane mark candidate. It is explanatory drawing which shows the example. It is a flowchart which shows the processing operation of the 2nd interpolation processing part.

Hereinafter, examples of embodiments of the driving assistance device and the driving assistance method according to the present invention will be described with reference to FIGS. 1 to 12.

FIG. 1 is a block diagram showing a configuration of a vehicle 10 including a traveling electronic control device 36 (hereinafter referred to as “traveling ECU 36”) as a driving assistance device according to an embodiment of the present invention.

In addition to the traveling ECU 36, the vehicle 10 (hereinafter, also referred to as “own vehicle 10”) includes a vehicle peripheral sensor group 20, a vehicle body behavior sensor group 22, a driving operation sensor group 24, a communication device 26, and a human machine. It has an interface 28 (hereinafter referred to as "HMI 28"), a driving force control system 30, a braking force control system 32, and an electric power steering system 34 (hereinafter referred to as "EPS system 34").

The vehicle peripheral sensor group 20 detects information about the periphery of the vehicle 10 (hereinafter, also referred to as “vehicle peripheral information Ic”). The vehicle peripheral sensor group 20 includes a plurality of external cameras 50, a plurality of radars 52, a LIDAR 54 (Light Detection And Ranging), and a global positioning system sensor 56 (hereinafter referred to as “GPS sensor 56”). included. Each captured image (hereinafter referred to as camera image 206) from the plurality of vehicle exterior cameras 50 is drawn in the corresponding image memory. Of course, a single camera may be used for recognizing the lane mark.

The plurality of vehicle exterior cameras 50 output image information images that capture the periphery (front, side, and rear) of the vehicle 10. The plurality of radars 52 output radar information radar indicating reflected waves for electromagnetic waves transmitted to the periphery (front, side, and rear) of the vehicle 10. The LIDAR 54 continuously emits a laser in all directions of the vehicle 10, measures the three-dimensional position of the reflection point based on the reflected wave, and outputs the three-dimensional information Ilider. The GPS sensor 56 detects the current position Pcur of the vehicle 10. The vehicle exterior camera 50, radar 52, LIDAR 54, and GPS sensor 56 are peripheral recognition devices that recognize vehicle peripheral information Ic.

The vehicle body behavior sensor group 22 detects information on the behavior of the vehicle 10 (particularly the vehicle body) (hereinafter, also referred to as “vehicle body behavior information Ib”). The vehicle body behavior sensor group 22 includes a vehicle speed sensor 60, a lateral acceleration sensor 62, and a yaw rate sensor 64.

The vehicle speed sensor 60 detects the vehicle speed V [km / h] of the vehicle 10. The lateral acceleration sensor 62 detects the lateral acceleration Glat [m / s / s] of the vehicle 10. The yaw rate sensor 64 detects the yaw rate Yr [rad / s] of the vehicle 10.

The driving operation sensor group 24 detects information related to the driving operation by the driver (hereinafter, also referred to as "driving operation information Io"). The driving operation sensor group 24 includes an accelerator pedal sensor 80, a brake pedal sensor 82, a steering angle sensor 84, and a steering torque sensor 86.

The accelerator pedal sensor 80 (hereinafter, also referred to as “AP sensor 80”) detects the operation amount θap (hereinafter, also referred to as “AP operation amount θap”) [%] of the accelerator pedal 90. The brake pedal sensor 82 (hereinafter, also referred to as “BP sensor 82”) detects the operation amount θbp of the brake pedal 92 (hereinafter, also referred to as “BP operation amount θbp”) [%]. The steering angle sensor 84 detects the steering angle θst (hereinafter, also referred to as “operation amount θst”) [deg] of the steering handle 94. The steering torque sensor 86 detects the steering torque Tst [Nm] applied to the steering handle 94.

The communication device 26 performs wireless communication with an external device. The external device here includes, for example, a traffic information server (not shown). The traffic information server provides traffic information such as traffic jam information, accident information, and construction information to each vehicle 10. Alternatively, the external device may include a route guidance server (not shown). The route guidance server generates or calculates a planned route Rv to the target point Pgoal on behalf of the traveling ECU 36 based on the current position Pcur of the vehicle 10 and the target point Pgoal received from the communication device 26.

The communication device 26 of the present embodiment is assumed to be mounted (or always fixed) on the vehicle 10, but can be carried to the outside of the vehicle 10 such as a mobile phone or a smartphone. It may be.

The HMI 28 accepts operation input from the occupant and presents various information to the occupant visually, audibly and tactilely. The HMI 28 includes an automatic operation switch 110 (hereinafter, also referred to as “automatic operation SW110”) and a display unit 112. The automatic operation SW110 is a switch for instructing the start and end of automatic operation control by the operation of an occupant. In addition to or in place of the automatic operation SW110, it is also possible to command the start or end of the automatic operation control by another method (voice input via a microphone (not shown) or the like). The display unit 112 includes, for example, a liquid crystal panel or an organic EL panel. The display unit 112 may be configured as a touch panel.

The driving force control system 30 includes an engine 120 (drive source) and a drive electronic control device 122 (hereinafter referred to as "drive ECU 122"). The AP sensor 80 and the accelerator pedal 90 described above may be positioned as a part of the driving force control system 30. The drive ECU 122 executes the drive force control of the vehicle 10 by using the AP operation amount θap and the like. In the driving force control, the driving ECU 122 controls the traveling driving force Fd of the vehicle 10 through the control of the engine 120.

The braking force control system 32 includes a braking mechanism 130 and a braking electronic control device 132 (hereinafter referred to as “braking ECU 132”). The BP sensor 82 and the brake pedal 92 described above may be positioned as part of the braking force control system 32. The brake mechanism 130 operates the brake member by a brake motor (or a hydraulic mechanism) or the like.

The braking ECU 132 executes braking force control of the vehicle 10 by using the BP operation amount θbp or the like. When controlling the braking force, the braking ECU 132 controls the braking force Fb of the vehicle 10 through the control of the brake mechanism 130 and the like.

The EPS system 34 includes an EPS motor 140 and an EPS electronic control device 142 (hereinafter referred to as "EPS-ECU 142"). The steering angle sensor 84, steering torque sensor 86, and steering handle 94 described above may be positioned as part of the EPS system 34.

The EPS-ECU 142 controls the EPS motor 140 in response to a command from the traveling ECU 36 to control the turning amount R of the vehicle 10. The turning amount R includes a steering angle θst, a lateral acceleration Glat, and a yaw rate Yr.

The traveling ECU 36 executes automatic driving control for driving the vehicle 10 to the target point Pgoal without requiring a driving operation by the driver, and includes, for example, a central processing unit (CPU). The ECU 36 has an input / output unit 150, a calculation unit 152, and a storage unit 154.

It is also possible to assign a part of the function of the traveling ECU 36 to an external device existing outside the vehicle 10. For example, the vehicle 10 itself does not have the action planning unit 172 and / or the map database 190, which will be described later, and can acquire the planned route Rv and / or the map information Imap from the route guidance server.

The input / output unit 150 performs input / output with devices other than the ECU 36 (sensor groups 20, 22, 24, communication device 26, etc.). The input / output unit 150 includes an A / D conversion circuit (not shown) that converts the input analog signal into a digital signal.

The calculation unit 152 performs calculation based on signals from the sensor groups 20, 22, 24, the communication device 26, the HMI 28, the ECUs 122, 132, 142, and the like. Then, the calculation unit 152 generates signals for the communication device 26, the drive ECU 122, the braking ECU 132, and the EPS-ECU 142 based on the calculation result.

As shown in FIG. 1, the calculation unit 152 of the travel ECU 36 includes a peripheral recognition unit 170, an action planning unit 172, and a travel control unit 174. Each of these units is realized by executing a program stored in the storage unit 154. The program may be supplied from an external device via the communication device 26. A part of the program may be composed of hardware (circuit parts).

The peripheral recognition unit 170 recognizes road boundaries (lane marks, guardrails, slopes, etc.) and peripheral obstacles (other vehicles, stationary objects, etc.) based on vehicle peripheral information Ic from the vehicle peripheral sensor group 20. For example, road boundaries are recognized based on image information Image. The peripheral recognition unit 170 recognizes the traveling lane of the vehicle 10 based on the recognized road boundary.

In addition, peripheral obstacles are recognized using the image information Image, the radar information Irader, and the three-dimensional information Iridar. Peripheral obstacles include moving objects such as other vehicles (other vehicles and the like) and stationary objects such as buildings and signs (for example, traffic lights). When the peripheral obstacle is a traffic light, the peripheral recognition unit 170 determines the color of the traffic light.

The action planning unit 172 calculates the planned route Rv of the own vehicle 10 to the target point Pgoal input via the HMI 28, and provides route guidance along the planned route Rv.

The travel control unit 174 controls the output of each actuator that controls the vehicle body behavior. The actuator referred to here includes an engine 120, a brake mechanism 130, and an EPS motor 140. The travel control unit 174 controls the behavior amount of the vehicle 10 (particularly the vehicle body) (hereinafter referred to as "vehicle body behavior amount Qb") by controlling the output of the actuator.

The vehicle body behavior amount Qb referred to here includes vehicle speed V, front-rear acceleration α [m / s / s], front-rear deceleration β [m / s / s], steering angle θst, lateral acceleration Glat, and yaw rate Yr. The acceleration α and the deceleration β can be calculated as the time derivative value of the vehicle speed V.

The travel control unit 174 includes a driving force control unit 180, a braking force control unit 182, and a turning control unit 184. The driving force control unit 180 controls the traveling driving force Fd (or acceleration α) of the vehicle 10 mainly by controlling the output of the engine 120. The braking force control unit 182 controls the braking force Fb (or deceleration β) of the vehicle 10 mainly by controlling the output of the braking mechanism 130. The turning control unit 184 mainly controls the turning amount R (or steering angle θst, lateral acceleration Glat, and yaw rate Yr) of the vehicle 10 by controlling the output of the EPS motor 140.

The storage unit 154 stores programs and data (including a map database 190) used by the calculation unit 152. Road map information (map information Imap) is stored in the map database 190 (hereinafter referred to as "map DB 190"). The map information Imap includes road information Iroad regarding the shape of the road and the like.

The storage unit 154 includes, for example, a random access memory (hereinafter referred to as “RAM”). As the RAM, a volatile memory such as a register and a non-volatile memory such as a flash memory can be used. Further, the storage unit 154 may have a read-only memory (hereinafter referred to as “ROM”) in addition to the RAM.

As described above, the traveling ECU 36 of the present embodiment executes automatic operation control. In the automatic driving control, the vehicle 10 is driven to the target point Pgoal without requiring a driving operation by the driver. However, in the automatic driving control, when the driver operates the accelerator pedal 90, the brake pedal 92 or the steering handle 94, the operation may be reflected in the driving. In the automatic driving control of the present embodiment, automatic driving force control, automatic braking force control, and automatic turning control are used in combination.

The automatic driving force control automatically controls the traveling driving force Fd of the vehicle 10. The automatic braking force control automatically controls the traveling driving force Fd of the vehicle 10. The automatic turning control automatically controls the turning of the vehicle 10. The turning of the vehicle 10 referred to here includes not only the case of traveling on a curved road, but also the right / left turn of the vehicle 10, the change of the traveling lane, the merging into another lane, and the maintenance of the traveling lane. Note that turning to maintain the traveling lane means turning (or steering) to maintain the vehicle 10 at a reference position (for example, the center in the vehicle width direction) in the vehicle width direction.

The automatic driving force control controls the traveling driving force Fd to drive the vehicle 10. At this time, the ECU 36 sets a target value (for example, a target engine torque) of the traveling driving force Fd, and controls the actuator (engine 120) according to the target value. Further, the ECU 36 sets an upper limit value αmax of the front-rear acceleration α of the vehicle 10 (hereinafter, also referred to as “acceleration upper limit value αmax”), and controls the traveling driving force Fd so that the front-rear acceleration α does not exceed the acceleration upper limit value αmax. To do.

The automatic braking force control controls the braking force Fb of the vehicle 10 to decelerate the vehicle 10. At this time, the ECU 36 sets a target value of the braking force Fb (for example, deceleration βtar before and after the target), and controls the actuator (brake mechanism 130) according to the target value. Further, the ECU 36 sets an upper limit value βmax of the deceleration β of the vehicle 10 (hereinafter, also referred to as “deceleration upper limit value βmax”) so that the deceleration β does not exceed the deceleration upper limit value βmax (rapid deceleration). Control the braking force Fb (so as not to overdo it).

The automatic turning control controls the turning amount R of the vehicle 10 to turn the vehicle 10. At this time, the ECU 36 sets a target value of the turning amount R (for example, a target steering angle θstar or a target lateral acceleration Glattar), and controls the actuator (EPS motor 140) according to the target value. Further, the ECU 36 sets an upper limit value Rmax of the turning amount R of the vehicle 10 (hereinafter, also referred to as “turning amount upper limit value Rmax”), and sets the turning amount R so that the turning amount R does not exceed the turning amount upper limit value Rmax. Control. The turning amount upper limit value Rmax is used, for example, in the form of the upper limit value θstmax of the steering angle θst or the upper limit value Glatmax of the lateral acceleration Glat.

Here, the overall flow of the automatic operation control of the present embodiment will be described with reference to the flowchart of FIG.

First, in step S11, the traveling ECU 36 determines whether or not to start the automatic operation. For example, the ECU 36 determines whether or not the automatic operation switch 110 (FIG. 1) has been switched from off to on. When starting the automatic operation (step S11: YES), the process proceeds to step S12. When the automatic operation is not started (step S11: NO), the current process is completed, and the process returns to step S11 after a predetermined time has elapsed.

In step S12, the ECU 36 sets the target point Pgoal. Specifically, the input of the target point Pgoal is received from the user (driver or the like) via the HMI 28. In step S13, the ECU 36 calculates the planned route Rv from the current position Pcur to the target point Pgoal. When step S13 is performed after step S21 described later, the ECU 36 updates the planned route Rv.

In step S14, the ECU 36 acquires vehicle peripheral information Ic, vehicle body behavior information Ib, and driving operation information Io from the sensor groups 20, 22, and 24. As described above, the vehicle peripheral information Ic includes the image information Image from the external camera 50, the radar information Irader from the radar 52, the three-dimensional information Irider from the LIDAR 54, and the current position Pcur from the GPS sensor 56. The vehicle body behavior information Ib includes the vehicle speed V from the vehicle speed sensor 60, the lateral acceleration Glat from the lateral acceleration sensor 62, and the yaw rate Yr from the yaw rate sensor 64. The operation operation information Io includes the AP operation amount θap from the AP sensor 80, the BP operation amount θbp from the BP sensor 82, the steering angle θst from the steering angle sensor 84, and the steering torque Tst from the steering torque sensor 86.

In step S15, the ECU 36 calculates the output upper limit value Pmax of each actuator. The actuator referred to here includes an engine 120, a brake mechanism 130, and an EPS motor 140.

Further, the upper limit value Pmax of the output Peng of the engine 120 (hereinafter, also referred to as “output upper limit value Pengmax”) is, for example, the upper limit value of the torque of the engine 120. The upper limit value Pmax of the output Pb of the brake mechanism 130 (hereinafter, also referred to as “output upper limit value Pbmax”) is, for example, the upper limit value of the braking force Fb. The upper limit value Pmax of the output Peps of the EPS motor 140 (hereinafter, also referred to as “output upper limit value Pepsmax”) is, for example, the upper limit value of the torque of the EPS motor 140. By using these output upper limit values Pmax (Pengmax, Pbmax, Pepsmax), it is possible to avoid excessive output and improve the ride quality of the occupant.

The output upper limit value Pmax is calculated based on the upper limit value Qbmax of the vehicle body behavior amount Qb. In step S15 of the present embodiment, limit control for switching the output upper limit value Pmax according to the vehicle speed V is executed.

In step S16, the ECU 36 calculates the travelable area of the own vehicle 10. The travelable area indicates an area in which the vehicle 10 can travel at present. For example, a region where the distance between the vehicle 10 and each peripheral object is equal to or greater than a predetermined value is indicated with reference to the reference point of the vehicle 10 (for example, the center of gravity of the vehicle 10 and the center of the line segment connecting the left and right rear wheels). In calculating the travelable area, the relationship with surrounding obstacles (particularly, front obstacles) of the vehicle 10 is also taken into consideration. In relation to peripheral obstacles, the ECU 36 performs peripheral monitoring control.

Further, the traveling ECU 36 performs peripheral monitoring control in, for example, detection of an intersection, interpolation of a virtual lane mark for an intersection, and the like. This peripheral monitoring control will be described later with reference to FIGS. 3 and later.

When the peripheral recognition unit 170 recognizes the red light, the area beyond the stop line in front of the traffic light can be excluded from the travelable area. Alternatively, the travelable area may be calculated simply by the relationship with surrounding obstacles (distance, etc.), and the travel restriction due to the red light may be reflected when calculating the target travel locus Ltar, which will be described later.

In step S17, the traveling ECU 36 calculates the target traveling locus Ltar. The target travel locus Ltar is a target value of the travel locus L of the vehicle 10. In the present embodiment, as the target travel locus Ltar, the optimum locus L satisfying various conditions is selected from the travelable regions calculated in step S16.

In step S18, the traveling ECU 36 calculates a target control amount (in other words, a target vehicle body behavior amount Qbtar) of each actuator based on the target traveling locus Ltar. The target vehicle body behavior amount Qbtar includes, for example, a target front-rear acceleration αtar, a target front-rear deceleration βtar, and a target lateral acceleration Glattar.

In step S19, the traveling ECU 36 controls each actuator (in other words, the vehicle body behavior amount Qb) using the target control amount calculated in step S18. For example, the driving force control unit 180 calculates the target output Pengtar (for example, the target engine torque) of the engine 120 (actuator) so as to realize the target front-rear acceleration αtar. Then, the driving force control unit 180 controls the engine 120 via the driving ECU 122 so as to realize the target output Pengtar.

Further, the braking force control unit 182 calculates the target output Pbtar of the brake mechanism 130 (actuator) so as to realize the target front-rear deceleration βtar. Then, the braking force control unit 182 controls the braking mechanism 130 via the braking ECU 132 so as to realize the target output Pbatar.

Further, the turning control unit 184 sets the target steering angle θstar so as to realize the target lateral acceleration Glattar. Then, the turning control unit 184 controls the EPS motor 140 (actuator) via the EPS-ECU 142 so as to realize the target steering angle θstar. In addition to or instead of turning by the EPS motor 140, it is also possible to turn the vehicle 10 by the torque difference between the left and right wheels (so-called torque vectoring).

In step S20, the ECU 36 determines whether or not to change the target point Pgoal or the planned route Rv. The case where the target point Pgoal is changed is the case where a new target point Pgoal is input through the operation of the HMI 28. The case where the planned route Rv is changed is, for example, a case where congestion occurs in the planned route Rv and it is necessary to set a detour. The occurrence of a traffic jam can be recognized by using, for example, the traffic jam information acquired from the traffic information server via the communication device 26.

When changing the target point Pgoal or the planned route Rv (step S20: YES), the process returns to step S13, and the planned route Rv based on the new target point Pgoal is calculated or the new planned route Rv is calculated. If the target point Pgoal or the planned route Rv is not changed (step S20: NO), the process proceeds to step S21.

In step S21, the traveling ECU 36 determines whether or not to end the automatic operation. The automatic driving is terminated, for example, when the vehicle 10 arrives at the target point Pgoal, or when the automatic driving switch 110 is switched from on to off. Alternatively, when the surrounding environment becomes difficult for automatic operation, the ECU 36 ends the automatic operation.

When the automatic operation is not ended (step S21: NO), the process returns to step S13, and the ECU 36 updates the scheduled route Rv based on the current position Pcur. When the automatic operation is terminated (step S21: YES), the process proceeds to step S22.

In step S22, the ECU 36 executes the termination process. Specifically, when the vehicle 10 arrives at the target point Pgoal, the ECU 36 notifies the driver or the like by voice, display, or the like via the HMI 28 that the vehicle 10 has arrived at the target point Pgoal. When the automatic operation switch 110 is switched from on to off, the ECU 36 notifies the driver or the like via the HMI 28 by voice, display, or the like that the automatic operation is finished. When the surrounding environment becomes difficult for automatic driving, the ECU 36 notifies the driver or the like by voice, display, or the like via the HMI 28.

As described above, the ECU 36 executes peripheral monitoring control when calculating the travelable area (step S16 in FIG. 2). The peripheral monitoring control is a control for avoiding contact with peripheral obstacles existing in the periphery (particularly in front of) of the vehicle 10 based on the information from the peripheral recognition unit 170. Further, when recognizing an intersection or the like or interpolating a virtual lane mark to the intersection or the like, the traveling ECU 36 executes peripheral monitoring control.

Hereinafter, peripheral monitoring control relating to recognition of intersections and the like and interpolation of virtual lane marks to intersections and the like will be described with reference to FIGS. 3 to 12.

Two examples (first peripheral recognition unit 170A and second peripheral recognition unit 170B) will be typically described with respect to the peripheral recognition unit 170.

As shown in FIG. 3, the first peripheral recognition unit 170A includes a specific scene detection unit 202 and a first interpolation processing unit 204A.

As shown in FIG. 3, the specific scene detection unit 202 has an interval of a predetermined distance or more on the camera image 206 (see FIG. 4A) drawn in the image memory by, for example, the outside camera 50 (see FIG. 1) that images the front. Is placed and an image of a specific scene (hereinafter referred to as a specific scene 208) in which a line (short line) exists on the back side and the front side, respectively, is detected.

The first interpolation processing unit 204A interpolates the lane mark candidate on the back side and the lane mark candidate on the front side based on the detection of the specific scene 208 by the specific scene detection unit 202.

Here, specifically, a case where an intersection is assumed as a specific scene 208 will be described with reference to FIGS. 4A to 5B.

As shown in FIG. 4A, the specific scene 208 detected by the specific scene detection unit 202 is the first lane mark from left to right in the upper region (the region indicating the back side) of the camera image 206. The candidate image (hereinafter referred to as the first lane mark candidate 210a) and the image of the second lane mark candidate (hereinafter referred to as the second lane mark candidate 210b) are arranged at intervals, and in the drawing area, In the lower area (the area indicating the front side), the image of the first lane mark candidate (hereinafter referred to as the third lane mark candidate 210c) and the image of the second lane mark candidate (hereinafter referred to as the third lane mark candidate 210c) from left to right. Hereinafter, it is referred to as a fourth lane mark candidate 210d) and is an image in which they are arranged at intervals.

Then, as shown in FIG. 3, the first interpolation processing unit 204A includes an image conversion unit 212, a lane mark candidate determination unit 216, and a first lane mark interpolation unit 218A.

The image conversion unit 212 converts the specific scene 208 into a bird's-eye view (BEV: bird's eye view) image (hereinafter referred to as BEV image 220) as shown in FIG. 4B. The BEV image 220 may be drawn in the same image memory or may be drawn in another image memory. When drawing in the same image memory, it is preferable to draw in a drawing area different from the drawing area of the specific scene 208.

As shown in FIG. 5A, the lane mark candidate determination unit 216 is in a direction orthogonal to the front third lane mark candidate 210c in the back side first lane mark candidate 210a and the front side third lane mark candidate 210c. It is determined whether or not the distance Da is within a predetermined range. For example, is the deviation between the first lane mark candidate 210a and the third lane mark candidate 210c, particularly the deviation (distance Da) in the direction orthogonal to the third lane mark candidate 210c within a predetermined range (for example, within 50 cm)? Judge whether or not. This predetermined range may be appropriately selected according to the road shape of the intersection and the like. The same applies hereinafter.

Further, as shown in FIG. 5B, the lane mark candidate determination unit 216 is orthogonal to the fourth lane mark candidate 210d on the front side in the second lane mark candidate 210b on the back side and the fourth lane mark candidate 210d on the front side. It is determined whether or not the distance Db in the direction is within a predetermined range. For example, it is determined whether or not the deviation between the second lane mark candidate 210b and the fourth lane mark candidate 210d, particularly the deviation in the direction orthogonal to the fourth lane mark candidate 210d (distance Db) is within a predetermined range.

A Hough transform can be used, for example, to obtain the first lane mark candidate 210a and the second lane mark candidate 210b on the back side, and the third lane mark candidate 210c and the fourth lane mark candidate 210d on the front side.

The first lane mark interpolation unit 218A determines in the lane mark candidate determination unit 216 that the above-mentioned distance Da between the first lane mark candidate 210a on the back side and the third lane mark candidate 210c on the front side is within a predetermined range. If so, as shown in FIG. 6A, the virtual left lane mark 224L is set so as to interpolate the first lane mark candidate 210a and the third lane mark candidate 210c, assuming that they are the same lane mark.

Similarly, in the lane mark candidate determination unit 216, the first lane mark interpolation unit 218A has the above-mentioned distance Db between the second lane mark candidate 210b on the back side and the fourth lane mark candidate 210d on the front side within a predetermined range. If it is determined to be present, as shown in FIG. 6A, the virtual right lane mark 224R is set so as to interpolate the second lane mark candidate 210b and the fourth lane mark candidate 210d by regarding the same lane mark. To do.

Here, two processing operations (first processing operation and second processing operation) for the first interpolation processing unit 204A will be described.

First, the first processing operation will be described with reference to FIG. 7. First, in step S101 of FIG. 7, the specific scene detection unit 202 detects the specific scene 208 (see FIG. 4A) from the camera image 206. When the specific scene 208 is detected, the process proceeds to step S102, and the image conversion unit 212 converts the image of the specific scene 208 into a BEV image 220 (see FIG. 4B).

In step S103, as shown in FIG. 5A, the lane mark candidate determination unit 216 calculates the distance Da in the direction orthogonal to the third lane mark candidate 210c in the first lane mark candidate 210a and the third lane mark candidate 210c. ..

In step S104, the lane mark candidate determination unit 216 determines whether or not the calculated distance Da is within a predetermined range. If it is within the predetermined range, the process proceeds to step S105, and the lane mark candidate determination unit 216 is orthogonal to the fourth lane mark candidate 210d in the second lane mark candidate 210b and the fourth lane mark candidate 210d, as shown in FIG. 5B. Calculate the distance Db in the direction of

In step S106, the lane mark candidate determination unit 216 determines whether or not the calculated distance Db is within a predetermined range. If it is within the predetermined range, the process proceeds to step S107, and the first lane mark interpolation unit 218 virtually left the left side so as to interpolate the first lane mark candidate 210a and the third lane mark candidate 210c as shown in FIG. 6A. Set the lane mark 224L. Further, the first lane mark interpolation unit 218 sets the virtual right lane mark 224R so as to interpolate the second lane mark candidate 210b and the fourth lane mark candidate 210d.

As a result, as shown in FIG. 6B, an image in which two lane marks LM1 and LM2 extending along the traveling direction of the own vehicle 10 are drawn in the intersection is obtained, and the action planning unit 172 and the like obtain the image of the intersection. It can be recognized as a virtual lane mark drawn inside. As a result, for example, lane keeping running in an intersection can be preferably performed.

If it is determined in step S104 or step S106 of FIG. 7 that the calculated distances Da and Db exceed the predetermined range, the current process is completed, and the process returns to step S101 after the elapse of the predetermined time.

In the above example, when the determination result in step S104 is not within the distance Da ≦ predetermined range (step S104: NO), the determination process in step S106 is not performed, but the determination result in step S104 does not matter, step S106. The determination process may be performed in.

Next, the second processing operation of the first interpolation processing unit 204A will be described with reference to FIG.

First, in steps S201 to S203 of FIG. 8, the same processing as in steps S101 to S103 of the first processing operation described above is performed.

After that, in step S204, the lane mark candidate determination unit 216 is orthogonal to the fourth lane mark candidate 210d in the second lane mark candidate 210b and the fourth lane mark candidate 210d, similarly to step S105 of the first processing operation described above. Calculate the distance Db in the direction of

In step S205, the lane mark candidate determination unit 216 determines whether or not the calculated distance Da is within a predetermined range. If it is within the predetermined range, the process proceeds to step S206, and as shown in FIG. 6A, the first lane mark interpolation unit 218A interpolates the first lane mark candidate 210a and the third lane mark candidate 210c on the virtual left side. Set the lane mark 224L.

When the process in step S206 is completed, or when it is determined in step S205 that the distance Da is not within the predetermined range, the process proceeds to step S207, and in step S207, the lane mark candidate determination unit 216 calculates. It is determined whether or not the distance Db is within a predetermined range. If it is within the predetermined range, the process proceeds to step S208, and the first lane mark interpolation unit 218A vertically interpolates the second lane mark candidate 210b and the fourth lane mark candidate 210d as shown in FIG. 6A. Set the lane mark 224R.

Next, the second peripheral recognition unit 170B will be described with reference to FIGS. 9 to 12. The duplicate description of the same configuration and function as the first peripheral recognition unit 170A will be omitted.

As shown in FIG. 9, the second peripheral recognition unit 170B includes a specific scene detection unit 202 and a second interpolation processing unit 204B.

Similar to the first interpolation processing unit 204A described above, the second interpolation processing unit 204B includes the lane mark candidate on the back side and the lane mark candidate on the front side based on the detection of the specific scene 208 by the specific scene detection unit 202. Is interpolated.

The second interpolation processing unit 204B includes the above-mentioned image conversion unit 212, scene division processing unit 226, interpolation processing determination unit 228, area connection processing unit 230, connection determination unit 232, and second lane mark interpolation unit. It has 218B.

As shown in FIG. 10A, the scene division processing unit 226 divides the BEV image 220 into three or more regions in the traveling direction of the own vehicle 10. FIG. 10A shows an example in which the BEV image 220 is divided into four regions (first region Z1, second region Z2, third region Z3, and fourth region Z4) in the traveling direction of the own vehicle 10.

In the interpolation processing determination unit 228, among the four divided regions (first region Z1 to fourth region Z4), whether there are lane mark candidates in the interpolated regions (second region Z2 and third region Z3). Judge whether or not. Candidates for lane marks include part of the broken white line. If there are lane mark candidates in the second region Z2 or the third region Z3, the second interpolation processing unit 204B does not perform the interpolation processing. The lane in which the white line of the broken line is drawn is drawn as a broken line on the image, and is recognized as a lane mark by the traveling ECU 36 or the like in the second area Z2 or the third area Z3 between the area on the back side and the area on the front side. Because it is done. As a result, the interpolation process can be performed only in the specific scene 208 which seems to be an intersection.

On the other hand, when there are no lane mark candidates in the intervening regions (second region Z2 and third region Z3), the region connection processing unit 230 uses the four divided regions (first region Z1 to fourth region Z4). ), The intervening regions (second region Z2 and third region Z3) are excluded, and the first region Z1 on the back side and the fourth region Z4 on the front side are connected. That is, as shown in FIG. 10B, a BEV image 220 (hereinafter, referred to as a connected image 234) in which the first region Z1 and the fourth region Z4 are connected is drawn.

In the connection determination unit 232, the first lane mark candidate 210a in the first region Z1 and the third lane mark candidate 210c in the fourth region Z4 are connected, and the second lane mark candidate 210b and the fourth in the first region Z1 are connected. It is determined whether or not the fourth lane mark candidate 210d of the region Z4 is connected. If both are connected, as shown in FIG. 10B, the second lane mark interpolation unit 218B recognizes the first lane mark candidate 210a and the third lane mark candidate 210c as one left lane mark 236L, and the second lane mark candidate 210a is recognized as one left lane mark 236L. The 2 lane mark candidate 210b and the 4th lane mark candidate 210d are recognized as one right lane mark 236R.

On the other hand, as shown in FIG. 11A, the first lane mark candidate 210a and the third lane mark candidate 210c are not connected by the connection determination unit 232, and the second lane mark candidate 210b and the fourth lane mark candidate 210b are not connected. When it is determined that the 210d is not connected, the second lane mark interpolation unit 218B performs the same processing as the first lane mark interpolation unit 218A described above as shown in FIG. 11B, and performs the same processing as the first lane mark interpolation unit 218A. A virtual left lane mark 224L is set so as to interpolate the mark candidate 210a and the third lane mark candidate 210c, and a virtual right lane mark is set so as to interpolate the second lane mark candidate 210b and the fourth lane mark candidate 210d. 224R may be set.

Here, the processing operation of the second interpolation processing unit 204B will be described with reference to the flowchart of FIG. The duplicate description of the step of performing the same processing as that of the first interpolation processing unit 204A is omitted.

First, in step S301 of FIG. 12, the specific scene detection unit 202 detects the specific scene 208 (see FIG. 4A) from the camera image 206. When the specific scene 208 is detected (step S301: YES), the process proceeds to step S302, and the image conversion unit 212 converts the image of the specific scene 208 into a BEV image 220 (see FIG. 4B).

In step S303, the scene division processing unit 226 divides the BEV image 220 into three or more regions in the traveling direction of the own vehicle 10, as shown in FIG. 10A. For example, it is divided into four regions (first region Z1 to fourth region Z4).

In step S304, the interpolation processing determination unit 228 is a candidate for a lane mark in an interpolated region (second region Z2 and third region Z3) among the four divided regions (first region Z1 to fourth region Z4). Determines if is present.

If there is no lane mark candidate (S304: YES), the process proceeds to step S305, and the area connection processing unit 230 is an area between the four divided areas (first area Z1 to fourth area Z4). Excluding (second region Z2 and third region Z3), as shown in FIG. 10B, the first region Z1 on the back side and the fourth region Z4 on the front side are connected. If there are lane mark candidates in the interpolating regions (second region Z2 and third region Z3) (step S304: NO), the processing in the second interpolation processing unit 204B ends.

In step S306, the connection determination unit 232 connects the first lane mark candidate 210a in the first region Z1 and the third lane mark candidate 210c in the fourth region Z4, and also connects the second lane mark candidate 210c in the first region Z1. It is determined whether or not 210b and the fourth lane mark candidate 210d in the fourth region Z4 are connected.

If both are connected (step S306: YES), the process proceeds to step S307, and the second lane mark interpolation unit 218B sets the first lane mark candidate 210a and the third lane mark candidate 210c as shown in FIG. 10B. Recognized as one left lane mark 236L. Similarly, the second lane mark candidate 210b and the fourth lane mark candidate 210d are recognized as one right lane mark 236R.

In step S306, it is determined that the first lane mark candidate 210a and the third lane mark candidate 210c are not connected, and the second lane mark candidate 210b and the fourth lane mark candidate 210d are not connected. If (step S306: NO), the process proceeds to step S308, and the second lane mark interpolation unit 218B performs the same processing as the first lane mark interpolation unit 218A described above as shown in FIG. The virtual left lane mark 224L is set so as to interpolate the lane mark candidate 210a and the third lane mark candidate 210c, and the virtual right lane is interpolated between the second lane mark candidate 210b and the fourth lane mark candidate 210d. Mark 224R is set.

If it is determined in step S306 that the first lane mark candidate 210a and the third lane mark candidate 210c are connected and the second lane mark candidate 210b and the fourth lane mark candidate 210d are not connected, it is determined. In step S307, the first lane mark candidate 210a and the third lane mark candidate 210c are recognized as one left lane mark 236L, and in step S308, the second lane mark candidate 210b and the fourth lane mark candidate 210d are interpolated. The virtual right lane mark 224R may be set as described above.

Of course, in step S306, it was determined that the first lane mark candidate 210a and the third lane mark candidate 210c are not connected, and the second lane mark candidate 210b and the fourth lane mark candidate 210d are connected. In this case, in step S307, the second lane mark candidate 210b and the fourth lane mark candidate 210d are recognized as one right lane mark 236R, and in step S308, the first lane mark candidate 210a and the third lane mark candidate 210c The virtual left lane mark 224L may be set so as to interpolate.

Further, the processing of steps S306 and S308 described above may be omitted. In this case, the lane mark on the back side and the lane mark on the front side may not be connected. However, by excluding the intervening regions (second region Z2 and third region Z3), the lane mark on the back side and the lane mark on the front side can be brought close to each other. Therefore, for example, it is possible to recognize the lane mark in a state where the interval is closer than that of the broken line lane mark, and it is possible to recognize the lane mark with high accuracy.

As described above, the driving assist device according to the present embodiment is a driving assist device that assists the driver's driving operation, and is a lane mark detecting unit (external camera 50 or the like) that detects a lane mark existing on the road. When a specific scene 208 in which lane mark candidates (first lane mark candidate 210a to fourth lane mark candidate 210d) are present on the back side and the front side at intervals of a predetermined distance or more is detected on the camera image 206. , Interpolation processing between the lane mark candidate on the back side (first lane mark candidate 210a, second lane mark candidate 210b) and the lane mark candidate on the front side (second lane mark candidate 210b, fourth lane mark candidate 210d). It has an interpolation processing unit (first interpolation processing unit 204A, second interpolation processing unit 204B).

Conventionally, when the white line on the front side is detected and then the white line on the back side is detected on the approximate straight line, it is recognized as a lane mark. Therefore, another object or dirt on the road can be detected at a distance from the intersection. It may be detected as the edge of the white line, or the white line edge on the right side on the front side and the white line edge on the left side on the back side may be connected by an approximate straight line. At intersections and the like, the length of the lane mark candidates to be extracted is short in the first place, and it is difficult to correctly recognize the lane mark on the front side.

However, in the present embodiment, it is specified that lane mark candidates (first lane mark candidate 210a to fourth lane mark candidate 210d) exist on the back side and the front side at intervals of a predetermined distance or more on the camera image 206, respectively. When the scene 208 is detected, the lane mark candidate on the back side (first lane mark candidate 210a, second lane mark candidate 210b) and the lane mark candidate on the front side (third lane mark candidate 210c, fourth lane mark candidate 210d) are detected. ), For example, a virtual lane mark can be set accurately in the intersection. That is, it is possible to make the traveling ECU 36 or the like recognize the virtual lane mark with high accuracy and with a small load, and it is possible to perform suitable automatic driving according to the plan of the own vehicle movement route.

In the present embodiment, the first interpolation processing unit 204A has a lane mark candidate on the back side (first lane mark candidate 210a, a second lane mark candidate 210b) and a lane mark candidate on the front side (third lane mark candidate 210c, When the fourth lane mark candidate 210d) is within a predetermined range in the direction orthogonal to the front lane mark candidate, interpolation processing is performed between the back lane mark candidate and the front lane mark candidate. ..

As a result, a pair of virtual lane marks can be set accurately at an intersection or the like where there is no actual lane mark, and the traveling ECU 36 or the like can recognize the virtual lane mark accurately and with a small load. This enables suitable automatic driving according to the plan of the vehicle movement route.

In this case, the interpolation process includes the lane mark candidate on the back side (first lane mark candidate 210a, second lane mark candidate 210b) and the lane mark candidate on the front side (third lane mark candidate 210c, fourth lane mark candidate 210d). ), A virtual lane mark is set, and the lane mark candidate on the back side and the lane mark candidate on the front side are connected.

In the present embodiment, the second interpolation processing unit 204B has a region in which the lane mark candidates (first lane mark candidate 210a, second lane mark candidate 210b) on the back side exist and a lane mark candidate (third lane) on the front side. The area between the area where the mark candidate 210c and the fourth lane mark candidate 210d) exist may be excluded, and the area on the back side and the area on the front side may be connected.

As a result, the first lane mark candidate 210a on the back side and the third lane mark candidate 210c on the front side are connected, and the second lane mark candidate 210b on the back side and the fourth lane mark candidate 210d on the front side are connected. It becomes possible. As a result, even if the sensor or the like is shaken, it is possible to reliably interpolate between the first lane mark candidate 210a on the back side and the third lane mark candidate 210c on the front side, and the third lane mark candidate on the back side. The two lane mark candidate 210b and the fourth lane mark candidate 210d on the front side can be interpolated.

Even if the first lane mark candidate 210a on the back side and the third lane mark candidate 210c on the front side are not connected, the second lane mark candidate 210b on the back side and the fourth lane mark candidate 210d on the front side are not connected. Even in this case, the distance between the first lane mark candidate 210a on the back side and the third lane mark candidate 210c on the front side, and the distance between the second lane mark candidate 210b on the back side and the fourth lane mark candidate 210d on the front side. Since the distance is shortened, even if the sensor or the like is shaken, the virtual left lane mark 224L is surely interpolated between the first lane mark candidate 210a on the back side and the third lane mark candidate 210c on the front side. In addition to being able to process, the virtual right lane mark 224R can be interpolated between the second lane mark candidate 210b on the back side and the fourth lane mark candidate 210d on the front side.

Further, the second interpolation processing unit 204B sets a lane mark in an interpolated region (second region Z2, third region Z3) among three or more regions (for example, the first region Z1 to the fourth region Z4) along the road. If no candidate is detected, interpolation processing is performed.

Scenes where lines exist on the back side and the front side include intersections and lanes with dashed lane marks. Since the lane on which the broken line lane mark is drawn is drawn as a broken line on the image, the traveling ECU 36 and the like can recognize it as a lane mark.

On the other hand, at intersections and the like, since the solid line lane mark and the broken line lane mark are not drawn, the traveling ECU 36 and the like cannot recognize the traveling lane. Therefore, when no lane mark candidate is detected in the interpolating region (second region Z2, third region Z3) among the three or more regions (first region Z1 to fourth region Z4), that is, the broken line lane mark. By performing interpolation processing when such as is not detected, it is possible to make the ECU or the like recognize it as a virtual lane mark.

That is, in a situation where the interpolation process does not need to be performed, the waste of performing the interpolation process can be eliminated, the recognition load by the ECU or the like can be suppressed, and recognition without a time lag can be performed.

It should be noted that the present invention is not limited to the above-described embodiment, and of course, it can be freely changed without departing from the gist of the present invention.

In the above example, the first lane mark candidate 210a and the third lane mark candidate 210c and the second lane mark candidate 210b and the fourth lane mark candidate 210d are interpolated with a straight line. , Each may be interpolated with a curve. The curve includes an arc shape, an elliptical arc shape, and the like.

Further, when an intersection exists on a wavy road, it may not be possible to image the lane mark candidates (first lane mark candidate 210a and third lane mark candidate 210c) on the back side. In such a case, the specific scene 208 is created by superimposing the map information Imap of the current position Pcur on the BEV image 220 of the image of the intersection including the captured lane mark candidate and drawing the lane mark candidate on the back side. It may be detected.

Examples of the lane mark include a white line and a broken line white line, but any lane mark can be applied, and for example, a yellow line and the like may be targeted.

Since this embodiment mainly sets a virtual lane mark based on peripheral monitoring, it can be used not only for automatic driving control but also for an alarm device that outputs an alarm when the vehicle deviates from the lane.

4A and 4B, 5A and 5B, 10A and 10B, 11A and 11B recognize the lane mark candidate as having a width, but it is not always necessary to have the width. The lane mark candidate may be recognized as not just a line.

In the above example, the specific scene is mainly detected based on the image captured by the external camera 50, but in addition, the specific scene 208 may be detected (acquired) from the map information Imap.

In the above-described embodiment, the BEV image 220 is created when the specific scene 208 is detected, but the present invention is not limited to this, and the BEV image 220 may be appropriately created or always created when necessary.

In the above-described embodiment, when the lane mark candidates are extracted in either the back side region or the front side region, the interpolation process is not performed, but the lane mark candidates are not extracted. The lane mark may be recognized by excluding only the area and connecting the extracted areas. This means that when the specific scene 208 is divided into many parts such as 4 divisions and 6 divisions, only the areas where the lane mark candidates do not exist are excluded, and the areas where the lane mark candidates exist are connected to recognize the lane marks. Can be mentioned. Of course, it can be applied even if it is not an intersection.

In the above-described embodiment, the vehicle peripheral sensor group 20 includes a plurality of vehicle exterior cameras 50, but the plurality of vehicle exterior cameras 50 may include a stereo camera that detects the front of the vehicle 10. Of course, a single camera may be used for recognizing the lane mark.

The vehicle body behavior sensor group 22 of the above embodiment includes, but is not limited to, the vehicle speed sensor 60, the lateral acceleration sensor 62, and the yaw rate sensor 64 (see FIG. 1). For example, it is possible to omit any one or more of the lateral acceleration sensor 62 and the yaw rate sensor 64.

The driving operation sensor group 24 of the above embodiment includes, but is not limited to, the AP sensor 80, the BP sensor 82, the steering angle sensor 84, and the steering torque sensor 86 (see FIG. 1). For example, it is possible to omit any one or more of the AP sensor 80, the BP sensor 82, the steering angle sensor 84, and the steering torque sensor 86.

In the above embodiment, the engine 120, the brake mechanism 130, and the EPS motor 140 are used as the actuators targeted by the automatic operation control (see FIG. 1), but the present invention is not limited to this. For example, it is possible to exclude any one or two of the engine 120, the brake mechanism 130, and the EPS motor 140 from the target of automatic driving control. When any of the actuators is excluded from the target of automatic driving control, the driver will control the actuator excluded from the target. Further, as described above, instead of the EPS motor 140, it is possible to turn using the torque difference between the left and right wheels. Of course, for example, lane keeping control and inter-vehicle distance maintenance control are also included in the automatic driving control.

In the above embodiment, automatic driving that does not require a driver's driving operation for any of acceleration, deceleration, and turning of the vehicle 10 has been described (see FIG. 2), but the present invention is not limited to this. For example, the present invention is applied to automatic driving that does not require the driver's driving operation only for any one or two of acceleration, deceleration, and turning of the vehicle 10, or automatic driving that assists the driver's driving operation. Is also possible.

10 ... Vehicle (own vehicle) 20 ... Vehicle peripheral sensor group 36 ... Traveling ECU (ECU) 50 ... External camera 170 ... Peripheral recognition unit 170A ... 1st peripheral recognition unit 170B ... 2nd peripheral recognition unit 172 ... Action planning unit 174 ... Travel control unit 202 ... Specific scene detection unit 204A ... First interpolation processing unit 204B ... Second interpolation processing unit 206 ... Camera image 208 ... Specific scene 210a to 210d ... First lane mark candidate to fourth lane mark candidate 212 ... Image conversion Unit 216 ... Lane mark candidate determination unit 218A ... First lane mark interpolation unit 218B ... Second lane mark interpolation unit 220 ... BEV image 224L ... Left lane mark 224R ... Right lane mark 226 ... Scene division processing unit 228 ... Intertrusion processing determination unit 230 ... Area connection processing unit 232 ... Connection determination unit 234 ... Connection images Z1 to Z4 ... First area to fourth area

Claims (6)

It is a driving assistance device that assists the driver's driving operation.
A lane mark detector that detects lane marks existing on the road,
When a specific scene in which lane mark candidates exist on the back side and the front side are detected at intervals of a predetermined distance or more on the image, the lane mark candidates on the back side and the lane mark candidates on the front side are interpolated. have a and the interpolation processing unit for processing,
In the interpolation processing unit, when the lane mark candidate on the back side and the lane mark candidate on the front side are within a predetermined range in the direction orthogonal to the lane mark candidate on the front side, the lane on the back side A driving assistance device characterized by interpolating between a mark candidate and a lane mark candidate on the front side. In the driving assistance device according to claim 1,
The interpolation processing sets a virtual line between the far side of the lane mark candidate and the front side of the lane mark candidate, to connect the front side of the lane mark candidate and the far side of the lane mark candidate drive assist device comprising a Turkey. In the driving assistance device according to claim 1 or 2.
The interpolation processing unit is
It is characterized in that the region between the region where the lane mark candidate on the back side exists and the region where the lane mark candidate on the front side exists is excluded, and the region on the back side and the region on the front side are connected. Driving assistance device. In the driving assistance device according to claim 1 or 2.
The interpolation processing unit is
Set 3 or more areas along the road
When the lane mark candidate is not detected in the area between the area where the lane mark candidate on the back side exists and the area where the lane mark candidate on the front side exists among the three or more areas, the interpolation process is performed. A driving assistance device characterized by performing. In the driving assistance device according to any one of claims 1 to 4.
A driving assistance device characterized in that the specific scene is an intersection. It is a driving assistance method that assists the driver's driving operation.
A step to detect a specific scene in which lane mark candidates exist on the back side and the front side at intervals of a predetermined distance or more on the image, and
A step of interpolating between the lane mark candidate on the back side and the lane mark candidate on the front side based on the detection of the specific scene, and
When the lane mark candidate on the back side and the lane mark candidate on the front side are within a predetermined range in the direction orthogonal to the lane mark candidate on the front side, the lane mark candidate on the back side and the lane mark candidate on the front side A driving assistance method comprising a step of interpolating between lane mark candidates.

Images