JP2023150513A - Vehicle control device, vehicle control method and program - Google Patents

Abstract

To properly change driving control of a vehicle even when a road section line recognized by a camera is different from the content of map information mounted on the own vehicle.SOLUTION: A vehicle control device comprises: an acquisition unit which acquires a camera image obtained by imaging a peripheral situation of a vehicle; a driving control unit which controls steering and acceleration/deceleration of the vehicle without depending on an operation of a driver of the vehicle on the basis of the camera image and map information; a mode decision unit which decides any of a plurality of driving modes including the first driving mode and the second driving mode as the driving mode of the vehicle; and a determination unit which determines whether or not a deviation exists between the road section line shown in the camera image and the road section line shown in the map information and determines whether or not the vehicle exists at a branch point shown in the map information.SELECTED DRAWING: Figure 1

Description

The present invention relates to a vehicle control device, a vehicle control method, and a program.

2. Description of the Related Art Conventionally, there has been known a technique for controlling the driving of a vehicle based on road markings recognized by a camera mounted on the vehicle. For example, in Patent Document 1, the vehicle is driven based on the recognized road marking line, and when the degree of recognition of the road marking line does not meet a predetermined standard, the vehicle is driven based on the trajectory of the preceding vehicle. The technology to do this is described.

JP2020-050086A

The technology described in Patent Document 1 controls the driving of the own vehicle based on road markings recognized by a camera and map information carried by the own vehicle. However, in the conventional technology, if the road markings recognized by the camera differ from the content of the map information carried by the own vehicle, the driving control of the vehicle may not be appropriately changed.

The present invention has been made in consideration of such circumstances, and it is possible to control the driving of a vehicle even when the road markings recognized by the camera and the content of the map information carried by the own vehicle are different. One of the objects is to provide a vehicle control device, a vehicle control method, and a program that can appropriately change the vehicle control method.

A vehicle control device, a vehicle control method, and a program according to the present invention employ the following configuration.
(1): A vehicle control device according to one aspect of the present invention includes an acquisition unit that acquires a camera image capturing a surrounding situation of a vehicle, and a control device that controls operations of a driver of the vehicle based on the camera image and map information. a driving control unit that controls steering and acceleration/deceleration of the vehicle without depending on the vehicle; and a driving control unit that determines the driving mode of the vehicle to be one of a plurality of driving modes including a first driving mode and a second driving mode. , the second driving mode is a driving mode in which the task imposed on the driver is lighter than the first driving mode, and is one of the plurality of driving modes including at least the second driving mode. A mode determining unit is controlled by the driving control unit, and changes the driving mode of the vehicle to a driving mode in which the task is more severe when the driver does not perform a task related to the determined driving mode. and determining whether there is a discrepancy between the road marking line shown in the camera image and the road marking line shown in the map information, and determining whether the vehicle is at a branch point shown in the map information. a determination unit that determines whether or not the mode determination unit determines that there is a discrepancy between the road division line shown in the camera image and the road division line shown in the map information; If it is determined that the vehicle is at a branch point indicated in the map information, a driving mode of the vehicle is determined based on the branch direction of the branch point and the direction of the road marking line indicated in the camera image. It is something.

(2): In the aspect of (1) above, when the degree of deviation between the branch direction of the branch point and the direction of the road marking line shown in the camera image is equal to or greater than a threshold, The second driving mode is changed to the first driving mode, and the first driving mode is continued using the road marking lines shown in the camera image.

(3): In the aspect of (1) or (2) above, when the branch direction of the branch point and the direction of the road marking line shown in the camera image are less than a threshold, The second driving mode is continued using the road markings shown in the map information.

(4): In any of the aspects (1) to (3) above, the second driving mode is a driving mode in which the driver is not tasked with holding an operator that receives a steering operation of the vehicle. The first driving mode is a driving mode in which the driver is tasked with at least the task of grasping the operator.

(5): In a vehicle control method according to another aspect of the present invention, a computer acquires a camera image capturing the surrounding situation of the vehicle, and based on the camera image and map information, the vehicle driver's operation is performed. controlling the steering and acceleration/deceleration of the vehicle without relying on the vehicle, determining the driving mode of the vehicle to one of a plurality of driving modes including a first driving mode and a second driving mode, and The second driving mode is a driving mode in which the task imposed on the driver is lighter than the first driving mode, and at least some of the plurality of driving modes including the second driving mode are This is done by controlling the steering and acceleration/deceleration of the vehicle without depending on the operation of the vehicle driver, and if the task related to the determined driving mode is not performed by the driver, the task becomes more severe. The driving mode of the vehicle is changed to a driving mode, and it is determined whether or not there is a discrepancy between the road marking line shown in the camera image and the road marking line shown in the map information. It is determined whether or not the vehicle is at a branch point indicated in the map information, and it is determined that there is a discrepancy between the road division line indicated in the camera image and the road division line indicated in the map information, and the vehicle is When it is determined that the vehicle is at a branch point indicated in the map information, a driving mode of the vehicle is determined based on a branch direction of the branch point and a direction of a road marking line indicated in the camera image. be.

(6): A program according to another aspect of the present invention causes a computer to acquire a camera image capturing a surrounding situation of a vehicle, and based on the camera image and map information, a program according to an operation of a driver of the vehicle. controlling the steering and acceleration/deceleration of the vehicle, determining the driving mode of the vehicle to one of a plurality of driving modes including a first driving mode and a second driving mode; The driving mode is a driving mode in which the task imposed on the driver is lighter than the first driving mode, and some of the plurality of driving modes including at least the second driving mode are This is done by controlling the steering and acceleration/deceleration of the vehicle without relying on the driver's operations, and if the task related to the determined driving mode is not performed by the driver, a driving mode with more severe tasks is selected. changes the driving mode of the vehicle, determines whether there is a discrepancy between the road marking line shown in the camera image and the road marking line shown in the map information, and changes the driving mode of the vehicle. It is determined whether or not the vehicle is at a branch point indicated in the information, and it is determined that there is a discrepancy between the road division line indicated in the camera image and the road division line indicated in the map information, and the vehicle is located at the branch point indicated in the map information. When it is determined that the vehicle is at a branch point indicated by the information, the driving mode of the vehicle is determined based on the branch direction of the branch point and the direction of the road marking line indicated in the camera image.

(7): A program according to another aspect of the present invention causes a computer to acquire a camera image capturing a surrounding situation of a vehicle, and based on the camera image and map information, a program according to another aspect of the invention controlling the steering and acceleration/deceleration of the vehicle, determining the driving mode of the vehicle to one of a plurality of driving modes including a first driving mode and a second driving mode; The driving mode is a driving mode in which the task imposed on the driver is lighter than the first driving mode, and some of the plurality of driving modes including at least the second driving mode are This is done by controlling the steering and acceleration/deceleration of the vehicle without relying on the driver's operations, and if the task related to the determined driving mode is not performed by the driver, a driving mode with more severe tasks is selected. changes the driving mode of the vehicle, determines whether there is a discrepancy between the road marking line shown in the camera image and the road marking line shown in the map information, and changes the driving mode of the vehicle. It is determined whether or not the vehicle is at a branch point indicated in the information, and it is determined that there is a discrepancy between the road division line indicated in the camera image and the road division line indicated in the map information, and the vehicle is located at the branch point indicated in the map information. When it is determined that the vehicle is at a branch point indicated by the information, the driving mode of the vehicle is determined based on the branch direction of the branch point and the direction of the road marking line indicated in the camera image.

According to (1) to (7), even if the road markings recognized by the camera differ from the map information carried by the own vehicle, it is possible to appropriately change the vehicle driving control. can.

1 is a configuration diagram of a vehicle system 1 using a vehicle control device according to an embodiment. 2 is a functional configuration diagram of a first control section 120 and a second control section 160. FIG. FIG. 3 is a diagram showing an example of the correspondence between a driving mode, a control state of the own vehicle M, and a task. FIG. 3 is a diagram illustrating an example of a scene in which the operation of the vehicle control device according to the embodiment is executed. FIG. 6 is a diagram showing another example of a scene in which the operation of the vehicle control device according to the embodiment is executed. 3 is a flowchart illustrating an example of the flow of operations executed by the vehicle control device according to the embodiment.

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments of a vehicle control device, a vehicle control method, and a program of the present invention will be described with reference to the drawings.

[overall structure]
FIG. 1 is a configuration diagram of a vehicle system 1 using a vehicle control device according to an embodiment. The vehicle on which the vehicle system 1 is mounted is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its driving source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using electric power generated by a generator connected to an internal combustion engine, or electric power discharged from a secondary battery or a fuel cell.

The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, and a vehicle sensor 40. , a navigation device 50, an MPU (Map Positioning Unit) 60, a driver monitor camera 70, a driving operator 80, an automatic driving control device 100, a driving force output device 200, a brake device 210, and a steering device 220. Equipped with. These devices and devices are connected to each other via multiplex communication lines such as CAN (Controller Area Network) communication lines, serial communication lines, wireless communication networks, and the like. Note that the configuration shown in FIG. 1 is just an example, and a part of the configuration may be omitted, or another configuration may be added.

The camera 10 is, for example, a digital camera that uses a solid-state imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to an arbitrary location of a vehicle (hereinafter referred to as own vehicle M) in which the vehicle system 1 is mounted. When photographing the front, the camera 10 is attached to the upper part of the front windshield, the rear surface of the room mirror, or the like. For example, the camera 10 periodically and repeatedly images the surroundings of the host vehicle M. Camera 10 may be a stereo camera.

The radar device 12 emits radio waves such as millimeter waves around the own vehicle M, and detects radio waves reflected by an object (reflected waves) to detect at least the position (distance and direction) of the object. The radar device 12 is attached to an arbitrary location on the own vehicle M. The radar device 12 may detect the position and velocity of an object using an FM-CW (Frequency Modulated Continuous Wave) method.

The LIDAR 14 irradiates light (or electromagnetic waves with a wavelength close to light) around the host vehicle M and measures scattered light. The LIDAR 14 detects the distance to the target based on the time from light emission to light reception. The irradiated light is, for example, pulsed laser light. LIDAR 14 is attached to any location of own vehicle M.

The object recognition device 16 performs sensor fusion processing on detection results from some or all of the camera 10, radar device 12, and LIDAR 14 to recognize the position, type, speed, etc. of the object. The object recognition device 16 outputs the recognition result to the automatic driving control device 100. The object recognition device 16 may output the detection results of the camera 10, radar device 12, and LIDAR 14 as they are to the automatic driving control device 100. The object recognition device 16 may be omitted from the vehicle system 1.

The communication device 20 uses, for example, a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), etc. to communicate with other vehicles existing around the own vehicle M, or wirelessly. Communicate with various server devices via a base station.

The HMI 30 presents various information to the occupant of the own vehicle M, and also accepts input operations from the occupant. The HMI 30 includes various display devices, speakers, buzzers, touch panels, switches, keys, and the like.

Vehicle sensor 40 includes a vehicle speed sensor that detects the speed of own vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects angular velocity around a vertical axis, a direction sensor that detects the direction of own vehicle M, and the like.

The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 holds first map information 54 in a storage device such as an HDD (Hard Disk Drive) or a flash memory. The GNSS receiver 51 identifies the position of the vehicle M based on signals received from GNSS satellites. The position of the own vehicle M may be specified or complemented by an INS (Inertial Navigation System) using the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, and the like. The navigation HMI 52 may be partially or completely shared with the aforementioned HMI 30. The route determination unit 53 determines, for example, a route (hereinafter referred to as A map route) is determined with reference to the first map information 54. The first map information 54 is, for example, information in which a road shape is expressed by links indicating roads and nodes connected by the links. The first map information 54 may include road curvature, POI (Point Of Interest) information, and the like. The route on the map is output to the MPU 60. The navigation device 50 may provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may be realized, for example, by the functions of a terminal device such as a smartphone or a tablet terminal owned by a passenger. The navigation device 50 may transmit the current position and destination to the navigation server via the communication device 20, and obtain a route equivalent to the route on the map from the navigation server.

The MPU 60 includes, for example, a recommended lane determining section 61, and holds second map information 62 in a storage device such as an HDD or a flash memory. The recommended lane determining unit 61 divides the route on the map provided by the navigation device 50 into a plurality of blocks (for example, divided into blocks of 100 [m] in the direction of vehicle travel), and refers to the second map information 62. Determine recommended lanes for each block. The recommended lane determining unit 61 determines which lane from the left the vehicle should drive. When there is a branch point in the route on the map, the recommended lane determining unit 61 determines a recommended lane so that the own vehicle M can travel on a reasonable route to the branch destination.

The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, information on the center of the lane or information on the boundary between the lanes. The second map information 62 also includes road information, traffic regulation information, address information (address/postal code), facility information, telephone number information, and information on prohibited sections where Mode A or Mode B, which will be described later, is prohibited. May be included. The second map information 62 may be updated at any time by the communication device 20 communicating with other devices. In this embodiment, it is assumed that the second map information 62 particularly includes position information of a lane branching point as meta information.

The driver monitor camera 70 is, for example, a digital camera using a solid-state imaging device such as a CCD or CMOS. The driver monitor camera 70 is positioned and oriented so that it can image the head of an occupant (hereinafter referred to as the driver) seated in the driver's seat of the own vehicle M from the front (in a direction that images the face). Can be installed at any location. For example, the driver monitor camera 70 is attached to the top of a display device provided at the center of the instrument panel of the own vehicle M.

The driving controls 80 include, for example, the steering wheel 82, an accelerator pedal, a brake pedal, a shift lever, and other controls. A sensor is attached to the driving operator 80 to detect the amount of operation or the presence or absence of the operation, and the detection result is sent to the automatic driving control device 100, the driving force output device 200, the brake device 210, and the steering device. 220 is output to some or all of them. The steering wheel 82 is an example of an "operator that accepts a steering operation by the driver." The operator does not necessarily have to be annular, and may be in the form of a deformed steering wheel, a joystick, a button, or the like. A steering wheel grip sensor 84 is attached to the steering wheel 82 . The steering wheel grip sensor 84 is realized by a capacitance sensor or the like, and automatically sends a signal that can detect whether or not the driver is gripping the steering wheel 82 (meaning that the driver is in contact with the steering wheel 82 in a state where force can be applied to it). Output to the operation control device 100.

The automatic driving control device 100 includes, for example, a first control section 120 and a second control section 160. The first control unit 120 and the second control unit 160 are each realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). In addition, some or all of these components are hardware (circuits) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit). (including circuitry), or may be realized by collaboration between software and hardware. The program may be stored in advance in a storage device such as the HDD or flash memory (a storage device equipped with a non-transitory storage medium) of the automatic driving control device 100, or may be stored in a removable device such as a DVD or CD-ROM. The information may be stored in a storage medium, and may be installed in the HDD or flash memory of the automatic driving control device 100 by attaching the storage medium (non-transitory storage medium) to a drive device. The automatic driving control device 100 is an example of a "vehicle control device," and the combination of the action plan generation section 140 and the second control section 160 is an example of a "driving control section."

FIG. 2 is a functional configuration diagram of the first control section 120 and the second control section 160. The first control unit 120 includes, for example, a recognition unit 130, an action plan generation unit 140, and a mode determination unit 150. The first control unit 120 realizes, for example, a function based on AI (Artificial Intelligence) and a function based on a model given in advance in parallel. For example, in the "recognize intersections" function, intersection recognition using deep learning, etc., and recognition based on pre-given conditions (pattern-matchable signals, road markings, etc.) are executed in parallel. This may be achieved by scoring and comprehensively evaluating. This ensures the reliability of automated driving.

The recognition unit 130 recognizes the position, speed, acceleration, etc. of objects around the own vehicle M based on information input from the camera 10, radar device 12, and LIDAR 14 via the object recognition device 16. do. The position of the object is recognized, for example, as a position on absolute coordinates with the origin at a representative point (such as the center of gravity or the center of the drive shaft) of the own vehicle M, and is used for control. The position of an object may be expressed by a representative point such as the center of gravity or a corner of the object, or may be expressed by an area. The "state" of an object may include the acceleration or jerk of the object, or the "behavioral state" (eg, whether it is changing lanes or is about to change lanes).

Further, the recognition unit 130 recognizes, for example, the lane in which the host vehicle M is traveling (driving lane). For example, the recognition unit 130 uses the road marking line pattern (for example, an array of solid lines and broken lines) obtained from the second map information 62 and the road marking lines around the own vehicle M recognized from the image captured by the camera 10. It recognizes the driving lane by comparing it with the pattern of Note that the recognition unit 130 may recognize driving lanes by recognizing not only road division lines but also road boundaries (road boundaries) including road division lines, road shoulders, curbs, median strips, guardrails, etc. . In this recognition, the position of the own vehicle M acquired from the navigation device 50 and the processing results by the INS may be taken into consideration. The recognition unit 130 also recognizes stop lines, obstacles, red lights, toll booths, and other road events.

When recognizing the driving lane, the recognition unit 130 recognizes the position and attitude of the host vehicle M with respect to the driving lane. For example, the recognition unit 130 calculates the relative position of the vehicle M with respect to the driving lane based on the deviation of the reference point of the vehicle M from the center of the lane and the angle made with respect to a line connecting the center of the lane in the traveling direction of the vehicle M. It may also be recognized as a posture. Instead, the recognition unit 130 recognizes the position of the reference point of the own vehicle M with respect to any side edge of the driving lane (road division line or road boundary) as the relative position of the own vehicle M with respect to the driving lane. You may.

In principle, the action plan generation unit 140 drives the vehicle M in the recommended lane determined by the recommended lane determination unit 61, and furthermore, the behavior plan generation unit 140 automatically (driver Generates a target trajectory to be traveled in the future (without relying on any operation). The target trajectory includes, for example, a velocity element. For example, the target trajectory is expressed as a sequence of points (trajectory points) that the vehicle M should reach. A trajectory point is a point that the own vehicle M should reach every predetermined travel distance (for example, about several [m]) along the road, and apart from that, it is a point that the own vehicle M should reach every predetermined distance traveled along the road (for example, about a few [m]). ) are generated as part of the target trajectory. Alternatively, the trajectory point may be a position to be reached by the host vehicle M at each predetermined sampling time. In this case, information on target speed and target acceleration is expressed by intervals between trajectory points.

The action plan generation unit 140 may set automatic driving events when generating the target trajectory. Autonomous driving events include constant-speed driving events, low-speed following driving events, lane change events, branching events, merging events, and takeover events. The action plan generation unit 140 generates a target trajectory according to the triggered event.

The mode determining unit 150 determines the driving mode of the host vehicle M to be one of a plurality of driving modes in which tasks imposed on the driver are different. The mode determining section 150 includes, for example, a determining section 152. The function of the determination unit 152 will be described later.

FIG. 3 is a diagram showing an example of the correspondence between driving modes, control states of the own vehicle M, and tasks. There are five driving modes of the host vehicle M, for example, mode A to mode E. The control state, that is, the degree of automation of the driving control of the own vehicle M is highest in mode A, then decreasing in order of mode B, mode C, and mode D, and mode E is the lowest. Conversely, the tasks imposed on the driver are the lightest in mode A, followed by mode B, mode C, and mode D, which become more severe in that order, and mode E is the most severe. Note that in modes D and E, the control state is not automatic driving, so the automatic driving control device 100 is responsible for ending the control related to automatic driving and transitioning to driving support or manual driving. The contents of each driving mode will be illustrated below.

In mode A, the vehicle is in an automatic driving state, and the driver is not required to monitor the front or hold the steering wheel 82 (in the figure, grip the steering wheel). However, even in mode A, the driver is required to be able to quickly shift to manual driving in response to a request from the system centered on the automatic driving control device 100. Note that the term "automatic driving" used herein means that both steering and acceleration/deceleration are controlled without depending on the driver's operations. The front means the space in the traveling direction of the own vehicle M that is visually recognized through the front windshield. Mode A is applicable, for example, to conditions such as when the own vehicle M is traveling at a predetermined speed (for example, about 50 [km/h]) or less on a motorway such as an expressway, and there is a vehicle in front to be followed. This is a driving mode that can be executed when the following conditions are satisfied, and is sometimes referred to as TJP (Traffic Jam Pilot). If this condition is no longer met, the mode determining unit 150 changes the driving mode of the host vehicle M to mode B.

In mode B, the driver is in a driving support state, and the driver is tasked with monitoring the front of the own vehicle M (hereinafter referred to as "forward monitoring"), but is not tasked with gripping the steering wheel 82. In mode C, the vehicle is in a driving support state, and the driver is tasked with the task of monitoring the front and the task of gripping the steering wheel 82. Mode D is a driving mode that requires a certain degree of driving operation by the driver regarding at least one of the steering and acceleration/deceleration of the own vehicle M. For example, in mode D, driving assistance such as ACC (Adaptive Cruise Control) and LKAS (Lane Keeping Assist System) is performed. In mode E, the vehicle is in a manual driving state in which both steering, acceleration and deceleration require driving operations by the driver. In both modes D and E, the driver is naturally tasked with monitoring the area in front of the own vehicle M.

The automatic driving control device 100 (and the driving support device (not shown)) executes an automatic lane change according to the driving mode. Auto lane changes include auto lane changes based on system requests (1) and auto lane changes based on driver requests (2). Auto lane change (1) includes an auto lane change for overtaking, which is performed when the speed of the vehicle in front is lower than the standard compared to the own vehicle's speed, and an auto lane change for proceeding toward the destination. Auto lane change (Auto lane change due to a change in the recommended lane). Auto lane change (2) changes the lane of the vehicle M toward the direction of operation when the driver operates the turn signal when conditions such as speed and positional relationship with surrounding vehicles are met. It is something that makes you

In mode A, automatic driving control device 100 performs neither automatic lane change (1) nor (2). The automatic driving control device 100 executes both automatic lane changes (1) and (2) in modes B and C. In mode D, the driving support device (not shown) executes automatic lane change (2) without executing automatic lane change (1). In mode E, neither automatic lane change (1) nor (2) is performed.

If the driver does not perform a task related to the determined driving mode (hereinafter referred to as the current driving mode), the mode determining unit 150 changes the driving mode of the own vehicle M to a driving mode in which the task is more severe.

For example, in mode A, if the driver is unable to shift to manual driving in response to a request from the system (for example, if the driver continues to look outside the permissible area, or if signs of difficulty in driving are detected) ), the mode determining unit 150 uses the HMI 30 to urge the driver to shift to manual driving, and if the driver does not respond, the vehicle M is brought to the shoulder of the road and gradually stopped, and automatic driving is stopped. I do. After automatic driving is stopped, the own vehicle enters the state of mode D or E, and it becomes possible to start the own vehicle M by manual operation by the driver. The same applies to "stop automatic operation" below. If the driver is not monitoring the road ahead in mode B, the mode determining unit 150 uses the HMI 30 to urge the driver to look ahead, and if the driver does not comply, the vehicle M is brought to the shoulder of the road and gradually stopped. , control such as stopping automatic operation. In mode C, if the driver is not monitoring the front or not gripping the steering wheel 82, the mode determining unit 150 uses the HMI 30 to have the driver monitor the front and/or grip the steering wheel 82. If the driver does not respond, the vehicle M is brought to the shoulder of the road and gradually stopped, and automatic driving is stopped.

The second control unit 160 controls the driving force output device 200, the brake device 210, and the steering device 220 so that the host vehicle M passes the target trajectory generated by the action plan generation unit 140 at the scheduled time. Control.

Returning to FIG. 2, the second control section 160 includes, for example, an acquisition section 162, a speed control section 164, and a steering control section 166. The acquisition unit 162 acquires information on the target trajectory (trajectory point) generated by the action plan generation unit 140, and stores it in a memory (not shown). The speed control unit 164 controls the travel driving force output device 200 or the brake device 210 based on the speed element associated with the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 according to the degree of curvature of the target trajectory stored in the memory. The processing of the speed control section 164 and the steering control section 166 is realized, for example, by a combination of feedforward control and feedback control. As an example, the steering control unit 166 performs a combination of feedforward control according to the curvature of the road in front of the host vehicle M and feedback control based on the deviation from the target trajectory.

The driving force output device 200 outputs driving force (torque) for driving the vehicle to the driving wheels. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, and the like, and an ECU (Electronic Control Unit) that controls these. The ECU controls the above configuration according to information input from the second control unit 160 or information input from the driving operator 80.

The brake device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the second control unit 160 or information input from the driving operator 80 so that brake torque corresponding to the braking operation is output to each wheel. The brake device 210 may include, as a backup mechanism, a mechanism that transmits hydraulic pressure generated by operating a brake pedal included in the driving operator 80 to a cylinder via a master cylinder. Note that the brake device 210 is not limited to the configuration described above, and may be an electronically controlled hydraulic brake device that controls an actuator according to information input from the second control unit 160 and transmits the hydraulic pressure of the master cylinder to the cylinder. Good too.

Steering device 220 includes, for example, a steering ECU and an electric motor. For example, the electric motor applies force to a rack and pinion mechanism to change the direction of the steered wheels. The steering ECU drives the electric motor in accordance with information input from the second control unit 160 or information input from the driving operator 80 to change the direction of the steered wheels.

[Operation of vehicle control device]
Next, the operation of the vehicle control device according to the embodiment will be explained. In the following description, it is assumed that the host vehicle M is traveling in the mode B driving mode. FIG. 4 is a diagram illustrating an example of a scene in which the operation of the vehicle control device according to the embodiment is executed. In FIG. 4, the own vehicle M is traveling on the lane L1 before a branch point, and represents a situation where the driving mode is set to mode B so that the vehicle M continues straight ahead.

While the host vehicle M is traveling in the lane L1, the recognition unit 130 recognizes the surrounding situation of the host vehicle M, particularly the road marking lines on both sides of the host vehicle M, based on the image captured by the camera 10. Hereinafter, the road marking line recognized based on the image captured by the camera 10 will be represented by CL (hereinafter referred to as "camera road marking line CL"), and the road marking line recognized based on the second map information 62. is represented by ML (hereinafter referred to as "map road marking line ML"). In FIG. 4, the map road division line ML deviates from the actual road division line AL, but this may be due to, for example, construction work or redrawing of the road division line. This indicates a situation where there is a deviation from AL.

The determining unit 152 determines whether or not there is a mismatch between the camera road marking line CL and the map road marking line ML while the host vehicle M is traveling. Here, the deviation means, for example, that the distance between the camera road marking line CL and the map road marking line ML is greater than or equal to a predetermined value. The determining unit 152 further determines whether the own vehicle M is at a branch point based on the position information of the branch point obtained from the second map information 62. More specifically, for example, the recognition unit 130 determines whether or not the own vehicle M is within a predetermined distance from the position information (for example, GPS coordinates) of the branch point, thereby determining whether the own vehicle M is at the branch point. Determine whether or not there is.

If it is determined that there is a discrepancy between the camera road marking line CL and the map road marking line ML, and it is determined that the host vehicle M is at a branch point, the mode determining unit 150 obtains information from the second map information 62. The driving mode of the own vehicle M is determined based on the branch direction of the branch point and the direction of the camera road marking line CL.

More specifically, for example, as shown in FIG. 4, the mode determining unit 150 determines the branching direction of the branching point obtained from the second map information 62 (the extending direction of the map road marking line ML) and the camera road marking line CL. If the calculated degree of deviation is equal to or greater than a threshold value, it is determined that the camera road marking line CL has higher reliability than the map road marking line ML. Generally, when the own vehicle M passes by a branch point, the recognized camera road marking line CL may be closer to the main road due to, for example, a special painting situation at the branch point. This is because there is a high frequency of erroneous recognition in which the vehicle is moved toward the branch road ML. That is, if the recognized camera road marking line CL is not located in the direction of the branch road ML, this means that the reliability of the camera road marking line CL is high.

Therefore, if the degree of deviation between the branch direction of the branch point obtained from the second map information 62 and the direction of the camera road marking line CL is greater than or equal to the threshold, the mode determining unit 150 changes the driving mode from mode B to mode B. C, and continues driving in mode C using the camera road marking line CL as a reference line. Alternatively, the mode determining unit 150 may continue traveling in the mode B driving mode without changing the driving mode from mode B, using the camera road marking line CL as a reference line. As a result, even if there is an unmatch between the camera road marking line CL and the map road marking line ML at a branch point, the vehicle M can be driven using a more reliable road marking line. can be controlled.

FIG. 5 is a diagram illustrating another example of a scene in which the operation of the vehicle control device according to the embodiment is executed. The scene in FIG. 5 represents a situation where the camera road marking line CL is drawn in a direction different from the lane L1 in which the host vehicle M originally travels as a result of the camera road marking line CL moving toward the direction of the branch road ML. There is. At this time, the determination unit 152 determines that the degree of deviation between the branch direction of the branch point obtained from the second map information 62 and the direction of the camera road marking line CL is less than the threshold value, and the mode determination unit 150 , the driving mode of mode B is continued using the map road marking line ML as a reference line. That is, if the degree of deviation between the branching direction of the branch point obtained from the second map information 62 and the direction of the camera road marking line CL is less than the threshold value, this means that the camera road marking line CL is in the direction of the branching road ML. In order to indicate that there is a possibility of erroneous recognition in which the vehicle approaches the vehicle, the driving mode of mode B is continued using the map road marking line ML. As a result, the host vehicle M can proceed straight in the lane L1 along the map road marking line ML without being guided by the camera road marking line CL that is closer to the branch road ML.

Alternatively, if the degree of deviation between the direction of the branch road ML and the direction of the camera road marking line CL is less than a threshold, the mode determining unit 150 notifies the occupants of the host vehicle M and then sets the driving mode. may be changed from mode B to mode C. For example, if the degree of deviation between the direction of the branch road ML and the direction of the camera road marking line CL is less than a threshold, the mode determination unit 150 transmits information indicating the possibility of misrecognition on the branch road ML to the own vehicle. It is also possible to notify the occupant of M and recommend the occupant to change to mode C, or to change to mode C after a certain period of time from the determination of the deviation.

Next, with reference to FIG. 6, a flow of operations executed by the vehicle control device according to the embodiment will be described. FIG. 6 is a flowchart illustrating an example of the flow of operations executed by the vehicle control device according to the embodiment. The processing according to this flowchart is executed in a predetermined cycle while the host vehicle M is traveling in the mode B driving mode using the camera road marking CL.

First, the mode determining unit 150 acquires the camera road marking line CL and the map road marking line ML via the recognition unit 130 (step S100). Next, the determining unit 152 determines whether there is a discrepancy between the acquired camera road marking line CL and the map road marking line ML (step S102).

Next, if it is determined that there is a discrepancy between the acquired camera road marking line CL and the map road marking line ML, the determining unit 152 determines whether the own vehicle M is at the branch point based on the second map information 62. It is determined whether or not there is (step S104). If it is determined that there is no discrepancy between the acquired camera road marking line CL and the map road marking line ML, or if it is determined that the host vehicle M is not at a branch point, the mode determining unit 150 performs processing. returns to step S100.

On the other hand, if it is determined that the host vehicle M is at a branch point, then the determination unit 152 determines whether the degree of deviation between the direction of the camera road marking line CL and the branch direction of the branch point is greater than or equal to the threshold value. It is determined whether or not (step S106). If it is determined that the degree of deviation between the direction of the camera road marking line CL and the branching direction of the branch point is less than the threshold, the mode determining unit 150 uses the map road marking line ML as a reference line to select a mode. It is determined to continue the operation mode B (step S108). On the other hand, if it is determined that the degree of deviation between the direction of the camera road marking line CL and the branching direction of the branch point is greater than or equal to the threshold value, the mode determining unit 150 changes the driving mode from mode B to mode C. (Step S110). At this time, the mode determining unit 150 temporarily discards the map road marking line ML, uses the camera road marking line CL as a reference line, and decides to continue the driving mode of mode C. Thereby, the processing of this flowchart ends.

According to the present embodiment described above, when a discrepancy occurs between the camera road marking line and the map road marking line and the own vehicle is at a branch point, the direction of the camera road marking line and the branch point are The driving mode of the own vehicle is controlled based on the branching direction. As a result, even if the road marking line recognized by the camera and the content of the map information carried by the own vehicle are different, the road marking line recognized by the camera and the content of the map information carried by the own vehicle Even if the values are different from each other, the driving control of the vehicle can be appropriately changed.

The embodiment described above can be expressed as follows.
a storage device that stores the program;
comprising a hardware processor;
the processor executing the computer-readable instructions to:
Obtain camera images of the surrounding situation of the vehicle,
Controlling the steering and acceleration/deceleration of the vehicle based on the camera image and map information without depending on the operation of the driver of the vehicle;
The driving mode of the vehicle is determined to be one of a plurality of driving modes including a first driving mode and a second driving mode, and the second driving mode is a mode in which the task imposed on the driver is The driving mode is a mild driving mode compared to the first driving mode, and a part of the plurality of driving modes including at least the second driving mode does not rely on the operation of the driver of the vehicle to operate and control the vehicle. This is performed by controlling acceleration and deceleration, and when the task related to the determined driving mode is not executed by the driver, changing the driving mode of the vehicle to a driving mode in which the task is more severe,
Determining whether there is a discrepancy between the road marking line shown in the camera image and the road marking line shown in the map information, and determining whether the vehicle is at a branch point shown in the map information. Determine,
When it is determined that there is a discrepancy between the road marking line shown in the camera image and the road marking line shown in the map information, and it is determined that the vehicle is at a branch point shown in the map information, determining a driving mode of the vehicle based on a branching direction of the branching point and a direction of a road marking line shown in the camera image;
A vehicle control device configured as follows.

Although the mode for implementing the present invention has been described above using embodiments, the present invention is not limited to these embodiments in any way, and various modifications and substitutions can be made without departing from the gist of the present invention. can be added.

10 Camera 12 Radar device 14 LIDAR
16 Object recognition device 100 Automatic driving control device 120 First control section 130 Recognition section 140 Action plan generation section 150 Mode determination section 152 Judgment section 160 Second control section

Claims (6)

an acquisition unit that acquires a camera image of the surrounding situation of the vehicle;
a driving control unit that controls steering and acceleration/deceleration of the vehicle based on the camera image and map information without relying on operations by a driver of the vehicle;
The driving mode of the vehicle is determined to be one of a plurality of driving modes including a first driving mode and a second driving mode, and the second driving mode is a mode in which the task imposed on the driver is The driving mode is a milder driving mode than the first driving mode, and a part of the plurality of driving modes including at least the second driving mode is controlled by the driving control unit, and the determined driving mode is a mode determining unit that changes the driving mode of the vehicle to a driving mode in which the task is more severe when the task related to the mode is not executed by the driver;
Determining whether there is a discrepancy between the road marking line shown in the camera image and the road marking line shown in the map information, and determining whether the vehicle is at a branch point shown in the map information. a determination unit that determines the
The mode determining unit determines that there is a discrepancy between the road marking line shown in the camera image and the road marking line shown in the map information, and the vehicle is located at a branch point shown in the map information. If it is determined that the driving mode of the vehicle is determined based on the branch direction of the branch point and the direction of the road marking line shown in the camera image,
Vehicle control device. The mode determining unit changes the second driving mode to the first driving mode when the degree of deviation between the branching direction of the branching point and the direction of the road marking line shown in the camera image is greater than or equal to a threshold value. mode and continuing the first driving mode using the road marking line shown in the camera image;
The vehicle control device according to claim 1. When the branching direction of the branch point and the direction of the road marking line shown in the camera image are less than a threshold, the mode determining unit controls the second driving mode using the road marking line shown in the map information. continue mode,
The vehicle control device according to claim 1 or 2. The second driving mode is a driving mode in which the driver is not tasked with holding an operator that receives a steering operation of the vehicle,
The first driving mode is a driving mode in which the driver is tasked with at least the task of grasping the operator;
The vehicle control device according to any one of claims 1 to 3. The computer is
Obtain camera images of the surrounding situation of the vehicle,
Controlling the steering and acceleration/deceleration of the vehicle based on the camera image and map information without depending on the operation of the driver of the vehicle;
The driving mode of the vehicle is determined to be one of a plurality of driving modes including a first driving mode and a second driving mode, and the second driving mode is a mode in which the task imposed on the driver is The driving mode is a mild driving mode compared to the first driving mode, and a part of the plurality of driving modes including at least the second driving mode does not rely on the operation of the driver of the vehicle to operate and control the vehicle. This is performed by controlling acceleration and deceleration, and when the task related to the determined driving mode is not executed by the driver, changing the driving mode of the vehicle to a driving mode in which the task is more severe,
Determining whether there is a discrepancy between the road marking line shown in the camera image and the road marking line shown in the map information, and determining whether the vehicle is at a branch point shown in the map information. Determine,
When it is determined that there is a discrepancy between the road marking line shown in the camera image and the road marking line shown in the map information, and it is determined that the vehicle is at a branch point shown in the map information, determining a driving mode of the vehicle based on a branching direction of the branching point and a direction of a road marking line shown in the camera image;
Vehicle control method. to the computer,
Obtain camera images of the vehicle's surroundings,
Controlling the steering and acceleration/deceleration of the vehicle based on the camera image and map information without depending on the operation of the driver of the vehicle;
The driving mode of the vehicle is determined to be one of a plurality of driving modes including a first driving mode and a second driving mode, and the second driving mode is a mode in which the task imposed on the driver is The driving mode is a mild driving mode compared to the first driving mode, and a part of the plurality of driving modes including at least the second driving mode does not rely on the operation of the driver of the vehicle to operate and control the vehicle. This is performed by controlling acceleration and deceleration, and when the driver does not perform a task related to the determined driving mode, changing the driving mode of the vehicle to a driving mode in which the task is more severe;
Determining whether there is a discrepancy between the road marking line shown in the camera image and the road marking line shown in the map information, and determining whether the vehicle is at a branch point shown in the map information. Let them judge the
When it is determined that there is a discrepancy between the road marking line shown in the camera image and the road marking line shown in the map information, and it is determined that the vehicle is at a branch point shown in the map information, determining a driving mode of the vehicle based on a branching direction of the branching point and a direction of a road marking line shown in the camera image;
program.

Images