JP7272257B2 - vehicle controller - Google Patents

Description

The present invention relates to a vehicle control device.

A planned travel route of the vehicle is generated based on the current position of the vehicle, the destination of the vehicle, and the map data for use in automatic driving control of the vehicle. The vehicle is controlled to travel along this planned travel route such that a safe distance is maintained between the vehicle and other vehicles.

The planned driving route includes, for example, a merging section where the vehicle enters from the road on which the vehicle is currently traveling to another road at the merging destination, a branch section where the vehicle exits from the road on which the vehicle is currently traveling to another road at the branch destination, Alternatively, road segments with reduced number of lanes may be included. In such places, when it becomes difficult to keep a safe distance between the vehicle and other vehicles, the vehicle control system changes the vehicle control from automatic control to manual control. In addition, the driver may be notified of the transfer of control.

For example, Patent Literature 1 discloses that when it is detected that the driver is awake based on an image captured by the driver, the vehicle is changed from automatic driving to manual driving using sound or light emission. We have proposed a vehicle control device that changes the driving of the vehicle from automatic driving to manual driving after notifying the driver.

Japanese Patent Application Laid-Open No. 2019-21220

However, even if the vehicle uses sound or light to notify the driver that the driving of the vehicle is to be changed from automatic driving to manual driving, the driver does not receive an output from the vehicle while the vehicle is being automatically driven. Because you may not pay enough attention to the information such as sound or light that you receive, you may fail to recognize the notification from the vehicle.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a vehicle control device that notifies a driver of an operation scheduled for a vehicle so that the driver can easily recognize the notification from the vehicle even when the vehicle is under automatic control.

According to one embodiment, a vehicle controller is provided. This vehicle control device includes a determination unit that determines whether or not an operation of the vehicle that should be notified to the driver is scheduled, based on the vehicle status information including the driving status information of the automatically controlled vehicle or the surrounding environment information. , when it is determined that an operation of the vehicle that should be notified to the driver is scheduled, an operation control that controls an operation unit that affects the behavior of the vehicle by operation by the driver, based on the operation that is scheduled for the vehicle. A notifying unit that notifies the driver of the expected operation of the vehicle by operating the unit in a state separated from the behavior of the vehicle, wherein moving the operation unit means moving the steering wheel and moving the accelerator pedal. and a notification unit including at least one of moving the brake pedal and moving the shift knob.

ADVANTAGE OF THE INVENTION The vehicle control apparatus which concerns on this invention is effective in being able to notify an operation|movement which a vehicle is scheduled to a driver so that it may be easy to recognize the notification from a vehicle, even when a vehicle is automatically controlled.

1 is a schematic configuration diagram of a vehicle control system in which a vehicle control device is implemented; FIG. FIG. 2 is a diagram illustrating an operating device and an operating portion of a vehicle whose behavior is affected by the operating device; 1 is a hardware configuration diagram of an electronic control device that is one embodiment of a vehicle control device; FIG. FIG. 4 is a functional block diagram of the processor of the electronic control unit regarding information notification processing; FIG. 11 is a diagram (part 1) illustrating processing for generating a driving lane plan in a merging section; FIG. 11 is a diagram (part 2) for explaining the process of generating a driving lane plan in a merging section; It is a figure explaining the icon which alert|reports the operation|movement which is scheduled to a vehicle to a driver. FIG. 2 is a diagram (part 1) for explaining an icon for informing the driver of the state of the vehicle; FIG. 2 is a diagram (part 2) for explaining an icon for informing the driver of the state of the vehicle; FIG. 3 is a diagram (part 3) for explaining an icon for informing the driver of the state of the vehicle; 4 is an operation flowchart of a vehicle control system including vehicle control processing;

The vehicle control device will be described below with reference to the drawings. This vehicle control device determines whether or not an operation of the vehicle that should be notified to the driver is scheduled, based on the vehicle status information including the driving status information of the automatically controlled vehicle or the surrounding environment information. The motion of the vehicle that should be notified to the driver includes, for example, the motion of moving from the current lane of the vehicle to an adjacent lane. During such vehicle operation, the distance between the vehicle and other vehicles or other objects may fall below a predetermined threshold. , the vehicle controller may transfer control of the vehicle to the driver. When the vehicle control device determines that a vehicle action to be notified to the driver is scheduled, the vehicle control device selects an operation unit that affects the behavior of the vehicle through operation by the driver, based on the scheduled operation of the vehicle. By moving the vehicle in a state separated from the behavior of the vehicle using the operation control unit that controls the vehicle, the driver is notified of the expected operation of the vehicle. Moving the operating unit includes at least one of moving the steering wheel, moving the accelerator pedal, moving the brake pedal, and moving the shift knob. As a result, the vehicle control device can notify the driver of the expected operation of the vehicle so that the driver can easily recognize the notification from the vehicle even when the vehicle is automatically controlled. By notifying the driver of the expected operation of the vehicle, the driver can understand, for example, that the control of the vehicle may be transferred to the driver or the operation to be performed after the transfer.

A vehicle control system having the vehicle control device of this embodiment has an automatic control driving mode in which the vehicle is driven under automatic control and a manual control driving mode in which the vehicle is driven by the driver. The vehicle control system in the automatic control driving mode switches the driving of the vehicle from the automatic control driving mode currently applied to the manual control driving mode when the driver accepts a control change notification requesting a change from automatic control to manual control. . As a result, when the vehicle cannot drive safely under automatic control, the driver can operate the vehicle under manual control to drive. In the manual control driving mode, the vehicle is manually controlled in at least one of driving, braking and steering.

FIG. 1 is a schematic configuration diagram of a vehicle control system in which a vehicle control device is implemented. FIG. 2 is a diagram for explaining an operating device and an operating portion of a vehicle whose behavior is affected by the operating device. Also, FIG. 3 is a hardware configuration diagram of an electronic control device, which is one embodiment of the vehicle control device.

In this embodiment, a vehicle control system 1 mounted on a vehicle 10 and controlling the vehicle 10 includes a camera 2 that captures an image in front of the vehicle, and LiDAR sensors 3a to 3d that are arranged on the front, rear, left, and right of the vehicle 10. and The vehicle control system 1 also includes a positioning information receiver 4, a map information storage device 5 for generating map information based on the positioning information output by the positioning information receiver 4, a user interface (UI) 6, and an operating device. 7, a navigation device 8, and an electronic control unit (ECU) 9, which is an example of a vehicle control device.

The camera 2, the LiDAR sensors 3a to 3d, the map information storage device 5, the UI 6, the operation device 7, the navigation device 8, and the ECU 9 communicate via an in-vehicle network 11 conforming to a standard such as a controller area network. Connected as possible.

The camera 2 is mounted, for example, in the vehicle interior of the vehicle 10 so as to face the front of the vehicle 10 . The camera 2 generates an image representing a predetermined area in front of the vehicle 10 at image information acquisition times set in a predetermined cycle. The generated image represents other vehicles around the vehicle 10 or features such as lane markings on the road surface included within a predetermined area in front of the vehicle 10 . The image produced by camera 2 may be a color image or a gray image. The camera 2 is an example of an imaging unit, and includes a two-dimensional detector composed of an array of photoelectric conversion elements sensitive to visible light, such as a CCD or C-MOS, and an object to be photographed on the two-dimensional detector. It has an imaging optical system that forms an image of the area. Each time the camera 2 generates an image, the camera 2 outputs the image and the image information acquisition time at which the image was generated to the ECU 9 via the in-vehicle network 11 . The image is used in the ECU 9 for processing for estimating the position of the vehicle and for processing for detecting other objects around the vehicle 10 .

The LiDAR sensors 3a to 3d are attached to the outer surface of the vehicle 10, for example, so as to face the front, left, rear, and right sides of the vehicle 10, respectively. Each of the LiDAR sensors 3a to 3d synchronously emits pulsed lasers toward the front, left, rear, and right sides of the vehicle 10 at reflected wave information acquisition times set in a predetermined cycle, Receive reflected waves reflected by reflecting objects. The time required for the reflected wave to return has information about the distance between the vehicle 10 and the feature located in the direction in which the laser was irradiated. Each of the LiDAR sensors 3a to 3d sends reflected wave information including the direction of laser irradiation and the time required for the reflected wave to return, along with the reflected wave information acquisition time when the laser was emitted, to the ECU 9 via the in-vehicle network 11. Output. The reflected wave information is used in processing for detecting other objects around the vehicle 10 in the ECU 9 .

The positioning information receiver 4 outputs positioning information representing the current position of the vehicle 10 . For example, the positioning information receiver 4 can be a GPS receiver. The positioning information receiver 4 has a positioning information receiving section 4a that receives GPS radio waves, and a processor 4b that outputs positioning information representing the current position of the vehicle 10 based on the GPS radio waves received by the positioning information receiving section 4a. . The processor 4b outputs the positioning information and the positioning information acquisition time at which the positioning information is acquired to the map information storage device 5 every time the positioning information receiving unit 4a acquires the positioning information at a predetermined reception cycle.

The map information storage device 5 has a processor (not shown) and a storage device (not shown) such as a magnetic disk drive or nonvolatile semiconductor memory. It stores wide-area map information of a relatively wide range (for example, a range of 10 to 30 km square) including . This wide-area map information is preferably high-precision map information including three-dimensional information on road surfaces, features such as lane markings on roads, information representing types and positions of structures, legal speed limits for roads, and the like. Note that the positions of features and structures on roads are represented, for example, by a world coordinate system with a predetermined reference position in real space as the origin. The processor receives wide area map information from an external server via a base station by wireless communication via a wireless communication device (not shown) included in the vehicle control system 1 according to the current position of the vehicle 10. Store in memory. Each time the positioning information is input from the positioning information receiver 4, the processor refers to the wide-area map information stored in the storage device to refer to a relatively narrow area (for example, 100 m to 10 km square) map information, positioning information, and positioning information acquisition time are output to the ECU 8 via the in-vehicle network 11 . Further, the processor of the map information storage device transmits the positioning information and the positioning information acquisition time to the navigation device 8 via the in-vehicle network 11 every time it receives the positioning information and the positioning information acquisition time from the positioning information receiver 4 . .

The UI 6 is controlled by the ECU 9 to display driving status information of the vehicle 10, map information, an operation planned for the vehicle, a state of the vehicle, or a control change notification requesting a change in the control of the vehicle 10 from automatic control to manual control. is notified to the driver, and an operation signal corresponding to the operation of the vehicle 10 by the driver is generated. The travel status information of the vehicle 10 includes information regarding the current and future routes of the vehicle, such as the current position of the vehicle, the planned travel route, and the plan to change lanes. Further, the travel status information of the vehicle 10 may include, for example, a travel lane plan, which will be described later. The UI 6 has, for example, a liquid crystal display or a touch panel as a notification device for notifying the driver of driving situation information, scheduled actions of the vehicle, vehicle state or control change notification, and the like. Moreover, UI6 has a touch panel or an operation button as an input device which inputs the operation information to the vehicle 10 from a driver, for example. The operation information includes, for example, the destination, the waypoint, the vehicle speed, other vehicle control information, and the driver's response to the control change notification. The UI 6 outputs the input operation information to the ECU 9 via the in-vehicle network 11 .

The operating device 7 influences the behavior of the vehicle 10 by being operated by the driver. As shown in FIG. 2, the operating device 7 includes a steering wheel 71, an accelerator pedal 72, a brake pedal 73, a shift knob 74, etc. for the driver to operate the vehicle 10 when the manual control driving mode is applied. The direction of travel of the vehicle 10 changes according to the operation of the steering wheel 71 by the driver. The acceleration of the vehicle 10 changes according to the operation of the accelerator pedal 72 by the driver. The deceleration of the vehicle 10 changes according to the operation of the brake pedal 73 by the driver. Operation of the shift knob 74 by the driver changes the gear ratio in the drive system of the vehicle 10 .

The steering wheel 71 and the clutch 42 are connected to each other via the first shaft 40 so that the steering force that the steering wheel 71 steers the steered wheels 46 can be transmitted to the clutch 42 . The first motor 41 and the first shaft 40 are connected to each other so that the power of the first motor 41 to rotate the steering wheel 71 can be transmitted to the first shaft 40 . The first motor 41 can rotate the steering wheel 71 under the control of the ECU 9 . The first motor 41 has an encoder (not shown), which detects the steering amount (rotational angle) of the steering wheel 71 by the driver's operation and outputs it to the ECU 9 . The clutch 42 and the steered wheels 46 are connected via the second shaft 43 and the gear portion 45 so that the steering force from the clutch 42 can be transmitted to the steered wheels 46 . The second motor 44 and the second shaft 43 are connected to each other so that the steering force with which the second motor 44 steers the steered wheels 46 can be transmitted to the second shaft 43 . The second motor 44 is controlled by the ECU 9 and can move the steered wheels 46 . The first shaft 40 and the second shaft 43 are connected via a clutch 42 so as to be able to transmit steering force. The clutch 42 can be controlled by the ECU 9 to connect the first shaft 40 and the second shaft 43 when the manual control operation mode is applied, and disconnect when the automatic control operation mode is applied. Therefore, when the automatic control driving mode is applied, the movement of the steering wheel 71 and the behavior of the steered wheels 46 are separated. When the manual control operation mode is applied, the second motor 44 can be controlled by the ECU 9 to apply a force to steer the steered wheels 46 so as to assist the driver's operation of the steering wheel 71 in accordance with the amount of steering. be. Note that the vehicle 10 may not have a clutch. In this case, when the manual control operation mode is applied, the ECU 9 can control the second motor 44 using a steering signal corresponding to the steering amount of the steering wheel 71 to move the steered wheels 46 .

The accelerator pedal 72 and the third motor 51 are connected via a first gear 50 that converts the amount of operation (accelerator opening) of the accelerator pedal 72 by the driver into the amount of rotation. The third motor 51 has an encoder (not shown) that generates an accelerator opening signal corresponding to the accelerator opening due to the operation of the accelerator pedal 72 by the driver and outputs the signal to the ECU 9 . The ECU 9 outputs the accelerator opening signal input from the third motor 51 to the engine of the vehicle 10 or the driving device 60 having a motor or the like. The third motor 51 can move the accelerator pedal 72 under the control of the ECU 9 . When the manual control operation mode is applied, the ECU 9 outputs to the driving device 60 an accelerator opening signal generated based on the operation of the accelerator pedal 72 by the driver. On the other hand, when the automatic control operation mode is applied, the ECU 9 outputs to the drive device 60 an accelerator opening signal generated based on an operation plan described later. When the automatic control driving mode is applied, the movement of the accelerator pedal 72 and the behavior of the driving device 60 are separated.

The brake pedal 73 and the fourth motor 53 are connected via a second gear 52 that converts the amount of operation (braking amount) of the brake pedal 73 by the driver into the amount of rotation. The fourth motor 53 has an encoder (not shown) that generates a brake signal corresponding to the amount of brake applied by the driver's operation of the brake pedal 73 and outputs the brake signal to the ECU 9 . The ECU 9 outputs the brake signal input from the fourth motor 53 to the brake 61 of the vehicle 10 . The fourth motor 53 can move the brake pedal 73 under the control of the ECU 9 . When the manual control operation mode is applied, the ECU 9 outputs to the brake 61 a brake signal generated based on the operation of the brake pedal 73 by the driver. On the other hand, when the automatic control operation mode is applied, the ECU 9 outputs to the brake 61 a brake signal generated based on the operation plan described later. When the automatic control operation mode is applied, the movement of the brake pedal 73 and the behavior of the brake 61 are separated.

The shift knob 74 is attached to the shift device 54 having guide grooves for guiding the movement of the shift knob 74 so that the shift position can be changed according to the driver's shift operation. The shift device 54 generates a shift position signal indicating the shift position of the shift knob 74 and outputs it to the ECU 9 . The shift device 54 also has a drive section (not shown) that drives the shift knob 74 to change the shift position under the control of the ECU 9 . When the manual control operation mode is applied, the ECU 9 outputs to the driving device 60 a shift position signal generated based on the shift position corresponding to the driver's shift operation. On the other hand, when the automatic control operation mode is applied, the ECU 9 outputs to the drive device 60 a shift position signal generated based on an operation plan, which will be described later. When the automatic control driving mode is applied, the movement of the shift knob 74 and the behavior of the drive device 60 of the brake 61 are decoupled.

The navigation device 8 generates a planned travel route from the current position of the vehicle 10 to the destination based on the navigation map information, the destination of the vehicle 10 and the current position of the vehicle 10 . The navigation device 8 has a memory (not shown) for storing navigation map information and a processor (not shown). The map information for navigation includes location information of links representing roads and location information of nodes connected by the links. The road layout of the planned travel route is represented by links representing roads and nodes connected by the links. The positions of links and nodes are represented, for example, by coordinates in the world coordinate system. The processor determines the current position of the vehicle 10 based on the map information for navigation stored in the memory, the destination of the vehicle 10 received from the UI 6, and the positioning information indicating the current position of the vehicle 10 received from the map information storage device 5. Generate a planned travel route from a position to a destination. The processor generates the planned travel route of the vehicle 10 using, for example, the Dijkstra method. The planned travel route includes information regarding positions such as right turns, left turns, merging, and branching. The processor newly generates a planned travel route for the vehicle 10 when a new destination is set, or when the current position of the vehicle 10 deviates from the planned travel route. The processor outputs the planned travel route to the ECU 9 via the in-vehicle network 11 every time it generates the planned travel route.

The ECU 9 controls running of the vehicle 10 according to the applied driving mode. In the present embodiment, when the automatic control driving mode is applied, the ECU 9 determines the operation of the vehicle 10 to be notified to the driver based on the vehicle status information including the driving status information of the automatically controlled vehicle 10 or the surrounding environment information. Determination processing for determining whether or not the behavior of the vehicle 10 to be notified to the driver is scheduled. By moving the operating device 7 that exerts an influence in a state separated from the behavior of the vehicle 10, a notification process is performed to notify the driver of the expected operation of the vehicle 10. FIG. Therefore, the ECU 9 has a communication interface 21 , a memory 22 and a processor 23 .

A communication interface (I/F) 21 is an example of a communication unit, and has an interface circuit for connecting the ECU 9 to the in-vehicle network 11 . That is, the communication interface 21 drives the first motor 41 that drives the steering wheel 71, the third motor 51 that drives the accelerator pedal 72, the fourth motor 53 that drives the brake pedal 73, or the shift knob 74 via the in-vehicle network 11. It is connected to a shift device 54 or the like that The communication interface 21 drives the first motor 41 that drives the steering wheel 71, the third motor 51 that drives the accelerator pedal 72, and the brake pedal 73 in order to inform the driver of the operation that is scheduled for the vehicle 10 from the ECU 9. Upon receiving a control signal for controlling the shift device 54 that drives the fourth motor 53 or the shift knob 74 , the received control signal is transmitted to the first motor 41 , the third motor 51 , the fourth motor 53 or the shift device 54 . Also, the communication interface 21 is connected to the camera 2, the map information storage device 5, and the like via the in-vehicle network 11. FIG. For example, every time an image and image information acquisition time are received from the camera 2 , the communication interface 21 passes the received image and image information acquisition time to the processor 23 . Further, the communication interface 21 passes the received map information, positioning information and positioning information acquisition time to the processor 23 every time it receives the map information, the positioning information and the positioning information acquisition time from the map information storage device 5 . The communication interface 21 also passes vehicle speed, acceleration and yaw rate received from a vehicle speed sensor, acceleration sensor and yaw rate sensor (not shown) to the processor 23 .

The memory 22 is an example of a storage unit, and has, for example, a volatile semiconductor memory and a nonvolatile semiconductor memory. The memory 22 stores various data used in information notification processing executed by the processor 23 of the ECU 9, installation position information such as the optical axis direction and mounting position of the camera 2, and internal data such as the focal length and angle of view of the imaging optical system. Store parameters, etc. In addition, the memory 22 stores internal parameters such as installation positions and operation ranges of the LiDAR sensors 3a to 3d. The memory 22 also stores the planned travel route received from the navigation device 8, the image received from the camera 2 and the image information acquisition time, the map information received from the map information storage device 5, the positioning information, the positioning information acquisition time, and the like. do.

The processor 23 has one or more CPUs (Central Processing Units) and their peripheral circuits. Processor 23 may further comprise other arithmetic circuitry such as a logic arithmetic unit, a math unit or a graphics processing unit. If the processor 23 has multiple CPUs, each CPU may have a memory. The processor 23 determines whether or not an operation of the vehicle 10 to be notified to the driver is scheduled based on the vehicle status information including the driving status information of the automatically controlled vehicle 10 or the surrounding environment information. When it is determined that an operation of the vehicle 10 to be notified to the driver is scheduled, the operation device 7 that affects the behavior of the vehicle 10 by operation by the driver is controlled based on the operation scheduled for the vehicle 10 . By moving the vehicle 10 in a state separated from the behavior of the vehicle 10, a notification process is performed to notify the driver of the operation expected for the vehicle 10.例文帳に追加Based on the image generated by the camera 2, the processor 23 also executes position estimation processing for estimating the position of the vehicle 10 at the image information acquisition time when this image was generated. In addition, the processor 23 updates the position of the vehicle 10 using vehicle state information such as the estimated position and vehicle speed of the vehicle 10 at the most recent image information acquisition time at the position determination time set at a predetermined cycle. Processor 23 controls the traveling motion of vehicle 10 based on the estimated position of vehicle 10 , the destination of vehicle 10 , and the relative positional relationship with other objects around vehicle 10 .

FIG. 4 is a functional block diagram of the processor 23 of the ECU 9 regarding vehicle control processing including determination processing and notification processing. The processor 23 includes a position estimation unit 31, an object detection unit 32, a lane planning unit 33, a driving planning unit 34, a vehicle control unit 35, an operation control unit 36, a determination unit 37, and a notification unit 38. , and a notification control unit 39 . All or part of these units of the processor 23 are, for example, functional modules implemented by computer programs running on the processor 23 . Alternatively, all or part of these units included in the processor 23 may be dedicated arithmetic circuits provided in the processor 23 . Further, among these units of the processor 23, the operation control unit 36, the determination unit 37, and the notification unit 38 execute information notification processing related to determination processing and notification processing.

A position estimation unit 31 of the processor 23 estimates the position of the vehicle 10 based on the features around the vehicle 10 . The position estimating unit 31 is provided in the image of the camera 2 with a matching area for detecting lane markings, which is an example of a feature around the vehicle 10, and a classifier for identifying the lane markings in the image. to detect lane markings. As the discriminator, for example, a deep neural network (DNN) pre-trained to detect lane markings represented in an input image can be used. Then, assuming the position and orientation of the vehicle 10, the position estimation unit 31 calculates the lane markings represented in the map information received from the map information generation device 5 by the camera 2 generated at the current image information acquisition time. projected onto the image of For example, the position estimating unit 31 determines the position of the vehicle 10 represented by the positioning information received from the positioning information receiver 5 at the current image information acquisition time, and the traveling direction of the vehicle 10 obtained immediately before. The attitude of 10 is assumed to be the assumed position and assumed attitude of the vehicle 10 . The position estimator 31 obtains a conversion formula from the world coordinate system to the camera coordinate system having the position of the camera 2 as the origin and the optical axis direction of the camera 2 as one axis direction, according to the assumed position and assumed orientation. Such a conversion formula is represented by a combination of a rotation matrix representing rotation between coordinate systems and a translation vector representing translation between coordinate systems. Then, the position estimator 31 converts the coordinates of the lane markings on the road around the vehicle 10 represented in the world coordinate system, which are included in the map information, into the coordinates of the camera coordinate system according to the conversion formula. Then, the position estimating unit 31 calculates the lane markings around the vehicle 10 expressed in the camera coordinate system based on the internal parameters of the camera 2 such as the focal length of the camera 2 by the camera generated at the current image information acquisition time. 2 image. Then, the position estimation unit 31 calculates the degree of matching between the lane markings detected from the image of the camera 2 and the lane markings around the vehicle 10 shown on the map. The position estimating unit 31 changes the assumed positions and assumed orientations by a predetermined amount, and executes the same coordinate system conversion, projection, and degree-of-match calculation processes as described above, thereby obtaining a plurality of assumed positions and assumed orientations. , the degree of matching between the lane markings around the vehicle 10 represented in the map information and the lane markings detected from the image is calculated. Then, the position estimator 31 identifies the assumed position and the assumed posture when the degree of matching is maximized, sets the assumed position as the estimated position of the vehicle 10, and estimates the traveling direction of the vehicle 10 based on the assumed posture. Find the azimuth angle.

Further, the position estimating unit 31 detects the position of the vehicle estimated at the image information acquisition time immediately before the position determination time, which is set in a cycle shorter than the cycle of the image information acquisition time at which the camera 2 generates an image. Based on the estimated position and the estimated azimuth of the vehicle 10 and the movement amount and movement direction of the vehicle 10 between the image information acquisition time and the position determination time, the estimated position of the vehicle 10 at the position determination time and the estimation of the vehicle 10 Estimate the azimuth. The position estimation unit 31 integrates the vehicle speed of the vehicle 10 to obtain the amount of movement of the vehicle 10 between the image information acquisition time and the position determination time, and integrates the yaw rate of the vehicle 10 to obtain the image information acquisition time and the position estimation unit 31 . The moving direction of the vehicle 10 between the position determination time is obtained. The position estimation unit 31 estimates the driving lane on the road on which the vehicle 10 is located based on the map information and the estimated position and azimuth angle of the vehicle 10 . For example, the position estimating unit 31 determines that the vehicle 10 is traveling in a lane specified by two adjacent lane markings that sandwich the center position of the vehicle 10 in the horizontal direction. Each time the position estimation unit 31 obtains the estimated position, the estimated azimuth angle, and the driving lane of the vehicle 10 at the position determination time, the position estimation unit 31 transmits these information to the object detection unit 32, the driving lane planning unit 33, the driving planning unit 34, the vehicle The control unit 35 and the determination unit 37 are notified. Note that, if there is no positioning information at the positioning reception time that matches the image information acquisition time described above, the position estimation unit 31 determines the movement amount and movement direction of the vehicle 10 between the image information acquisition time and the positioning reception time. Based on this, the estimated position of the vehicle 10 and the attitude of the vehicle 10 at the image information acquisition time may be estimated.

The object detection unit 32 of the processor 23 detects other objects around the vehicle 10 and their types based on the image generated by the camera 2 . Other objects include other vehicles driving around vehicle 10 . The object detection unit 32 detects an object represented in the image by, for example, inputting the image generated by the camera 2 into the classifier. As a discriminator, for example, a deep neural network (DNN) pre-trained to detect an object represented in an input image can be used. The object detection unit 32 may use a discriminator other than the DNN. For example, the object detection unit 32 receives, as an identifier, a feature amount (for example, Histograms of Oriented Gradients, HOG) calculated from a window set on an image, and the window represents an object to be detected. A pre-trained support vector machine (SVM) may be used to output the confidence that Alternatively, the object detection unit 32 may detect an object region by performing template matching between a template representing an object to be detected and an image. Also, the object detection unit 32 may detect other objects around the vehicle 10 based on reflected wave information output by the LiDAR sensors 3a to 3d. Further, the object detection unit 32 obtains the orientation of the other object with respect to the vehicle 10 based on the position of the other object in the image generated by the camera 2, and this orientation and the reflected waves output by the LiDAR sensors 3a to 3d Based on the information, the distance between this other object and the vehicle 10 may be determined. Based on the current position of the vehicle 10 and the distance and bearing of the other object with respect to the vehicle 10, the object detection unit 32 estimates the position of the other object expressed, for example, in the world coordinate system. In addition, the object detection unit 32 detects objects from the latest image by associating other objects detected from the latest image generated by the camera 2 with objects detected from past images according to tracking processing based on optical flow. It may track other objects that have been hit. Then, the object detection unit 32 may obtain the trajectory of another object being tracked based on the position represented by the world coordinate system of the object in the latest image from the past images. The object detection unit 32 can estimate the speed of another object relative to the vehicle 10 based on the change in the position of the other object over time. Also, the object detection unit 32 can estimate the acceleration of the other object based on the change in the speed of the other object over time. Furthermore, the object detection unit 32 identifies the lane in which the other object is traveling based on the lane markings represented by the map information and the position of the other object. For example, the object detection unit 32 determines that the other object is traveling in the lane specified by two adjacent lane markings that sandwich the horizontal center position of the other object. The object detection unit 32 notifies the operation planning unit 34 and the determination unit 37 of information indicating the type (for example, vehicle) of the detected other object, information indicating its position, speed, acceleration, and driving lane.

The driving lane planning unit 33 of the processor 23 generates the map information and the purpose of the vehicle 10 in the most recent driving section (for example, 10 km) selected from the planned driving route at the driving lane plan generation time set at a predetermined cycle. Based on the planned traveling route to the ground and the current position of the vehicle 10, a lane in the road on which the vehicle 10 travels is selected, and a traveling lane plan representing the planned traveling lane along which the vehicle 10 travels is generated. In addition, the driving lane planning unit 33 determines whether it is necessary to change lanes in the most recent driving section selected from the planned driving route based on the map information, the planned driving route, and the current position of the vehicle 10. judge. In addition, the driving lane planning unit 33 may further use surrounding environment information or vehicle state information to determine whether a lane change is necessary. The surrounding environment information includes the positions and speeds of other vehicles traveling around the vehicle 10 . The vehicle state information includes the current position of the vehicle 10, vehicle speed, acceleration, traveling direction, and the like. The travel lane planning unit 33 determines whether or not a lane change is necessary within a predetermined section of the planned travel route, and includes the presence or absence of a lane change and, if the lane change is included, the lane before the change and the lane after the change. Generate driving lane plans. The driving lane planning unit 33 notifies the operation planning unit 34 and the determining unit 37 of the driving lane plan each time it generates the driving lane plan. Specifically, the driving lane planning unit 33 selects the nearest driving section from the planned driving route notified from the navigation device 8 at the driving lane plan generation time set at a predetermined cycle, and select lanes in the road on which the vehicle 10 is traveling to generate a travel lane plan. In addition, based on the planned travel route and the current position of the vehicle 10, the travel lane planning unit 33 allows the vehicle 10 to move from the road on which the vehicle 10 is currently traveling to another road to which the vehicle 10 merges (merge). The presence or absence of an event position where at least one of the following events occurs: turning right, turning left by the vehicle 10, and exiting the road on which the vehicle 10 is currently traveling to another road at a branch destination (branching). judge. The driving lane planning unit 33 determines whether or not it is necessary to change the lane when the driving section includes the event position. Specifically, the driving lane planning unit 33 determines whether or not the lane for executing the event at the event position is the same as the lane in which the vehicle 10 is currently traveling, and changes the lane if they are different. It determines that it is necessary, and generates a driving lane plan including the lanes before and after the change. In addition, if there is another vehicle traveling in the same lane as the lane in which the vehicle 10 is traveling, and the vehicle 10 continues to travel in the same lane, the traveling lane planning unit 33 determines that the vehicle 10 and the other vehicle are traveling in the same lane. When a collision with a vehicle is predicted, it is determined that it is necessary to change lanes, and a driving lane plan including the lanes before and after the change is generated.

Referring to FIG. 5, which shows an operation example of processing for generating a driving lane plan by the driving lane planning unit 33, the vehicle 10 enters from the road on which the vehicle 10 is currently traveling to another road (merging). It is explained below.

In the example shown in FIG. 5 , when the driving section includes a merging section that is the event position, the driving lane planning unit 33 calculates an event in the merging section based on the map information, the planned travel route, and the current position of the vehicle 10. is the same as the lane in which the vehicle 10 is currently traveling. When the lane in which the vehicle 10 is currently traveling is different from the lane in which the event is to be executed in the merging section, the traveling lane planning unit 33 determines that it is necessary to change the lane, and determines that the vehicle 10 is currently traveling. Generate a driving lane plan that includes moving from the lane of the road to the lane for executing the event in the merging section.

In the example shown in FIG. 5, the planned travel route 503 of the vehicle 10 includes a route 503a on the road 501 and a route 503b on the road 502 where the road 501 joins. A current position 500 of the vehicle 10 is on a route 503a. A route 503b is a route along which the vehicle 10 will travel in the future. The current driving section 504 includes a merging section 505 where the road 501 on which the vehicle 10 is currently traveling joins another road 502 . The driving lane planning unit 33 determines a merging section 505 where the vehicle 10 is currently traveling from the road 501 to another merging road 502 as an event position within the driving section 504 . The travel lane planning unit 33 is notified from the position estimating unit 31 that the current position 500 of the vehicle 10 is on the lane 501a. Since the driving section 504 includes the merging section 505, the driving lane planning unit 33 determines whether or not it is necessary to change the lane. In the merging section 505, the vehicle 10 moves from the lane 501a of the current road 501 to the lane 502a of the merging road 502, so the driving lane planning unit 33 executes the event at the event position. It is determined that the lane 502a for the vehicle 10 is different from the lane 501a on which the vehicle 10 is currently traveling. Therefore, the travel lane planning unit 33 determines that it is necessary to change the lane from the lane 501a of the road 501 on which the vehicle is currently traveling to the lane 502a of the road 502 . Then, the driving lane planning unit 33 generates a driving lane plan including changing lanes from the lane 501a of the road 501 to the lane 502a of the road 502 until the vehicle 10 reaches the merging section 505 in the driving section 504. .

Furthermore, the process of generating a driving lane plan including lane changes by the driving lane planning unit 33 will be described below with reference to FIG. 6 . In the example shown in FIG. 6, the vehicle 10 is traveling on the lane 601a of the road 601. In the example shown in FIG. The road 601 merges with the road 602 at the confluence section 605 . In the merging section 605, the lane 601a of the road 601, which is the merging lane, and the lane 602a of the road 602, which is the main road, are adjacent to each other, so the vehicle 10 can move from the lane 601a to the lane 602a. A starting point P1 of the merging section 605 is a position where the connection between the lane 601a and the lane 602a starts. The end point P4 of the merging lane is the end point of lane 601a, which is the merging lane. The driving lane plan includes vehicle 10 changing lanes from lane 601 a of road 601 to lane 602 a of road 602 in this merging section 605 .

When the vehicle 10 applies the automatic driving control mode, in the merging section 605, if it is unlikely that a safe distance can be secured between the vehicle 10 and the other vehicle, a safe distance between the vehicle 10 and the other vehicle is maintained. Continue driving in the current merging lane without changing lanes until the distance can be secured. Therefore, in preparation for the case where it is determined that the lane change cannot be performed by automatic control in the merging section 605, the driving lane planning unit 33 sets the mode change decision position P3 for deciding to stop the lane change by automatic control of the vehicle 10 as follows: It is determined on the lane 601 a of the merging section 605 . The driving lane planning unit 33 sets the mode change decision position P3 to the position of the minimum distance from the end point P4 of the merging section 605 at which the driver who has been handed over the control of the vehicle 10 can operate the vehicle 10 for lane change. decide to be For example, the driving lane planning unit 33 determines the time required for the driver to recognize the control change notification of the vehicle 10, start the operation of the vehicle 10 for lane change, and complete the lane change (for example, 4 seconds). and the current vehicle speed (for example, the average vehicle speed over the last few seconds) may be determined as the mode change determination position P3. In addition, the driving lane planning unit 33 may include, in the mode change determination position P3, a margin distance that allows the driver to operate the vehicle 10 with sufficient time under manual control. This margin distance may be determined based on vehicle speed or may be a fixed value. A control change section D2 between the give-up determination position P3 and the end point P4 of the merging section 605 is a section for transferring control of the vehicle 10 to the driver.

In the merging section 605, the vehicle 10 attempts to change the lane from the lane 501a to the lane 502a by automatic control in the automatically controlled section D1 between the starting point P1 of the merging section 605 and the mode change determination position P3. Therefore, the driving lane planning unit 33 drives so as to change the lane from the lane 601a of the road 601 to the lane 602a of the road 602 in the automatic control section D1 between the starting point P1 of the merging section 605 and the mode change determination position P3. Generate lane plans.

The operation planning unit 34 generates a predetermined operation based on the map information, the driving lane plan, the current position of the vehicle 10, the surrounding environment information, and the vehicle state information at the operation plan generation time set at a predetermined cycle. A driving plan for the vehicle 10 is generated for the time ahead (for example, 5 seconds). The driving plan is expressed as a set of target positions of the vehicle 10 and target vehicle speeds at the target positions at each time from the current time to a predetermined time ahead. The cycle of the driving plan generation time is preferably shorter than the cycle of the driving lane plan generation time. The operation planning unit 34 may determine the target vehicle speed based on the vehicle speed input by the driver or the legal speed of the lane in which the vehicle is traveling. The operation planning unit 34 notifies the vehicle control unit 35 of the operation plan each time it generates the operation plan. The operation planning unit 34 uses a prediction filter such as a Kalman filter to estimate a future trajectory based on the detected recent trajectory of the other vehicle, and the lane in which the detected other vehicle is traveling and the estimated Based on the relative distance calculated from the trajectory, a driving plan for the vehicle 10 is made so that the relative distance from the vehicle 10 to another vehicle is equal to or greater than a predetermined distance and the operation of the vehicle 10 satisfies predetermined constraints. to generate Predetermined restrictions include that the amount of change in velocity per unit time, the amount of change in acceleration per unit time, or the amount of change in yaw rate per unit time is equal to or less than an upper limit value. The driving plan unit 34 may generate a plurality of driving plans based on the driving lane plan. In this case, the driving plan unit 34 may select the driving plan that minimizes the sum of the absolute values of the acceleration of the vehicle 10 from among the plurality of driving plans. The driving plan unit 34 notifies the vehicle control unit 35 of the driving plan.

When the driving lane plan includes a lane change in which the vehicle 10 moves between lanes, the operation planning unit 34 selects one or more destinations after the lane change for moving from the current lane 601a to the adjacent lane 602a. merge target position is determined on the destination lane. The operation planning unit 34 determines the merging target position of the vehicle 10 and the target A driving plan is generated as a set of target vehicle speeds.

In the example shown in FIG. 6 , the operation planning unit 34 generates an operation plan to go to the confluence target position 603 . The operation planning unit 34 ensures that the vehicle 10 has completed the lane change from the merging lane to the main lane between the merging target position 603 set when planning the first lane change in the merging section 605 and the mode change determination position P3. is determined as a merging completion target position P2. For example, the operation planning unit 34 may determine the merging completion target position P2 at a position a predetermined distance before the mode change determination position P3. This predetermined distance may vary, for example, depending on vehicle speed. The operation planning unit 34 notifies the determination unit 37 of the merging completion target position P2. A driver merging preparation section D2 between the merging completion target position P2 and the mode change determination position P3 is a section in which the driver starts preparations for starting operation of the vehicle 10 manually.

Note that the driving plan unit 34 generates a driving plan for the vehicle 10 so that the vehicle 10 continues to travel in the current lane when the driving lane plan does not include a lane change.

When the automatic operation control mode is applied, the vehicle control unit 35 controls the vehicle 10 based on the position of the vehicle 10 at the position determination time, the vehicle speed and yaw rate, and the notified driving plan (including the lane change plan). Each part of the vehicle 10 is controlled so as to travel along the planned travel route. For example, the vehicle control unit 35 obtains the steering angle, acceleration, and angular acceleration of the vehicle 10 according to the notified driving plan and the current vehicle speed and yaw rate of the vehicle 10, and obtains the steering angle, acceleration, and angular acceleration. , steering amount, accelerator opening, shift position, or brake amount. The vehicle control unit 35 then outputs a steering signal corresponding to the set steering amount to the second motor 44 that controls the steered wheels 46 of the vehicle 10 . Further, the vehicle control unit 35 obtains the fuel injection amount according to the set accelerator opening, and outputs an accelerator opening signal corresponding to the fuel injection amount to the driving device 60 of the vehicle 10 . Vehicle control unit 35 also outputs a shift position signal indicating the shift position to drive device 60 . The drive device 60 selects a gear ratio corresponding to the shift position indicated by the control signal, and drives the drive wheels (not shown) at this gear ratio. Alternatively, the vehicle control unit 35 outputs a brake signal corresponding to the set brake amount to the brake 61 of the vehicle 10 . In addition, when the manual operation control mode is applied, the vehicle control unit 35 generates an accelerator opening signal, a shift position signal, and a brake signal according to the operation of the operation device 7 by the driver, and controls each part of the vehicle 10. .

When the driving plan includes a set of target trajectories and target vehicle speeds for changing lanes, the vehicle control unit 35 controls the traveling operation of the vehicle 10 so as to change lanes.

The operation control section 36 controls the operation device 7 . When the manual control operation mode is applied, the operation control unit 36 operably connects the operation device 7 and the action parts of the vehicle 10 operated by the operation device 7 with the operation device 7 . Further, when the automatic control operation mode is applied, the operation control unit 36 separates the operating device 7 from the operating parts of the vehicle 10 operated by the operating device 7 so that they are not operated by the operating device 7 . Specifically, when the manual control operation mode is applied, the operation control unit 36 controls the clutch 42 to connect the first shaft 40 and the second shaft 43, and the steering wheel 71 and the steered wheels 46 are connected to each other. The steering wheel 46 is operably connected by 71 . When the automatic control operation mode is applied, the clutch 42 is controlled to disconnect the first shaft 40 and the second shaft 43 and disconnect the steered wheels 46 so that they are not operated by the steering 71 . In addition, when the manual control operation mode is applied, the operation control unit 36 outputs an accelerator opening signal generated based on the driver's operation of the accelerator pedal 72 to the drive device 60, so that the accelerator pedal 72 and the drive device 60 is operably connected to the drive device 60 by the accelerator pedal 72 . When the automatic control operation mode is applied, the operation control unit 36 outputs an accelerator opening signal generated based on the operation plan to the driving device 60, and disconnects the driving device 60 so that it is not operated by the accelerator pedal 72. Further, when the manual control operation mode is applied, the operation control unit 36 outputs a brake signal generated based on the driver's operation of the brake pedal 73 to the brake 61, and the brake pedal 73 and the brake 61 are connected to the brake pedal. 73 operably connects brake 61 . When the automatic control operation mode is applied, the operation control unit 36 outputs a brake signal generated based on the operation plan to the brake 61 to disconnect the brake 61 from being operated by the brake pedal 73 . Furthermore, when the manual control operation mode is applied, the operation control unit 36 outputs to the drive device 60 a shift position signal generated based on the shift position corresponding to the shift operation of the driver, and the shift knob 74 and the drive device 60 are shifted. is operably connected to drive 60 by shift knob 74 . When the automatic control operation mode is applied, the operation control unit 36 outputs a shift position signal generated based on the operation plan to the driving device 60 and disconnects the driving device 60 from being operated by the shift knob 74.

When the automatic control driving mode is applied, the determination unit 37 schedules the operation of the vehicle 10 to be notified to the driver based on the vehicle status information including the driving status information of the automatically controlled vehicle 10 or the surrounding environment information. Determine whether or not Specifically, the determination unit 37 schedules the operation of the vehicle 10 to be notified to the driver at the determination time set at a predetermined cycle, based on the driving plan included in the driving status information of the vehicle status information. Determine whether or not The period of determination time is preferably shorter than the period of lane plan generation time. For example, when the driving plan includes a lane change, the determination unit 37 determines that an operation of the vehicle 10 to be notified to the driver is scheduled. When changing lanes, the distance between the vehicle 10 and other vehicles may fall below a predetermined threshold value. may be transferred. Then, the driver to whom control of the vehicle has been transferred is expected to operate the vehicle 10 so as to avoid a collision with another vehicle or the like, and cause the vehicle 10 to perform a planned operation. In this manner, when there is a possibility that control of the vehicle will be transferred to the driver based on the vehicle status information, the determination unit 37 determines that an operation of the vehicle 10 that should be notified to the driver is scheduled, and performs control. The operation that is presumed to be caused by the transferred driver operating the vehicle 10 to cause the vehicle 10 to operate, or the operation of the vehicle 10 when the automatic driving control mode is applied, is determined as the operation of the vehicle 10 that should be notified to the driver. do. The determination unit 37 notifies the notification unit 38 of the operation of the vehicle 10 to be notified to the driver. Next, an operation example of the determination unit 37 will be described below using first to fifth examples.

(first example)
In the example shown in FIG. 6, since the driving plan includes a lane change in the merging section 605, the determining unit 37 determines that the operation of the vehicle 10 to be notified to the driver is not performed before reaching the starting point P1 of the merging section. determined to be scheduled. The determination unit 37 determines a lane change as an action of the vehicle 10 to be notified to the driver. Also, while the vehicle 10 is traveling in the merging section 605, the determination unit 37 determines that the operation of the vehicle 10 to be notified to the driver is scheduled because the driving plan includes a lane change. The determination unit 37 determines a lane change as an action of the vehicle 10 to be notified to the driver. The determination unit 37 determines a braking operation as the operation of the vehicle 10 to be notified to the driver.

In the example shown in FIG. 6, the driving planning unit 34 controls the driving device 60 using the vehicle control unit 35 while the vehicle 10 is traveling in the automatic control section D1 in preparation for the planned transfer of control to the driver. vehicle speed may be reduced by increasing the transmission ratio of the The drive with engine 60 uses engine braking to reduce vehicle speed by increasing the transmission ratio, and the drive with motor 60 increases the transmission ratio and regenerates to reduce vehicle speed. The determination unit 37 determines the operation of reducing the vehicle speed as the operation of the vehicle 10 to be notified to the driver. In the example shown in FIG. 6, the operation planning unit 34 operates the brake 61 so that the vehicle 10 stops before reaching the end point P4 of the merging section while the vehicle 10 is traveling in the control change section D2. There is The determination unit 37 determines a braking operation as the operation of the vehicle 10 to be notified to the driver.

(Second example)
Based on the current position and speed of the vehicle 10 included in the driving status information of the vehicle status information and the positions and speeds of other vehicles included in the surrounding environment information, the determination unit 37 determines the location of the vehicle 10 to be notified to the driver. Determine whether an action is scheduled. This second example, and the third and fourth examples described below are specific examples of this determination process. Specifically, the determination unit 37 determines the distance between the vehicle 10 and another vehicle traveling in front of the vehicle 10 and the relative speed between the vehicle 10 and the other vehicle. estimates the collision time required to collide with another vehicle, and if the collision time is within the first collision time, it is determined that an action of the vehicle 10 to be notified to the driver is scheduled. In order to ensure the safety of the vehicle 10, the driver to whom control has been transferred or the operation planning unit 34 is considered to decelerate in order to avoid approaching another vehicle traveling in front of the vehicle 10. The determination unit 37 determines a braking operation as the operation of the vehicle 10 to be notified to the driver.

(Third example)
Further, the determining unit 37 determines whether the other vehicle is closer to the vehicle 10 based on the distance between the vehicle 10 and another vehicle following the vehicle 10 and the relative speed between the vehicle 10 and the other vehicle. A collision time is estimated, and if the collision time is within the first collision time, it is determined that an action of the vehicle 10 to be notified to the driver is scheduled. Note that the first collision time in the third example may be different from the time in the second example. In order to ensure the safety of the vehicle 10, the driver to whom control has been transferred or the operation planning unit 34 may decide to avoid other vehicles following the vehicle 10 from approaching the vehicle 10, so that the current Since it is conceivable that the vehicle will change lanes by moving from the driving lane to an adjacent lane, the determination unit 37 determines lane changes as the operation of the vehicle 10 that should be notified to the driver.

(Fourth example)
Further, if the distance between the vehicle 10 and another vehicle following the vehicle 10 continues for the first duration or longer, the determination unit 37 should notify the driver. It is determined that an operation of the vehicle 10 is scheduled. This is because there is a possibility that the driver of another vehicle following the vehicle 10 is driving in a so-called rushing manner. In order to ensure the safety of the vehicle 10, the driver to whom control has been transferred or the operation planning unit 34 accelerates in order to secure a distance from other vehicles that may be driving in a rush. Therefore, the determination unit 37 determines the acceleration operation as the operation of the vehicle 10 to be notified to the driver.

(Fifth example)
If the driver of the other vehicle is estimated to be dozing off while driving based on the trajectory of the other vehicle included in the surrounding environment information, the determining unit 37 determines that the operation of the vehicle 10 to be notified to the driver is scheduled. and judge. For example, if the frequency (the number of times per predetermined time) of generating a trajectory in which another vehicle travels either left or right and then returns to the center is equal to or greater than the first frequency, the determination unit 37 , the driver of the vehicle is assumed to be dozing off while driving. In order to ensure the safety of the vehicle 10, the driver to whom control has been transferred or the driving planning unit 34 slows down to secure a distance from other vehicles that may be dozing off while driving. Therefore, the determination unit 37 determines the braking operation as the operation of the vehicle 10 to be notified to the driver.

(6th example)
If the determining unit 37 determines that a downhill with an average gradient equal to or greater than a predetermined threshold continues for a first distance or longer based on the three-dimensional information of the road surface included in the driving condition information of the vehicle condition information, It is determined that an operation of the vehicle 10 to be notified to the driver is scheduled. In order to ensure the safety of the vehicle 10, either the driver to which control has been transferred or the driving planner 34 increases the gear ratio and uses engine braking or the like to slow down the long downhill. As a result, the vehicle speed is considered to be decelerated, so the determination unit 37 determines the operation of increasing the gear ratio as the operation of the vehicle 10 to be notified to the driver.

If the determining unit 37 determines that the operation of the automatically driven vehicle that should be notified to the driver is scheduled, based on the vehicle status information, the determination unit 37 determines the urgency level for notifying the driver of the operation scheduled for the vehicle 10. A determination unit 371 may be provided. Specifically, the urgency determination unit 371 sets the urgency to Determine high, otherwise determine low urgency. Next, an operation example of the urgency determination unit 371 will be described below for the first to fifth examples described above.

(first example)
In the first example shown in FIG. 6 , the urgency determining unit 371 determines that the vehicle 10 may fail to change lanes in the section before reaching the starting point P1 of the merging section and in the automatically controlled section D1. Since the distance from the current position of 10 to the end point P4 of the merging section is longer than a predetermined threshold value, it is determined that the urgency is low. On the other hand, when the vehicle 10 reaches the merging completion target position P2, the urgency determination unit 371 determines that the urgency is high because the distance from the current position of the vehicle 10 to the end point P4 of the merging section is equal to or less than the predetermined threshold value. and decide. That is, for example, in the example shown in FIG. 6, the urgency determining unit 371 determines that the urgency is high when the vehicle 10 is in the control change section D3.

(Second example)
In the second example, the urgency determining unit 371 determines that the urgency is high when the estimated collision time is shorter than the second collision time, which is shorter than the first collision time. determine low.

(Third example)
In the third example, the urgency determination unit 371 determines that the urgency is high when the estimated collision time is shorter than the second collision time, which is shorter than the first collision time. determine low.

(Fourth example)
In the fourth example, the urgency determining unit 371 determines that the urgency is high when the state of being at or below the predetermined distance is longer than the second duration, which is longer than the first duration. determined to be low.

(Fifth example)
In the fifth example, the urgency determining unit 371 determines that the urgency is high when the meandering frequency of other vehicles is equal to or higher than the second frequency, which is higher than the first frequency, and otherwise the urgency is low. and decide.

(6th example)
In the sixth example, the urgency determining unit 371 determines that the urgency is high when it is determined that a downhill with an average gradient equal to or greater than a predetermined threshold continues for a second distance longer than the first distance. and otherwise determine that the urgency is low.

When the determination unit 37 determines that the operation of the vehicle 10 to be notified to the driver is scheduled, the notification unit 38 affects the behavior of the vehicle 10 by the driver's operation based on the scheduled operation of the vehicle 10 . By moving the operation device 7 in a state separated from the behavior of the vehicle 10 using the operation control unit 36 that controls the operation device 7, the driver is notified of the expected operation of the vehicle 10. FIG. For example, the notification unit 38 may be moved so that the movement (movement amount or speed) of the operation device 7 becomes more intense over time. Further, the notification unit 38 may determine the motion of the operation device 7 based on the expected motion of the vehicle 10 and the degree of urgency. Next, operation examples of the notification unit 38 will be described below for the above-described first to fifth examples.

(first example)
In the first example shown in FIG. 6 , the notification unit 38 uses the operation control unit 36 to control the steering wheel 71 for changing the traveling direction of the vehicle 10 based on the scheduled lane change operation of the vehicle 10 . The driver is notified of a lane change, which is an example of an expected action of the vehicle 10, by moving in a state separated from the behavior. When the automatic operation control mode is applied, the operation control unit 36 controls the clutch 42 to disconnect the first shaft 40 and the second shaft 43 . The notification unit 38 uses the operation control unit 36 to rotate the steering wheel 71 to the right and left about the neutral position so as to vibrate at a predetermined angle and at a predetermined speed. is notified to the driver. Further, the operation control unit 36 may notify the driver of the scheduled lane change of the vehicle 10 by rotating the steering wheel 71 from the neutral position toward the destination lane in a vibrating manner. When the degree of urgency is high, the operation control unit 36 may rotate the steering wheel 71 at a larger angle or at a higher speed than when the degree of urgency is low. In the merging section of the lanes, it is highly likely that the driver will pay attention to the steering wheel 71 that changes the traveling direction of the vehicle by the driver's operation in manual control even when the automatic control driving mode is applied. Therefore, the notification unit 38 can notify the driver that the lane change is scheduled by moving the steering wheel 71 .

In addition, the notification unit 38 displays icons as shown in FIGS. 7A to 7C using the UI 6 to notify the driver that the direction of travel is scheduled to be changed. good too. As a result, the driver can be alerted to the steering wheel 71 that is performing the notification operation. The notification unit 38 may display the arrow in the drawing so that the color changes to indicate that the control signal is transmitted from the ECU 9 to the steered wheels 46 . Incidentally, when the automatic operation control mode is applied, the notification unit 38 displays icons such as those shown in FIGS. 8 (A) to 8 (C) using the UI 6, and the automatic operation control mode is applied. The driver may be notified of this.

In addition, the notification unit 38 moves the shift knob 74 or the brake pedal 73 together with the steering wheel 71 using the operation control unit 36 in a state separated from the behavior of the vehicle 10, so that the driver is notified of the expected operation of the vehicle 10. may be notified to The notification unit 38 may notify the driver of a decrease in vehicle speed, which is an example of an expected operation of the vehicle 10 , by operating the shift knob 74 using the operation control unit 36 . For example, the notification unit 38 may match the timing at which the shift knob 74 is first moved with the timing at which the operation planning unit 34 uses the vehicle control unit 35 to increase the gear ratio of the driving device 60 . In addition, the notification unit 38 may notify the driver of a braking operation, which is an example of an operation expected in the vehicle 10 , by moving the brake pedal 73 using the operation control unit 36 .

(Second example)
In the second example, the notification unit 38 uses the operation control unit 36 to move the brake pedal 73 based on the expected braking operation of the vehicle 10, thereby causing the braking operation, which is an example of the expected operation of the vehicle 10. is notified to the driver. When the degree of urgency is high, the operation control unit 36 may move the brake pedal 73 at a faster speed or at shorter intervals than when the degree of urgency is low. When the vehicle 10 is approaching another vehicle traveling in front, the driver should pay attention to the brake pedal 73, which is operated by the driver in manual control to decelerate the vehicle, even when the automatic control driving mode is applied. likely to be Therefore, the notification unit 38 can notify the driver that the braking operation is scheduled by moving the brake pedal 73 .

(Third example)
In the third example, the notification unit 38 uses the operation control unit 36 to separate the steering 71 that changes the traveling direction of the vehicle 10 from the behavior of the vehicle 10 based on the scheduled lane change operation of the vehicle 10 . The driver is notified of a lane change, which is an example of an operation expected for the vehicle 10, by moving the vehicle 10 in a state where the vehicle 10 is held. When the degree of urgency is high, the operation control unit 36 may rotate the steering wheel 71 at a larger angle or at a higher speed than when the degree of urgency is low. When another vehicle following the vehicle 10 is approaching, the driver automatically controls the steering wheel 71 for changing the traveling direction of the vehicle by operating the driver in manual control in order to move away from the other vehicle. It is highly likely that they are paying attention even when driving mode is applied. Thereby, the notification unit 38 can notify the driver that the lane change is scheduled by moving the steering wheel 71 .

(Fourth example)
In the fourth example, the notification unit 38 uses the operation control unit 36 to move the accelerator pedal 72 based on the expected acceleration operation of the vehicle 10, thereby causing the acceleration operation, which is an example of the expected operation of the vehicle 10. is notified to the driver. When the degree of urgency is high, the operation control unit 36 may move the accelerator pedal 72 at a faster speed or at shorter intervals than when the degree of urgency is low. When there is another vehicle around the vehicle 10 that may be driving in a rush, the accelerator pedal 72 is operated by the driver under manual control to accelerate the vehicle in order to move away from the other vehicle. , the driver is likely to pay attention even when the automatic control driving mode is applied. Thereby, the notification unit 38 can notify the driver that the acceleration operation is scheduled by moving the accelerator pedal 72 .

(Fifth example)
In the fifth example, the notification unit 38 moves the brake pedal 73 using the operation control unit 36 based on the expected braking operation of the vehicle 10, thereby causing the braking operation, which is an example of the expected operation of the vehicle 10. is notified to the driver. When the degree of urgency is high, the operation control unit 36 may move the brake pedal 73 at a faster speed or at shorter intervals than when the degree of urgency is low. When another vehicle that may be drowsy driving is around the vehicle 10, the brake pedal 73 that decelerates the vehicle by being operated by the driver under manual control in order to move away from the other vehicle. , the driver is likely to pay attention even when the automatic control driving mode is applied. Thereby, the notification unit 38 can notify the driver that the deceleration operation is scheduled by moving the brake pedal 73 .

(6th example)
In the sixth example, the notification unit 38 is an example of an operation expected for the vehicle 10 by moving the shift knob 74 using the operation control unit 36 based on an operation expected to increase the gear ratio of the vehicle 10. The driver is notified of the operation to increase the gear ratio. When the degree of urgency is high, the operation control section 36 may move the shift knob 74 at a faster speed or at shorter intervals than when the degree of urgency is low. In long downhill conditions, the driver may turn the shift knob 74 under manual control to decelerate the vehicle by operating the shift knob 74 to slow the vehicle using engine braking or the like. It is likely that they are paying attention even when applying. As a result, the notification unit 38 can notify the driver that the operation of decelerating by increasing the gear ratio is scheduled by moving the shift knob 74 .

If the lane change has not been completed when the vehicle 10 reaches the merging completion target position P2, the notification control unit 39 sends a control change notification requesting that the control of the vehicle 10 is changed from automatic control to manual control. The UI 6 is used to notify the driver. The notification control unit 39 determines when the vehicle 10 reaches the merging completion target position P2 based on the current position of the vehicle 10, the driving plan, and the map information, and determines when the vehicle 10 is currently traveling. Based on the current lane and the driving lane plan, it is determined whether or not the lane change has been completed.

For example, if the lane change has not been completed when the vehicle 10 reaches the merging completion target position P2, the notification control unit 39 uses the UI 6 to display a notification that the lane cannot be changed by automatic control and a control change notification. do.

The UI 6 inputs an operation signal of the driver's response to the control change notification, and outputs it to the ECU 9 via the in-vehicle network 11 . When the driver approves the control change notification, the operation planning unit 34 switches the operation of the vehicle 10 from the applied automatic control operation mode to the manual control operation mode. The notification control unit 39 displays icons as shown in FIGS. 9A to 9C using the UI 6 to notify the driver of switching from the automatic control operation mode to the manual control operation mode. may The driver who has accepted the control change notification operates the vehicle 10 to change lanes after switching to the manual control driving mode. Preferably, in the manual control driving mode, at least the steering is manually controlled. Alternatively, in the manual control driving mode, steering, acceleration and braking may be manually controlled. The notification control unit 39 displays icons as shown in FIGS. 10A to 10C using the UI 6 to notify the driver that the manual operation control mode is applied. . The notification unit 38 may display the arrow in the drawing so that the color changes to indicate that the control signal is transmitted from the steering wheel 71 to the steered wheels 46 . On the other hand, if the driver does not approve the control change notification, the driving planning unit 34 continues driving the vehicle 10 in the automatic control driving mode. When the vehicle 10 reaches the mode change determination position P3, the driving planning unit 34 determines that the vehicle 10 is at the end point of the merging section 605 if the lane change has not been completed and the driver has not approved the control change notification. Vehicle 10 may be controlled to stop before reaching P4.

FIG. 11 is an operation flowchart of vehicle control processing including determination processing and notification processing executed by the processor 23 . In the operation flowchart shown below, the processing of steps S106 and S107 corresponds to the control change notification processing when it is determined that the automatic control cannot change the lane.

First, the navigation device 8 generates a planned travel route from the current position of the vehicle 10 to the destination based on the navigation map information, the destination of the vehicle 10, and the current position of the vehicle 10 (step S101). .

Next, the position estimator 31 of the processor 23 obtains the estimated position and the estimated azimuth angle of the vehicle 10 at each position determination time (step S102).

Next, the object detection unit 32 of the processor 23 detects other objects around the vehicle 10 based on the image generated by the camera 2 and the reflected wave information generated by the LiDAR sensors 3a to 3d (step S103 ).

Next, the driving lane planning unit 33 of the processor 23 determines whether or not it is necessary to change the lane based on the map information and the current position of the vehicle 10 in the driving section of the planned driving route. selects a lane in the road on which the travels, and generates a travel lane planning part (step S104).

Next, the driving plan unit 34 of the processor 23 generates a driving plan for the vehicle 10 up to a predetermined time ahead at a driving plan generation time set at a predetermined cycle (step S105).

Next, the determination unit 37 of the processor 23 determines whether an operation of the vehicle 10 to be notified to the driver is scheduled based on the vehicle status information including the driving status information of the automatically controlled vehicle 10 or the surrounding environment information. (step S106).

Next, when the notifying unit 38 of the processor 23 determines that the behavior of the vehicle 10 to be notified to the driver is scheduled, the behavior of the vehicle 10 is affected by the driver's operation based on the scheduled behavior of the vehicle 10. is operated in a state separated from the behavior of the vehicle 10 by using the operation control unit 36 that controls the operation unit, thereby informing the driver of the expected operation of the vehicle 10 (step S107).

As described above, the vehicle control device determines whether the vehicle operation to be notified to the driver is planned based on the vehicle status information including the driving status information of the automatically controlled vehicle or the surrounding environment information. determine whether or not When the vehicle control device determines that a vehicle action to be notified to the driver is scheduled, the vehicle control device selects an operation unit that affects the behavior of the vehicle through operation by the driver, based on the scheduled operation of the vehicle. By moving the vehicle in a state separated from the behavior of the vehicle using the operation control unit that controls the vehicle, the driver is notified of the expected operation of the vehicle. As a result, the vehicle control device can notify the driver of the expected operation of the vehicle so that the driver can easily recognize the notification from the vehicle even when the vehicle is automatically controlled. By notifying the driver of the expected operation of the vehicle, the driver can understand, for example, that the control of the vehicle may be transferred to the driver or the operation to be performed after the transfer.

Next, modified examples 1 to 3 of the above-described embodiment will be described below. In Modified Example 1, the notification unit 38 stops the operation of the operating device 7 being controlled when it is determined that the driver has recognized the operation of the operating device 7 by the driver. For example, the notification unit 38 may detect that the driver has recognized the operation of the operating device 7 based on an image of the driver's face taken using a camera (not shown) arranged in the vehicle compartment. . When the notification unit 38 detects that the driver's line of sight is facing the operation device 7 operating for notification for a predetermined time or longer, the notification unit 38 determines that the driver has recognized the operation of the operation device 7. You can judge. Further, when the steering wheel 71 has a gripping sensor that detects the driver's gripping, the notification unit 38 uses this gripping sensor to detect that the driver is gripping the steering wheel 71 for a predetermined time or longer. It may be determined that the operation of the device 7 has been recognized. Furthermore, if a recognition button (not shown) that is operated when the driver recognizes the operation of the operation device 7 is arranged in the vehicle interior, the notification unit 38 detects that the driver has operated the recognition button. , it may be determined that the driver has recognized the operation of the operating device 7 .

In Modification 2, when switching from the automatic control operation mode to the manual control operation mode, the vehicle 10 is controlled by combining the manual control operation and the automatic control operation between the automatic control operation mode and the manual control operation mode. The switching mode to be performed is set. When this switching mode is applied, when the vehicle 10 is controlled in the automatic control operation mode, the vehicle control unit 35 determines a target steering amount, a target accelerator opening, Alternatively, each part of the vehicle 10 is controlled by setting a switching target steering amount, a switching target accelerator opening, or a switching target brake amount that is smaller than the target brake amount. As a result, the driver is encouraged to compensate for the steering amount of the steering wheel 71, compensate for the accelerator opening of the accelerator pedal 72, or compensate for the braking amount of the brake pedal 73 while confirming the surrounding conditions of the vehicle 10. be done. The vehicle control unit 35 increases the ratio of the driver's involvement in control over time, and finally switches to the manual control driving mode. For example, the vehicle control unit 35 sets a switching target steering amount to a value obtained by subtracting the product A×α of a predetermined angle A and a coefficient α from the target steering amount determined based on the driving plan. The vehicle control unit 35 increases the value of the coefficient α as time elapses, and finally brings the target steering amount and the product A×α into agreement. Control of the vehicle 10 is then switched to a fully manual control operating mode. As a result, the driver can gradually get used to the operation of the vehicle 10 while being assisted by the automatic control, so that the vehicle 10 can be safely switched from the automatic control operation mode to the manual control operation mode.

In Modified Example 3, when a collision between the vehicle 10 and an object such as another vehicle is predicted based on the vehicle status information, the notification unit 38 causes the determination unit 37 to notify the driver of the expected operation of the vehicle 10. Even if it is determined to notify, it does not notify. When the vehicle 10 is predicted to collide with an object such as another vehicle, for example, the notification unit 38 detects, based on the vehicle status information, that another vehicle is approaching the lane to which the vehicle is moving due to the lane change. If so, the operation of moving the steering wheel 71 is not performed. This prevents the vehicle 10 from inadvertently approaching another vehicle by switching to the manual driving mode and operating the steering wheel 71 when the driver recognizes the lane change. Further, based on the vehicle condition information, the reporting unit 38 determines that even if the collision time required for the vehicle 10 to collide with another vehicle traveling ahead is within the first collision time, the following vehicle will If the collision time with the vehicle 10 is within the first collision time, the accelerator pedal 72 is not moved. Similarly, based on the vehicle condition information, the notification unit 38 may notify the vehicle 10 that the vehicle 10 is following another vehicle even if the collision time required for the vehicle 10 to collide with another vehicle traveling ahead is within the first collision time. is within the first collision time, the brake pedal 73 is not moved. As a result, the driver, recognizing that another vehicle ahead or behind is approaching, switches to the manual operation mode and operates the accelerator pedal 72 or the brake pedal 73, thereby causing the vehicle 10 to approach another vehicle carelessly. You can prevent things from happening.

In the present invention, the vehicle control device of the embodiment described above can be modified as appropriate without departing from the gist of the present invention. Moreover, the technical scope of the present invention is not limited to those embodiments, but extends to the invention described in the claims and equivalents thereof.

For example, in the above-described embodiment, the notification unit uses the operation control unit to move the operation unit in a state separated from the behavior of the autonomous vehicle, thereby informing the driver of the expected operation of the autonomous vehicle. Doing is not limited to the above description. For example, the notification unit may set the lighting time of the panel display in the vehicle indicating the blinker display for notifying the surroundings of the lane change to be longer than the normal lighting time, and notify the driver of the operation scheduled for the vehicle. good.

1 Vehicle Control System 2 Cameras 3a to 3d LiDAR Sensor 4 Positioning Information Receiver 4a Positioning Information Receiver 4b Processor 5 Map Information Storage Device 6 User Interface (UI)
7 operating device 8 navigation device 9 electronic control unit (ECU)
10 vehicle 21 communication interface 22 memory 23 processor 31 position estimation unit 32 object detection unit 33 traveling lane planning unit 34 operation planning unit 35 vehicle control unit 36 operation control unit 37 judgment unit 371 urgency determination unit 38 notification unit 39 notification control unit 40 First shaft 41 First motor 42 Clutch 43 Second shaft 44 Second motor 45 Gear portion 46 Steering wheel 50 First gear 51 Third motor 52 Second gear 53 Fourth motor 54 Shift device 60 Driving device 61 Brake

Claims (1)

Based on the vehicle condition information including the driving condition information of the automatically controlled vehicle or the surrounding environment information , it is determined whether or not there is a possibility of transferring the control of the vehicle to the driver, and the control of the vehicle is transferred to the driver. If there is a possibility of transferring control to the driver, it is determined that the operation of the vehicle that should be notified to the driver is scheduled, and if there is no possibility of transferring the control of the vehicle to the driver, the operation of the vehicle that should be notified to the driver a determination unit that determines that is not scheduled ;
When it is determined that an operation of the vehicle that should be notified to the driver is scheduled, an operation unit that affects the behavior of the vehicle when operated by the driver is controlled based on the operation that is scheduled for the vehicle. a notification unit for notifying a driver of an expected operation of the vehicle by operating the operation control unit to operate the vehicle in a state separated from the behavior of the vehicle, wherein moving the operation unit moves the steering wheel a notification unit including at least one of moving an accelerator pedal, moving a brake pedal, and moving a shift knob;
A vehicle control device having

Images