JP2022096232A - Vehicle control device, vehicle control method and program - Google Patents

Abstract

To properly determine whether to let a vehicle overtake a preceding vehicle while suppressing a processing load from increasing.SOLUTION: A vehicle control device comprises: a recognition part which recognizes a circumferential situation of a vehicle; and a drive control part which controls acceleration and deceleration, and steering of the vehicle not through operation by a passenger of the vehicle. The drive control part comprises an overtake control part which determines whether or not to overtake a preceding vehicle traveling ahead of the vehicle in the same lane with the vehicle by comparing a value obtained by integrating a plurality of evaluation values that a plurality of evaluation functions output with a threshold value. Each of the plurality of evaluation functions outputs the evaluation values once a plurality of index values indicative of the circumferential situation of the vehicle recognized by the recognition part are input, and output tendencies are adjusted in a previous environment through reinforced learning.SELECTED DRAWING: Figure 5

Description

The present invention relates to a vehicle control device, a vehicle control method, and a program.

Research and practical application of technologies for automatically driving vehicles are underway. Patent Document 1 describes determining the position of the lane change destination when the vehicle is automatically changed to the lane. It also describes how to select a lane change destination that is close to the driver's feeling by learning the driver's driving behavior.

Japanese Unexamined Patent Publication No. 2019-217828

The invention described in Patent Document 1 does not mention a suitable timing for overtaking a vehicle in front while traveling while maintaining a target speed. If the judgment as to whether or not to overtake the preceding vehicle is performed by a simple If-Then rule, the relationship with the surrounding vehicles is not always maintained favorably, and as a typical example, When changing lanes once and returning to the original lane change, the distance from the following vehicle may become too short at the lane change destination.

On the other hand, research is underway on making more advanced decisions using reinforcement learning. Reinforcement learning has a high degree of freedom in model construction and enables complicated judgments, but it has a high processing load to be mounted on an in-vehicle computer, and problems such as processing delay and increased power consumption may occur.

The present invention has been made in consideration of such circumstances, and is a vehicle control device capable of appropriately determining whether or not to allow a vehicle to overtake a vehicle in front while suppressing an increase in processing load. One of the purposes is to provide a vehicle control method and a program.

The vehicle control device, the vehicle control method, and the program according to the present invention have adopted the following configurations.
(1): The vehicle control device according to one aspect of the present invention includes a recognition unit that recognizes the surrounding conditions of the vehicle, and an operation control unit that controls acceleration / deceleration and steering of the vehicle without depending on the operation of the occupants of the vehicle. , And the operation control unit compares a value obtained by integrating a plurality of evaluation values output by a plurality of evaluation functions with a threshold value to travel in front of the vehicle in the same lane as the vehicle. It is provided with an overtaking control unit that determines whether or not to overtake the vehicle, and each of the plurality of evaluation functions is input with one of a plurality of index values indicating the surrounding conditions of the vehicle recognized by the recognition unit. The evaluation value is output, and the output tendency is adjusted by reinforcement learning in the pre-environment.

(2): In the embodiment of (1) above, the overtaking control unit determines to overtake the preceding vehicle when the product of the plurality of evaluation values is equal to or greater than the threshold value, and the vehicle is in front of the vehicle. It overtakes running vehicles.

(3): In the embodiment (1) or (2), the plurality of index values include the difference between the target speed of the vehicle and the speed of the vehicle in front.

(4): In any of the above embodiments (1) to (3), the plurality of index values include the distance between the vehicle and the vehicle in front.

(5): In any of the above embodiments (1) to (4), the plurality of index values are rearward of the vehicle and the adjacent lane to which the vehicle changes lanes when overtaking. It includes the distance to the adjacent rear vehicle in.

(6): In any one of the above (1) to (5), each of the plurality of evaluation functions has its output tendency adjusted by the Actor-Critic method.

(7): In the vehicle control method according to another aspect of the present invention, the vehicle control device recognizes the surrounding situation of the vehicle and controls the acceleration / deceleration and steering of the vehicle without depending on the operation of the occupants of the vehicle. The control is to overtake the preceding vehicle traveling in front of the vehicle on the same lane as the vehicle by comparing the value obtained by integrating the plurality of evaluation values output by the plurality of evaluation functions with the threshold value. Each of the plurality of evaluation functions, including determining whether or not, outputs an evaluation value when one of a plurality of index values indicating the recognized surrounding conditions of the vehicle is input. The output tendency is adjusted by reinforcement learning in the pre-environment.

(8): The program according to another aspect of the present invention causes a computer mounted on the vehicle to recognize the surrounding conditions of the vehicle and control the acceleration / deceleration and steering of the vehicle without depending on the operation of the occupants of the vehicle. The control is to overtake the preceding vehicle traveling in front of the vehicle on the same lane as the vehicle by comparing the value obtained by integrating the plurality of evaluation values output by the plurality of evaluation functions with the threshold value. Each of the plurality of evaluation functions, including determining whether or not, outputs an evaluation value when one of a plurality of index values indicating the recognized surrounding conditions of the vehicle is input. , The output tendency is adjusted by reinforcement learning in the pre-environment.

According to the above aspects (1) to (8), it is possible to suitably control the vehicle to overtake the vehicle in front while suppressing the increase in the processing load.

It is a block diagram of the vehicle system 1 using the vehicle control device which concerns on embodiment. It is a functional block diagram of the 1st control unit 120 and the 2nd control unit 160. It is a figure for demonstrating the scene where overtaking control is performed, and various definitions. It is a figure for demonstrating the operation which the own vehicle M overtakes the preceding vehicle mA. It is a figure which shows the outline of the overtaking determination processing by the overtaking control unit 142. It is a figure for demonstrating the learning process of the evaluation function in a pre-environment. It is a flowchart which shows an example of the flow of the process executed by the overtaking control unit 142.

Hereinafter, embodiments of the vehicle control device, vehicle control method, and program of the present invention will be described with reference to the drawings.

[overall structure]
FIG. 1 is a configuration diagram of a vehicle system 1 using the vehicle control device according to the embodiment. The vehicle on which the vehicle system 1 is mounted is, for example, a vehicle such as a two-wheeled vehicle, a three-wheeled vehicle, or a four-wheeled vehicle, and the drive source thereof is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates by using the power generated by the generator connected to the internal combustion engine or the discharge power of the secondary battery or the fuel cell.

The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, and a vehicle sensor 40. , A navigation device 50, an MPU (Map Positioning Unit) 60, a driving controller 80, an automatic driving control device 100, a traveling driving force output device 200, a braking device 210, and a steering device 220. These devices and devices are connected to each other by a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like. The configuration shown in FIG. 1 is merely an example, and a part of the configuration may be omitted or another configuration may be added.

The camera 10 is, for example, a digital camera using a solid-state image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to an arbitrary position on the vehicle on which the vehicle system 1 is mounted (hereinafter referred to as the own vehicle M). When photographing the front, the camera 10 is attached to the upper part of the front windshield, the back surface of the rear-view mirror, and the like. The camera 10 periodically and repeatedly images the periphery of the own vehicle M, for example. The camera 10 may be a stereo camera.

The radar device 12 radiates radio waves such as millimeter waves around the own vehicle M, and also detects radio waves (reflected waves) reflected by the object to detect at least the position (distance and direction) of the object. The radar device 12 is attached to an arbitrary position of the own vehicle M. The radar device 12 may detect the position and velocity of the object by the FM-CW (Frequency Modulated Continuous Wave) method.

The LIDAR 14 irradiates the periphery of the own vehicle M with light (or an electromagnetic wave having a wavelength close to that of light) and measures the scattered light. The LIDAR 14 detects the distance to the target based on the time from light emission to light reception. The emitted light is, for example, a pulsed laser beam. The LIDAR 14 is attached to any position on the own vehicle M.

The object recognition device 16 performs sensor fusion processing on the detection results of a part or all of the camera 10, the radar device 12, and the LIDAR 14, and recognizes the position, type, speed, and the like of the object. The object recognition device 16 outputs the recognition result to the automatic operation control device 100. The object recognition device 16 may output the detection results of the camera 10, the radar device 12, and the LIDAR 14 to the automatic operation control device 100 as they are. The object recognition device 16 may be omitted from the vehicle system 1.

The communication device 20 communicates with another vehicle existing in the vicinity of the own vehicle M by using, for example, a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), or wirelessly. Communicates with various server devices via the base station.

The HMI 30 presents various information to the occupants of the own vehicle M and accepts input operations by the occupants. The HMI 30 includes various display devices, speakers, buzzers, touch panels, switches, keys and the like.

The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the own vehicle M, an acceleration sensor that detects the acceleration, a yaw rate sensor that detects the angular velocity around the vertical axis, an orientation sensor that detects the direction of the own vehicle M, and the like.

The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a routing unit 53. The navigation device 50 holds the first map information 54 in a storage device such as an HDD (Hard Disk Drive) or a flash memory. The GNSS receiver 51 identifies the position of the own vehicle M based on the signal received from the GNSS satellite. The position of the own vehicle M may be specified or complemented by an INS (Inertial Navigation System) using the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, and the like. The navigation HMI 52 may be partially or wholly shared with the above-mentioned HMI 30. The route determination unit 53, for example, has a route from the position of the own vehicle M (or an arbitrary position input) specified by the GNSS receiver 51 to the destination input by the occupant using the navigation HMI 52 (hereinafter,). The route on the map) is determined with reference to the first map information 54. The first map information 54 is, for example, information in which a road shape is expressed by a link indicating a road and a node connected by the link. The first map information 54 may include road curvature, POI (Point Of Interest) information, and the like. The route on the map is output to MPU60. The navigation device 50 may provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may be realized by, for example, the function of a terminal device such as a smartphone or a tablet terminal owned by an occupant. The navigation device 50 may transmit the current position and the destination to the navigation server via the communication device 20 and acquire a route equivalent to the route on the map from the navigation server.

The MPU 60 includes, for example, a recommended lane determination unit 61, and holds the second map information 62 in a storage device such as an HDD or a flash memory. The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into a plurality of blocks (for example, divides the route into 100 [m] units with respect to the vehicle traveling direction), and refers to the second map information 62. Determine the recommended lane for each block. The recommended lane determination unit 61 determines which lane to drive from the left. When a branch point exists on the route on the map, the recommended lane determination unit 61 determines the recommended lane so that the own vehicle M can travel on a reasonable route to proceed to the branch destination.

The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, information on the center of the lane, information on the boundary of the lane, and the like. Further, the second map information 62 may include road information, traffic regulation information, address information (address / zip code), facility information, telephone number information, and the like. The second map information 62 may be updated at any time by the communication device 20 communicating with another device.

The driving controller 80 includes, for example, an accelerator pedal, a brake pedal, a shift lever, a steering wheel, a deformed steering wheel, a joystick, and other controls. A sensor for detecting the amount of operation or the presence or absence of operation is attached to the operation controller 80, and the detection result is the automatic operation control device 100, or the traveling driving force output device 200, the brake device 210, and the steering device. It is output to a part or all of 220.

The automatic operation control device 100 includes, for example, a first control unit 120 and a second control unit 160. The first control unit 120 and the second control unit 160 are each realized by executing a program (software) by a hardware processor such as a CPU (Central Processing Unit). In addition, some or all of these components are hardware (circuits) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit). It may be realized by the part; including circuitry), or it may be realized by the cooperation of software and hardware. The program may be stored in advance in a storage device (a storage device including a non-transient storage medium) such as an HDD or a flash memory of the automatic operation control device 100, or is detachable such as a DVD or a CD-ROM. It is stored in a storage medium, and the storage medium (non-transient storage medium) may be installed in the HDD or flash memory of the automatic operation control device 100 by being attached to the device in the drive device. The automatic driving control device 100 is an example of a "vehicle control device", and a combination of an action plan generation unit 140 and a second control unit 160 is an example of a "driving control unit".

FIG. 2 is a functional configuration diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130 and an action plan generation unit 140. The recognition unit 130 includes an index value derivation unit 132. The action plan generation unit 140 includes an overtaking control unit 142. The functions of the index value derivation unit 132 and the overtaking control unit 142 will be described later. The first control unit 120, for example, realizes a function by AI (Artificial Intelligence) and a function by a model given in advance in parallel. For example, the function of "recognizing an intersection" is executed in parallel with the recognition of an intersection by deep learning or the like and the recognition based on predetermined conditions (there are signals that can be matched with patterns, road markings, etc.). It may be realized by scoring and comprehensively evaluating. This ensures the reliability of autonomous driving.

The recognition unit 130 recognizes the position, speed, acceleration, and other states of objects around the own vehicle M based on the information input from the camera 10, the radar device 12, and the LIDAR 14 via the object recognition device 16. do. The position of the object is recognized as, for example, a position on absolute coordinates with the representative point (center of gravity, center of drive axis, etc.) of the own vehicle M as the origin, and is used for control. The position of the object may be represented by a representative point such as the center of gravity or a corner of the object, or may be represented by a represented area. The "state" of an object may include the object's acceleration, jerk, or "behavioral state" (eg, whether it is changing lanes or is about to change lanes).

Further, the recognition unit 130 recognizes, for example, the lane (traveling lane) in which the own vehicle M is traveling. For example, the recognition unit 130 has a road lane marking pattern (for example, an arrangement of a solid line and a broken line) obtained from the second map information 62 and a road lane marking around the own vehicle M recognized from the image captured by the camera 10. By comparing with the pattern of, the driving lane is recognized. The recognition unit 130 may recognize the traveling lane by recognizing not only the road marking line but also the running road boundary (road boundary) including the road marking line, the shoulder, the median strip, the guardrail, and the like. .. In this recognition, the position of the own vehicle M acquired from the navigation device 50 and the processing result by the INS may be added. The recognition unit 130 also recognizes stop lines, obstacles, red lights, tollhouses, and other road events.

When recognizing a traveling lane, the recognition unit 130 recognizes the position and posture of the own vehicle M with respect to the traveling lane. The recognition unit 130 determines, for example, the deviation of the reference point of the own vehicle M from the center of the lane and the angle formed with respect to the line connecting the center of the lane in the traveling direction of the own vehicle M with respect to the relative position of the own vehicle M with respect to the traveling lane. And may be recognized as a posture. Instead, the recognition unit 130 recognizes the position of the reference point of the own vehicle M with respect to any side end portion (road division line or road boundary) of the traveling lane as the relative position of the own vehicle M with respect to the traveling lane. You may.

In principle, the action plan generation unit 140 travels in the recommended lane determined by the recommended lane determination unit 61, and the own vehicle M automatically (driver) so as to be able to respond to the surrounding conditions of the own vehicle M. Generate a target track to travel in the future (regardless of the operation of). The target trajectory contains, for example, a velocity element. For example, the target track is expressed as an arrangement of points (track points) to be reached by the own vehicle M in order. The track point is a point to be reached by the own vehicle M for each predetermined mileage (for example, about several [m]) along the road, and separately, for a predetermined sampling time (for example, about 0 comma number [sec]). ) Target velocity and target acceleration are generated as part of the target trajectory. Further, the track point may be a position to be reached by the own vehicle M at the sampling time for each predetermined sampling time. In this case, the information of the target velocity and the target acceleration is expressed by the interval of the orbital points.

The action plan generation unit 140 may set an event for automatic driving when generating a target trajectory. Autonomous driving events include constant speed driving events, low speed following driving events, lane change events, branching events, merging events, takeover events, and the like. The action plan generation unit 140 generates a target trajectory according to the activated event.

The second control unit 160 sets the traveling driving force output device 200, the brake device 210, and the steering device 220 so that the own vehicle M passes the target trajectory generated by the action plan generation unit 140 at the scheduled time. Control.

Returning to FIG. 2, the second control unit 160 includes, for example, an acquisition unit 162, a speed control unit 164, and a steering control unit 166. The acquisition unit 162 acquires the information of the target trajectory (orbit point) generated by the action plan generation unit 140 and stores it in a memory (not shown). The speed control unit 164 controls the traveling driving force output device 200 or the brake device 210 based on the speed element associated with the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 according to the degree of bending of the target trajectory stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is realized by, for example, a combination of feedforward control and feedback control. As an example, the steering control unit 166 executes a combination of feedforward control according to the curvature of the road in front of the own vehicle M and feedback control based on the deviation from the target track.

The traveling driving force output device 200 outputs a traveling driving force (torque) for the vehicle to travel to the drive wheels. The traveling driving force output device 200 includes, for example, a combination of an internal combustion engine, a motor, a transmission, and the like, and an ECU (Electronic Control Unit) that controls them. The ECU controls the above configuration according to the information input from the second control unit 160 or the information input from the operation controller 80.

The brake device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to the information input from the second control unit 160 or the information input from the operation controller 80 so that the brake torque corresponding to the braking operation is output to each wheel. The brake device 210 may include a mechanism for transmitting the hydraulic pressure generated by the operation of the brake pedal included in the operation operator 80 to the cylinder via the master cylinder as a backup. The brake device 210 is not limited to the configuration described above, and is an electronically controlled hydraulic brake device that controls the actuator according to the information input from the second control unit 160 to transmit the hydraulic pressure of the master cylinder to the cylinder. May be good.

The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, exerts a force on the rack and pinion mechanism to change the direction of the steering wheel. The steering ECU drives the electric motor according to the information input from the second control unit 160 or the information input from the operation controller 80, and changes the direction of the steering wheel.

[Overtaking control]
Hereinafter, the overtaking control performed by the automatic operation control device 100 will be described. The overtaking control unit 142 of the action plan generation unit 140 operates, for example, when a constant-speed traveling event is being executed. The overtaking control unit 142 once changes the lane of the own vehicle M to the adjacent lane in order to overtake the preceding vehicle traveling in front of (immediately before) the own vehicle M on the same lane as the own vehicle M, and the own vehicle M changes the lane. Return to the original lane when the vehicle is sufficiently ahead of the vehicle in front. "Immediately before" means that there is no vehicle in between.

FIG. 3 is a diagram for explaining a scene in which overtaking control is performed and various definitions. In the figure, mA is the vehicle in front, L1 is the lane in which the vehicle M is located, L2 is the adjacent lane, and the vehicle M is the lane to which the vehicle M changes lanes when overtaking the vehicle mA in front. VM is the speed of the own vehicle _M , and V _mA is the speed of the preceding vehicle mA. Further, mB is an adjacent lane forward vehicle in front of the own vehicle M in the adjacent lane L2, mC is an adjacent lane rear vehicle in the adjacent lane L2 behind the own vehicle M, and mD is a following vehicle. .. The term "rear" means, for example, the rear as a result of comparing the positional relationship of representative points (center of gravity, etc.) with respect to the road extension direction (X direction in the figure). Further, it may mean that the front end portion of the own vehicle M is behind the rear end portion in the road extending direction (X direction in the figure). The same applies to the front. If there are adjacent lanes on both sides of the lane in which the own vehicle M is located, the lane to be changed may be determined by some method, and the determined lane may be used as the adjacent lane.

As a result of the above-mentioned recognition process, the index value deriving unit 132 of the recognition unit 130 has (1) a speed difference ΔV (= VM) obtained by subtracting the speed _VmA of the preceding vehicle _mA from the target speed VM * of the own vehicle _M. * -V _mA ), (2) Distance D _MA between own vehicle M and preceding vehicle mA, (3) Distance D _MC between own vehicle M and vehicle behind the adjacent lane D MC, etc. are derived. The target speed _VM * is determined based on the legal speed, the set speed set by the occupant, and the like. The distance D _MA is the distance between the front end of the own vehicle M and the rear end of the front vehicle mA in the road extending direction, but the representative point of the own vehicle M and the representative point of the front vehicle mA It may be a distance related to the direction of extension of the road. Further, the distance DMC is the distance between the rear end of the own vehicle M and the front end of the front vehicle _mC in the road extending direction, but the representative point of the own vehicle M and the representative point of the front vehicle mC. It may be a distance related to the direction of extension of the road. The index value derivation unit 132 may derive only a part of the index values of (1) to (3) described above, and is another index value as long as it indicates the environment of the own vehicle M at the time of overtaking. May be derived. For example, the index value derivation unit 132 is a position (horizontal position) relating to the distance between the own vehicle M and the vehicle in front of the adjacent lane mB, the degree of fluctuation in the speed of the preceding vehicle mA (for example, dispersion), and the road width direction (Y direction in the figure). ) Etc. may be derived as an index value.

The overtaking control unit 142 acquires the evaluation value for each index value by inputting each of the plurality of index values into the evaluation function for each index value. Then, the overtaking control unit 142 determines whether or not to overtake the preceding vehicle mA by comparing the value obtained by integrating the plurality of evaluation values with the threshold value. More specifically, the overtaking control unit 142 determines to overtake the preceding vehicle mA when the product of a plurality of evaluation values is equal to or greater than the threshold value, and causes the own vehicle M to overtake the preceding vehicle mA.

FIG. 4 is a diagram for explaining an operation in which the own vehicle M overtakes the preceding vehicle mA. The overtaking control unit 142 first causes the own vehicle M to change lanes in front of the vehicle behind the adjacent lane mC in the adjacent lane L2, and then changes lanes in front of the preceding vehicle mA in the original lane L1. Since the speed control and steering control at this time do not form the core of the present invention, the description thereof will be omitted. When determining whether or not to execute such an operation, the automatic driving control device 100 makes a determination according to at least the positions and speeds of the vehicle in front, the vehicle in front of the adjacent lane, mB, and the vehicle behind in the adjacent lane, mC, and their changes. In order not to make the occupants feel uncomfortable, a high degree of judgment is required.

Therefore, the overtaking control unit 142 determines whether or not to overtake the preceding vehicle mA by using a plurality of evaluation functions whose output tendency is adjusted by reinforcement learning in the prior environment. The pre-environment refers to a simulation environment or a test environment using an actual vehicle before the software is applied to the own vehicle M. Each of the plurality of evaluation functions outputs the evaluation value when the index value is input. FIG. 5 is a diagram showing an outline of an overtaking determination process by the overtaking control unit 142. The overtaking control unit 142 inputs index values 1, 2, 3, ... n such as speed difference ΔV, distance D _MA , distance D _MC , etc. into the corresponding evaluation functions 1, 2, 3, ..., N, respectively. The total power of the obtained evaluation values E1 to En is calculated. Then, when the total product is equal to or higher than the threshold value Th, it is determined to overtake the preceding vehicle mA. The value output by each evaluation function is a value indicating the desire of the occupant of the own vehicle M to overtake. The value output by each evaluation function is adjusted to take a value between zero and one, for example.

[Preliminary environment]
FIG. 6 is a diagram for explaining the learning process of the evaluation function in the pre-environment. The learning process of the evaluation function in the pre-environment is executed by any computer device. An actual environment such as a test course may be used instead of the simulation environment. First, as the Actor 300, a function equivalent to that of the overtaking control unit 142 (which is determined to be overtaken when the product of the evaluation values obtained by inputting the index value included in the state is equal to or greater than the threshold value) is prepared. The state which is the output of the simulation environment reflecting the determination result of the Actor 300 is input to the Actor 300 and the reward function 310.

The reward function 310 inputs an index value included in the state (a part or all of the index value input to the actor may overlap or may be a completely different index value) into the reward function for each index value. Then, a plurality of reward function values R1, R2, R3, ..., Rm are obtained. The index values in the reward function 310 are, for example, a speed difference ΔV # (= _VM * _-VM ) obtained by subtracting the actual speed VM from the target speed VM * of the own vehicle _M , and the own vehicle _M and the preceding vehicle mA. The distance D _MA , the distance D _MD between the own vehicle M and the following vehicle mD, and the like, and each reward function outputs a reward function value representing the cruising comfort of the own vehicle M. Then, the infinite product of the reward function values R1, R2, R3, ..., Rm is output to Critic320 as the integrated reward function value Rtotal.

The Critic 320 estimates the future sum of rewards based on the input integrated reward function value Rtotal, evaluates the appropriateness of the behavior of the Actor 300, and adjusts the parameters of the evaluation function of the Actor 300 based on the evaluation result. Then, the evaluation function of the Actor 300 at the stage where the adjustment is sufficiently completed is mounted on the own vehicle M as the evaluation function used by the overtaking control unit 142. The Critic320 performs a process using a DNN (Deep Neural Network) for such a process. The treatment of Critic320 is based on, for example, a known Actor-Critic method, and detailed description thereof will be omitted. Although such processing has a high processing load, since it is performed in a pre-environment before it is mounted on a vehicle, there is no problem such as a processing load when it is mounted on a vehicle. In addition, the state value can be accurately estimated by performing advanced processing such as DNN. In addition, instead of the Actor-Critic method, another reinforcement learning method may be used.

[Processing flow]
FIG. 7 is a flowchart showing an example of the flow of processing executed by the overtaking control unit 142. The processing of this flowchart is, for example, repeatedly executed while the constant speed running event is being executed.

First, the overtaking control unit 142 acquires a plurality of index values from the index value derivation unit 132 (step S100), and inputs the plurality of index values to the corresponding evaluation functions (step S102). Then, the overtaking control unit 142 determines whether or not the infinite product of the evaluation values is equal to or greater than the threshold value Th (step S104). When the total product of the evaluation values is equal to or greater than the threshold value Th, the overtaking control unit 142 causes the own vehicle M to overtake the preceding vehicle mA (step S106).

According to the embodiment described above, the recognition unit (130, 132) that recognizes the peripheral situation of the vehicle (M) and the operation control unit that controls acceleration / deceleration and steering of the vehicle without depending on the operation of the occupant of the vehicle (13, 132). 140, 160), and the operation control unit travels in front of the vehicle on the same lane as the vehicle by comparing a value obtained by integrating a plurality of evaluation values output by a plurality of evaluation functions with a threshold value. It is equipped with an overtaking control unit (142) that determines whether or not to overtake the vehicle in front, and each of the plurality of evaluation functions evaluates that a plurality of index values indicating the surrounding conditions of the vehicle recognized by the recognition unit are input. Since the value is output and the output tendency is adjusted by reinforcement learning in the pre-environment, it is appropriately determined whether or not the vehicle overtakes the vehicle in front while suppressing the increase in the processing load. be able to.

The embodiment described above can be expressed as follows.
A storage device that stores the program and
With a hardware processor,
By executing the program stored in the storage device by the hardware processor.
Recognize the surrounding situation of the vehicle,
It controls the acceleration / deceleration and steering of the vehicle without depending on the operation of the vehicle occupants.
The control is to overtake the preceding vehicle traveling in front of the vehicle on the same lane as the vehicle by comparing the value obtained by integrating the plurality of evaluation values output by the plurality of evaluation functions with the threshold value. Including deciding whether or not
Each of the above-mentioned evaluation functions
When a plurality of index values indicating the surrounding conditions of the vehicle recognized by the recognition unit are input, the evaluation value is output.
The output tendency is adjusted by reinforcement learning in the pre-environment.
A vehicle control unit configured as such.

Although the embodiments for carrying out the present invention have been described above using the embodiments, the present invention is not limited to these embodiments, and various modifications and substitutions are made without departing from the gist of the present invention. Can be added.

100 Automatic operation control device 130 Recognition unit 132 Index value derivation unit 140 Action plan generation unit 142 Overtaking control unit

Claims (8)

A recognition unit that recognizes the surrounding conditions of the vehicle and
It is equipped with an operation control unit that controls acceleration / deceleration and steering of the vehicle without depending on the operation of the occupants of the vehicle.
Whether the driving control unit overtakes the preceding vehicle traveling in front of the vehicle on the same lane as the vehicle by comparing the value obtained by integrating the plurality of evaluation values output by the plurality of evaluation functions with the threshold value. Equipped with an overtaking control unit that determines whether or not
Each of the above-mentioned evaluation functions
When one of a plurality of index values indicating the surrounding conditions of the vehicle recognized by the recognition unit is input, the evaluation value is output.
The output tendency is adjusted by reinforcement learning in the pre-environment.
Vehicle control unit. The overtaking control unit determines to overtake the preceding vehicle when the product of the plurality of evaluation values is equal to or greater than the threshold value, and causes the vehicle to overtake the preceding vehicle.
The vehicle control device according to claim 1. The plurality of index values include the difference between the target speed of the vehicle and the speed of the vehicle in front.
The vehicle control device according to claim 1 or 2. The plurality of index values include the distance between the vehicle and the vehicle in front.
The vehicle control device according to any one of claims 1 to 3. The plurality of index values include the distance between the vehicle and an adjacent rear vehicle that is behind the vehicle in the adjacent lane to which the vehicle changes lanes when overtaking.
The vehicle control device according to any one of claims 1 to 4. Each of the plurality of evaluation functions has its output tendency adjusted by the Actor-Critic method.
The vehicle control device according to any one of claims 1 to 5. The vehicle control device,
Recognize the surrounding situation of the vehicle,
It controls the acceleration / deceleration and steering of the vehicle without depending on the operation of the vehicle occupants.
The control is to overtake the preceding vehicle traveling in front of the vehicle on the same lane as the vehicle by comparing the value obtained by integrating the plurality of evaluation values output by the plurality of evaluation functions with the threshold value. Including deciding whether or not
Each of the above-mentioned evaluation functions
When one of a plurality of index values indicating the surrounding conditions of the recognized vehicle is input, an evaluation value is output.
The output tendency is adjusted by reinforcement learning in the pre-environment.
Vehicle control method. On the computer installed in the vehicle,
Recognize the surrounding situation of the vehicle
The acceleration / deceleration and steering of the vehicle are controlled without depending on the operation of the vehicle occupants.
The control is to overtake the preceding vehicle traveling in front of the vehicle on the same lane as the vehicle by comparing the value obtained by integrating the plurality of evaluation values output by the plurality of evaluation functions with the threshold value. Including letting you decide whether or not
Each of the above-mentioned evaluation functions
When one of a plurality of index values indicating the surrounding conditions of the recognized vehicle is input, an evaluation value is output.
The output tendency is adjusted by reinforcement learning in the pre-environment.
program.

Images