JP7589134B2 - Vehicle control device - Google Patents

Description

The present invention relates to a vehicle control device that controls a vehicle based on information from a sensor that detects the outside world.

A technology is known that detects the number of vehicles using images captured by a camera and estimates the state of the collection of vehicles traveling ahead of the vehicle based on the detection results (see Patent Document 1).

JP 2021-33757 A

During traffic jams, it may not be possible to capture all of the vehicles due to occlusion. Therefore, with conventional technology, it is difficult to estimate the traffic jam due to the effects of occlusion.

A vehicle control device according to one aspect of the present invention includes a camera that captures an image of the external world surrounding the host vehicle to acquire position information of targets; a detector that acquires position information of targets based on reflected waves from detection objects within the camera's imaging area; a setting unit that sets a control target to be used for driving control of the host vehicle based on the position information acquired by the camera and the detector, respectively; a determination unit that determines a congestion state of other lanes adjacent to the host vehicle's lane based on the position information acquired by the camera and the detector, respectively; and a restriction unit that, when the determination unit determines that the other lanes are congested, restricts the setting unit to exclude other vehicles in the vehicle line from the control targets that are newly set in a specific area including the boundaries of the host lane and other lanes along the vehicle line that constitutes the congestion.

The present invention makes it possible to reduce the effects of occlusion during traffic jams.

1 is a block diagram showing an outline of the overall configuration of a vehicle control system for an autonomous driving vehicle having a vehicle control device according to an embodiment; 1 is a block diagram showing a configuration of a main part of a vehicle control device according to an embodiment; FIG. 4 is a diagram illustrating an example of a detection range detected by a group of external sensors. 10 is a flowchart showing an example of a process performed by a traffic congestion determination/confirmation unit. 10 is a flowchart showing an example of a process performed by a traffic congestion determination/confirmation unit. FIG. 4 is a diagram for explaining a traffic congestion determination by a sensor. FIG. 4 is a diagram for explaining integrated congestion determination. FIG.

Hereinafter, an embodiment of the present invention will be described with reference to FIG. 1 and FIG.
The vehicle control device according to the embodiment of the present invention can be applied to both vehicles having an automatic driving function, i.e., automatic driving vehicles, and manually driven vehicles not having an automatic driving function. Manually driven vehicles include vehicles equipped with a driving assistance function. An example of applying the vehicle control device to an automatically driven vehicle will be described below.
The vehicle to which the vehicle control device according to the embodiment is applied may be referred to as the host vehicle to distinguish it from other vehicles.

The vehicle may be an engine vehicle that uses an internal combustion engine (engine) as its driving source, an electric vehicle that uses a driving motor as its driving source, or a hybrid vehicle that uses an engine and a driving motor as its driving sources. The vehicle (autonomous vehicle) can be driven not only in an autonomous driving mode in which no driving operation by the driver is required, but also in a manual driving mode (driving assistance functions can be used) in which the driver operates the vehicle.

<Outline of Autonomous Driving Configuration>
First, a schematic configuration related to autonomous driving will be described. Fig. 1 is a block diagram showing a schematic overall configuration of a vehicle control system 100 for an autonomous driving vehicle having a vehicle control device according to an embodiment. As shown in Fig. 1, the vehicle control system 100 mainly includes a controller 10, an external sensor group 1 communicatively connected to the controller 10 via a CAN communication line or the like, an internal sensor group 2, an input/output device 3, a positioning unit 4, a map database 5, a navigation device 6, a communication unit 7, and an actuator AC for driving.

The external sensor group 1 is a collective term for a number of sensors (external sensors) that detect the external situation, which is information about the surroundings of the vehicle. For example, the external sensor group 1 includes a LiDAR (Laser Imaging Detection and Ranging) that detects the position (distance and direction from the vehicle) of an object around the vehicle by irradiating a laser beam and detecting the reflected light, a Radar that detects the position of an object around the vehicle by irradiating an electromagnetic wave and detecting the reflected wave, and a camera that has an imaging element such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) sensor and captures an image of the surroundings of the vehicle. The imaging element is also called an image sensor. Lidar and radar can detect objects within the imaging area of the camera. Lidar and radar may also be called detectors.

The internal sensor group 2 is a collective term for multiple sensors (internal sensors) that detect the driving state of the vehicle. The internal sensor group 2 includes, for example, a vehicle speed sensor that detects the driving speed (vehicle speed) of the vehicle, an acceleration sensor that detects the acceleration in the forward/rearward and left/right directions of the vehicle, a gyro sensor that detects the rotation and change in direction of the vehicle as angular velocity, and a rotation speed sensor that detects the rotation speed of the driving source. The internal sensor group 2 also includes sensors that detect the driving operations of the driver in manual driving mode, such as the operation of the accelerator pedal, the operation of the brake pedal, and the operation of the steering wheel.

The input/output device 3 is a general term for devices that receive commands from the driver and output information to the driver. For example, the input/output device 3 includes various switches through which the driver inputs various commands by operating operating members, a microphone through which the driver inputs commands by voice, a display that provides information to the driver via a displayed image, and a speaker that provides information to the driver by voice.

The positioning unit (GNSS; Global Navigation Satellite System unit) 4 has a positioning sensor that receives positioning signals transmitted from positioning satellites. The positioning sensor can also be included in the internal sensor group 2. The positioning satellite is an artificial satellite such as a GPS satellite or a quasi-zenith satellite. The positioning unit 4 measures the current position (latitude, longitude, altitude) of the vehicle using the positioning information received by the positioning sensor.

The map database 5 is a device that stores general map information used in the navigation device 6, and is configured, for example, with a hard disk or a semiconductor device. The map information includes road position information, road shape information (curvature, etc.), and position information of intersections and branching points.
It should be noted that the map information stored in the map database 5 is different from the highly accurate map information stored in the storage unit 12 of the controller 10 .

The navigation device 6 is a device that searches for a target route on roads to a destination input by the driver and provides guidance along the target route. The destination is input and guidance along the target route is performed via the input/output device 3. The target route is calculated based on the current position of the vehicle measured by the positioning unit 4 and map information stored in the map database 5. The current position of the vehicle can also be measured using the detection values of the external sensor group 1, and the target route can be calculated based on this current position and high-precision map information stored in the memory unit 12.

The communication unit 7 communicates with various servers (not shown) via networks including wireless communication networks such as the Internet and mobile phone networks, and acquires map information, driving history information, traffic information, and the like from the servers periodically or at any time. Networks include not only public wireless communication networks, but also closed communication networks set up for each specified management area, such as wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), and the like. The acquired map information is output to the map database 5 and the memory unit 12, where the map information is updated.

Actuator AC is a driving actuator for controlling the driving of the host vehicle. When the driving source for driving is an engine, actuator AC includes a throttle actuator that adjusts the opening of the throttle valve (throttle opening) of the engine. When the driving source for driving is a driving motor, actuator AC includes the driving motor. Actuator AC also includes a brake actuator that operates the braking device of the host vehicle and a steering actuator that drives the steering device.

The controller 10 is composed of an electronic control unit (ECU). More specifically, the controller 10 includes a computer having an arithmetic unit 11 such as a CPU (microprocessor), a storage unit 12 such as a ROM (Read Only Memory) and a RAM (Random Access Memory), and other peripheral circuits (not shown) such as an I/O interface. Note that multiple ECUs with different functions, such as an engine control ECU, a traction motor control ECU, and a braking device ECU, can be provided separately, but for convenience, in FIG. 1, the controller 10 is shown as a collection of these ECUs.

High-precision road map information is stored in the memory unit 12. This road map information includes road position information, road shape (curvature, etc.), road gradient information, intersection and branch point position information, number of lanes (which may also be called driving lanes), driving lane width and position information for each driving lane (information on the center position of the driving lane and boundary line of the driving lane position), information on landmarks (traffic lights, signs, buildings, etc.) as markers on the map, and road surface profile information such as unevenness of the road surface. Landmark information includes information on the landmark's shape (outline), characteristics, position, etc.

The calculation unit 11 has the following functional components: a vehicle position recognition unit 13, an external environment recognition unit 14, an action plan generation unit 15, a driving control unit 16, and a traffic congestion determination/confirmation unit 17.

The vehicle position recognition unit 13 recognizes the position of the vehicle on the map (vehicle position) based on the vehicle position information obtained by the positioning unit 4 and the map information in the map database 5. The vehicle position may be recognized using the map information stored in the memory unit 12 and the surrounding information of the vehicle detected by the external sensor group 1, thereby making it possible to recognize the vehicle position with high accuracy. Note that when the vehicle position can be measured by a sensor installed externally on or beside the road, the vehicle position can also be recognized by communicating with the sensor via the communication unit 7.

The external environment recognition unit 14 recognizes the external situation around the vehicle based on signals from a group of external sensors 1 such as a lidar, radar, and camera. For example, it recognizes the positions, speeds, and accelerations of surrounding vehicles (vehicles in front, behind, and to the sides) traveling around the vehicle, the positions of surrounding vehicles stopped or parked around the vehicle, and the positions and states of other objects. Other objects include signs, traffic lights, markings such as road dividing lines and stop lines, buildings, guardrails, utility poles, signs, pedestrians, bicycles, tunnel entrances, etc. The states of other objects include the color of traffic lights (red, green, yellow), the movement speed and direction of pedestrians and bicycles, etc.

The object that is the target of detection by the external sensor group 1 is called a target. Targets include both people and objects, and both moving and stationary objects. The external environment recognition unit 14 performs integrated processing (fusion processing) of the detection data of different types of sensors (e.g., cameras and detectors) that make up the external sensor group 1, determines whether the same target has been detected by each sensor, and derives position data of the target. For example, when the same target is detected, fusion processing such as coordinate conversion of the detection data, data complementation, and averaging is performed to derive position data of the target. This allows the position of the target to be recognized with high accuracy.

The action plan generating unit 15 generates a running trajectory (target trajectory) of the vehicle from the current time to a predetermined time ahead based on, for example, the target route calculated by the navigation device 6, the map information stored in the memory unit 12, the vehicle position recognized by the vehicle position recognition unit 13, and the external situation (target object) recognized by the external environment recognition unit 14. When there are multiple trajectories that are candidates for the target trajectory on the target route, the action plan generating unit 15 selects an optimal trajectory that complies with laws and regulations and satisfies criteria such as efficient and safe running, and sets the selected trajectory as the target trajectory. Then, the action plan generating unit 15 generates an action plan according to the generated target trajectory. The action plan generating unit 15 generates various action plans corresponding to overtaking running to overtake a preceding vehicle, lane change running to change the running lane, following running to follow a preceding vehicle, lane keeping running to maintain the running lane without deviating, constant speed running, deceleration running or acceleration running, etc. When generating a target trajectory, the behavior plan generation unit 15 first determines the driving mode and generates the target trajectory based on the driving mode.

In the autonomous driving mode, the driving control unit 16 controls each actuator AC so that the host vehicle runs along the target trajectory generated by the behavior plan generating unit 15. More specifically, the driving control unit 16 calculates a required driving force for obtaining the target acceleration per unit time calculated by the behavior plan generating unit 15, taking into account the running resistance determined by the road gradient, etc., in the autonomous driving mode. Then, for example, the driving control unit 16 feedback-controls the actuators AC so that the actual acceleration detected by the internal sensor group 2 becomes the target acceleration. In other words, the actuators AC are controlled so that the host vehicle runs at the target vehicle speed and the target acceleration.
In the manual driving mode, the driving control unit 16 controls each actuator AC in response to a driving command (such as a steering operation) from the driver acquired by the internal sensor group 2 .

The congestion judgment/determination unit 17 judges the congestion state based on signals from the sensors (camera and detector) that constitute the external sensor group 1. In the embodiment, it judges the congestion state in other lanes adjacent to the lane in which the vehicle is traveling on a highway or the like (if the lane is adjacent to the left in the direction of travel, it is called the left lane, and if the lane is adjacent to the right in the direction of travel, it is called the right lane). Details of the judgment process will be described later.

In the external environment recognition unit 14 according to the embodiment, position data of the same target is obtained based on the detection values of a group of different types of external sensors 1, such as a camera and a radar. However, in order to prevent overdetection of the target, after the target is detected, the target is not immediately treated as a control target used for vehicle control, but is set as a control target when the target is continuously detected for a predetermined time (referred to as a third predetermined time) or more. For example, when a camera repeatedly captures images at a predetermined cycle to obtain multiple frames of camera images, if the number of times the target is continuously detected from the camera images of consecutive frames exceeds a third threshold (= third predetermined time / predetermined cycle), it is determined that the target has been continuously detected for the third predetermined time or more, and is set as a control target. The third predetermined time may be changed according to the distance from the vehicle to the object, etc.

However, when congestion occurs in the left or right lane of the lane in which the vehicle is traveling, the external environment recognition unit 14 may overdetect part of the line of vehicles that makes up the congestion in the left or right lane as a new target (vehicle).
For example, if the above-mentioned overdetection occurs during adaptive cruise control (ACC), which automatically performs both accelerator control and brake control of the vehicle based on the recognition results of the preceding vehicle and drives the vehicle following the preceding vehicle while maintaining an appropriate distance between the vehicle and the preceding vehicle, brake control will be performed based on the target generated by the overdetection even if there is no congestion in the lane in which the vehicle is traveling.

In order to suppress overdetection of targets during the above-described traffic jams, the vehicle control device according to the embodiment is configured as follows: That is, in the vehicle control device according to the embodiment, the traffic jam determination/determination unit 17 determines the traffic jam state of other lanes adjacent to the own lane.
When the traffic jam determination/confirmation unit 17 has determined that there is a traffic jam, the external environment recognition unit 14 treats the vehicle lines that make up the traffic jam in the other lanes as a single mass. In principle, the external environment recognition unit 14 does not treat targets (vehicles) newly detected near the boundary between the own lane and the other lanes along the area of the single mass of vehicles (called a specific area) as control targets. However, exceptions are also defined in which the external environment recognition unit 14 treats targets detected in the specific area as control targets.
On the other hand, if the traffic congestion judgment/confirmation unit 17 has determined that there is no traffic congestion, the external environment recognition unit 14 treats the detected object as a control object as usual if the detected object is detected continuously for a third predetermined time or more.
Such a vehicle control device will be described in detail below, focusing on the processing performed by the traffic congestion determination/confirmation unit 17.

FIG. 2 is a block diagram showing the main configuration of a vehicle control device 30 according to an embodiment. This vehicle control device 30 constitutes part of the vehicle control system 100 of FIG. 1. As shown in FIG. 2, the vehicle control device 30 has a camera 1a, a first detector 1b-1, a second detector 1b-2, a third detector 1b-3, a fourth detector 1b-4, a fifth detector 1b-5, a controller 10, and an actuator AC. The camera 1a and the first detector 1b-1 to the fifth detector 1b-5 correspond to the six sensors that make up the external sensor group 1.

Figure 3 is a diagram illustrating the detection range detected by external sensor group 1 (six sensors). The vehicle 200 equipped with the vehicle control system 100 is traveling forward (upward in Figure 3) in the vehicle's own lane 60. The lane adjacent to the right side of the vehicle's own lane 60 is the right lane 70, and the lane adjacent to the left side of the vehicle's own lane 60 is the left lane 50.

The dashed line area marked with the symbol R1a indicates the detection range (corresponding to the imaging area) of camera 1a. The dashed line area marked with the symbol R1b-1 indicates the detection range of the first detector 1b-1. The dashed line area marked with the symbol R1b-2 indicates the detection range of the second detector 1b-2. The dashed line area marked with the symbol R1b-3 indicates the detection range of the third detector 1b-3. The dashed line area marked with the symbol R1b-4 indicates the detection range of the fourth detector 1b-4. The dashed line area marked with the symbol R1b-5 indicates the detection range of the fifth detector 1b-5.

2 is a monocular camera having an imaging element such as the CCD or CMOS sensor, and constitutes a part of the external sensor group 1 in FIG. 1. The camera 1a is, for example, an omnidirectional camera (also called a 360-degree camera) attached to a predetermined position in the front of the vehicle 200. The camera 1a continuously captures the front space, left and right lateral spaces, and rear space of the vehicle 200 included in the imaging area R1a, and acquires an image (camera image) of the target. The target includes vehicles traveling around the vehicle 200 and structures. The external environment recognition unit 14 can recognize the position and type of the target based on the camera image. For example, if the horizontal direction of the camera image obtained by converting the front space of the omnidirectional image into two dimensions is the x direction and the vertical direction is the y direction, the position of the target in the vehicle width direction can be obtained from the position in the x direction on the camera image, and the position in the height direction and the traveling direction of the target can be obtained from the position in the y direction. In other words, the position data (position information) of the target can be acquired by the camera 1a. The same applies to the left and right lateral spaces and the rear space.

The first detector 1b-1 to the fifth detector 1b-5 are detectors that detect the distance from the vehicle 200 to a target based on the reflected waves from the detection target (target), and include either radar or lidar, or both. The external environment recognition unit 14 can acquire target position data (position information) based on the center position of the vehicle 200 from the detection data of each detector 1b-1 to 1b-5. The position data includes target position and speed data. The detection ranges R1b-1 to R1b-5 of the first detector 1b-1 to the fifth detector 1b-5 are included in the imaging area R1a of the camera 1a. Therefore, when the target detected by the camera 1a and the target detected by the first detector 1b-1 to the fifth detector 1b-5 are the same, the position and speed of the target can be derived by performing sensor fusion processing.

The controller 10 in FIG. 2 has a traffic jam determination/confirmation unit 17, a driving control unit 16, and an external environment recognition unit 14 as functional components carried out by the calculation unit 11 (FIG. 1). The traffic jam determination/confirmation unit 17 includes a sensor-specific determination unit 17a, an integrated determination unit 17b, and a determination determination unit 17c. The external environment recognition unit 14 includes a setting unit 14a and a restriction unit 14b.

<Explanation of the flow chart>
The details of the congestion determination and congestion confirmation process for the left lane 50 and right lane 70 adjacent to the vehicle's own lane 60, performed by the congestion determination/confirmation unit 17, will be described with reference to the flowcharts shown in FIGS.

Figures 4 and 5 are flowcharts showing an example of the processing executed by the controller 10 in Figure 2, particularly the processing executed by the congestion determination/confirmation unit 17. The processing shown in the flowcharts in Figures 4 and 5 is performed based on the most recent camera image and the most recent detection data, for example, when the camera 1a and the first detector 1b-1 to the fifth detector 1b-5 start capturing images or detecting position information at a predetermined cycle, respectively.

In step S10 of FIG. 4, the congestion judgment/determination unit 17 counts the number of objects in the adjacent lanes (left lane 50 and right lane 70) and proceeds to step S20. The congestion judgment/determination unit 17 sends a command to the external environment recognition unit 14 to, for example, perform a predetermined image processing on the camera image and count the number of vehicles appearing in the left lane 50 and right lane 70 from the camera image after image processing. The image processing may include at least one of binarization, edge detection, feature extraction processing, etc.

In step S20, the traffic jam judgment/determination unit 17 compares the number of landmarks counted with a predetermined value. If the number of landmarks > the predetermined value is true, the traffic jam judgment/determination unit 17 makes a positive judgment at step S20 and proceeds to step S30, and if the number of landmarks > the predetermined value is not true, the traffic jam judgment/determination unit 17 makes a negative judgment at step S20 and proceeds to step S180 in FIG. 5.

In step S30, the traffic jam determination/confirmation unit 17 extracts vehicles based on camera images or detection data for each of the six sensors that make up the external sensor group 1, and proceeds to step S40. The traffic jam determination/confirmation unit 17 sends a command to the external environment recognition unit 14 to extract vehicles based on the camera images from camera 1a and based on the detection data from each of the first detector 1b-1 to the fifth detector 1b-5.

In step S40, the traffic jam judgment/determination unit 17 rearranges the extracted vehicles in ascending order in the direction of travel and proceeds to step S50. The rearrangement of the vehicles is performed for each of the six sensors that make up the external sensor group 1, dividing them into the left lane 50 and the right lane 70.

In step S50, the congestion judgment/determination unit 17 calculates the vehicle distance between the rearranged vehicles and proceeds to step S60. The vehicle distance is calculated based on the center position of each vehicle that has been converted for each of the six sensors that make up the external sensor group 1, and is divided into the left lane 50 and the right lane 70 for each sensor.

In step S60, the traffic jam judgment/determination unit 17 groups vehicles whose inter-vehicle distance is less than a predetermined distance, and proceeds to step S70. The grouping is performed by dividing the six sensors that make up the external sensor group 1 into a left lane 50 and a right lane 70.

In step S70, the congestion judgment/determination unit 17 compares the number of vehicles in the group with a predetermined number. If the number of vehicles in the group > the predetermined number is true, the congestion judgment/determination unit 17 makes a positive judgment at step S70 and proceeds to step S80, and if the number of vehicles in the group > the predetermined number is not true, the congestion judgment/determination unit 17 makes a negative judgment at step S70 and proceeds to step S180 in FIG. 5. The above judgment is performed for each of the six sensors that make up the external sensor group 1, divided into the left lane 50 and the right lane 70. The predetermined number may be determined for each of the six sensors.

In step S80, the traffic jam judgment/determination unit 17 compares the absolute speeds of all vehicles in the group with a predetermined speed. If the absolute speeds of all vehicles are equal to or less than the predetermined speed, the traffic jam judgment/determination unit 17 makes a positive judgment in step S80 and proceeds to step S90, and if the absolute speeds of all vehicles are equal to or less than the predetermined speed, the traffic jam judgment/determination unit 17 makes a negative judgment in step S80 and proceeds to step S180 in FIG. 5. The above judgment is performed for each of the six sensors constituting the external sensor group 1, separately for the left lane 50 and the right lane 70. The predetermined speed may be set commonly for all six sensors.

In step S90, the traffic congestion determination/confirmation unit 17 performs a sensor-specific traffic congestion determination in the sensor-specific determination unit 17a as follows. When the conditions in both steps S70 and S80 are satisfied, the traffic congestion determination/confirmation unit 17 determines the possibility of traffic congestion for each of the left lane 50 and the right lane 70, and for each of the six sensors that make up the external sensor group 1. FIG. 6A is a diagram illustrating the results of the sensor-specific traffic congestion determination.

6A, the left column indicates the type of sensor, and the value in parentheses indicates the direction of the detection range of the sensor as viewed from the center position of the host vehicle 200.
In Fig. 6A, the center column shows the result of the congestion judgment by the sensor for the left lane 50. "Congestion" indicates that it has been judged that there is a possibility of congestion. (1) is a flag indicating that there is a possibility of congestion. "X" indicates that it has been judged that there is no possibility of congestion. (0) is a flag indicating that there is no possibility of congestion.

In Fig. 6A, the right column shows the results of the congestion determination by the sensor for the right lane 70. "X" indicates that it has been determined that there is no possibility of congestion. (0) is a flag indicating that there is no possibility of congestion. "Congestion" indicates that it has been determined that there is a possibility of congestion. (1) is a flag indicating that there is a possibility of congestion.
When the traffic congestion determination/confirmation unit 17 performs the above-described sensor-specific traffic congestion determination by the sensor-specific determination unit 17a, the process proceeds to step S100. Note that the "traffic congestion" and "x" shown in Fig. 6A are examples of the determination contents and are unrelated to the number of vehicles illustrated around the vehicle 200 illustrated in Fig. 3.

In step S100, the congestion judgment/determination unit 17 performs an integrated congestion judgment for each of the left and right lanes in the integrated judgment unit 17b based on the results of the congestion judgment by sensor, as follows. FIG. 6B is a diagram illustrating the results of the integrated congestion judgment. In FIG. 6B, the center column shows the result of the integrated congestion judgment for the left lane 50. "TRUE" indicates that it has been provisionally judged that there is congestion. In FIG. 6B, the right column shows the result of the integrated congestion judgment for the right lane 70. "FALSE" indicates that it has been provisionally judged that there is no congestion.

In the embodiment, the angle of view of the camera 1a covers the entire area in which congestion is determined. In other words, the area in which congestion is determined is set in the imaging area R1a of the camera 1a. In contrast, the first detector 1b-1 that detects the area in front of the vehicle 200 and the second detector 1b-2 to the fifth detector 1b-5 that detect the area in front or behind the vehicle 200 on the left and right sides respectively cover a part of the area in which congestion is determined. Therefore, when it is determined that there is a possibility of congestion based on the camera image (flag (1)) and also that there is a possibility of congestion based on at least one of the output data of the first detector 1b-1 to the fifth detector 1b-5 (flag (1)), the integrated determination unit 17b provisionally determines that there is congestion in that lane.
Moreover, when it is determined based on the camera image that there is no possibility of congestion (flag (0)), the integrated judgment unit 17b provisionally judges that there is no congestion in that lane. Furthermore, when it is determined based on all of the output data from the first detector 1b-1 to the fifth detector 1b-5 that there is no possibility of congestion (flag (0)), the integrated judgment unit 17b provisionally judges that there is no congestion in that lane. After the integrated judgment unit 17b makes the above integrated congestion judgment, the process proceeds to step S110 in FIG. 5.
The integrated congestion determination performed by the integrated determination unit 17b can be summarized as follows.
(I) A logical sum (OR) of flags based on the output data of the first detector 1b-1 to the fifth detector 1b-5 is calculated.
(II) A logical product (AND) of the result of the calculation in (I) above and a flag based on the camera image obtained by the camera 1a is calculated.
(III) If the result of the calculation in (II) above is (1), it is provisionally determined that the lane is congested.
(IV) If the result of the calculation in (II) above is (0), it is provisionally determined that there is no congestion in that lane.

In steps S110 to S180 in FIG. 5, the congestion determination/confirmation unit 17 performs a confirmation determination of whether or not there is a congestion in the confirmation determination unit 17c. FIG. 7 is a diagram showing the conditions for performing the processes in steps S110 to S180 and the contents of the processes.

In step S110, the determination unit 17c determines whether the current congestion judgment is "TRUE". If the most recent integrated congestion judgment is "TRUE", the determination unit 17c makes a positive judgment at step S110 and proceeds to step S120, and if the most recent integrated congestion judgment is not "TRUE", the determination unit 17c makes a negative judgment at step S110 and proceeds to step S160. The above judgment is performed separately for the left lane 50 and the right lane 70.

In step S120, the determination determination unit 17c determines whether the previous traffic congestion determination is "FALSE". If the most recent traffic congestion determination (performed in step S180 described below) is "FALSE", the determination determination unit 17c determines step S120 as "YES" and proceeds to step S130, and if the most recent traffic congestion determination (performed in step S150 described below) is not "FALSE", the determination determination unit 17c determines step S120 as "NO" and proceeds to step S150. The above determination is performed separately for the left lane 50 and the right lane 70.

In step S130, the determination unit 17c determines whether the previous congestion judgment is "TRUE". If the previous integrated congestion judgment is "TRUE", the determination unit 17c makes a positive judgment at step S130 and proceeds to step S140, and if the previous integrated congestion judgment is not "TRUE", the determination unit 17c makes a negative judgment at step S130 and proceeds to step S180. The above judgment is performed separately for the left lane 50 and the right lane 70.

In step S140, the determination unit 17c compares the number of times the traffic jam determination "TRUE" is made with the first threshold value. If the number of consecutive "TRUE" determinations > the first threshold value is met, the determination unit 17c makes a positive determination at step S140 and proceeds to step S150, and if the number of consecutive "TRUE" determinations > the first threshold value is not met, the determination unit 17c makes a negative determination at step S140 and proceeds to step S180. When proceeding to step S150, the counter that counts the number of times the traffic jam determination "TRUE" is made is reset to its initial value of 0. When proceeding to step S180, the counter that counts the number of times the traffic jam determination "TRUE" is made is incremented by +1.

The case where the number of consecutive "TRUE" judgments > the first threshold value is established corresponds to the case where, when the processing in Figures 4 and 5 is repeated at a predetermined period, the provisional judgment that there is a traffic jam based on the integrated traffic jam judgment continues for longer than a predetermined time (= first threshold value x predetermined period).
The above determination is made separately for the left lane 50 and the right lane 70.

In step S150, the determination unit 17c determines that there is a traffic jam and ends the process in Fig. 5. "TRUE" indicates that it has been determined that there is a traffic jam.
The above-mentioned determination is made separately for the left lane 50 and the right lane 70.

In step S160, which is reached after making a negative judgment at step S110 described above, the determination judgment unit 17c judges whether the previous congestion determination is "TRUE". If the most recent congestion determination (performed at step S150 above) is "TRUE", the determination judgment unit 17c makes a positive judgment at step S160 and proceeds to step S170, and if the most recent congestion determination (performed at step S180 described below) is not "TRUE", the determination judgment unit 17c makes a negative judgment at step S160 and proceeds to step S180. The above judgment is performed separately for the left lane 50 and the right lane 70.

In step S170, the determination unit 17c compares the number of times the traffic jam judgment is "FALSE" with the second threshold. If the number of consecutive "FALSE" judgments > the second threshold is satisfied, the determination unit 17c judges step S170 to be positive and proceeds to step S180, whereas if the number of consecutive "FALSE" judgments > the second threshold is not satisfied, the determination unit 17c judges step S170 to be negative and proceeds to step S150. When proceeding to step S180, the counter that counts the number of times the traffic jam judgment is "FALSE" is reset to its initial value of 0. When proceeding to step S150, the counter that counts the number of times the traffic jam judgment is "FALSE" is incremented by +1.

The case where the number of consecutive "FALSE" judgments > the second threshold value is established corresponds to the case where, when the processing in Figures 4 and 5 is repeated at a predetermined period, the provisional judgment that there is no congestion based on the integrated congestion judgment continues for longer than a predetermined time (= second threshold value x predetermined period).
The second threshold is set to a value larger than the first threshold. The reason for this is that the first threshold is intended to suppress overdetection, whereas the second threshold is intended to provide hysteresis. By configuring in this way, it is possible to prevent a phenomenon in which a definite determination that there is a traffic jam and a definite determination that there is no traffic jam are frequently repeated.
The above determination is made separately for the left lane 50 and the right lane 70.

In step S180, the determination unit 17c determines that there is no traffic congestion and ends the process in Fig. 5. "FALSE" indicates that it has been determined that there is no traffic congestion.
The above-mentioned determination is made separately for the left lane 50 and the right lane 70.

<Exceptions when it is determined that there is a traffic jam>
As described above, when the determination determination unit 17c of the traffic jam determination/determination unit 17 has determined that there is a traffic jam, the external environment recognition unit 14 treats the vehicle train C (FIG. 3) that constitutes the traffic jam in the other lanes (left lane 50, right lane 70) as a single mass. Regarding targets (vehicles) that are newly detected near the boundary between the vehicle train 60 and the other lanes (left lane 50, right lane 70) along the area (specific area) of the single mass of vehicle train C, the setting unit 14a does not set them as control targets in principle because they are restricted by the restriction unit 14b.
However, when the determination determination unit 17c of the congestion determination/determination unit 17 has determined that there is a congestion, for example, when the angle (angle with respect to the traveling direction) of the target detected in the specific area is detected by at least one of the six sensors constituting the external sensor group 1, the restriction by the restriction unit 14b is released as an exception. Therefore, the setting unit 14a sets the target as a control target.

As a specific example, this corresponds to a case where a vehicle that has left a jammed line of vehicles C forcibly cuts in front of the vehicle 200 in the vehicle's lane 60. By providing an exception, braking control is performed based on the object that has cut in, making it possible to control the vehicle 200 to decelerate appropriately.

According to the embodiment described above, the following advantageous effects are achieved.
(1) A vehicle control system 100 having a vehicle control device 30 according to an embodiment includes a camera 1a that captures an image of the external environment around the vehicle 200 to acquire position information of a target, a first detector 1b-1 to a fifth detector 1b-5 that acquire position information of the target based on a reflected wave from a detection object within an imaging area of the camera 1a, a setting unit 14a that sets a control target to be used for driving control of the vehicle 200 based on the position information respectively acquired by the camera 1a and the first detector 1b-1 to the fifth detector 1b-5, and a control unit 15a that controls the vehicle 200 based on the position information respectively acquired by the camera 1a and the first detector 1b-1 to the fifth detector 1b-5. The vehicle is equipped with a congestion judgment/determination unit 17 as a judgment unit that judges the congestion state of the left lane 50 and the right lane 70, which are other lanes adjacent to the vehicle's lane 60 in which the vehicle 200 is traveling, based on the position information respectively acquired by the devices 1b-5, and a restriction unit 14b that restricts the setting unit 14a from newly setting a control target in a specific area including the boundary between the left lane 50 and/or the right lane 70 along the vehicle queue C that constitutes the congestion and the vehicle's lane 60 when the congestion judgment/determination unit 17 judges that the left lane 50 and/or the right lane 70 is congested.
With this configuration, it is possible to suppress the effects of occlusion during congestion. For example, when occlusion occurs in which a part of a group of vehicles C that constitutes a congestion is temporarily hidden by another vehicle in the traffic line C, and the hidden vehicle reappears, setting this vehicle as a new control target is restricted. In other words, overdetection of a part of the traffic line C in the congestion as a new target is suppressed. As a result, braking control of the host vehicle 200 based on a control target generated by overdetection is prevented even when there is no congestion in the host lane 60 in which the host vehicle 200 is traveling.

(2) In the vehicle control device 30, the camera 1a and the first detector 1b-1 to the fifth detector 1b-5 each repeatedly acquire positional information at a predetermined period, and the traffic congestion judgment/determination unit 17 as a judgment unit repeatedly judges the traffic congestion state in the other lanes, that is, the left lane 50 and the right lane 70, based on the most recent positional information acquired by the camera 1a and the first detector 1b-1 to the fifth detector 1b-5, respectively.
With this configuration, the congestion judgment/determination unit 17 is able to repeatedly judge the congestion state in the left lane 50 and the right lane 70 appropriately based on new position information.

(3) In the vehicle control device 30, the congestion determination/confirmation unit 17 as a determination unit includes a sensor-specific determination unit 17a that determines whether there is a possibility of congestion in the left lane 50 and the right lane 70 as other lanes in the image capture area R1a of the camera 1a and whether there is a possibility of congestion in the left lane 50 and the right lane 70 in the detection ranges R1b-1 to R1b-5 of the first detector 1b-1 to the fifth detector 1b-5, and a sensor-specific determination unit 17a that determines whether there is a possibility of congestion in the left lane 50 and/or the right lane 70 in the image capture area R1a of the camera 1a. The device includes an integrated determination unit 17b that provisionally determines that the left lane 50 and/or the right lane 70 are congested when it is determined that there is a possibility of congestion of 0 and when it is determined that there is a possibility of congestion in the left lane 50 and/or the right lane 70 in the detection ranges R1b-1 to R1b-5 of the detector, and a definitive determination unit 17c that definitively determines that the left lane 50 and/or the right lane 70 are congested when the number of consecutive provisional determinations by the integrated determination unit 17b that the left lane 50 and/or the right lane 70 are congested exceeds a first threshold value.
By providing the sensor-specific determination unit 17a, it is possible to simplify the determination process compared to a case in which the determination of the possibility of traffic congestion is not performed separately for each sensor.
In addition, by having the integrated judgment unit 17b, when it is judged that there is a possibility of traffic congestion only in the imaging area R1a of the camera 1a or when it is judged that there is a possibility of traffic congestion only in the detection ranges R1b-1 to R1b-5 of the first detector 1b-1 to the fifth detector 1b-5, it is possible to increase the reliability of the judgment that there is a possibility of traffic congestion compared to a case where it is immediately provisionally judged that there is a traffic congestion in the left lane 50 and/or the right lane 70.
Furthermore, by having the determination judgment unit 17c, it is possible to appropriately judge the congestion state while avoiding overdetection, etc., compared to a case where a definitive judgment is immediately made that the left lane 50 and/or the right lane 70 are congested based on a single provisional judgment that the lane is congested.

(4) In the vehicle control device 30, the integrated judgment unit 17b provisionally judges that the left lane 50 and/or the right lane 70 are not congested when the sensor-specific judgment unit 17a judges that there is no possibility of congestion in the left lane 50 and/or the right lane 70 in the imaging area R1a of the camera 1a, or when the detection ranges R1b-1 to R1b-5 of the first detector 1b-1 to the fifth detector 1b-5 judge that there is no possibility of congestion in the left lane 50 and/or the right lane 70, and the final judgment unit 17c definitively judges that the left lane 50 and/or the right lane 70 are not congested when the number of consecutive provisional judgments that the left lane 50 and/or the right lane 70 are not congested after a definitive judgment that the left lane 50 and/or the right lane 70 are congested exceeds a second threshold value.
With this configuration, it is possible to appropriately judge the congestion state while avoiding overdetection, etc., compared to a case where a definitive judgment is immediately made that the left lane 50 and/or the right lane 70 are not congested based on a single provisional judgment that the lane is not congested.

(5) In the vehicle control device 30, the second threshold value is greater than the first threshold value.
With this configuration, it is possible to prevent a phenomenon in which a definite judgment that there is a traffic jam and a definite judgment that there is no traffic jam are frequently repeated.

The above embodiment can be modified in various ways, and modifications will be described below.
(Variation 1)
In the above embodiment, the camera 1a is configured as an omnidirectional camera, but it does not necessarily have to be an omnidirectional camera. For example, the camera 1a may be configured to acquire four camera images captured by a first camera 1a-1 that captures an image of the space in front of the vehicle 200, a second camera 1a-2 that captures an image of the left and right lateral spaces of the vehicle 200, a third camera 1a-3, and a fourth camera 1a-4 that captures an image of the rear space. When the four camera images are stitched together, they correspond to a camera image of the omnidirectional space.

(Variation 2)
In the above embodiment, examples have been given of using radar or lidar as the first detector 1b-1 to the fifth detector 1b-5, but the detectors may have any configuration as long as they are configured to acquire target position information based on reflected waves from a detection object in the imaging area R1a of the camera 1a.

The above description is merely an example, and the present invention is not limited to the above-mentioned embodiment and modifications, as long as the characteristics of the present invention are not impaired. The above-mentioned embodiment and modifications can be combined in any manner.

1a camera, 1b-1 to 1b-5 detectors 1 to 5, 10 controller, 14 external environment recognition unit, 14a setting unit, 14b restriction unit, 16 driving control unit, 17 congestion judgment/determination unit, 17a sensor-specific judgment unit, 17b integrated judgment unit, 17c determination unit, 30 vehicle control device, AC actuator

Claims (6)

A camera that captures an image of the external environment around the vehicle and acquires position information of a target;
a detector that acquires position information of a target based on a reflected wave from a detection object within an imaging area of the camera;
a setting unit that sets a control target to be used for driving control of the host vehicle based on the position information acquired by the camera and the detector, respectively;
a determination unit that determines a congestion state of another lane adjacent to the lane in which the host vehicle is traveling based on the position information acquired by the camera and the detector, respectively;
and a restriction unit that, when the judgment unit determines that the other lane is congested , restricts the setting unit to exclude other vehicles in the traffic jam from the control targets that are newly set in a specific area that includes the boundaries of the other lane and the vehicle's own lane along the traffic jam. 2. The vehicle control device according to claim 1,
The detector further acquires information indicating an intrusion of the other vehicle into the own lane,
the restriction unit releases the restriction on the setting unit when the detector acquires information indicating the interrupt.
A vehicle control device comprising: 3. The vehicle control device according to claim 1,
The camera and the detector each repeatedly acquire position information at a predetermined cycle;
The vehicle control device, wherein the determination unit repeatedly determines the congestion state of the other lane based on the most recent position information acquired by the camera and the detector, respectively. 4. The vehicle control device according to claim 3 ,
The determination unit is
a sensor-specific determination unit that determines whether or not there is a possibility of congestion in the other lane in an imaging area of the camera and whether or not there is a possibility of congestion in the other lane in a detection range of the detector;
an integrated determination unit that provisionally determines that the other lane is congested when the sensor-specific determination unit determines that there is a possibility of congestion in the other lane in the imaging area of the camera and determines that there is a possibility of congestion in the other lane in the detection range of the detector; and
A vehicle control device comprising: a confirmation judgment unit that makes a definitive judgment that the other lane is congested when the number of times the integrated judgment unit continues to make the provisional judgment that the other lane is congested exceeds a first threshold value. 5. The vehicle control device according to claim 4 ,
The integrated determination unit provisionally determines that the other lane is not congested when the sensor-specific determination unit determines that there is no possibility of congestion in the other lane in the imaging area of the camera, or determines that there is no possibility of congestion in the other lane in the detection range of the detector,
The vehicle control device is characterized in that the definitive judgment unit makes a definitive judgment that the other lane is not congested when the number of consecutive provisional judgments that the other lane is not congested after the definitive judgment that the other lane is congested exceeds a second threshold value. 6. The vehicle control device according to claim 5 ,
The control device for a vehicle, wherein the second threshold value is greater than the first threshold value.

Images