JP2024049862A - Vehicle control device - Google Patents

Abstract

[Problem] To provide a vehicle control device that realizes optimal driving assistance even when the sensor's recognition function is temporarily degraded due to a change in the environment surrounding the vehicle. [Solution] The vehicle control device includes a recognition degradation detection unit that detects the cause of the degradation of the sensor's recognition function based on the vehicle's environment, a sensor information storage unit that stores sensor information including at least sensor specification information and installation information, an acquisition unit that acquires detection results of at least one of a target detectable by the sensor whose recognition function is degraded and an area in which the target is present based on the cause of the degradation and the sensor information, a recognition unit that refers to pattern information that defines a plurality of patterns indicating a combination of sensors used to recognize the target corresponding to each driving assistance function, and recognizes the target based on the detection results by the sensors included in each pattern, and a vehicle control unit that executes the driving assistance function corresponding to the pattern for which the recognition result was obtained based on the target recognition result by the recognition unit. [Selected Figure] Figure 3

Description

The present invention relates to a vehicle control device.

Nowadays, vehicles may be equipped with sensors for monitoring the surroundings in order to realize driving assistance functions for vehicles such as automatic emergency braking (AEB) and adaptive cruise control (ACC). In recent years, in order to add functions and improve the performance of driving assistance functions, multiple sensors for monitoring the surroundings may be adopted for the purpose of detecting directions other than the vehicle's front and improving the reliability of the detection results. In other words, when an object is detected by a single or multiple sensors, in order to determine without fail whether it is a detection target for realizing a driving assistance function, such as whether it is an obstacle that needs to be avoided, it is necessary to grasp the exact position of the object and accurately determine whether it is a detection target (object to be controlled). In this way, the technology of collating the detection results of an object (target) by multiple sensors is called sensor fusion.

As an example of such sensor fusion technology, a technology has been proposed that recognizes pedestrians based on the results of pedestrian recognition by image recognition processing of an onboard camera and the results of target recognition by a laser radar (see, for example, Patent Document 1).

JP 2009-237898 A

However, each sensor has its strengths and weaknesses in detection, and the detection accuracy of each sensor may decrease depending on the vehicle's surrounding environment (driving environment), for example, weather conditions (rainfall, snowfall, fog, strong winds, sunlight, etc.). In such cases, even when multiple sensors are detecting the same object, some sensors may be able to maintain good detection while other sensors may experience decreased detection accuracy. In such cases, it may be difficult to determine in which direction (area) relative to the vehicle the detection accuracy of other vehicles is decreasing, or in which direction (area) the detection accuracy of pedestrians is decreasing. As a result, even when sensor fusion technology is used, this may cause erroneous detection or misjudgment of objects, resulting in decreased driving assistance accuracy.

The present invention has been made in consideration of the above-mentioned problems, and aims to provide a vehicle control device that can more appropriately grasp the recognition status of the sensor and realize optimal driving assistance even when the surrounding environment (driving environment) of the vehicle changes and the sensor's recognition function (detection reliability) temporarily decreases.

In order to solve the above-mentioned problems and achieve the object, the vehicle control device according to the present invention is a vehicle control device that executes one or more driving assistance functions based on detection signals received from multiple types of sensors that detect targets around the vehicle, and includes a recognition degradation detection unit that detects factors for degradation of the recognition function of the sensors based on the environment surrounding the vehicle, a sensor information storage unit that stores sensor information including at least specification information and installation information of the sensors, an acquisition unit that acquires detection results of at least one of the targets detectable by the sensors whose recognition function is degraded and the area in which the targets exist based on the factors for degradation and the sensor information, a recognition unit that recognizes the targets based on the detection results by the sensors included in each pattern by referring to pattern information that defines multiple patterns indicating combinations of the sensors used to recognize the targets corresponding to each driving assistance function, and a vehicle control unit that executes the driving assistance function corresponding to the pattern from which the recognition result was obtained based on the recognition result of the targets by the recognition unit.

The present invention provides a vehicle control device that can more appropriately grasp the sensor's recognition status and provide optimal driving assistance even when the vehicle's surroundings (driving environment) change and the sensor's recognition function (detection reliability) temporarily declines.

FIG. 1 is an exemplary schematic diagram showing the layout and detection ranges of various sensors of a vehicle to which a vehicle control device according to an embodiment can be applied. FIG. 2 is an exemplary schematic diagram showing an electrical configuration of a vehicle to which the vehicle control device according to the embodiment can be applied. FIG. 3 is an exemplary schematic block diagram showing a functional configuration of the vehicle control device (drive control ECU) according to the embodiment. FIG. 4 is an exemplary diagram showing characteristics of various sensors that can be used in the vehicle control device according to the embodiment. FIG. 5 is an exemplary diagram showing the performance of various sensors that can be used in the vehicle control device according to the embodiment. FIG. 6 is an exemplary diagram showing a pattern table that can be used in the vehicle control device according to the embodiment. FIG. 7 is an exemplary schematic plan view showing a change in the recognition area of a target when the recognition function of a sensor is degraded in the vehicle control device according to the embodiment. FIG. 8 is an exemplary flowchart showing the flow of operations of the vehicle control device (drive control ECU) according to the embodiment.

Below, with reference to Figures 1 to 8, an embodiment of a vehicle control device according to the present invention will be described in detail. Furthermore, the present invention is not limited to the following embodiment, and the components in the following embodiment include those that a person skilled in the art would easily come up with, those that are substantially the same, and those that are within the scope of what is called equivalent. Furthermore, various omissions, substitutions, modifications, and combinations of the components can be made without departing from the spirit of the following embodiment.

(Layout and detection range of various sensors on the vehicle)
1 is a diagram showing an example of the arrangement and detection range of various sensors of a vehicle to which a vehicle control device according to an embodiment can be applied. With reference to FIG. 1, an example of the arrangement and detection range of various sensors of a vehicle 1 according to the present embodiment will be described.

As shown in FIG. 1, the vehicle 1 is equipped with a plurality of types of sensors, and executes at least one of the driving assistance functions, such as automatic emergency braking (AEB), adaptive cruise control (ACC), or misstep prevention function, based on the detection results of the sensors. The automatic emergency braking (AEB) is a function that automatically brakes the vehicle to avoid a collision when a collision with an obstacle is predicted, or reduces the strength of the collision even if a collision with an obstacle is unavoidable, thereby reducing damage to the vehicle. The adaptive cruise control (ACC) is a function that realizes following driving while maintaining the distance from another vehicle in front by automatically accelerating and decelerating the vehicle within a preset vehicle speed range during long-distance travel on a highway, etc. The misstep prevention function is a function that determines that the driver has misstepped the accelerator pedal when there is a target such as a person or vehicle in front of the vehicle and the target is detected by a sensor when the vehicle 1 starts, and the driver mistakenly presses the accelerator pedal hard if the pressing speed is equal to or greater than a predetermined speed and the pressing amount is equal to or greater than a predetermined amount, closes the electronic throttle to control the engine speed, and automatically applies the brakes after a predetermined time. As shown in FIG. 1(a), the vehicle 1 is equipped with, as an example, a forward camera sensor 2, four sonar sensors 3, a forward millimeter wave radar 4, two front side millimeter wave radars 5, two rear side millimeter wave radars 6, and four surround view cameras 7.

The forward camera sensor 2 is provided on the upper part of the windshield of the vehicle 1, and is an imaging device that recognizes targets, such as vehicles, motorcycles, and pedestrians in front, within the detection range 2a shown in FIG. 1(b). The sonar sensor 3 is provided on the corners of the front bumper and the rear bumper of the vehicle 1, and is a sensor that detects targets by emitting ultrasonic waves within the detection range 3a shown in FIG. 1(b). The forward millimeter wave radar 4 is provided on the back side of the front bumper of the vehicle 1, and is a radar device that detects targets in front of the vehicle 1 by emitting millimeter wave radio waves toward the front of the vehicle 1 and receiving the reflected radio waves that return within the detection range 4a shown in FIG. 1(b).

The front-side millimeter-wave radar 5 is provided at the corners of the front bumper of the vehicle 1 and is a radar device that detects targets in the front side of the vehicle 1 by irradiating millimeter-wave radio waves toward the front side of the vehicle 1 and receiving the reflected radio waves in the detection range 5a shown in FIG. 1B. The rear-side millimeter-wave radar 6 is provided at the corners of the rear bumper of the vehicle 1 and is a radar device that detects targets in the rear side of the vehicle 1 by irradiating millimeter-wave radio waves toward the rear side of the vehicle 1 and receiving the reflected radio waves in the detection range 6a shown in FIG. 1B. The surround-view camera 7 is provided on the front side, back door side, and left and right side mirrors of the vehicle 1 and is an imaging device that generates an overhead image of the vehicle 1 from the sky using images captured in the wide-angle imaging range 7a shown in FIG. 1B.

In this way, the vehicle 1 realizes the various driving assistance functions described above by mounting multiple types of sensors and using sensor fusion to compare the detection results of targets from each sensor. Note that the various sensors of the vehicle 1 shown in FIG. 1 are merely examples, and the number, types, and placement of the sensors are not limited to those shown in FIG. 1.

(Vehicle Electrical Configuration)
2 is a diagram showing an example of an electrical configuration of a vehicle 1 to which the vehicle control device according to the embodiment can be applied. The electrical configuration of the vehicle 1 according to the embodiment will be described with reference to FIG.

As shown in FIG. 2, vehicle 1 has multiple ECUs (Electronic Control Units) for controlling various parts. Each ECU has a microcontroller unit (microcomputer), which includes, for example, a CPU (Central Processing Unit), a non-volatile memory such as a flash memory, and a volatile memory such as a DRAM (Dynamic Random Access Memory).

Specifically, as shown in FIG. 2, the vehicle 1 includes a drive ECU 11, a steering ECU 12, a brake ECU 13, a meter ECU 14, a body ECU 15, a driving control ECU 31, a camera ECU 32, a sonar ECU 33, and a radar ECU 34. The ECUs are connected to each other via a bus line 19 so that they can communicate with each other. The bus line 19 realizes communication based on a serial communication protocol such as CAN (Controller Area Network). Note that the serial communication protocol is not limited to CAN, and other serial communication protocols may also be applied.

The vehicle 1 also includes a drive unit 21, a steering angle sensor 22a, a steering unit 22b, a braking unit 23, a function switch 24, headlamps 25, a positioning signal receiver 35, a vehicle speed sensor 36, a display unit 37, a speaker 38, a rain sensor 39, etc. The vehicle 1 also includes a first monocular camera 42a, a second monocular camera 42b, a third monocular camera 42c, a first stereo camera 42d, a second stereo camera 42e, a sonar sensor 43, a first millimeter wave radar 44a, and a second millimeter wave radar 44b as sensors used to realize the driving assistance function.

The drive ECU 11 is an ECU that controls the drive device 21 of the vehicle 1 based on input accelerator pedal signals, etc. The drive device 21 is connected to the drive ECU 11 and has at least one of an engine or a motor as a drive source. The drive device 21 also includes, for example, an electronic throttle that adjusts the amount of intake air to control the output of the engine by electronically controlling the throttle opening of the engine. The drive device 21 also includes a transmission that changes the speed of the drive force from the drive source and outputs it as necessary.

The steering ECU 12 is an ECU that receives a detection signal from the steering angle sensor 22a and controls the steering device 22b of the vehicle 1. The steering angle sensor 22a is connected to the steering ECU 12 and is a sensor that detects the angle of the steering wheel. The steering device 22b is connected to the steering ECU 12 and is an electric power steering device that transmits the torque of an electric motor to a steering mechanism. The steering mechanism includes, for example, a rack-and-pinion steering gear, and is configured such that when the rack shaft moves in the vehicle width direction due to the torque of the electric motor, the left and right steered wheels turn left and right in accordance with the movement of the rack shaft.

The brake ECU 13 is an ECU that controls the braking device 23 of the vehicle 1. The braking device 23 is connected to the brake ECU 13 and is a hydraulic or electric braking device. For example, if the braking device 23 is a hydraulic type, the braking device 23 is equipped with a brake actuator, and the function of the brake actuator distributes hydraulic pressure to the wheel cylinders of the brakes provided on each wheel, and the hydraulic pressure is used to apply braking force from each brake to the wheels, including the drive wheels.

The meter ECU 14 is an ECU that controls each part of the meter panel of the vehicle 1. The meter panel includes instruments that display the vehicle speed and engine RPM, and displays such as a liquid crystal display for displaying various information. The meter ECU 14 is also connected to a function switch 24 for enabling or disabling various driving assistance functions, for example. Note that some driving assistance functions may, by their nature, be enabled at all times, regardless of whether they are switched on or off by the function switch 24.

The body ECU 15 is an ECU that controls the left and right turn signals and door lock motors, which need to be operated even when the vehicle ignition switch is off. In FIG. 2, the headlamps 25 are connected to the body ECU 15 and turn on according to commands from the body ECU 15.

The driving control ECU 31 is an ECU that is the center of control of the driving support functions, and is an example of a vehicle control device. The driving control ECU 31 executes various driving support functions based on sensor fusion, which collates target detection results based on detection signals from various sensors aggregated by the camera ECU 32, sonar ECU 33, and radar ECU 34. The driving control ECU 31 also includes a memory 41a and an external I/F 41b. As shown in FIG. 2, the driving control ECU 31 is also connected to a positioning signal receiver 35, a vehicle speed sensor 36, a display device 37, a speaker 38, a rain sensor 39, and the like.

Memory 41a is a non-volatile storage device such as a flash memory that stores a pattern table that specifies the combination patterns of various sensors used in the sensor fusion of the driving control ECU 31, and sensor information including the specifications, mounting position, mounting angle, and other installation information of various sensors. A pattern table exists for each driving assistance function and for each detection target (vehicle, motorcycle, pedestrian, etc.) in that driving assistance function. Sensor information also exists for each sensor. The pattern table and sensor information will be described later.

The external I/F 41b is an interface for performing data communication with an external device 51, which is an external information processing device. The external I/F 41b is an interface that complies with standards such as Ethernet (registered trademark) or USB (Universal Serial Bus). The external device 51 is an information processing device such as a PC (Personal Computer) that is used for processing such as saving, editing, and updating the pattern table and sensor information stored in the memory 41a.

The camera ECU 32 is an ECU to which the first monocular camera 42a, the second monocular camera 42b, the third monocular camera 42c, the first stereo camera 42d, and the second stereo camera 42e are connected, and which receives and processes image signals captured by each camera to generate image data. The camera ECU 32 transmits image data obtained by processing the image signals received from each camera to the driving control ECU 31. The first monocular camera 42a, the second monocular camera 42b, and the third monocular camera 42c are camera sensors capable of continuously capturing still images of a search range in front of or behind the vehicle 1 at a predetermined frame rate. The first stereo camera 42d and the second stereo camera 42e are camera sensors capable of measuring the distance to a target from the parallax of images captured by two imaging units arranged in parallel equiposition.

The sonar ECU 33 is an ECU to which the sonar sensor 43 is connected, and which receives and processes the detection signal of a target obtained by the sonar sensor 43. The sonar ECU 33 transmits data obtained by processing the detection signal received from the sonar sensor 43 to the driving control ECU 31. The sonar sensor 43 is a sensor that detects a target by emitting ultrasonic waves.

The radar ECU 34 is connected to the first millimeter wave radar 44a and the second millimeter wave radar 44b, and is an ECU that receives and processes target detection signals obtained by each millimeter wave radar. The radar ECU 34 transmits data obtained by processing the detection signals received from each millimeter wave radar to the driving control ECU 31. The first millimeter wave radar 44a and the second millimeter wave radar 44b are radar devices that detect targets present around the vehicle 1 by emitting millimeter wave band radio waves and receiving the reflected radio waves.

Note that the various sensors of vehicle 1 shown in FIG. 2 are merely examples, and the number, types, and placement of sensors are not limited to those shown in FIG. 2.

The positioning signal receiving unit 35 is a receiving device that receives positioning signals from positioning satellites based on the Global Navigation Satellite System (GNSS). The positioning signal receiving unit 35 is communicatively connected to the driving control ECU 31 via, for example, a USB standard communication cable, and outputs the received positioning signals to the driving control ECU 31. The driving control ECU 31 detects the location where the vehicle 1 is located based on the positioning signals received from the positioning signal receiving unit 35. An example of the GNSS is the Global Positioning System (GPS).

The vehicle speed sensor 36 is a sensor that is installed, for example, near the wheels of the vehicle 1 and generates a vehicle speed pulse that indicates the rotation speed or number of rotations of the wheel. The vehicle speed sensor 36 is communicatively connected to the driving control ECU 31 and outputs the generated vehicle speed pulse to the driving control ECU 31. The driving control ECU 31 determines the vehicle speed of the vehicle 1 by counting the vehicle speed pulses received from the vehicle speed sensor 36.

The display device 37 is a display device such as an LCD (Liquid Crystal Display) or an OELD (Organic Electro-Luminescent Display) that is installed on the dashboard or the like in the passenger compartment of the vehicle 1 and displays object recognition information, etc. The display device 37 is connected to the driving control ECU 31 so as to be able to communicate with it.

The speaker 38 is an audio device that is installed in the passenger compartment of the vehicle 1 and outputs sounds and voices. The speaker 38 is connected to the driving control ECU 31 so as to be able to communicate with it.

The rain sensor 39 can be used as one of the sensors that detects the surrounding environment (driving environment) of the vehicle 1. The rain sensor 39 is, for example, an optical (infrared) sensor that detects rainfall on the vehicle 1, and is arranged, for example, inside the windshield of the vehicle 1. In this case, when there are no raindrops on the windshield, the infrared light irradiated from the light emitting unit is reflected by the windshield surface and enters the light receiving unit. On the other hand, when there are raindrops on the windshield, the infrared light passes through the raindrops, so the amount of light entering the light receiving unit is reduced. In other words, the presence of raindrops and the amount of raindrops (degree of rainfall) can be determined according to the amount of light incident on the light receiving unit, and the generated detection signal is output to the driving control ECU 31. The driving control ECU 31 determines the surrounding environment (rainfall situation) of the vehicle 1 based on the detection signal received from the rain sensor 39. For example, it becomes possible to consider that the recognition accuracy of a sensor that performs sensing using an image has decreased due to rainfall. In addition, when the detection signal of the rain sensor 39 is used, the decrease in the recognition accuracy (detection reliability) of the sensor may be determined in stages according to the amount of rainfall.

The detection of the surrounding environment (driving environment) of the vehicle 1 is not limited to the rain sensor 39, and may be performed based on image data acquired from the camera ECU 32, for example. When image data is used, for example, when the contrast of the image (image data) sent from the camera ECU 32 is lower than a predetermined level compared to a standard image (a standard image prepared in advance when visibility is good) for a predetermined period of time, it can be determined that there is a possibility of poor visibility due to rainfall, snowfall, fog, etc. In this case, it can be determined that the recognition level of the target is worse than the level at which the driving assistance of the vehicle 1 can be performed without hindrance. In other words, the image data itself sent from the camera ECU 32 can be used to determine the surrounding environment (driving environment) of the vehicle 1. Similarly, when the edge included in the image (image data) sent from the camera ECU 32 is more difficult to distinguish than the standard image by a predetermined level (blurred state) for a predetermined period of time, it can be determined that there is a possibility of poor visibility due to rainfall, snowfall, fog, etc. In this case, it can also be determined that the recognition level of the target is worse than the level at which the driving assistance of the vehicle 1 can be performed without hindrance. That is, the image data itself sent from the camera ECU 32 can be used to determine the surrounding environment (driving environment) of the vehicle 1. In addition, since the detection signal sent from the sonar ECU 33 is a signal based on the reflection of sound waves, if the surrounding conditions of the vehicle 1 are exposed to stronger winds than during normal driving, the pattern of the detection signal clearly changes from that when there are no strong winds. Based on the degree of change, the surrounding environment (driving environment) of the vehicle 1 can be determined (determined whether or not there is a strong wind state). In this way, if a detection value that does not reach the detection value threshold (standard value) predetermined for each sensor is continuously detected for a predetermined period of time, it can be determined that the surrounding environment of the vehicle 1 has deteriorated. By recognizing the change (deterioration) in the surrounding environment of the vehicle 1, various processes for driving assistance control according to the surrounding environment (driving environment) of the vehicle 1, which will be described later, can be realized.

Note that the electrical configuration of vehicle 1 shown in FIG. 2 is an example, and does not need to include all of the components shown in FIG. 2, or may include other components. Furthermore, the various ECUs shown in FIG. 2 are not limited to being independent hardware, and may be configured as an integrated ECU. For example, the driving control ECU 31 and the camera ECU 32 may be configured as a single integrated ECU, and the ECU may have the functions of both the driving control ECU 31 and the camera ECU 32.

(Functional block configuration and operation of vehicle driving control ECU)
FIG. 3 is an exemplary schematic block diagram showing a functional configuration of a vehicle control device (driving control ECU). FIG. 4 is a diagram showing an example of characteristics of various sensors that can be used in the vehicle control device according to the embodiment. FIG. 5 is a diagram showing an example of performance of various sensors that can be used in the vehicle control device according to the embodiment. FIG. 6 is a diagram showing an example of a pattern table that can be used in the vehicle control device according to the embodiment. The functional block configuration and operation of the driving control ECU 31 of the vehicle 1 according to the embodiment will be described with reference to FIGS. 3 to 6.

As shown in FIG. 3, the driving control ECU 31 has a setting unit 61, a memory unit 62, a recognition unit 63, a vehicle control unit 64, and an output control unit 65. The recognition unit 63 includes a control determination unit 63a, a sensor recognition degradation detection unit 63b, a recognizable area calculation unit 63c, and a recognizable area integration unit 63d. The memory unit 62 includes a pattern table memory unit 62a and a sensor information memory unit 62b. The vehicle control unit 64 includes a variable processing unit 64a.

The setting unit 61 is a functional unit that executes setting processes such as saving, editing, and updating the pattern tables and sensor information stored in the memory unit 62 in response to commands from the external device 51.

As described above, the pattern table (an example of pattern information) stored in the pattern table storage unit 62a is prepared for each driving assistance function and for each detection target (vehicle, motorcycle, pedestrian, etc.) in the driving assistance function, and is a table that specifies the combination patterns of the above-mentioned various sensors. Here, the above-mentioned various sensors, namely, the first monocular camera 42a, the second monocular camera 42b, the third monocular camera 42c, the first stereo camera 42d, the second stereo camera 42e, the sonar sensor 43, the first millimeter wave radar 44a, and the second millimeter wave radar 44b, each have different characteristics. For example, FIG. 4 shows an example of the characteristics of each type of sensor. In FIG. 4, the characteristics of the sensor are, for example, the sensing method, the detection distance, the lateral resolution, the detectable object, and the weather dependency. For example, in the case of a monocular sensor, the sensing method is based on an image, the detection distance is up to a hundred and several tens of meters, the lateral resolution is particularly excellent, the detectable object has slightly poor performance, and the weather dependency is moderate. In addition, it can be seen that millimeter wave radar is particularly superior to sonar sensors and laser radar in terms of detection distance. Note that in Figure 4, sensors of the same type are described as having the same characteristics, but it goes without saying that sensors of the same type can have different characteristics.

As shown in FIG. 4, the characteristics of various sensors differ depending on the type, so it is judged whether they are suitable for performing various performances required for performing various driving support functions, such as vehicle identification performance, pedestrian identification performance, and distance measurement performance. FIG. 5 shows the suitability of various performances required for performing various driving support functions, judged according to the characteristics of the first monocular camera 42a, the second monocular camera 42b, the third monocular camera 42c, the first stereo camera 42d, the second stereo camera 42e, the sonar sensor 43, the first millimeter wave radar 44a, and the second millimeter wave radar 44b mounted on the vehicle 1. For example, FIG. 5 shows that the first monocular camera 42a, the second monocular camera 42b, and the first stereo camera 42d are suitable for performing vehicle identification performance, the second millimeter wave radar 44b is somewhat unsuitable, and the third monocular camera 42c, the first millimeter wave radar 44a, the second stereo camera 42e, and the sonar sensor 43 are unsuitable. It is also shown that any sensor other than the second monocular camera 42b is suitable for achieving distance measurement performance. Note that the method of displaying the suitability of various performances shown in FIG. 5 is merely an example, and the suitability evaluation may be expressed, for example, by a numerical evaluation value.

In this way, the developer organizes in advance the suitability of various performances required for the execution of various driving assistance functions as shown in FIG. 5 according to the characteristics of various sensors mounted on the vehicle 1. Then, the developer creates a pattern table via the external device 51 for each driving assistance function and for each detection target in the driving assistance function, based on the suitability of the various sensors for the various performances, that specifies the pattern of the combination of sensors used to detect the detection target corresponding to the various performances (i.e., the combination of sensors that may confirm the detection of the detection target). FIG. 6 shows a pattern table corresponding to the vehicle AEB as a driving assistance function as an example of a pattern table. Since this vehicle AEB detects a vehicle as a detection target and includes vehicle identification processing and distance measurement processing, the developer sets the pattern table for the vehicle AEB according to the suitability of the various sensors for the vehicle identification performance and distance measurement performance shown in FIG. 5. The pattern table may be created by the external device 51 and then stored or updated in the storage unit 62 (pattern table storage unit 62a) by the setting unit 61, or the pattern table stored in the storage unit 62 may be edited by the setting unit 61 in response to a command from the external device 51.

As a criterion for the pattern of combinations of various sensors specified in the pattern table corresponding to the vehicle AEB, the developer sets, for example, a sensor combination that satisfies either the following condition (1) or condition (2) as a sensor combination that may confirm detection for the vehicle.

Condition (1): A combination of a sensor whose vehicle identification performance is rated as "good" and whose distance measurement performance is rated as "good" Condition (2): A combination of a sensor whose vehicle identification performance is rated as "△" and two other sensors whose distance measurement performance is rated as "good"

Figure 6 shows an example in which patterns (1) to (14) are set in the pattern table as patterns of sensor combinations that satisfy the above conditions. For example, in pattern (1), the suitability of the vehicle identification performance and the distance measurement performance of the first monocular camera 42a are both "o", so condition (1) is satisfied and the detection of the vehicle can be confirmed by itself. In pattern (2), the suitability of the vehicle identification performance of the second monocular camera 42b is "o", and the suitability of the distance measurement performance of the third monocular camera 42c is "o", so this combination satisfies condition (1) and may confirm the detection of the vehicle. In pattern (7), the suitability of the vehicle identification performance of the second millimeter wave radar 44b is "△", and the suitability of the distance measurement performance of the third monocular camera 42c and the first millimeter wave radar 44a is "o", so this combination satisfies condition (2) and may confirm the detection of the vehicle.

Note that the above conditions for defining the sensor combination patterns are merely examples, and patterns may be defined by other conditions. In addition, in FIG. 6, patterns (1) to (14) are defined in the pattern table, but it is not necessary to use all of these patterns for target recognition. In this case, unnecessary patterns can be deleted from the pattern table.

In addition, when an existing sensor is removed or a new sensor is installed in response to a change in the specifications of the vehicle 1, the developer may use the external device 51 to change the pattern table via the setting unit 61. For example, when an existing sensor is removed, the developer may delete a column corresponding to the relevant sensor in the pattern table and reconfigure a pattern of a combination of sensors that satisfies the above-mentioned conditions. In addition, when a new sensor is installed, the developer may add a new column for the sensor to the pattern table and additionally set a pattern of a combination of sensors that satisfies the above-mentioned conditions. Alternatively, the developer may create a pattern table in advance that corresponds to the maximum number and types of sensors that are expected to be installed, and edit and set the contents of the patterns specified in the pattern table in response to the removal of a sensor or the installation of a new sensor. In this case, the effort of deleting or adding columns in the pattern table can be omitted.

Note that the pattern table shown as an example in FIG. 6 is information in table format, but is not limited to table format, and any format of information (an example of pattern information) may be used as long as it can specify a pattern of combinations of various sensors.

The sensor information storage unit 62b also stores sensor information including at least the specification information and installation information of each of the above-mentioned sensors. As described above, each sensor has different characteristics so that various detection information can be efficiently acquired. In addition, the installation position and installation attitude (installation angle) for appropriately and efficiently acquiring the required detection information differ for each sensor. Therefore, the sensor information storage unit 62b stores the sensing method, detection distance, lateral resolution, detectable objects, and weather-dependent characteristics as specification information for each sensor. The weather-dependent characteristics are information indicating in which weather the detection reliability decreases for each sensor. For example, when the sensing method is an image, when the surrounding conditions of the vehicle 1 are foggy or rainy, the detection reliability decreases. In this case, for example, the degree of fog or rainfall may be ranked, and a weighting coefficient for the detection reliability for each rank may be specified. Additionally, the installation position (e.g., the installation position on the front bumper or rear bumper) and installation attitude (e.g., the detection direction in the vehicle width direction and the angle in the vertical direction, etc.) are stored as installation information associated with the specification information for each sensor in the sensor information storage unit 62b.

The storage unit 62 is a functional unit that stores pattern tables and sensor information corresponding to each driving assistance function and each detection target in the driving assistance function. The storage unit 62 is realized by the memory 41a shown in FIG. 2. The storage unit 62 may also be realized by a storage device such as an external HDD (Hard Disk Drive) or SSD (Solid State Drive).

The recognition unit 63 is a functional unit that refers to a pattern table in the memory unit 62 that corresponds to the driving assistance function enabled by the vehicle control unit 64, and uses sensor fusion to compare the detection results according to each pattern defined in the pattern table to recognize targets. Target recognition involves, for example, recognizing the position of the target and identifying the type of the target. The recognition unit 63 outputs the recognition results for the target to the vehicle control unit 64 and the output control unit 65.

First, a case where each sensor normally performs the various functions shown in FIG. 4, that is, a case where the environment around the vehicle 1 is in a state where the detection reliability of each sensor can be appropriately obtained (for example, a state where there is no fog or rainfall) will be described. For example, when the driving assistance function enabled in the vehicle control unit 64 is the AEB for the vehicle, the recognition unit 63 refers to a pattern table corresponding to the AEB for the vehicle in the storage unit 62 (that is, corresponding to the AEB as the driving assistance function and the vehicle as the detection target). Next, the recognition unit 63 compares the positions of the targets recognized by the sensors belonging to each pattern specified in the pattern table and the types of the identified targets. Then, the recognition unit 63 sets the position and type of the target that match as the final recognition result as a result of the comparison. For example, when patterns (1) to (3) are set in the pattern table, the recognition unit 63 compares the positions of the targets recognized by the patterns (1) to (3), and sets the position of the target as the final recognition result if they match. Similarly, the recognition unit 63 compares the types of targets identified by the patterns (1) to (3), and if there is a match, the type of target is the final recognition result. Note that even if the detection results to be compared (for example, the detection results of the target position) do not necessarily match exactly, the recognition unit 63 may consider them to match and use them as the final recognition result as long as the difference is less than a predetermined value or is very small.

The vehicle control unit 64 is a functional unit that executes various driving assistance functions based on the recognition results by the recognition unit 63, and controls the running of the vehicle 1 by outputting commands to the drive ECU 11, steering ECU 12, brake ECU 13, etc. Furthermore, when display output by the display device 37 or output of a warning sound or voice guidance by the speaker 38 is required depending on the driving assistance function, the vehicle control unit 64 outputs a command to the output control unit 65 to cause such output. Note that the driving assistance functions executed by the vehicle control unit 64 are not limited to being multiple, and a single driving assistance function may be executed.

The output control unit 65 is a functional unit that controls the display of the display device 37 and the sound or audio output of the speaker 38. For example, the output control unit 65 may cause the recognition result received from the recognition unit 63 to be displayed on the display device 37. In addition, the output control unit 65 causes the display device 37 to output a display, or the speaker 38 to output a warning sound or voice guidance, etc., depending on the driving assistance function executed by the vehicle control unit 64.

However, as described above, the recognition (detection) accuracy (detection reliability) of each sensor may decrease due to changes (mainly deterioration of the surrounding environment) in the surrounding environment (driving environment) of the vehicle 1. Therefore, in the vehicle control device of this embodiment, even if the recognition accuracy (detection reliability) of the sensor decreases, appropriate driving assistance control is realized in response to the change in the surrounding environment (driving environment) of the vehicle 1. Note that the change in the surrounding environment (driving environment) of the vehicle 1 can be detected by a dedicated sensor such as the rain sensor 39. In other embodiments, the change in the surrounding environment (driving environment) of the vehicle 1 can also be detected based on a decrease in the recognition (detection) accuracy of various sensors connected to the camera ECU 32, sonar ECU 33, and radar ECU 34. Note that the decrease in the recognition accuracy of the sensor also includes the stop of the recognition function.

For example, in the case of a monocular camera or stereo camera that uses images as a sensing method, the detection performance (recognition accuracy, detection reliability) of targets such as vehicles and pedestrians may decrease due to the influence of weather conditions such as rain, snow, fog, and sunlight shining in at sunrise or sunset. Similarly, distance measurement performance may decrease. As a result, this may cause erroneous determination of the presence or absence of a target or erroneous measurement of the distance to the target. Therefore, the recognition unit 63 of this embodiment includes a control determination unit 63a that controls at least one of the recognition unit 63 and the vehicle control unit 64 based on the surrounding environment of the vehicle 1, and is capable of responding even when the recognition accuracy (detection reliability) of the sensor decreases.

The control determination unit 63a can change the range of driving assistance control targets or change the driving assistance itself to be controlled depending on the surrounding environment of the vehicle 1, for example, depending on the recognition accuracy (detection reliability) of the sensor in the recognition unit 63.

The sensor recognition degradation detection unit 63b has a function of detecting factors of degradation of the recognition function of the sensor based on the surrounding environment of the vehicle 1. As described above, the sensor recognition degradation detection unit 63b can detect whether the surrounding environment of the vehicle 1 has deteriorated based on the detection result of the rain sensor 39 and image data acquired from the camera ECU 32. For example, when the detection value of the sensor reaches a predetermined deterioration level (reliability reduction level) set for each sensor, the sensor recognition degradation detection unit 63b identifies the sensor whose reliability has decreased and associates the cause of the deterioration (e.g., fog, rainfall, etc.) with the sensor and stores it in the memory area. In this case, the reliability reduction level may be ranked, for example, small, medium, large, etc., or classified in detail by numerical value. Information on the sensor whose reliability has decreased and the cause of the deterioration may be stored in the memory unit 62, etc.

The recognizable area calculation unit 63c functions as an acquisition unit that acquires at least one of the detection results of a target detectable by a sensor with a deteriorated recognition function and the area in which the target exists, based on the deterioration factor and the sensor information. That is, the recognizable area calculation unit 63c acquires the specification information and installation information corresponding to the sensor with a deteriorated reliability from the sensor information storage unit 62b based on the detection result of the sensor recognition deterioration detection unit 63b, and calculates the recognition area in which the detection result can be used even in a state in which the reliability is deteriorated. FIG. 7 is an exemplary and schematic plan view showing a change in the recognition area of a target when the recognition function (reliability) of the sensor is deteriorated. For example, it is assumed that the detection capability of the front camera sensor 2 provided on the upper part of the windshield of the vehicle 1 when the surrounding environment is good is the detection range 2a. In this case, the recognizable area calculation unit 63c calculates the recognition range in which the detection result can be used based on the detection result of the sensor recognition deterioration detection unit 63b, for example, the detection of fog (including the detection of the degree of fog), the specification information of the front camera sensor 2, the installation information, etc. In the case of FIG. 7, the reduced detection range 2aa of the front camera sensor 2 is shown as a result of the calculation by the recognizable area calculation unit 63c. That is, the recognition unit 63 considers that the range indicated by the detection range 2aa is the reliable detectable area of the front camera sensor 2 in the current surrounding environment. In addition, the recognizable area calculation unit 63c changes the recognizable target when the recognizable target changes due to the change in the detectable area. For example, if the surrounding environment is not deteriorated and it is possible to detect vehicles and pedestrians, but due to the deterioration of the surrounding environment (reduced detection range), it is no longer possible to identify vehicles but pedestrians are still possible to identify, the recognizable target is changed. The recognizable area calculation unit 63c similarly calculates reliable detection ranges and detection targets based on the current surrounding environment for other sensors. In addition, in FIG. 7, the sonar sensor 3 is shown as being for short-distance detection and is not affected by the current fog (no decrease in detection reliability).

The recognizable area integration unit 63d functions as a recognition unit that refers to pattern information that defines multiple patterns indicating combinations of sensors used to recognize targets corresponding to each driving assistance function, and recognizes targets based on the detection results by the sensors included in each pattern. That is, the recognizable area integration unit 63d integrates the detection ranges of each sensor calculated by the recognizable area calculation unit 63c. By integrating the detection ranges, it becomes possible to confirm a match with a pattern in a pattern table used for driving assistance, and it is possible to determine the detection area of targets (vehicles, pedestrians, etc.) in the current surrounding environment. That is, it is possible to improve the discrimination between areas where detection reliability can be ensured and areas where detection reliability cannot be ensured.

In this case, even if the recognition accuracy (detection reliability) of a sensor required for a specific driving assistance is reduced and it is deemed that there is a possibility of erroneous judgment or erroneous control during driving assistance, the control determination unit 63a changes (specifically reduces) the control range of the driving assistance so that the sensor results are used for driving assistance within a range in which detection reliability can be maintained. As a result, it becomes possible to suppress erroneous judgment and erroneous recognition in driving assistance.

The control determination unit 63a may change the execution mode of the driving assistance function in the vehicle control unit 64 according to the detection accuracy (detection reliability) of the sensor. The control determination unit 63a may request the variable processing unit 64a of the vehicle control unit 64 to change the control intervention amount and intervention timing of the driving assistance according to the detection accuracy (detection reliability) of the sensor. For example, when it is considered that the recognition accuracy of targets such as vehicles and pedestrians is reduced due to fog, rainfall, etc. by the processing of the sensor information storage unit 62b, the recognizable area calculation unit 63c, the recognizable area integration unit 63d, etc., the control determination unit 63a may limit the intervention amount in the driving assistance control to a distance where the target can be reliably recognized based on the processing result of the recognizable area integration unit 63d. For example, when the driving assistance function is steering control, the torque to be applied is reduced or the control steering angle is reduced. Similarly, the control determination unit 63a may delay the timing of intervention in the driving assistance control until the target can be reliably recognized. Also, for example, in the case of automatic braking control such as automatic emergency braking, the start timing of the automatic brake may be delayed until the target can be reliably recognized. In other words, the braking operation of the vehicle 1 is left to the manual operation of the driver. In this case, too, erroneous detection by a sensor with reduced recognition accuracy and the resulting erroneous judgment can be suppressed, and inappropriate driving assistance control can be prevented from being executed even when recognition accuracy has decreased.

When the detection range of the sensor is changed (reduced) by the recognition unit 63, or when the execution mode of the driving support function is changed, the variable processing unit 64a of the vehicle control unit 64 executes prohibition control of various driving support functions, change control of the intervention amount, change control of the intervention timing, etc. based on the change result. The vehicle control unit 64 continues the control (driving) of the vehicle 1 by outputting appropriate commands to the drive ECU 11, steering ECU 12, brake ECU 13, etc. even when the sensor function is reduced. In addition, the vehicle control unit 64 may output a command to the output control unit 65 requesting display output by the display device 37 or output of a warning sound or guidance voice by the speaker 38 in accordance with the change process by the variable processing unit 64a. In this case, for example, a message such as "The sensor recognition accuracy is decreasing. The intervention of the driving support control is being restricted. Please drive while paying attention to the surrounding situation" or "The sensor recognition accuracy is decreasing. The driving support control will be temporarily reduced or stopped" is output. In addition, an image notifying the sensor whose recognition accuracy is decreasing may be displayed on the display unit 37. For example, an overhead image showing the detection range of the vehicle 1 and the sensor as shown in (b) in FIG. 1 may be displayed, and the detection range where the recognition accuracy has decreased may be displayed in flashing red, or conversely, the detection range where the recognition accuracy has decreased may be hidden. In another example, driving assistance using a sensor with a decreased recognition accuracy (detection reliability) may be temporarily prohibited. In this case, the guidance message may be, for example, "The sensor recognition accuracy has decreased. Currently, pedestrians can be detected, but the detection accuracy of the vehicle has decreased. Following driving control for the vehicle ahead will be stopped. Please drive while paying attention to the surrounding situation." In addition, even if the recognition accuracy (detection reliability) of a sensor has decreased, detection reliability may be ensured if the detection target is changed. For example, although it cannot identify vehicles, it may be possible to detect obstacles. In such a case, it may be possible to change the target of driving assistance based on the detection result from direct driving control to driving assistance such as warnings by alarms, etc.

In this way, even if the surrounding environment (driving environment) of the vehicle 1 changes and the sensor's recognition function (detection reliability) temporarily declines, it is possible to more appropriately grasp the recognition situation by the sensor and provide optimal driving assistance.

The setting unit 61, recognition unit 63, vehicle control unit 64, and output control unit 65 shown in FIG. 3 are realized, for example, by the CPU of the driving control ECU 31 shown in FIG. 2 executing a program. Some or all of these functional units may be realized by hardware such as a logic circuit.

Furthermore, each functional unit of the driving control ECU 31 shown in FIG. 3 is a conceptual representation of the function, and is not limited to this configuration. For example, multiple functional units illustrated as independent functional units in FIG. 3 may be configured as one functional unit. On the other hand, the function of one functional unit in FIG. 3 may be divided into multiple units and configured as multiple functional units. For example, the control determination unit 63a may be provided separately from the recognition unit 63, the sensor information storage unit 62b, the recognizable area calculation unit 63c, and the recognizable area integration unit 63d may be integrated into one recognition area processing unit, and the variable processing unit 64a may be provided separately from the vehicle control unit 64.

(Operation flow of the vehicle driving control ECU)
8 is a flowchart showing an example of an operation flow of the vehicle control device (drive control ECU) of the vehicle 1 according to the embodiment. The operation flow of the drive control ECU 31 of the vehicle 1 according to the embodiment will be described with reference to FIG.

<Step S11>
The developer creates a pattern table in advance via the external device 51 for each driving assistance function and for each detection target in the driving assistance function, the pattern table defining a combination of sensors used to detect the detection target corresponding to each performance, based on the suitability of each sensor for each performance. Then, the setting unit 61 stores the pattern table in the pattern table storage unit 62a of the storage unit 62 in response to a command from the external device 51. Also, when an existing sensor is removed or a new sensor is installed in response to a change in the specifications of the vehicle 1, the developer uses the external device 51 to change and set the pattern table via the setting unit 61. Then, the process proceeds to step S12.

<Step S12>
If the driver of the vehicle 1 has started driving (step S12: Yes), the process proceeds to step S13. If the driver has not started driving the vehicle 1 (step S12: No), the process waits.

<Step S13>
When the sensor recognition degradation detection unit 63b determines that the sensor does not have a functional degradation (deterioration in detection reliability) based on the detection results from the sensors that detect the surrounding environment (driving environment) of the vehicle 1, such as the rain sensor 39, and the detection results indicating a deterioration in the recognition accuracy of the various sensors for sensing targets such as vehicles and pedestrians (step S13: No), the process proceeds to step S14. In this case, the various sensors are the first monocular camera 42a, the second monocular camera 42b, the third monocular camera 42c, the first stereo camera 42d, the second stereo camera 42e, the sonar sensor 43, the first millimeter wave radar 44a, the second millimeter wave radar 44b, etc. Also, when the sensor recognition degradation detection unit 63b determines that the sensor has a functional degradation (deterioration in detection reliability) (step S13: Yes), the process proceeds to step S17.

<Step S14>
The control determination unit 63a (recognition unit 63) refers to a pattern table corresponding to the enabled driving support functions and the detection targets of the driving support functions in the storage unit 62. Then, the control determination unit 63a obtains the results of the detection process by the sensors belonging to each pattern defined in the pattern table. Then, the process proceeds to step S15.

<Step S15>
The control determination unit 63a performs sensor fusion to compare the positions of the targets obtained as the detection results of each pattern and the types of the identified targets. The control determination unit 63a determines the positions and types of the targets that match as the final recognition results. The control determination unit 63a then outputs the recognition results for the targets to the vehicle control unit 64 and the output control unit 65. Then, the process proceeds to step S16.

<Step S16>
The vehicle control unit 64 executes various driving assistance functions based on the recognition result by the control determination unit 63a, and controls the running of the vehicle 1 by outputting commands to the drive ECU 11, the steering ECU 12, the brake ECU 13, etc.

<Step S17>
In step S13, if it is determined that the sensor has a reduced function (step S13: Yes), the control determination unit 63a modifies the pattern table to be referenced. For example, if it is determined that the recognition accuracy of the second monocular camera 42b has decreased, the pattern table is modified to refer to a pattern including the second monocular camera 42b. Then, the process proceeds to step S18. Note that if it is determined that the recognition accuracy of multiple sensors has decreased, the pattern table is modified to refer to the patterns including each of the sensors in the same manner.

<Step S18>
The recognizable area calculation unit 63c acquires specification information and installation information corresponding to the sensor whose reliability has decreased from the sensor information storage unit 62b based on the detection result of the sensor recognition deterioration detection unit 63b, and calculates the recognizable object and the recognizable area whose detection result can be used even in a state where the reliability has decreased. Then, the process proceeds to step S19.

<Step S19>
The recognizable area integration unit 63d integrates the recognizable objects and recognizable areas for all sensors according to a pattern table corresponding to the driving assistance to be performed. That is, it determines the recognizable objects (targets) and recognizable areas even in a state where the detection reliability of the sensor is reduced. Then, the process proceeds to step S20.

<Step S20>
The control determination unit 63a notifies (notifies) the user (driver, etc.) of the recognizable objects and recognizable areas integrated by the recognizable area integration unit 63d, and controls intervention in driving assistance control. For example, it executes changes to the target range of driving assistance, changes to detectable targets, changes to the control amount and control timing of driving assistance, etc., and temporarily ends this flow.

(Effects of this embodiment)
As described above, the vehicle control device (driving control ECU 31) of the vehicle 1 according to this embodiment is a device that executes one or more driving support functions based on detection signals received from multiple types of sensors that detect targets around the vehicle 1. The recognition unit 63 includes a recognition degradation detection unit (sensor recognition degradation detection unit 63b) that detects a factor of degradation in the recognition function of the sensor based on the surrounding environment of the vehicle 1, an acquisition unit (recognizable area calculation unit 63c) that acquires detection results of at least one of targets detectable by a sensor whose recognition function is degraded and an area in which the target exists based on the degradation factor and sensor information, and a recognition unit (recognizable area integration unit 63d) that recognizes targets based on detection results by sensors included in each pattern by referring to pattern information that defines multiple patterns indicating combinations of sensors used to recognize targets corresponding to each driving support function. The recognition unit 63 also includes a sensor information storage unit 62b that stores sensor information including at least sensor specification information and installation information. The recognition unit 63 includes a vehicle control unit 64 that executes a driving support function corresponding to the pattern in which the recognition result is obtained based on the recognition result of the target by the recognition unit (recognizable area integration unit 63d). In this way, even if the surrounding environment (driving environment) of the vehicle 1 changes and the sensor's recognition function is temporarily reduced, the sensor's recognition status can be more appropriately grasped to achieve optimal driving assistance.

The recognition unit (recognizable area integration unit 63d) can also determine whether or not there is an area where the driving assistance function can be applied based on the target recognition results. As a result, it becomes possible to suppress erroneous determinations and erroneous recognitions in driving assistance.

The vehicle control unit 64 can limit the intervention of the driving assistance function in areas other than the applicable area. As a result, it is possible to reliably avoid malfunctions or erroneous control in driving assistance based on erroneous judgments or erroneous recognitions.

1 Vehicle 31 Driving control ECU
32 Camera ECU
33 Sonar ECU
34 Radar ECU
Description of the Reference Signs 41a Memory 42a First monocular camera 42b Second monocular camera 42c Third monocular camera 42d First stereo camera 42e Second stereo camera 43 Sonar sensor 44a First millimeter wave radar 44b Second millimeter wave radar 51 External device 61 Setting unit 62 Storage unit 62a Pattern table storage unit 62b Sensor information storage unit 63 Recognition unit 63a Control determination unit 63b Sensor recognition degradation detection unit 63c Recognizable area calculation unit 63d Recognizable area integration unit 64 Vehicle control unit 64a Variable processing unit 65 Output control unit

Claims (3)

A vehicle control device that executes one or more driving assistance functions based on detection signals received from a plurality of types of sensors that detect targets around a vehicle,
a recognition degradation detection unit that detects a cause of degradation of the recognition function of the sensor based on an environment surrounding the vehicle;
a sensor information storage unit that stores sensor information including at least specification information and installation information of the sensor;
an acquisition unit that acquires a detection result of at least one of the target detectable by the sensor having the deteriorated recognition function and an area in which the target exists, based on the cause of the deterioration and the sensor information;
a recognition unit that refers to pattern information defining a plurality of patterns indicating a combination of the sensors used to recognize the target, corresponding to each of the driving assistance functions, and recognizes the target based on the detection results by the sensors included in each of the patterns;
a vehicle control unit that executes the driving assistance function corresponding to the pattern for which the recognition result is obtained based on the recognition result of the target by the recognition unit;
A vehicle control device comprising: The vehicle control device according to claim 1, wherein the recognition unit determines whether or not the driving assistance function is applicable based on the recognition result of the target. The vehicle control device according to claim 2, wherein the vehicle control unit limits intervention of the driving assistance function in areas other than the applicable area.

Images