JP2019158662A - Target recognition device and vehicle driving control system - Google Patents

Abstract

To suppress spray from being recognized as a target in target recognition processing of recognizing a target around a vehicle.SOLUTION: Both a lidar and a camera are used in target recognition processing. A first target is detected based on a lidar measurement result, and a second target is detected based on a camera photographing image. When the first target and the second target are capable of being fused, a determination is made whether an integrated target is spray or not. A target frame is a frame of the first target projected on the camera photographing image. If the integrated target is the spray, the image within the target frame is supposed to have features specific to the spray. Then, an amount of feature relevant to the target frame is computed based on a camera photographing result. When the amount of feature is equal to or more than a threshold, it is determined that the integrated target is the spray, and the integrated target is excluded from the target.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a target recognition device and a vehicle travel control system mounted on a vehicle.

  Patent Document 1 discloses an obstacle detection device for a vehicle using a laser radar. The obstacle detection device includes a roll-up detection means for detecting the water roll-up by the vehicle tire. The hoisting detection means detects the hoisting of moisture by the tire based on the laser light intensity, the wiper operating speed, the increase in rainfall, and the like. Then, the obstacle detection device controls the output of the laser radar based on the output signal of the hoisting detection means.

JP-A-7-198851

  Consider “target recognition processing” for recognizing targets around a vehicle. The result of the target recognition process can be used for vehicle travel control such as automatic driving control. Therefore, it is desired to improve the accuracy of the target recognition process.

  As one factor that lowers the target recognition accuracy, it is conceivable that the water splash is generated as the vehicle travels. For example, when the vehicle travels when it is raining or when water is accumulated on the road surface, water is taken up by the tires and splashes occur. Such splashing may be erroneously recognized as a target around the vehicle.

  According to the technique disclosed in Patent Document 1 above, for example, splashing is determined based on the laser light intensity. However, in that case, it is not possible to distinguish between a rear vehicle that does not have a reflector and a splash. As another example, splashing is determined based on an increase in wiper operating speed or rainfall. However, even if it is not raining, if water accumulates on the road surface, splashing occurs. Therefore, even if reference is made to the increase in the wiper operating speed and the rainfall, it is not always possible to correctly determine the splashing.

  One object of the present invention is to provide a technique capable of suppressing that a splash is erroneously recognized as a target in a target recognition process for recognizing a target around a vehicle.

In one aspect of the present invention, a target recognition device mounted on a vehicle is provided.
The target recognition device
With the rider,
A camera whose field of view overlaps with the rider;
And a control device that performs target recognition processing.
The target recognition process
A first target detection process for detecting a first target based on a measurement result by the rider;
A second target detection process for detecting a second target based on an image captured by the camera;
A fusion process for integrating the first target and the second target as an integrated target;
A splash determination process for determining whether or not the integrated target is a splash generated as the vehicle travels;
And an exclusion process for excluding the integrated target from the targets around the vehicle when the integrated target is determined to be the splash.
A target frame is a frame representing the first target projected on the image.
The target area is the area of the target frame.
The flow amount is the size of the optical flow of the feature point of the image in the target frame.
The first feature amount is a ratio of the average luminance of the image in the target frame to the average luminance of the image outside the target frame.
The second feature amount is a degree of increase in the target area with respect to an increase in the speed of the vehicle.
The third feature amount is a degree of increase in the flow amount with respect to an increase in the speed of the vehicle.
The conditions for determining the integrated target as the splash in the splash determination process are that the first feature value is equal to or greater than a first threshold, the second feature value is equal to or greater than a second threshold, and the third It includes at least one of the feature amount being equal to or greater than the third threshold value.

  According to the present invention, both the rider and the camera are used for the target recognition process. The first target is detected based on the measurement result by the rider, and the second target is detected based on the image captured by the camera. When the first target and the second target can be fused, it can be accurately determined whether or not the integrated target is splashed.

  Specifically, the target frame and the feature amount are considered in the splash determination process. The target frame is a frame of the first target projected on the image captured by the camera. If the integrated target is a splash, the image in the target frame should have characteristics specific to the splash. Therefore, the feature quantity related to the target frame is calculated based on the imaging result by the camera. And when a feature-value is more than a threshold value, it determines with an integrated target being a splash.

  With such a method, it is possible to accurately determine the splashing generated as the vehicle travels. When it is determined that the integrated target is splash, the integrated target is excluded from the targets around the vehicle. Thereby, it is suppressed that a splash is misrecognized as a target. That is, the accuracy of the target recognition process is improved.

It is a conceptual diagram for demonstrating the outline | summary of embodiment of this invention. It is a block diagram which shows the structural example of the target recognition apparatus which concerns on embodiment of this invention. It is a flowchart which shows the target recognition process which concerns on embodiment of this invention. It is a conceptual diagram for demonstrating the splash determination process in the target recognition process which concerns on embodiment of this invention. It is a block diagram which shows the structural example of the vehicle travel control system which concerns on embodiment of this invention.

  Embodiments of the present invention will be described with reference to the accompanying drawings.

1. Overview FIG. 1 is a conceptual diagram for explaining an overview of the present embodiment. The vehicle 1 has a target recognition function for recognizing surrounding targets. A typical example of a target around the vehicle 1 is the surrounding vehicle 2. FIG. 1 illustrates a preceding vehicle 2A, a following vehicle 2B, and a parallel running vehicle 2C.

  When the vehicle 1 travels when it is raining or when water is accumulated on the road surface, water is wound up by the tires and splashes occur. Such splashing may be erroneously recognized as a target around the vehicle 1. For example, the splash may be erroneously recognized as the following vehicle 2B or the parallel running vehicle 2C.

  In this way, the splashing that occurs as the vehicle 1 travels becomes a factor that lowers the target recognition accuracy. Therefore, the present embodiment provides a technique that can improve the determination accuracy of splashing and suppress the erroneous recognition of the splashing as a target.

2. Target Recognition Device FIG. 2 is a block diagram illustrating a configuration example of the target recognition device 10 according to the present embodiment. The target recognition device 10 is mounted on the vehicle 1 and performs target recognition processing for recognizing targets around the vehicle 1. The target recognition device 10 includes a rider (LIDAR: Laser Imaging Detection and Ranging) 20, a camera 30, a vehicle state sensor 40, and a control device 50.

  The rider 20 measures the position (distance and direction) of the reflection point using the laser pulse. More specifically, the rider 20 sequentially outputs (scans) laser pulses in a plurality of directions. When the laser pulse is reflected at the reflection point, the reflected light of the laser pulse returns to the rider 20. The rider 20 receives the reflected light of the laser pulse. The distance and direction of the reflection point can be calculated from the light reception state of the reflected light. The measurement result by the rider 20 is sent to the control device 50.

  The camera 30 images the surroundings of the vehicle 1. The field of view of the camera 30 at least partially overlaps the field of view (measurement range) of the rider 20. Image information captured by the camera 30 is sent to the control device 50.

  In addition, the installation position and the number of riders 20 and cameras 30 are not particularly limited. For example, the rider 20 and the camera 30 are installed on the front surface, the side surface, and the rear surface of the vehicle 1.

  The vehicle state sensor 40 detects the state of the vehicle 1. For example, the vehicle state sensor 40 includes a wheel speed sensor, a vehicle speed sensor, and the like. The wheel speed sensor detects the rotational speed of each wheel of the vehicle 1. The vehicle speed sensor detects the speed of the vehicle 1. The vehicle state information detected by the vehicle state sensor 40 is sent to the control device 50.

  The control device 50 is a microcomputer including a processor and a storage device. The control device 50 is also called an ECU (Electronic Control Unit). When the processor executes the control program stored in the storage device, the processing by the control device 50 is realized. The processing by the control device 50 includes target recognition processing for recognizing targets around the vehicle 1. Hereinafter, the target recognition process by the control apparatus 50 is demonstrated.

3. Target Recognition Processing FIG. 3 is a flowchart showing the target recognition processing according to the present embodiment. The flow shown in FIG. 3 is repeatedly executed every fixed cycle.

3-1. Step S100 (target detection process)
The control device 50 detects a target around the vehicle 1. Specifically, the control device 50 detects the target based on the measurement result by the rider 20 (step S110). The target detected based on the measurement result by the rider 20 is hereinafter referred to as “first target T1”. The control device 50 acquires detection information (distance, direction, size, etc.) of the first target T1. Further, the control device 50 may calculate the relative speed of the first target T1.

  Moreover, the control apparatus 50 detects the target around the vehicle 1 by analyzing the image imaged with the camera 30 (step S120). The target detected based on the image picked up by the camera 30 is hereinafter referred to as “second target T2”. The control device 50 acquires detection information (distance, direction, size, etc.) of the second target T2. Further, the control device 50 may calculate the relative speed of the second target T2.

3-2. Step S200 (fusion process)
When the first target T1 and the second target T2 satisfy predetermined conditions, the two targets can be recognized as the same target. Such a process is called “fusion”. The predetermined condition includes, for example, that the position difference between the first target T1 and the second target T2 is within an error range. The predetermined condition may further include that the difference in relative speed between the first target T1 and the second target T2 is within an error range.

  The control device 50 determines whether or not a predetermined condition is satisfied, that is, whether or not fusion is possible (step S210). When the fusion is possible (step S210; Yes), the control device 50 integrates the first target T1 and the second target T2 as the integrated target TF (step S220). Thereafter, the process proceeds to step S300.

  On the other hand, when fusion is not possible (step S210; No), the control device 50 recognizes each of the first target T1 and the second target T2 as a single target (step S230). Thereafter, the process proceeds to step S500.

3-3. Step S300 (spray determination process)
The control device 50 determines whether or not the integrated target TF is splashed as the vehicle 1 travels. When it is determined that the integrated target TF is splashing (step S300; Yes), the process proceeds to step S400. In other cases (step S300; No), the process proceeds to step S500. In the example shown in FIG. 3, step S300 includes the following steps S310 to S330.

3-3-1. Step S310
FIG. 4 is a conceptual diagram for explaining step S310. FIG. 4 shows an image captured by the camera 30. In the example shown in FIG. 4, splashing occurs as the vehicle 1 travels, and the image includes splashing. In step S100 (target detection process), the splash (collection of water droplets) is detected as one target by a known clustering process.

  In step S <b> 310, the control device 50 projects the first target T <b> 1 on the image captured by the camera 30. The installation positions of the rider 20 and the camera 30 and the camera parameters of the camera 30 are known. Based on such known parameters, the first target T1 can be projected on the image captured by the camera 30. A “target frame FR” in FIG. 4 is a frame representing the first target T1 projected on the image.

3-3-2. Step S320
If the integrated target TF is a splash, the image in the target frame FR should have a characteristic characteristic of the splash. Therefore, the control device 50 calculates the “feature amount C” related to the target frame FR based on the imaging result of the camera 30. Hereinafter, some examples of the feature amount C will be described.

<First feature amount C1>
If the integrated target TF is splashed, the area inside the target frame FR is considered brighter than the area outside the target frame FR. From this point of view, “average image brightness” is used as the first feature amount C1. For example, the first feature amount C1 is defined as follows.

  First feature amount C1 = (average luminance of the image inside the target frame FR) / (average luminance of the image outside the target frame FR)

  It can be said that the larger the first feature amount C1, the higher the probability that the integrated target TF is splashing.

<Second feature amount C2>
As the vehicle speed (the speed of the vehicle 1) increases, the amount and momentum of splashing increases. Therefore, if the integrated target TF is splashed, it is considered that the target frame FR increases as the vehicle speed increases. On the other hand, if the integrated target TF is the succeeding vehicle 2B, the target frame FR should be reduced because the inter-vehicle distance increases as the vehicle speed increases. From such a viewpoint, the “target area” that is the area of the target frame FR is used as the second feature amount C2. For example, the second feature amount C2 is defined as follows.

  Second feature amount C2 = (target area @ Vmax−target area @ Vmin) / (Vmax−Vmin)

  Here, Vmax and Vmin are the maximum value and the minimum value of the vehicle speed in a certain period, respectively, and are detected by the vehicle state sensor 40. It can also be said that the second feature amount C2 is the degree of increase in the target area with respect to the increase in the vehicle speed. The larger the second feature amount C2, the higher the probability that the integrated target TF is splashing.

<Third feature amount C3>
Instead of the target area, attention may be paid to the optical flow of the feature points of the image in the target frame FR. The size of the optical flow of the feature points of the image within the target frame FR is hereinafter referred to as “flow amount”. If the integrated target TF is splashed, the flow amount is considered to increase as the vehicle speed increases. From such a viewpoint, the third feature amount C3 is defined as follows, for example.

  Third feature amount C3 = (flow amount @ Vmax−flow amount @ Vmin) / (Vmax−Vmin)

  Here, Vmax and Vmin are the maximum value and the minimum value of the vehicle speed in a certain period, respectively, and are detected by the vehicle state sensor 40. It can also be said that the third feature amount C3 is the degree of increase in the flow amount with respect to the increase in the vehicle speed. The larger the third feature amount C3 is, the higher the probability that the integrated target TF is splashing.

3-3-3. Step S330
The control device 50 determines whether the determination condition is satisfied. The determination condition includes at least one of the following first to third conditions.
<First condition> The first feature amount C1 is equal to or greater than a first threshold value.
<Second condition> The second feature amount C2 is equal to or greater than the second threshold value.
<Third condition> The third feature amount C3 is equal to or greater than a third threshold value.
The determination condition may be that all of the first to third conditions are satisfied. Alternatively, the determination condition may be that some of the first to third conditions are satisfied.

  When the determination condition is satisfied (step S330; Yes), the control device 50 determines that the integrated target TF is splashing (step S300; Yes). In this case, the process proceeds to step S400. On the other hand, when the determination condition is not satisfied (step S330; No), the control device 50 determines that the integrated target TF is not sprayed (step S300; No). In this case, the process proceeds to step S500.

3-4. Step S400 (exclusion process)
The control device 50 excludes the integrated target TF from the targets around the vehicle 1.

3-5. Step S500 (target information output process)
The control device 50 outputs information (position, direction, size, etc.) regarding the recognized target. The recognized targets are a single target (first target T1, second target T2) and an integrated target TF. In the case of the integrated target TF, the front / rear position of the first target T1 may be used as the front / rear position, and the lateral position of the second target T2 may be used as the lateral position.

4). Effect As described above, according to the present embodiment, both the rider 20 and the camera 30 are used for the target recognition process. The first target T1 is detected based on the measurement result by the rider 20, and the second target T2 is detected based on the image captured by the camera 30. When the first target T1 and the second target T2 can be fused, it is possible to accurately determine whether or not the integrated target TF is splashed.

  Specifically, the target frame FR and the feature amount C are considered in the splash determination process. The target frame FR is a frame of the first target T1 projected on the image captured by the camera 30. If the integrated target TF is a splash, the image in the target frame FR should have a characteristic characteristic of the splash. Therefore, the feature amount C related to the target frame FR is calculated based on the imaging result by the camera 30. And when the feature-value C is more than a threshold value, it determines with the integrated target TF being a splash.

  With such a method, it is possible to accurately determine the splashes generated as the vehicle 1 travels. When it is determined that the integrated target TF is splashing, the integrated target TF is excluded from the targets around the vehicle 1. Thereby, it is suppressed that a splash is misrecognized as a target. That is, the accuracy of the target recognition process is improved.

5). Vehicle Travel Control System The result of the target recognition process is used for vehicle travel control. For example, the result of the target recognition process is used for automatic operation control. In the automatic driving control, for example, whether or not the lane change can be performed is determined based on the recognition result of the surrounding vehicle 2 (see FIG. 1). Therefore, increasing the accuracy of the target recognition process is important in automatic operation control. As another example, the result of the target recognition process may be used for driving support control such as ACC (Adaptive Cruise Control) or LTA (Lane Tracing Assist).

  FIG. 5 is a block diagram illustrating a configuration example of the vehicle travel control system 100 according to the present embodiment. The vehicle travel control system 100 is mounted on the vehicle 1 and performs vehicle travel control for controlling the travel of the vehicle 1.

  More specifically, the vehicle travel control system 100 includes a travel device 60 in addition to the target recognition device 10 described above. The traveling device 60 includes a steering device, a driving device, and a braking device. The steering device steers the wheels. The driving device is a power source that generates a driving force. Examples of the driving device include an electric motor and an engine. The braking device generates a braking force.

  The control device 50 performs vehicle travel control based on the result of the target recognition process performed by the target recognition device 10. The vehicle travel control includes steering control and acceleration / deceleration control. The control device 50 performs steering control by appropriately operating the steering device. In addition, the control device 50 performs acceleration / deceleration control by appropriately operating the drive device and the braking device. It can be said that the control device 50 and the traveling device 60 constitute a “vehicle traveling control device 70” that performs vehicle traveling control.

  According to the present embodiment, the accuracy of the target recognition process by the target recognition device 10 is improved. Therefore, the accuracy of automatic driving control and driving support control using the result of the target recognition process is also improved. This contributes to the improvement of the driver's reliability with respect to the automatic driving control and driving support control.

DESCRIPTION OF SYMBOLS 1 Vehicle 2 Peripheral vehicle 10 Target recognition apparatus 20 Rider 30 Camera 40 Vehicle state sensor 50 Control apparatus 60 Travel apparatus 70 Vehicle travel control apparatus 100 Vehicle travel control system

Claims (1)

A target recognition device mounted on a vehicle,
With the rider,
A camera whose field of view overlaps with the rider;
A control device for performing target recognition processing,
The target recognition process
A first target detection process for detecting a first target based on a measurement result by the rider;
A second target detection process for detecting a second target based on an image captured by the camera;
A fusion process for integrating the first target and the second target as an integrated target;
A splash determination process for determining whether or not the integrated target is a splash generated as the vehicle travels;
An exclusion process for excluding the integrated target from the targets around the vehicle when the integrated target is determined to be the splash;
A target frame is a frame representing the first target projected on the image,
The target area is the area of the target frame,
The flow amount is the size of the optical flow of the feature point of the image in the target frame,
The first feature amount is a ratio of the average luminance of the image in the target frame to the average luminance of the image outside the target frame,
The second feature amount is a degree of increase in the target area with respect to an increase in the speed of the vehicle,
The third feature amount is a degree of increase in the flow amount with respect to an increase in the speed of the vehicle,
The conditions for determining the integrated target as the splash in the splash determination process are that the first feature value is equal to or greater than a first threshold, the second feature value is equal to or greater than a second threshold, and the third A target recognition apparatus including at least one of the feature amount being equal to or greater than a third threshold value.

Images