TWI559267B - Method of quantifying the reliability of obstacle classification - Google Patents

Description

Method for reliable measurement of obstacle classification

The invention relates to a method for quantifying the reliability of object classification, in particular to a method for reliably classifying obstacle classifications applied to a perceptual fusion system.

At present, the functions of the vehicle computer are becoming more and more perfect. In order to improve the safety of driving and the future of autonomous driving, the reliability of the detection and classification of the obstacles in front of the vehicle is particularly important. For various objects such as cars, pedestrians, bicycles, and utility poles, the classification items are determined according to the system settings. In this way, the system can determine whether to prompt the brakes, automatically brake, or perform other actions according to the classification of the obstacles.

FIG. 1 is a block diagram of detecting front obstacles and perceptual fusion in the prior art, the camera 10 captures the road image in front, and the plurality of sensors 11 and 12 can detect the distance of the obstacle in front or capture the vehicle. The body signal of the vehicle itself, and the obstacle distance detected by the plurality of sensors 11, 12 can obtain the obstacle height, the contour and the like, and then, based on the road image, the obstacle information and the body signal, on the one hand, Each of the obstacle information 13 is analyzed, and the position of the obstacle is calculated, and the classification information 15 is calculated. On the other hand, the detection confidence level 14 is estimated based on the road image, the obstacle information and the body signal, which are the front obstacles. Accuracy, and the detection of confidence fusion 16 , and finally output information 17, this output information 17 may include the probability of the obstacle does exist and the coordinate position, and the possible categories of obstacles. However, the system calculates whether the confidence of detection is correct, and there is no mechanism to reconfirm, and directly believes in the fusion result of detecting confidence.

Passing trust to detect confidence, if there is a misjudgment, it will have serious consequences. In the real case, as shown in Figure 2, the vehicle computer on vehicle 18a has a system for detecting obstacles in front and classifying warnings. The system maintains a safe distance from the preceding vehicle 18b, while the right lane drives a large tanker 18c. When the tanker 18c passes by the vehicle 18a, the millimeter wave reflected by the preceding vehicle 18b passes by the tanker. Diffuse occurred on the 18c, and the system of the vehicle 18a received a diffuse millimeter wave and made a judgment that "the possibility of collision with the preceding vehicle is high", and then an automatic emergency brake was made, and the rear vehicle 18d did not respond. A rear-end collision occurred. However, in fact, there is no vehicle in the vicinity of the front, but the tanker 18c is driving in the adjacent right lane. The system misjudges the noise as a vehicle, causing a false brake.

Therefore, how to make the detection confidence and classification confidence have higher accuracy, and to provide more reference for the fusion of information to avoid further misjudgment is an important issue at present. The invention proposes a method for reliably classifying obstacle classification, and the specific architecture and its implementation manner will be described in detail below:

The main object of the present invention is to provide a method for reliably classifying obstacle classification, which quantifies the reliability of obstacle classification information, improves classification accuracy, and avoids the security mechanism of the back-end active safety system due to mistaking information. Malfunction, system failure.

Another object of the present invention is to provide a method for reliably classifying obstacle classification, which combines information captured by a distance measuring sensor, an image capturing unit, and a body signal sensor to obtain all sensors. A location information, a classification information, and a classification confidence information for each obstacle.

A further object of the present invention is to provide a method for reliably classifying obstacle classification, and further comprising a classification failure filtering mechanism. If the quantitative obstacle classification reliability is less than a preset value, it represents a classification error, and the obstacle is skipped. On the contrary, if the reliability of the obstacle classification is greater than or equal to the preset value, the system will notify the driver, and if it is an automatic driving system, it can automatically brake.

In order to achieve the above object, the present invention provides a method for reliably classifying obstacles, which is applied to a sensory fusion system in a vehicle computer, which is connected to an image capturing unit and a body signal. Detector and complex ranging sensor, the method comprises the following steps: vehicle The computer receives at least one obstacle obstacle information, at least one image information corresponding to the obstacle information, and a plurality of body signals, and uses a classifier to classify the obstacle information, the image information, and the body signal; the sensing fusion system measures the distance The detection result of the sensor respectively calculates a detection confidence; reads the accuracy of the classifier, and uses the detection confidence and accuracy to calculate the classification confidence of each obstacle corresponding to each obstacle sensor; Convergence calculation is carried out for all classification confidences, and the reliability of obstacle classification for each obstacle is quantified by each ranging sensor; after any obstacle classification reliability is obtained, a classification failure is performed according to the obstacle classification reliability. The filtering mechanism eliminates obstacles whose obstacle classification reliability is less than a preset value.

The detection confidence is the probability that the obstacle detected by the ranging sensor is a physical object. The detection confidence system uses the ranging sensor to perform coordinate tracking on the obstacles currently detected, and then compares one of the actual values received with the tracking values to define the obstacles that are actually present in the current situation. Probability. Tracking, comparing, and defining the confidence of the obstacle is calculated using a Joint Integrated Probabilistic Data Association (JIPDA).

In the present invention, the fusion computing system calculates the reliability of the obstacle classification by using the classification confidence, the accuracy of the ranging sensor, and the probability of continuous detection of at least one obstacle, wherein the continuous detection probability of the obstacle is ranging sensing. The probability of the same obstacle being continuously detected.

According to the above, if there is no image to judge the obstacle classification, for example, only the radar detection information, the method for judging the obstacle continuous detection probability includes the following steps: suppose that it is determined whether the obstacle is a car, first receiving the distance measurement The information of the obstacle detected by the sensor; whether the obstacle in the information of the previous obstacle is the same, if not, the obstacle is not the car, and if so, whether the same obstacle is continuously detected If the same obstacle is detected continuously for more than a preset number of times, it is determined that the obstacle is a vehicle, and if not, the obstacle is determined not to be a vehicle.

When the calculated obstacle classification reliability is greater than or equal to the preset value, the vehicle computer will inform the driver of the obstacle information in front of the vehicle in an audible, tactile or visual manner, and the vehicle computer informs The driver's obstacle is the probability of a car or a pedestrian. At the same time, the perceptual fusion system will return to the previous step, take another obstacle classification reliability, and continue to judge whether it is less than the preset value.

The classification failure filtering mechanism in the steps of the present invention includes the following steps: capturing the reliability of the obstacle classification of any obstacle; determining whether the reliability of the obstacle classification obtained is less than a preset value, and if so, viewing the obstacle Misclassification and filtering for classification, and vice versa, the reliability of obstacle classification for another obstacle is judged again.

The fusion computing step in the step of the invention comprises the steps of: introducing, for a specific obstacle, all ranging sensors to detect the classification confidence, accuracy and the probability of continuous obstacle detection of a specific obstacle; according to the specific obstacle The presence or absence of the situation calculates the confidence of each ranging sensor; the specific obstacle includes four detection situations: empty set, presence, non-existence, possible existence or non-existence, and the confidence of the previous step is substituted. Calculate a fusion confidence based on the four detection conditions; and calculate the probability of existence of one of the specific obstacles after fusion based on the fusion confidence.

The purpose, technical content, features and effects achieved by the present invention will be more readily understood by the detailed description of the embodiments.

10‧‧‧ camera

11‧‧‧ Sensor

12‧‧‧ Sensors

13‧‧‧Resolve obstacle information

14‧‧‧ respective estimates of confidence

15‧‧‧Location, classification and integration information

16‧‧‧Detective Convergence Fusion

17‧‧‧ Output information

18a‧‧ Vehicles

18b‧‧‧Before the car

18c‧‧‧ tanker

18d‧‧‧ After the car

20‧‧‧Image capture unit

22‧‧‧Ranging sensor

24‧‧‧ Body Signal Sensor

26‧‧‧Car computer

27‧‧‧ classifier

28‧‧‧Perceptual Fusion System

Figure 1 is a block diagram of the detection of pre-vehicle obstacles and perceptual fusion in the prior art.

Figure 2 is a schematic diagram of misjudgment using methods of the prior art.

Figure 3 is a system architecture diagram of a method for reliably classifying obstacle classifications according to the present invention.

Figure 4 is a flow chart of a method for reliably classifying obstacle classifications according to the present invention.

Figure 5 is a flow chart for determining the probability of continuous detection of obstacles in the present invention.

Figure 6 is a flow chart of the convergence of confidence in the present invention.

Figure 7 is a flow chart of the classification failure filtering mechanism in the present invention.

The invention provides a method for reliably classifying obstacles. When each sensor detects an obstacle in front, it can track the position information through the obstacle, quantify the detection confidence of each sensor, and combine the senses. The classification confidence of the detector is combined to quantify the reliability of the obstacle classification, so that the system can perform classification error filtering to improve the accuracy and reliability of the overall classification.

FIG. 3 is a system architecture diagram of a method for reliably classifying an obstacle classification according to the present invention. The vehicle computer 26 includes a classifier 27 and a perceptual fusion system 28, and the vehicle computer 26 is connected to an image capture system. The unit 20, a body signal sensor 24 and a plurality of ranging sensors 22, the distance measuring sensor 22 is a radar or a laser radar (Lidar), which acquires obstacle information of at least one obstacle in front of the vehicle, and captures the image. The unit 20 can capture at least one image information corresponding to the obstacle information, and the body signal sensor 24 can obtain multiple body signals, including the vehicle speed, the steering wheel angle and the like.

Figure 4 is a flow chart of a method for reliably classifying obstacle classifications according to the present invention. Step S10: The vehicle computer receives at least one obstacle obstacle information, at least one image information corresponding to the obstacle information, and a plurality of body signals, and classifies the obstacle information, the image information, and the body signal by using a classifier. The device is a function module in the vehicle computer; and as described in step S12, the sensing fusion system calculates a detection confidence for the detection result of the ranging sensor, and the detection confidence represents the ranging sensing. The obstacles detected by the device are the probability of physical objects. Each ranging sensor will detect the confidence of the obstacles. If multiple obstacles are detected, multiple detection confidences will be obtained. In the present invention, the detection confidence is performed by the ranging sensor for coordinate tracking of the obstacle currently detected, that is, the position information of the obstacle is obtained, and then one of the actual value and the tracking value is subsequently received. The comparison, in order to define the probability that the tracked obstacle actually exists in the present, the present invention regards this probability as detecting confidence. Tracking, comparing, and defining the confidence of the obstacle is calculated using a Joint Integrated Probabilistic Data Association (JIPDA).

Next, step S14 reads the accuracy of the classifier, which is determined by the classifier developer.

Step S16 uses the detection confidence level and the classifier accuracy read in the previous step to calculate the classification confidence of each obstacle corresponding to each obstacle sensor, and the classification confidence of each ranging sensor Multiply the confidence of the classifier by the confidence factor.

In step S18, all the classification confidence levels are combined and calculated, and the reliability of the obstacle classification of each obstacle sensor for each obstacle is quantized. In this step, the perceptual fusion system first defines the detection of each obstacle, including four cases: , { },{ },{ , }},among them Representative empty set, { } represents the presence of obstacles, { } means that the obstacle does not exist, { , } represents that the obstacle may or may not exist. At the same time, three parameters are required to calculate the obstacle classification reliability of a specific obstacle. The three parameters include the confidence of each ranging sensor in the classification of a specific obstacle. Degree, accuracy of the distance measuring sensor and at least one obstacle continuous detection probability, wherein the accuracy of the distance measuring sensor is the accuracy provided by the manufacturer at the factory, and the accuracy of each ranging sensor In general, the accuracy is unlikely to reach 100%. If the user (driver) feels that the current accuracy is different from the initial value, for example, the accuracy is reduced, the accuracy can be manually adjusted, so the distance sensing is performed. The accuracy of the device is a preset preset value, and the obstacle continuous detection probability is the probability that the ranging sensor continuously detects the same obstacle. If the image capturing unit 20 is provided on the vehicle, the image can be directly taken from the image. To determine whether the obstacle is a car, the distance sensor is only for auxiliary judgment. However, if the vehicle does not have an image capturing unit to capture the image, it can only be judged entirely by the distance measuring sensor. Material company The detection probability is determined as shown in FIG. 5, and if it is determined whether the classification of the obstacle is a vehicle, the following steps are included: Step S30 receives the obstacle information detected by the ranging sensor; and Step S32 compares and Whether the obstacle in the previous obstacle information is the same, if not, the obstacle is not the car as described in step S34, and if so, it is determined in step S36 whether the same obstacle is continuously detected for more than a predetermined number of times. If the same obstacle is continuously detected for more than the preset number of times, it is determined that the obstacle is a vehicle as described in step S38, and if not, the obstacle is determined not to be a vehicle as described in step S34.

The fusion calculation in step S18 is performed by the (Dempster-Shafer) theory. For the fusion of all the information of a specific obstacle, please refer to the fusion calculation flowchart of FIG. 6, including the following steps: Step S182 introduces all ranging sensors to detect the classification confidence, accuracy and the specific obstacle. The obstacle continuously detects the probability; step S184 calculates a basic belief assignment according to the presence or absence of the specific obstacle; step S186 includes the empty set, presence, nonexistence, possible There are four detection situations, such as the presence or absence of the above, the confidence level of the step S184 is substituted, and a fusion confidence is calculated according to the four detection conditions; and the step S188 calculates the existence probability of the object of the specific obstacle after the fusion according to the fusion confidence degree. The probability of existence of this object is the reliability of the classification of the obstacle in the present invention.

The formula of the fusion calculation is as follows: First, step S182 introduces the probability of continuous detection of obstacles. , the accuracy of the distance sensor And the confidence confidence of each obstacle sensor corresponding to the obstacle p ⁱ ( x). Then, step S184 is performed to calculate the confidence of each ranging sensor, as shown in the following formulas (1) and (2):

Where m _i ({ }) the confidence level when the obstacle exists, m _i ({ }) is the confidence level when the obstacle does not exist. Here, only obstacles exist { } and does not exist { } two conditions, regardless of the empty set And obstacles may or may not exist { , The status of }.

Then, in step S186, the fusion confidence is calculated, and only the { }with{ } two sets, as shown in the following equation (3), where A={ }, B={ }:

Finally, according to the fusion confidence, the probability of existence of one of the specific obstacles after fusion is calculated, as shown in the following formula (4): Where Bel _F ({ }) In case of failure to determine the presence or absence of an obstacle, Pl _F ({ }) In consideration of the inability to determine the presence or absence of an obstacle, this part is the weighting of the probability of existence, and then takes the average of the various conditions, that is, the probability of existence of the object that obtains the obstacle, that is, the reliability of the obstacle classification.

The last step of FIG. 4 is as described in step S20, and a classification failure filtering mechanism is performed according to the obstacle classification reliability. When the obstacle classification reliability is less than a preset value, the detection or classification of the obstacle is unreliable. Therefore, the perceptual fusion system will exclude this obstacle. For the specific process, please refer to Figure 7. Step S202 first captures the obstacle classification reliability of any obstacle. The reliability of the obstacle classification is the fusion of all ranging sensors. And then, in step S204, it is determined whether the acquired obstacle classification reliability is less than a preset value, and if so, the obstacle is regarded as a classification misjudgment and filtered as in step S206; otherwise, the process returns to step S202 to capture The obstacle classification reliability of the other obstacle is judged again until all the obstacles detected in front of the vehicle have undergone the classification failure filtering mechanism. Step S202 further includes the system notifying the driver of obstacle information in front of the vehicle in an audible, tactile or visual manner, and the vehicle computer further informs the driver of the probability that the obstacle is a car or a pedestrian, and therefore, if the automatic assisted driving system is Can automatically brake.

In the above step S204, the preset value of the obstacle classification reliability can be adjusted by the driver. For example, if the driver starts the semi-automatic driving, the preset value needs to be increased, for example, more than 70% is needed to avoid When the preset value is lowered, most obstacles will be judged to be the car, and the vehicle-controlled semi-automatic driving system will continue to brake urgently; while the driver drives the car and only uses the obstacle classification reliability as an auxiliary reference, The preset value is lowered, for example, 30~50%. As a result, the sensing fusion system often sends a notice that the front obstacle is a car, should be decelerated or braked, but the driver can decide whether to slow down or brake.

For example, if the ranging sensor is a radar, the detection confidence is 0.9999, and the score is The classifier accuracy is 0.87, and its classification confidence is 0.9999*0.87=0.8699, while the image capture unit has a detection confidence of 0.94 and the classifier accuracy is 0.95. The classification confidence is 0.94*0.95=0.893. The reliability of the obstacle classification obtained after the fusion calculation is 0.895, and the reliability of any obstacle classification is judged to be greater than a preset value. If the preset value is 0.6, since 0.895 is greater than 0.6, the obstacle is judged as a vehicle. Established to inform the driver of this obstacle information.

Compared with the prior art, after the prior art obtains the confidence and classification of the obstacles, the system will act, but does not reconfirm the high degree of credibility of the classification. When the classification fails, it will lead to The noise is misjudged as a vehicle and the safety mechanism is erroneously activated. The emergency brake causes the rear vehicle to catch up. In contrast, the technique of the present invention first quantifies the probability that the distance measuring sensor detects the obstacle as a physical object (ie, Detecting confidence), then using the confidence of confidence and the accuracy of the classifier to calculate the confidence of the classification, and finally increasing the mechanism of the reliability of the classification of the quantitative obstacles, correcting the problem of the original classification failure, and improving the classification accuracy. To avoid misoperation of the back-end system, and even if there is no image, it can rely on radar, laser radar and other ranging sensors to classify obstacles. It will not lose its protection ability because there is no image, and greatly improve classification reliability and driving safety.

The above is only the preferred embodiment of the present invention and is not intended to limit the scope of the present invention. Therefore, any changes or modifications of the features and spirits of the present invention should be included in the scope of the present invention.

Claims (12)

A method for reliably classifying an obstacle classification, which is applied to a sensing fusion system of a vehicle computer, which is connected to an image capturing unit, a body signal sensor and a plurality of ranging sensors The method includes the following steps: the vehicle computer receives at least one obstacle obstacle information, at least one image information corresponding to the obstacle information, and a plurality of body signals, and uses a classifier to information about the obstacle and the image information. And classifying the body signals; the sensing fusion system respectively calculates a detection confidence for the detection results of the ranging sensors; and calculating the confidence by using the detection confidence and the accuracy of the classifier One of the ranging sensors classifies confidence corresponding to one of each of the obstacles; and performs fusion calculation on all of the class confidences to quantify all of the ranging sensors for each of the obstacles An obstacle classification reliability, the fusion calculation comprising the steps of: utilizing the classification confidence, the accuracy of the ranging sensors, and the at least one obstacle continuous detection probability calculation Calculating a confidence level according to the confidence of the distance measuring sensors; and calculating a probability of existence of an object of the specific obstacle after the fusion according to the fusion confidence; And performing a classification failure filtering mechanism according to the obstacle classification reliability, and removing the obstacle whose reliability is less than a preset value. The method for reliably classifying the obstacle classification according to claim 1, wherein the detecting confidence is performed by using the ranging sensors respectively for coordinate tracking of the obstacles currently detected, and then receiving the subsequent One of the actual values is compared with the complex tracking value to define the chasing The probability that the obstacle is actually present in the moment. A method for reliably classifying obstacle classification according to claim 2, wherein tracking, comparing, and defining the confidence of the obstacles are related using a common integrated probability data (Joint Integrated Probabilistic Data Association, JIPDA) ) Calculation. The method for reliably classifying obstacles according to claim 1, wherein the detection confidence is a probability that the obstacles detected by the ranging sensors are physical objects. The method for reliably classifying an obstacle classification according to claim 1, wherein the obstacle continuous detection probability is a probability that the ranging sensors continuously detect the same obstacle. The method for reliably classifying obstacles according to claim 1 or 5, wherein the method for judging the probability of continuous detection of the obstacle comprises the steps of: receiving the obstacle information detected by the ranging sensor; Comparing with the previous one of the obstacle information in the obstacle information is the same one, if not, determining that the obstacle is not a car, and if so, determining whether the same obstacle is continuously detected for more than a preset number of times; If it is continuously detected that the same obstacle exceeds the preset number of times, it is determined that the obstacle is a vehicle, and if not, it is determined that the obstacle is not a vehicle. The method for reliably classifying obstacle classification according to claim 1, wherein the accuracy of the ranging sensors is a preset value set in advance. The method for reliably classifying an obstacle classification according to claim 1, wherein the classification failure filtering mechanism comprises the following steps: capturing the reliability of the obstacle classification of any of the obstacles; and determining the obstacle to be taken Whether the classification reliability is less than the preset value; and if so, the obstacle is regarded as a misclassification and filtering, and vice versa, the obstacle classification reliability of the other obstacle is again determined. The method for reliably classifying an obstacle classification according to claim 1, wherein in the classification failure filtering mechanism, when the obstacle classification reliability is greater than or equal to the preset value, the vehicle is notified in an audible, tactile or visual manner. The obstacle information in front of the driver. The method for reliably classifying an obstacle classification according to claim 9, wherein in the classification failure filtering mechanism, when the obstacle classification reliability is greater than or equal to the preset value, the vehicle computer further informs the driver of the obstacle The probability of being a car or a pedestrian. The method for reliably classifying obstacle classification according to claim 1, wherein the ranging sensors are radar or laser radar (Lidar). The method for reliably classifying obstacle classification according to claim 1, wherein the distance sensing is calculated by using the classification confidence, the accuracy of the ranging sensors, and the probability of continuous detection of the obstacle. The step of confidence is a specific obstacle for one of the obstacles, including the empty detection, presence, non-existence, possible existence or non-existence, and the like. The confidence of the ranging sensor is substituted, and the fusion confidence is calculated according to the four detection conditions.