EP1436640A2

EP1436640A2 - Object detecting device

Info

Publication number: EP1436640A2
Application number: EP02774378A
Authority: EP
Inventors: Goetz Braeuchle; Martin Heinebrodt; Juergen Boecker
Original assignee: Robert Bosch GmbH
Current assignee: Robert Bosch GmbH
Priority date: 2001-10-05
Filing date: 2002-09-17
Publication date: 2004-07-14
Also published as: DE10149115A1; WO2003031228A3; US7012560B2; JP2005505074A; US20050062615A1; WO2003031228A2

Abstract

The invention concerns a device for detecting objects designed for a driving aid system in motor vehicles. The inventive object detecting device comprises at least two sensor systems (13; 14; 16L, 16R) which measure data (d1, j1; d2, j2; d3, y3) concerning the location or the displacement state of the objects (24) in the neighbourhood of the vehicle (10) and whereof the detection zones (18, 20) are mutually overlapping. The inventive object detecting device is characterized in that it comprises an error detection device which controls the coherence of the data measured by the sensor systems(12; 14; 16L, 16R) and emits an error signal when it identifies an incoherence.

Description

Whether it is a data acquisition device

L O

The invention relates to an object detection device for driver assistance systems in motor vehicles, with at least two sensor systems that measure data about the location and / or state of motion of objects in the vicinity of vehicle L5 and whose detection areas overlap one another.

Motor vehicles are increasingly being equipped with driver assistance systems that support and relieve the driver of driving the vehicle. An example of one

20 Assistance system is a so-called ACC system (Adaptive Cruise Control), which automatically regulates the speed of the vehicle to a desired speed selected by the driver or, if a vehicle in front is present, adjusts the speed so that a suitable one, with help

25 of a distance sensor monitored distance to the vehicle in front is observed. Other examples of driver assistance systems are collision warning devices, automatic lane guidance systems (LKS; Lane Keeping System) that recognize lane markings and the vehicle through

30 Automatically keep intervention in the steering in the middle of the lane, sensor-assisted parking aids and the like. All of these assistance systems require a sensor system with which information about the surroundings of the vehicle can be recorded, as well as evaluation units with which this information

35 can be evaluated and interpreted appropriately. In particular, these devices must be able to detect objects in the vicinity of the vehicle, for example other vehicles and other obstacles, and to record data which identify the location and, if appropriate, the state of motion of these objects. The sensor systems and the associated evaluation units should therefore be referred to collectively as an object detection device.

Examples of sensor systems used in such object detection devices are

Radar systems and their light-optical counterparts, so-called lidar systems, as well as stereo camera systems. With radar systems, the distance of the object along the line of sight can be measured by evaluating the transit time of the radar echo. By evaluating the Doppler shift of the radar echo, the relative speed of the object on the line of sight can also be measured directly. With a direction-sensitive radar system, for example a multi-beam radar, it is also possible ^'direction data to be recorded over the objects, for example, the azimuth angle relative to a through the adjustment of the

Radar sensor defined reference axis. With stereo camera systems, directional data and, by evaluating the parallax, distance data can also be obtained. By evaluating the raw data measured directly by these sensor systems, data can be calculated which indicate the distance of the object in the direction of travel and the transverse offset of the object relative to the center of the lane or to the current straight-ahead direction of the vehicle.

Since all these known sensor systems have their particular strengths and weaknesses in the acquisition of the required measurement data, it is expedient to use several sensor systems which complement one another.

In the case of ACC systems, it is generally known that the measured To subject raw data to a plausibility analysis in order to decide or at least indicate probabilities as to whether the detected object is a relevant obstacle or an irrelevant object, for example a traffic sign on the edge of the road. Under certain circumstances, the implausibility of the recorded data can also be an indication of a defect in the sensor system.

Generally, however, it is with the known ones

Object detection devices not possible to reliably detect misalignments or other defects in the sensor systems that impair the functionality of the assistance system.

The object of the invention is therefore a

To create object detection device with which it is possible to detect defects in the sensor systems more precisely and reliably during operation and thus to improve the functional reliability of the assistant system.

This object is achieved according to the invention by an error detection device which checks the data measured by the sensor systems for their freedom from contradictions and outputs an error signal when a contradiction is detected.

The invention is based on the consideration that, in the case of several sensor systems with overlapping detection areas, it repeatedly occurs that objects are located in the overlapping area. In this case, the sensor systems, which operate independently of one another, provide redundant information which enables error detection during the running operation of the device. If the sensor systems involved work correctly, the information they provide must be compatible with one another within certain error limits. If not, then if so If data contradict each other, it can be concluded that at least one of the sensor systems involved is defective and an error signal is output. This error signal can be used in the simplest case, alert the driver through a visual or audible indication of the malfunction and possibly a self-shutdown of ^• assistance system trigger. According to a development of the invention, however, an automatic error correction can also be carried out with the aid of this error signal.

The invention thus enables continuous self-diagnosis of the object detection device during normal driving operation and thus a substantial improvement in traffic safety of the assistance system that uses the data of the object detection device.

Advantageous refinements of the invention result from the subclaims.

In a preferred embodiment, the object detection device has, in addition to a sensor system for the long-range, which is formed, for example, by a 77 GHz radar system or a lidar system, a sensor system for the short-range, which has a shorter range but detects a larger angular range, so that blind spots largely avoided at close range. The sensor system for the close range can also be formed by a radar system or by a lidar system or also by a video sensor system, for example a stereo camera system with two electronic cameras.

In a modified embodiment, there can be three mutually independent sensor systems whose detection areas have a common overlap area. In this case, there are objects in the common overlap area, in addition to error detection, there is a simple possibility of "majority decision" to identify the faulty sensor system and, if necessary, to correct the data, the adjustment or the calibration of the faulty system.

In certain circumstances, even in embodiments with only two sensor systems, automatic identification of the faulty system and automatic

L0 error correction possible, especially with a

Plausibility evaluation taking into account the particular characteristics of the physical measuring principles used in the sensor systems involved. For example, using radar and lidar systems is a relatively accurate one

L5 distance measurement possible, while the distance measurement with the aid of a stereo camera system, particularly with larger distances, is subject to greater error tolerances and critically depends on the camera adjustment. In the event of a discrepancy, there is therefore a high probability of an error in the

20 stereo camera system. Conversely, a video system allows a relatively precise measurement of the transverse offset of a vehicle in front, while the transverse offset measurement with the aid of a radar or lidar system is critically dependent on the adjustment of the radar or lidar sensor. In this case, a discrepancy therefore speaks for a defect in the radar or lidar system.

In practice, the area in which the detection areas of the sensor systems overlap will be one

30 area that is special for the assistance system

Relevance is. For example, in the case of an ACC system, the sensor systems for the short and long range will preferably be designed such that they overlap in the distance range which corresponds to the typical safety distance from a vehicle traveling in front. In this case, achieve an automatic error correction or an improvement of the measuring accuracy also in that the data supplied by the various sensor systems are weighted according to their respective reliability and then combined to a final result.

According to a development of the invention, it is expedient to store the error signals supplied by the error detection device together with the associated, contradicting L0 measurement data and thus to generate error statistics which facilitate the diagnosis when repairing or maintaining the object detection device.

Exemplary embodiments of the invention are explained in more detail below with reference to L5 of the drawing.

Show it:

Fig. 1 is a schematic representation of the detection areas of several sensor systems that are on a motor vehicle

20 are installed;

Fig. 2 is a block diagram of an object detection device according to the invention; and FIG. 3 shows a sketch to explain the consequences of error adjustments of different sensor systems.

25

FIG. 1 schematically shows a top view of the front part of a motor vehicle 10 which is equipped with three sensor systems working independently of one another, namely a long-range radar 12, a short-range radar 14 and one

SO video system formed by two cameras 16L and 16R. The long-range radar 12 has a detection range 18 with a range of, for example, 150 m and a detection angle of 15 °, while the short-range radar 14 has a detection range 20 with a range of, for example, 50

15 m and a detection angle of 40 °. Between these Detection areas 18, 20, which are not shown to scale in the drawing, there is an overlap area. 22. The detection area of the video system formed by the cameras 16L, 16R, which in summary with the

5 denotes reference numeral 16. is not shown in the drawing, but includes the overlap area 22 (in good visibility conditions). An object 24, which is located in this overlap region 22, can therefore be detected by all three sensor systems.

0

The detection area 18 of the long-range radar 12 is symmetrical with respect to a reference axis 18A, which - with correct adjustment of the radar sensor - runs parallel to a main axis H which runs in the longitudinal direction through the center of the vehicle 10

.5 runs. The detection area 20 of the

Short-range radar 14 symmetrical to a reference axis 20A, which is also parallel to the main axis H and to the reference axis 18A.

! 0 The long-range radar 12 measures the distance dl to the object 24 and the relative speed of the object 24 relative to the vehicle 10 and the azimuth angle jl of the object 24 relative to the reference axis 18A. Correspondingly, the short-range radar 14 measures the distance d2 from the object 24, the relative speed of the

25 object 24 along the line of sight from the radar sensor to the object and the azimuth angle j2 of the object 24 relative to the reference axis 20A.

In video system 16, the images of object. 24 recorded by cameras 16L, 16R 0 are electronically evaluated. The known as such evaluation software of such stereo camera systems is able to identify the object 24 in the images recorded by both cameras and, based on the parallactic shift, the position of the object 24 in a two-dimensional coordinate system (parallel to the Road level). In this way, the video system 16 provides the vertical distance d3 of the object 24 from the vehicle 10 (ie, from the base line of the cameras 16L, 16R) and the transverse offset y3 of the object 24 with respect to the 5 main axis H.

The location coordinates of the object 24 can thus be determined with the aid of the three sensor systems 12, 14, 16 in three mutually independent ways. The radar systems

L0 measured polar coordinates can be determined by a simple

Convert coordinate transformation to Cartesian coordinates, as in the example shown by the pair of coordinates (d3, y3). The three sets of coordinates measured independently of one another can now be compared with one another, and

L5 if these coordinates contradict each other, this indicates that one of the three sensor systems is malfunctioning. The faulty system can also be identified on the basis of the different coordinate set.

The relative speed 24 of the object can also be determined by temporally deriving the distance d3 measured with the video system 16. Since the visual rays from the radar sensors to the object 24, along which the relative speeds are measured with the aid of the Doppler effect, are not exactly parallel to the

25 are the main axis H, the three measured

Relative speeds differ slightly from each other. However, this deviation is usually negligible in the distance ratios that occur in practice. If necessary, it can be converted to Cartesian

> 0 coordinates are corrected so that the measured speed data can also be compared and compared.

Figure 2 shows a block diagram of a 5 object detection device, the long-range radar 12, the Short-range radar 14, video system 16 and associated evaluation units 26, 28, 30 and also one

Error detection device 32 and a correction device 34 comprises. The evaluation units 26, 28, 30, the 5 error detection device 32 and the correction device 34 can be formed by electronic circuits, by microcomputers or also by software modules in a single microcomputer.

L0 The evaluation unit 26 determines from the raw data supplied by the long-range radar 12 the distances dli, the relative speeds vli and the azimuth angles jli of all objects that are located in the detection area 18 of the long-range radar 12. The index i is used here

L5 Identification of the individual objects. The evaluation unit 26 also calculates the transverse offsets yli of the various objects from the distance data and azimuth angles.

In an analogous manner, the evaluation unit 28 determines the distances 20 d2i, the relative speed v2i, the azimuth angles j2i and the transverse offsets y2i of all objects that are located in the detection area 20 of the short-range radar 14.

The evaluation unit 30 first determines the azimuth angles jLi 25 and jRi of the objects detected by the cameras 16L, 16R. These azimuth angles are defined analogously to the azimuth angles j1 and j2 in FIG. 1, that is, they indicate the angle between the respective line of sight to the object and a straight line parallel to the main axis H. From the azimuth angles jLi and jRi, the distances d3i and the transverse offsets y3i and - by time derivation of the distance data - the relative speeds v3i are calculated on the basis of the known distance between cameras 16L and 16R.

15 determined by the three evaluation units 26, ^'28, 30 Distances dli, d2i and d3i are fed to a distance module 36 of the error detection device 32. Correspondingly, the relative speed data vli, v2i and v3i are fed to a speed module 38 and the transverse offset data yli, y2i and y3i to a transverse offset module 40. An angle module 42 of the error detection device 32 evaluates the azimuth angles jli, j2i, jLi and jRi.

The various modules of the error detection device 32 are connected to one another and have access to all of the data that are fed to the error detection device 28 from any of the evaluation units. The data connections shown in the drawing only relate to the data whose processing is in the foreground in the module concerned.

When the evaluation unit 26 reports the distance dil of a detected object (with the index i) to the distance module 36, the distance module 36 first checks using the associated transverse offset yli whether the object in question is also in the detection area 20 of the short-range radar 14 and / or in Detection area of the video system 16 is located. If this is the case, the distance module checks whether data for this object are also available from the evaluation units 28, 30. The identification of the objects is facilitated in that the distance module 36 can track the change in the distance data over time. If, for example, an object is initially only detected by the long-range radar 12 and then enters the detection range 20 of the short-range radar 14, it can be expected that the evaluation unit 28 reports the occurrence of a new object, which can then be identified with the tracked object. In order to eliminate ambiguities, the criterion can also be used that the location coordinates transmitted by the various evaluation units for the same object must at least roughly match. If distance data for the same object are available from several sensor systems, the distance module 36 checks whether these distance data match within the respective error limits 5. It should be borne in mind that the

The limits of error are variable. For example, the cross offset data yli are relatively imprecise at a large object distance, because the long-range radar 12 has only a limited angular resolution and even slight deviations in the

L0 measured azimuth angle lead to a considerable deviation of the associated cross offset. If the distance data match within the error limits, the matching value di is transmitted to a downstream assistance system 44, for example an ACC system.

L5 The output value di can be a weighted average of the distance data dli, d2i and d3i, the weights being greater the more reliable the data of the sensor system in question are.

20 The transmitted by the evaluation units 28 and 30

Distance data d2i and d3i are evaluated by distance module 36 in a manner corresponding to data dli by evaluation unit 26. If, for example, an object is initially only detected by short-range radar 14 and then in

25 immerses the detection area 18 of the long-range radar 12, the distance module 36 first tracks the change in the data arriving from the evaluation unit 28 and then checks whether corresponding data also arrives from the evaluation unit 26 at the expected time.

30

If the expected data from one of the evaluation units 26, 28, 30 fail to appear, that is to say if a sensor system does not detect an object, even though this object would have to be in the detection range based on the data of the other systems, the distance module 36 outputs an error signal Fdj , The index j identifies the sensor system from which no data was obtained. The error signal Fdj thus indicates that the sensor system in question may have failed or is "blind". 5

If the distance module 36 contains all expected distance data, but this data differs from one another by more than the error limits, then the error signal Fdj is also output. In this case, the error signal Fdj also indicates L0 ^' from which sensor systems the deviating data were obtained and how large the deviation is.

If there are distance data from at least two sensor systems that overlap within the error limits, L5 can be used to form the distance value di from this data and output it to the assistance system 44, although an error was detected and the error signal Fdj was generated.

The mode of operation of the speed module 38 is analogous to the mode of operation of the distance module 36 described above, only that here not the distance data, but the

Speed data vli, v2i and v3i are compared with one another in order to form a speed value vi therefrom, which is output to the assistance system 44, and / or to output an error signal Fvj, which indicates a discrepancy between the measured relative speeds.

The mode of operation of the transverse offset module 40 is largely the same as that of the distance module 36 and

30 of the speed module 38 analog. However, no output of an error signal is provided here in the example shown, because the transverse offset data are only derived data that are calculated from the measured azimuth angles, so that the error detection is primarily based on the

35 azimuth angle should be turned off. Accordingly, the azimuth angles jli, j2i, jLi and jRi are compared separately in the angle module 42. When comparing these azimuth angles, the deviations to 5 must of course be taken into account, which inevitably result for a given object from the object distance and the different positions of the relevant sensors or cameras on the baseline. If, taking these deviations into account, there remains a discrepancy that exceeds the error limits, a will

-0 Ffk error signal output. In this case, the index k (k = 1 - 4) identifies the camera 16L or 16R or the radar sensor, the azimuth angle of which or not its _. other azimuth angles. If, despite the discrepancy, a sufficiently reliable determination of the

L5 transverse offset is possible, a corresponding value yi for the transverse offset is output by the transverse offset module 40 to the assistance system 44.

The error signals Fdj, Fvj and Ffk are in the example shown

20 supplied to the correction device 34. Provided that from the

Error signals with sufficient certainty shows which of the three sensor systems is responsible for the discrepancy, and if it follows from the type and size of the error found that this error is due to a recalibration of the

25 concerned sensor system can be corrected, so a

Correction signal K is output to the associated evaluation unit 26, 28 or 30. Optionally, the correction signal can also be output directly to the long-range radar 12, the short-range radar 14 or the video system 16. 0

An example of a systematic error that can be corrected by recalibration is a misalignment of a radar sensor or a camera, which leads to a pivoting of the relevant reference axis, for example 18A or 2.0A, and thus to an incorrect measurement of the azimuth angle. In this In this case, the calibration in the relevant evaluation unit can be changed so that the misalignment is corrected and the correct azimuth angle is obtained again. The misadjustment should nevertheless be remedied during the next repair 5, since the misadjustment also leads to an undesired one

Changes in the detection range leads, but the functionality of the system can be maintained for the time being through the recalibration.

In the example shown, the correction device 34 has a statistics module 46 which stores the error signals Fdj, Fvj and Fjk which occurred during the operation of the device and thus documents the type and size of all errors which have occurred. This data is then available for a repair or

-5 Maintenance of the device is available for diagnostic purposes.

In addition, in the example shown, the statistics module 46 has the function of deciding whether an error can be corrected automatically or whether there is an error that cannot be remedied and an optical or acoustic error message F to the driver

20 must be issued to indicate the malfunction.

For example, error message F will be output if the signals received from error correction device 28 indicate a total failure of one of the sensor systems. The functions of the statistics module 46 offer the possibility 5 of not outputting the error message F immediately in the event of a discrepancy occurring only once or sporadically, but instead of outputting the error message only if discrepancies of the same type occur with a certain frequency. In this way, the robustness of the device is increased considerably. 0

FIG. 3 shows examples of how a misadjustment of a sensor affects the measurement result.

For example, if the main axis 18A is rotated by the angle Dj is pivoted, the measured azimuth angle j1 is too large by this angle, and the long-range radar 12 does not “see” the object 24 in the actual position, but in the position 24 ′ shown in broken lines. This results in 5 an error Dyl for the measured transverse offset. This error is greater the further object 24 is away from vehicle 10.

If, on the other hand, the left camera 16L of the video system has a .0 misalignment of the same size, the associated azimuth angle jL is falsified by the same angular deviation Dj as is indicated in FIG. 3 by a line of sight S drawn in broken lines. The video system 16 then sees the object 24 at the intersection of the visual beams of the .5 two cameras, that is to say in the position 24 ″. It can be seen that in this case the error Dy3 measured for the transverse offset is significantly smaller. On the other hand, the misadjustment of the camera 16L leads to a considerable error Dd3 in the measurement of the distance. ! 0

These relationships can be used in the described device for automatic error correction, even if only two sensor systems are present.

! 5 If, for example, there is a misadjustment of the long-range radar 12, a comparison of the transverse offset data of the long-range radar 12 and the video system 16 results in a clear discrepancy Dyl, while the distance data measured with the same systems are essentially consistent

10 are. This suggests that the error is due to a misalignment of the radar system and not to a misadjustment of a camera. The misalignment can even be determined quantitatively from the measured size of the error and by recalibrating the radar sensor

15 or the associated evaluation unit 26 are corrected. If, on the other hand, there is a misadjustment of the camera 16L, this can be recognized from a large discrepancy Dd3 in the distance data with the consistency of the cross offset data being largely consistent. In this case, the error can be corrected by recalibrating the video system.

In the event of contradictory measurement results, other criteria can also be used to decide which of the sensor systems involved is defective, in particular also in those cases in which only data from two sensor systems are available for the object or only two sensor systems at all. are present on the vehicle.

If, for example, as in FIG. 3, the transverse offset of the object 24 with respect to the reference axis 18A is not 0, the azimuth angle j1 is approximately inversely proportional to the object distance. In this case, the rate of change of the azimuth angle j1 is dependent on the relative speed of the object, which can be measured directly using the radar system. However, if the object 24 is actually on the main axis 18A and the transverse offset is only simulated by a misalignment Dj of the sensor system, the measured (apparent) azimuth angle is independent of the distance and the relative speed. Accordingly, there is also a discrepancy between the measured and the theoretically predicted change rate of the azimuth angle on the basis of the relative speed, even when the object is actually offset. This discrepancy suggests a misadjustment of the sensor system.

With video system 16 there is also the possibility of measuring the distance-dependent change in the apparent size of object 24. This apparent change in size is directly proportional to the relative speed and almost the other way round proportional to the distance. An error in the distance measurement, which is caused by incorrect adjustment of a camera, can then be recognized from the fact that the apparent change in size does not match the measured change in distance.

Since the type of object, for example a car or a truck, can also be recognized with the aid of the camera system 16 and the typical actual size of such objects is at least approximately known, it can also be checked whether the distance of the object measured with the camera system 16 is also the measured apparent size of the object is compatible.

If color tooth markings are also recognized with the aid of the camera system 16, this information can also be used for automatic error detection and error correction. In particular, the transverse offset of one's own vehicle relative to the center of the lane can be recognized from the recognized lane markings. It is to be expected that this transverse offset has the value 0 on a statistical average. If the evaluation in the statistics module 46 shows that the measured

The transverse position of one's own vehicle constantly deviates in one direction from the center of the road, this indicates a misadjustment of one or both cameras. Of course, this applies all the more if the transverse offset of an object measured with the camera system deviates in the same sense from the transverse offset measured with another sensor system.

Claims

claims

1. Object detection device for driver assistance systems in motor vehicles, with at least two sensor systems (12, 14, 16), the data (dl, vl, jl; d2, v2, j2; jL, jR) about the location and / or state of motion of objects (24) around the

L5 measure the vehicle (10) and their detection areas (18, 20) overlap one another, characterized by an error detection device (32) which checks the data measured by the sensor systems (12, 14, 16) for freedom from contradictions and upon detection of a contradiction

20 outputs an error signal (Fdj, Fvj, Ffk).

2. Object detection device according to claim 1, characterized in that the sensor systems (12; 14; 16) are designed as a radar, lidar or video system or a combination thereof 5.

3. Object detection device according to claim 1, characterized in that the sensor systems comprise a sensor system (12) for the far range and a sensor system (14; 16) for the near range.

4. Object detection device according to claim 3, characterized in that the sensor device (12) for the far range is a radar system or lidar system. 5

5. Object detection device according to claim 3 or 4, characterized in that the sensor system (14; 16) for the close range is a radar system, a lidar system or a video system.

6. Object detection device according to one of the preceding claims, characterized in that one of the sensor systems comprise a video system (16) with two cameras (16L, 16R) for detecting a stereo image of the object (24).

7. Object detection device according to one of the preceding claims, characterized in that the

Error detection device (32) uses the data of at least one sensor system (12) to check whether the object is in the overlap area (22) of the detection area (18) of this sensor system with the detection area (20) of another sensor system (14), and the error signal (Fdj , F j, Fjk) outputs if the object (24) is not located by the other sensor system (14) or if the data measured by this sensor system (14) taking into account the error limits of the data measured by the first sensor system (12) differ.

8. Object detection device according to one of the preceding claims, characterized in that the

Error detection device (32) for objects (24) located in the overlap area (22) of several sensor systems (12, 14, 16), the data measured by these sensor systems is weighted according to their reliability and the corresponding weighted data to data (di, vi , yi) combined, which are output to the assistance system (44).

9. Object detection device according to one of the preceding claims, characterized in that the error detection device (32) is designed to use the type of contradiction found to identify the sensor system that is causing the error.

10. Object detection device according to claim 9, characterized 5 by a correction device (34) for correcting the detected error by readjusting or recalibrating the sensor system which causes the error, or an evaluation unit (26, 28, 30) belonging to this sensor system.

.0 11. Object detection device according to claim 9 or 10, characterized by a statistics module (46) for recording and storing the detected errors.

12. Object detection device according to claim 11, characterized in that the statistics module (46) is designed to output an error message (F) to the driver when an error detected by the error detection device (32) occurs with a certain statistical frequency.