DE102008061301B4 - Driver assistance with merged sensor data - Google Patents

Abstract

Driver assistance system for a vehicle, the driver assistance system having: a radar sensor (101); an optical sensor (102); a control unit (103); the radar sensor (101) for detecting an object in the area surrounding the vehicle and for forwarding corresponding object data to the control unit (103);wherein the optical sensor (102) is designed to acquire optical data about the object;wherein the control unit (103) is designed to validate the object data on the basis of the optical data; and provision is made for the object data of the radar sensor (101) and the object data of the optical sensor (102) to be merged with one another; andwherein the driver assistance system is designed to carry out a driving intervention depending on the validation of the object data,wherein the validation comprises a classification of the object,where the object is a neighboring vehicle,andthe driver assistance system for determining a driving path of the neighboring vehicle on the basis of Validation is done.

Description

field of invention

The invention relates to assistance and safety technology for vehicles. In particular, the invention relates to a driver assistance system for a vehicle, a vehicle with a driver assistance system, a method for carrying out a driving intervention of a driver assistance system in a vehicle, a program element and a computer-readable medium.

Technological background

From the Südwest Presse, "The car of the future thinks with you", June 4, 1996, a vehicle emerges that has an autopilot. With the help of a video camera arranged in the vehicle and image processing algorithms, this determines information about distances to objects on the road and the lane of the vehicle. Furthermore, traffic signs recorded by the video camera can also be recognized. It has turned out to be disadvantageous that the image processing is often not reliable. In addition, high investments are required.

Modern motor vehicles are increasingly being equipped with functions and services that support the driver. The functions that support the driver, such as ABS, ESP, ACC and automatic emergency call, provide requested services with a predefined process. The services, on the other hand, provide a service on request, whereby the way in which the service is provided is usually left open. For example, with the navigation service it is completely open how the destination will be reached if the service request is made to guide the vehicle to a selected destination. The functions and services mentioned have so far been predominantly driver-related, since they support the driver of the individual vehicle in controlling the vehicle dynamics, in calling for assistance and in finding a destination. On the other hand, services that support the driver due to fleet issues are virtually unknown. A few examples are ARI, RDS-TMC and TEGARON, which are essentially used to provide traffic information. The technical facilities required for the implementation of the above functions and services are usually only designed for the special function or service and therefore cannot be used more than once.

Out of DE 39 41 402 A1 a method for detecting and averting a risk of collision between a vehicle moving ahead in traffic and a vehicle moving behind is known, in which each vehicle continuously transmits at least one signal containing information about its state of motion in terms of current speed and/or current acceleration in the opposite direction to the respective direction of travel emits the rear hemisphere, in which the signal received by each vehicle when it enters the transmission range of the vehicle is subjected to a comparison with the signal, evaluated with regard to the presence of a risk of collision, and in which the signal and/or a risk of collision as a result of the evaluated comparison signaling comparison signal a signaling device informing the vehicle driver and/or other following road users and/or a vehicle changing the state of motion of the vehicle in the sense of avoiding a collision pulling device applied.

Out of DE 298 03 684 U1 a safety system for motor vehicles on the road is also known, with at least one vehicle transmitter in a first vehicle, with a transmission means for emitting at least one specific safety signal, with a triggering means for determining whether a safety-related event is occurring in the vehicle, with the carrier of the signal, which the emits transmission means, represents an electromagnetic wave in the non-visible range, and with at least one information receiver in a second vehicle with a receiving part for receiving safety-relevant information that is transmitted by a vehicle transmitter, and at least one warning device in the vehicle, in which the triggering means at least is connected to the following systems in the vehicle: brake detector, on/off switch for the hazard warning lights and is set up in such a way that it transmits information to the transmission means at least about the reason for the triggering and the transmission means of the Vehicle transmitter emits this information, and the warning device has at least two different representations for different warning or emergency situations.

The DE 10 2005 009 146 A1 describes a driver assistance system for motor vehicles, with a sensor device for detecting the environment, at least two assistance functions and a data processing device that evaluates the measured data and generates at least one specific environmental hypothesis for each assistance function.

DE 44 23 966 A1 discloses an obstacle detection system for a motor vehicle, which detects obstacles in front of the motor vehicle by means of laser radar and CCD camera and information on the level of danger between the motor vehicle and each obstacle to carry out a risk assessment.

Further describes DE 102 38 936 A1 a device for controlling a system component of a generic information technology system of a motor vehicle.

Summary of the Invention

It is an object of the invention to specify improved driver assistance for vehicles.

A driver assistance system, a vehicle with a driver assistance system, a method for carrying out a driving intervention of a driver assistance system in a vehicle, a program element and a computer-readable medium according to the features of the independent claims are specified. Further developments of the invention result from the dependent claims.

The exemplary embodiments described relate equally to the driver assistance system, the vehicle, the method, the program element and the computer-readable medium. In other words, the following with regard to z. B. implement the features mentioned on the driver assistance system in the method, the program element or the computer-readable medium, and vice versa.

According to an exemplary embodiment of the invention, a driver assistance system for a vehicle is specified, which has a radar sensor and an optical sensor as well as a control unit. The radar sensor is designed to detect an object in the area surrounding the vehicle and to forward the corresponding object data to the control unit. The optical sensor is designed to capture optical data relating to the object, with the control unit being designed to validate the object data on the basis of the optical data. Furthermore, the driver assistance system is designed to carry out a driving intervention depending on the validation of the object data.

In other words, both the measurement data from the radar sensor and the optical sensor are used to control the driver assistance system. With the help of the radar sensor data, for example, the relative position of the object to the vehicle and its speed can be determined. The optical sensor data can be used to check what kind of object it is. In this way it can be determined, for example, whether it is a vehicle or a pedestrian.

Depending on the end result achieved (position determination, speed determination and validation), the driver assistance system is then activated.

Of course, multiple optical sensors and multiple radar sensors can also be provided. A lidar sensor can also be provided instead of or in addition to the radar sensor. In addition, an ultrasonic sensor can be provided.

The objects captured and detected by means of the radar sensor (and/or the other sensors) are therefore validated by the visual sensor and the driver assistance system is activated by this validation.

According to the invention, the measurement data from the radar sensor and the measurement data from the optical sensor are merged with one another. This data fusion can take place within the control unit or in an upstream fusion module. In this case, it is a one-step procedure (see also 3 ). A Kalman filter is used for this, for example. It is not necessary for the radar sensor data to be subsequently validated on the basis of the optical data.

According to a further exemplary embodiment of the invention, the risk of the vehicle colliding with the object is estimated on the basis of the validation and the object data as well as other predefined parameters. Furthermore, the vehicle is braked until the vehicle comes to a standstill if the risk of collision is not equal to zero or at least exceeds a predetermined threshold value.

In other words, a driver assistance unit is activated in such a way that the vehicle is braked to a complete standstill when there is a risk of a collision as a result of the validation based on the specified parameters of the driver assistance system.

Furthermore, the validation of the object data includes a classification of the object. For example, it can be determined whether it is a pedestrian, a bicycle or a motor vehicle. It is also possible to distinguish whether it is a truck or a passenger car. Motorcycles can also be classified or identified.

According to a further exemplary embodiment of the invention, the optical sensor is designed to detect light in the visible spectral range. For example, it is a video sensor, e.g. B. in the form of a mono video camera or a stereo video camera.

If the object is a neighboring vehicle, for example a vehicle driving ahead, the driver assistance system is designed to determine a driving path of the neighboring vehicle on the basis of the validation.

According to a further exemplary embodiment of the invention, the driver assistance system is designed to regulate the speed of the vehicle on the basis of the validation and the object data.

According to a further exemplary embodiment of the invention, the driver assistance system for accelerating the vehicle is designed on the basis of the validation and the object data.

According to a further exemplary embodiment of the invention, the driver assistance system also has digital map data that is stored in a corresponding memory. In this exemplary embodiment, the validation also takes place on the basis of data from the corresponding digital map.

If the position of the vehicle is required for this purpose, this can be supplied, for example, by a satellite navigation device (GPS device). It should be noted that in the context of the present invention, GPS is representative of all Global Navigation Satellite Systems (GNSS), such as GPS, Galileo, GLONASS (Russia), Compass (China), IRNSS (India), ...

According to a further exemplary embodiment of the invention, a vehicle is provided with a driver assistance system as described above.

According to a further exemplary embodiment of the invention, a method for carrying out a driving intervention of a driver assistance system in a vehicle is specified, in which an object in the area surrounding the vehicle is detected by a first sensor and corresponding object data are generated. Furthermore, optical data about the object are recorded or generated by a second sensor. In a next step, the object data is validated on the basis of the optical data and a driving intervention is carried out depending on the validation of the object data.

According to a further exemplary embodiment of the invention, a program element is specified which, when it is executed on a processor of a driver assistance system, instructs the driver assistance system to carry out the method steps specified above.

The program element z. B. be part of software that is stored on a processor of the driver assistance system. The processor can also be the subject of the invention. Furthermore, this exemplary embodiment of the invention comprises a program element which uses the invention right from the start, as well as a program element which causes an existing program to use the invention by means of an update.

According to a further exemplary embodiment of the invention, a computer-readable medium is specified, on which a program element is stored which, when it is executed on a processor of a driver assistance system, instructs the driver assistance system to carry out the method steps described above.

One aspect of the invention can be seen in the fact that a dynamic driving speed control system, also known as adaptive cruise control (ACC), or also a so-called full speed range ACC (FSRA) or also a so-called stop and go ACC, is an autonomous Braking function is added. This autonomous braking function uses the validation data based on the images from the optical sensor (so-called Target Validated Braking).

The decisive factor here is the combination of ACC and Target Validated Braking. The term Target Validated Braking means that the vehicle uses the sensors in the vehicle to brake depending on the objects in the surrounding area.

For this purpose, an additional camera is used in the vehicle, which is used to distinguish vehicles in the area from other radar targets. Pedestrians can also be identified. This makes it possible to automatically validate stationary vehicles with the camera and automatically initiate appropriate braking. In contrast, a radar sensor can only reliably detect stationary objects if they have previously moved.

Furthermore, even the smallest changes in the appearance of the vehicle in front are recognized. In this way, obstacles that have moved between one's own vehicle and the vehicle in front while the vehicle in front is stationary, for example, can be evaluated as a termination criterion. This is particularly advantageous for pedestrians as objects to be recognized.

An advantage of the invention is the possibility of carrying out automatic braking maneuvers with an existing and known ACC system. In particular, the start-up process can also be significantly simplified. Since the camera recognizes the object, the vehicle can always be approached autonomously if the object has been confirmed (e.g. by the driver) or is known from the previous situation. This means that the driver no longer has to confirm each time before starting off. Due to the relatively simple operation and universal application, driving comfort is increased and the driver's workload is relieved.

With the radar sensor, distance and speed of the object can be well detected. The optical sensor, for example in the form of a video sensor, can be used to determine the lateral offset (i.e. the lateral distance from the center axis of the vehicle) and the object type, and to identify the lane. The quality of the ACC control is improved by merging the two or more sensor data sets. In this way it is possible to better predict the driving path of one's own vehicle or that of the vehicle in front via the lane and thus to detect cutting-in and cutting-out processes more reliably. In response, the speed and distance can be adjusted earlier. In this way, it is possible to more closely match the behavior of the ACC to that of a human driver.

All obstacles of a relevant size that have a radar signature can be detected. This applies in particular to obstacles larger than approximately 200 mm x 200 mm. The driver assistance system according to the invention and in particular the special design as an FSRA can thus detect all such objects that have the desired radar signature, including pedestrians. Depending on the performance of the radar, a fusion with ultrasonic sensors can also take place.

At the same time, the objects (e.g. the vehicles driving ahead) can be classified with the optical sensor (i.e. the camera). This makes it possible, similar to FSRA with Target Validated Braking, to drive off autonomously in a traffic jam without driver confirmation. Obstacles that move between the vehicles are then already recognized by the merged sensors (or a downstream evaluation or control unit).

A further advantage of the invention is the improvement in the driving path prediction of ACC and the reliable detection of cutting-in and cutting-out processes.

Exemplary embodiments of the invention are described below with reference to the figures.

character list

• 1 shows a representation of a driver assistance system according to an embodiment of the invention.
• 2 shows a vehicle according to an embodiment of the invention.
• 3 shows a flowchart of a method according to an embodiment of the invention.

Detailed description of exemplary embodiments

The representations in the figures are schematic and not to scale.

In the following description of the figures, the same reference numbers are used for the same or similar elements.

1 shows a driver assistance system 100 according to an embodiment of the invention. Driver assistance system 100 has a central control unit 103 . A radar sensor 101, a lidar sensor 104, an ultrasonic sensor 105 and an optical sensor 102 are connected to the control unit 103, although not all of the sensors have to be present. These sensors can be used to monitor the area around the vehicle. The individual sensor data or sensor data sets can be merged with one another in a fusion module 109, which is part of the control unit or is connected upstream of the control unit.

Since each of the sensors detects partially different measured variables (e.g. the radar sensor detects the speed and the relative distance of the object and the camera detects its shape), the fusion of the sensor data leads to a comprehensive result, which allows a very precise identification of object types and object movements as well as the corresponding object distances to the vehicle. The fusion result can then be fed to an adaptive cruise control ACC 108, which then carries out a corresponding driving intervention.

Furthermore, the data from a digital map can be used to analyze the situation. The satellite navigation receiver 107 for receiving positioning signals from, for example, Galileo satellites or GPS satellites, and the navigation unit 106 are provided for this purpose. The satellite navigation receiver 107 is with the Control unit 103 connected. The navigation unit 107 is also connected to the control unit 103 . Furthermore, there is a direct connection between the navigation unit 106 and the satellite navigation receiver 107. The GPS signals can thus be transmitted directly to the control unit 103.

The term "digital maps" also includes maps for advanced driver assistance systems (ADAS, Advanced Driver Assistance Systems) without navigation taking place.

At this point it should also be pointed out that the data transmission between the individual components can be both wired and wireless.

2 shows a vehicle 201 with a driver assistance system 100 according to an embodiment of the invention. An adjacent vehicle 202 drives ahead of the vehicle 201 . Driver assistance system 100 enables vehicle 201 to carry out autonomous braking interventions, up to and including emergency braking. In particular, stationary vehicles can also be identified. Pedestrians who push their way between the individual vehicles in a traffic jam, for example, can also be identified. In this way, a stop and go operation can be carried out fully automatically.

3 shows a flowchart of a method according to an embodiment of the invention. In step 301, an object in the area surrounding the vehicle is detected by a first sensor (for example a radar sensor) and corresponding object data are generated. In step 302, which can take place at the same time as step 301 or also before or after, optical data from the object are recorded (for example with the aid of a camera). The optical data and the object data are transferred to a control unit and evaluated in step 303. Here, the position of the object relative to the host vehicle is determined, the direction of movement and the speed of the object are determined, and the type of object is identified (validation). In addition, a driving path of the object is predicted with high accuracy.

In step 304, an autonomous driving intervention is then carried out by an ACC unit based on the calculations.

Additionally, it should be noted that "comprising" and "comprising" do not exclude other elements or steps, and "a" or "an" does not exclude a plurality. Furthermore, it should be pointed out that features or steps that have been described with reference to one of the above exemplary embodiments can also be used in combination with other features or steps of other exemplary embodiments described above. Any reference signs in the claims should not be construed as limitations.

Claims (10)

Driver assistance system for a vehicle, having the driver assistance system: a radar sensor (101); an optical sensor (102); a control unit (103); wherein the radar sensor (101) is designed to detect an object in the area surrounding the vehicle and to forward corresponding object data to the control unit (103); wherein the optical sensor (102) is designed to acquire optical data about the object; wherein the control unit (103) is designed to validate the object data on the basis of the optical data; and it is provided that the object data of the radar sensor (101) and the object data of the optical sensor (102) are merged with one another; and wherein the driver assistance system is designed to carry out a driving intervention depending on the validation of the object data, where the validation includes a classification of the object, wherein the object is an adjacent vehicle, and the driver assistance system is designed to determine a driving path of the neighboring vehicle on the basis of the validation. driver assistance system claim 1 , designed to: Estimate a collision risk of the vehicle based on the validation and the object data and other specified parameters; and performing vehicle deceleration to a stop if the risk of collision exceeds a predetermined threshold. Driver assistance system according to one of the preceding claims, wherein the optical sensor (102) is designed to detect light in the visible range. Driver assistance system according to one of the preceding claims, wherein the driver assistance system is designed to regulate the speed of the vehicle on the basis of the validation and the object data. Driver assistance system according to one of the preceding claims, wherein the driver assistance system for accelerating the vehicle based on the validation and the object data. Driver assistance system according to one of the preceding claims, further comprising: a digital card; whereby the validation is also based on data from the digital map. Vehicle with a driver assistance system according to one of Claims 1 until 6 . Method for carrying out a driving intervention of a driver assistance system in a vehicle, the method having the steps: detecting an object in the vicinity of the vehicle by a first sensor (101, 104, 105) and generating corresponding object data; acquiring optical data about the object by a second sensor (102); validation of the object data based on the optical data, the validation including a classification of the object and the object being a neighboring vehicle; Merging the object data from the first sensor (101, 104, 105) and the object data from the second sensor (102); Determination of a driving path of the neighboring vehicle on the basis of the validation, as well as Carrying out a driving intervention depending on the validation of the object data. Program element that, when executed on a processor of a driver assistance system, directs the driver assistance system to carry out the following steps: detecting an object in the vicinity of the vehicle by a first sensor (101, 104, 105) and generating corresponding object data; acquiring optical data about the object by a second sensor (102); validation of the object data based on the optical data, the validation including a classification of the object and the object being a neighboring vehicle; Merging the object data from the first sensor (101, 104, 105) and the object data from the second sensor (102); Determination of a driving path of the neighboring vehicle on the basis of the validation, as well as Carrying out a driving intervention depending on the validation of the object data. Computer-readable medium storing a program element which, when executed on a processor of a driver assistance system, directs the driver assistance system to carry out the following steps: detecting an object in the vicinity of the vehicle by a first sensor (101, 104, 105) and generating corresponding object data; acquiring optical data about the object by a second sensor (102); validation of the object data based on the optical data, the validation including a classification of the object and the object being a neighboring vehicle; Merging the object data from the first sensor (101, 104, 105) and the object data from the second sensor (102); Determination of a driving path of the neighboring vehicle on the basis of the validation, as well as Carrying out a driving intervention depending on the validation of the object data.

Images