DE10346573B4 - Environment detection with compensation of self-movement for safety-critical applications - Google Patents

Abstract

A method for detecting the surroundings of a vehicle (30) in current driving situations in traffic, in which objects (31, 33, 34, 36) already detected from the surroundings are tracked, and the tracking of these objects (31, 33, 34, 36) by a Corrected coordinate transformation of these objects is simplified, with the steps: a) Acquisition of inertia measured variables with inertia sensors (3) attached to the vehicle (30), which allow the determination of at least one of the following current movement variables of the vehicle's own movement: a1) pitching movement, a2) yawing movement , a3) rolling movement, b) including (23) correction data resulting from an evaluation of the inertia measurement variables in calculations when tracking the objects (31, 33, 34, 36) that have already been detected, and c) continuing the control of objects that have already been detected (31, 33, 34, 36), which have disappeared from the detection range of at least one sensor during a predetermined period of time, whereby safety function nen are triggered due to the approach of the vehicle (30) to an object (34, 31) that is currently not in the detection range of a sensor if a prediction (38, 35) of the corresponding object state, taking into account the vehicle's own movement (30), is likely to result in a collision makes appear.

Description

State of the art

The present invention relates to the field of telematics and in particular to a driver assistance system with detection of the vehicle surroundings, an object that has already been detected being tracked.

Methods for supporting the driver of a motor vehicle are currently being increasingly developed. There is intensive research activity in particular in the area of sensor-based environment detection. Objects such as trees, pedestrians, vehicles and lane markings are recognized as such in the vicinity of a vehicle and the results are used to support the driver. Sensors such as video cameras, lidar, distance cameras, infrared cameras and radar are used to detect the surroundings. The tracking or recognition of objects that have already been recorded turns out to be particularly problematic. On the one hand, the objects themselves have their own movement, so that their own position changes over time; on the other hand, the movement of the sensor-carrying vehicle must also be taken into account.

This finding or tracking of objects that have already been detected is important in order to obtain reliable information about their existence, position and movement. The certainty of existence increases with the number of observations, because it is likely that real objects can be confirmed in several measurements, while it is unlikely that "ghost objects" caused by measurement errors will persist over several measurements.

In addition, the accuracy of the determination of position and existence is improved or even only made possible if several measurements are available for each object.

In the currently known methods for detecting the surroundings, data from wheel speed sensors and steering angle sensors are used to include the vehicle's own movements.

In the one in the German patent application specification DE 100 30 421 A1 disclosed methods for sensing the surroundings, such sensors are used, for example, to measure the yaw movement about the vertical axis. With the measured yaw rate, turning angle-dependent corrections of position data of an object are carried out by means of coordinate transformation. However, pitching movements around the transverse axis, for example when braking hard, or rolling movements around the longitudinal axis, caused for example by uneven road surfaces, are not recorded. This leads to considerable inaccuracies in the method of DE 100 30 421 A1 .

However, wheel speed sensors and steering angle sensors are unsuitable for more extreme driving situations. In the event of a full braking with locking tires, a completely wrong acceleration value is measured. The same applies when the tires spin on black ice. Furthermore, a steering angle sensor cannot be used if the vehicle starts to skid.

In such situations, the tracked objects are lost to the system because they are not in the sensor area in which the automatic environment detection assumes them. The entire sensor area must then be searched again for the objects, which requires considerable computing effort and thus valuable time in extreme situations.

These disadvantages of the known methods lead to incorrect results or no results at all, particularly in extreme driving situations, including accident situations. In such cases, it is particularly useful to support the driver due to the correct and quick detection of the surroundings. The driver can often no longer react quickly enough or he is acting incorrectly due to stress. Important information from the area around the vehicle can also be missing.

From the DE 100 33 599 A1 a position detecting device for a vehicle is known which detects the position of an object in a real space coordinate system. A plurality of objects are extracted from an image obtained by cameras. A pitch-dependent correction amount is calculated on the basis of displacement amounts of the position of the plurality of objects in their height direction. Positions of the plurality of objects are corrected on the basis of the pitch-dependent correction contribution.

From the US 2002/0130953 A1 an imaging device is known which is trained to recognize objects relevant to navigation, such as street signs or house numbers, by means of operator input and computer control. Optional lighting in the visible, infrared, ultraviolet light range or in another spectrum improves imaging, especially at night. Optional processing can be applied to the image to increase brightness, sharpness and / or size and / or to counteract positional or other distortions or errors. Computer-controlled motion tracking influenced by pattern recognition algorithms with optional artificial intelligence and / or freeze frame function and / or optical or digital image stabilization is used to stabilize the view of moving vehicles.

From the U.S. 6,157,294 A a method for detecting the surroundings of a vehicle in current driving situations in traffic is known, in which objects already detected from the surroundings are tracked, the tracking of these objects being simplified by a corrected coordinate transformation of these objects. In addition, inertia measurement variables are recorded with inertia sensors attached to the vehicle, which allow the vehicle to be pitched. Furthermore, correction data that result from an evaluation of the inertia measured variables are included in calculations when tracking the objects that have already been detected.

Advantages of the invention

In the following, reference is made to the claims.

With the measures of the independent claims, a method and system is created which, through the inventive combination of the vehicle environment sensor system with inertia sensor system, whose signals are used to compensate for the movement of the sensor-carrying vehicle, advantageously ensures a functional environment detection, especially in extreme traffic situations. An object can therefore be reliably tracked further in the context of the detection of the surroundings even in unusual situations. Inertial sensors are understood to be sensors that detect inertial forces due to the associated accelerations, such as the sensor with the product designation DRS-MM1.1, as sold by the applicant himself.

The inertia measurements that can be measured with inertia sensors include, for example, longitudinal acceleration, e.g. through braking or forward acceleration, lateral acceleration, e.g. when cornering, up / down acceleration and, in particular, nodding movements, yawing movements and rolling movements. The possibility according to the invention of such a comprehensive detection of the movement of a vehicle and the use of the correction data obtained therefrom in the calculation when tracking the objects once detected improves the quality of the tracking of the object considerably.

An advantageous additional step for the method according to the invention is the continuation of the control of objects that have already been detected, which have disappeared from the detection area of at least one sensor during a predetermined period of time. This means that predicted object states are still available for objects that can no longer be observed and that have disappeared from the detection area or are covered by other objects, on the basis of which safety functions can be activated if the prediction, taking into account the vehicle's own movement, appears likely to collide with the object leaves.

Advantageous refinements, developments and improvements of the respective subject matter of the invention are specified in the subclaims.

Yaw rate sensors and / or acceleration sensors are preferably used as inertia sensors. This makes it possible to record the intrinsic movement as precisely and completely as is required for the method according to the invention in order to be able to eliminate the disruptive influences of the intrinsic movement on the tracking of the objects in the vicinity of the sensor-carrying vehicle.

With an advantageous further development of the method according to the invention, in which an object state is predicted from the point of view of the sensor-carrying vehicle, taking the correction data into account, the object state can be predicted reliably and precisely, in particular according to position and movement, and thus better tracked. Either the correction data recorded by the inertia sensors to compensate for the intrinsic movement are applied to the predicted object state, or the compensation for the intrinsic motion of the vehicle is carried out directly on the basis of the sensor data acquisition.

Due to the reliable and precise prediction of an object state, only a narrower area, in which the object is suspected according to the prediction, needs to be searched for a later measurement of the surroundings in order to recognize the object there again. So that we save computational effort and time.

An additional, advantageous development of the method according to the invention is the further step of evaluating the current driving situation for the purpose of initiating measures adapted to the situation. These include, in particular, the triggering of safety functions of the vehicle. An advantageous embodiment of these functions is the automatic intervention in the driving dynamics of the vehicle by actuating predetermined actuators, such as braking, steering, accelerating, horn or headlight flasher, if the assessment of the situation shows that an accident occurs without intervention large, predetermined probability would occur. These measures can help ensure that accidents are avoided.

A particularly preferred variant of the system according to the invention is the use of a control unit with the above functions with an interface to inertia sensors that are also used by other functions in the vehicle that support the driver, since the inertia sensors and their data are then used multiple times. For example, inertia sensors are also used in vehicle dynamics control (ESP) and headlight control.

drawings

Exemplary embodiments of the invention are explained with the aid of the drawings.

• 1 eine Blockdarstellung einer Ausgestaltung des erfindungsgemäßen Verfahrens zur Umfelderfassung eines Fahrzeugs;
• 2 eine Blockdarstellung einer weiteren Ausgestaltung des erfindungsgemäßen Verfahrens zur Umfelderfassung eines Fahrzeugs;
• 3 zwei aus der Vogelperspektive skizzierte, zu aufeinander folgenden Zeitpunkten t1 (links) und t2 (rechts) beispielhaft erfasste Verkehrsituationen mit einem Fahrzeug, das eine Ausgestaltung des erfindungsgemäßen Systems enthält;
• 4 die zwei zu aufeinander folgenden Zeitpunkten t1 und t2 gemäß 3 erfassten Verkehrssituationen skizziert aus Sicht eines Videosensors des Fahrzeugs;
• 5 die Prädiktion der Positionen von im Zeitpunkt t1 im Sensorbild (linkes Bild) erfassten Objekten auf das skizzierte Sensorbild im Zeitpunkt t2 (rechtes Bild) ohne Kompensation der Eigenbewegung des Fahrzeugs;
• 6 eine Skizze mit Sensorbild zum Zeitpunkt t2 und mit Prädiktion der Position von früher beobachteten Objekten nach einer Variante der erfindungsgemäßen Kompensation der Eigenbewegung des Fahrzeugs;
• 7 eine Skizze mit Sensorbild zum Zeitpunkt t2 mit Prädiktion der Position von früher beobachteten Objekten nach einer weiteren Variante der erfindungsgemäßen Kompensation der Eigenbewegung des Fahrzeugs; und
• 8 eine Blockdarstellung einer Ausgestaltung eines erfindungsgemäßen Systems mit Schnittstelle zu von anderen Systemen des Fahrzeugs genutzten Trägheitssensoren.

Show it

• 1 a block diagram of an embodiment of the method according to the invention for detecting the surroundings of a vehicle;
• 2 a block diagram of a further embodiment of the method according to the invention for detecting the surroundings of a vehicle;
• 3 two traffic situations with a vehicle, which contains an embodiment of the system according to the invention, sketched from a bird's eye view and recorded by way of example at successive times t1 (left) and t2 (right);
• 4th the two at successive times t1 and t2 according to 3 detected traffic situations outlined from the point of view of a video sensor of the vehicle;
• 5 the prediction of the positions of objects detected at time t1 in the sensor image (left image) on the sketched sensor image at time t2 (right image) without compensation for the vehicle's own movement;
• 6th a sketch with a sensor image at time t2 and with prediction of the position of previously observed objects according to a variant of the inventive compensation for the vehicle's own movement;
• 7th a sketch with a sensor image at time t2 with prediction of the position of previously observed objects according to a further variant of the inventive compensation for the vehicle's own movement; and
• 8th a block diagram of an embodiment of a system according to the invention with an interface to inertial sensors used by other systems of the vehicle.

Description of exemplary embodiments

In the figures, the same reference symbols denote the same or functionally identical components.

1 and 2 show block diagrams of two alternative embodiments of the method according to the invention. The same applies to both variants: The environment sensors 1 , consisting of video and / or radar and / or lidar sensors and / or distance camera and / or infrared camera record the surroundings of the own vehicle and supply their data to a system 5 to object detection. This system contains an algorithm 13th to find and track objects based on current sensor data 7th but also to the time step "T" 9 previous sensor data 11 can fall back on. “Finding” means the first time a (new) object is discovered. “Tracking” means finding objects that have already been “discovered”.

The retrieval is supported or made possible by the fact that on the basis of at least one around the time step "T" 17th previous property condition 19th the current state of the property 15th with a predictor 21 can be predicted. A predicted object state contains at least the estimated location of the object. The object state can also contain movement data such as speed vector, acceleration vector, axis of rotation, angular speed and angular acceleration as well as other time derivatives of these variables. In addition, the object state can contain other variables that can be useful for tracking the object, such as B. the spatial orientation of the object and / or the estimated object dimensions.

The coordinate systems belonging to two successive points in time z. B. the coordinate systems rigidly connected to the sensor-carrying vehicle are compensated 23 the proper movement of the vehicle clearly transferred into one another. This transformation 23 between the two coordinate systems is described by a three-dimensional translation vector and a (3,3) -rotation matrix. This rotation matrix contains three independent quantities that are interpreted as rotation angles around the Cartesian coordinate axes. Overall, the transformation is 23 determined by six independent quantities. These six parameters of transformation 23 , also referred to as correction data, are obtained according to the invention from the data from the inertia sensors 3 determined measured inertia variables.

To determine all six parameters, there are three rotation rate sensors and three acceleration sensors. If only less Inertial sensors 3 are available, the detection of the vehicle's own movement is restricted accordingly, such as pitching, yawing or rolling movement.

The result of the object tracking is a current object status 15th , the z. B. can be passed on to safety and comfort systems in the vehicle in the form of an object list.

For the compensation according to the invention 23 the proper movement by means of the inertia sensors 3 certain transformation parameters are used in the in 1 the variant shown, the compensation 23 the proper motion applied to the predicted state of the object. The prediction based on the coordinate system of the earlier observation is suitably compensated so that the algorithm for object tracking 13th can appropriately restrict its search area to a small region around a predicted spatial area. The order of the blocks 21 and 23 is arbitrary.

The following explanations will now be given using the “video camera” sensor selected as an example. However, the person skilled in the art will understand that other sensors, such as the lidar, range image cameras, infrared cameras and radar already mentioned at the outset, can also be used and can systematically provide similar results.

At the in 2 The variant shown is the compensation according to the invention 23 the proper movement applied directly to the sensor data. The coordinate transformation to compensate for the intrinsic movement is converted in such a way that it is used directly in the coordinate system of the environment sensors. It is irrelevant whether the compensation is "forward" to the earlier sensor data 11 or "backwards" to the current sensor data 7th is applied.

In relation to image data, this means for the “backwards” compensation, for example, that the _{image data recorded at the later point in time t 2} are converted as if they had been recorded from the camera perspective from the earlier point in time t1.

It is also possible to compensate “forwards”, that is to say to convert the image data recorded at time t1 as if they had been recorded from the camera perspective at time t2. With the compensation forwards, there are estimates of the object states, such as the distances of the objects from the camera and object dimensions, which can be used for a more precise compensation.

When compensating backwards, predicted object states can be used. This utilization of the object states is shown by dashed lines in the block diagram.

The block 23 in 2 has two input and two output arrows, which are to be understood as alternatives, on the one hand for the compensation "backwards" and on the other hand for the compensation "forwards".

In reference to 3 until 7th traffic situations selected by way of example are used below in order to illustrate the mode of operation of the present invention.

3 shows the initial situation at time t1 from a bird's eye view on the left. Two vehicles 30th , 36 come towards each other. In the right lane of the vehicle carrying the sensor 30th (bottom right, limited by a right lane delimitation 31 and a left lane boundary 33 there is a pedestrian 34 . The sensor-carrying vehicle 30th has a sensor range angle in the front direction 32 of 40 degrees, for example. It now leads, as on the right in 3 shown an evasive maneuver to the left to avoid the collision with the pedestrian 34 to avoid. The vehicle 30th is already in a spin motion.

The situation at the later point in time t2 \ is shown on the right. Both vehicles 30th , 36 have moved on, the pedestrian 34 Not. The sensor-carrying vehicle 30th has changed its orientation considerably as a result of the sling motion. The pedestrian 34 is no longer in the sensor range. The left lane boundary 33 that delimits the opposite lane is just in the sensor area.

4th shows the same scene with an oncoming vehicle 36 and pedestrians 34 at times t1 and t2 from the point of view of a video sensor. Other detectors such as infrared sensors can be used just as well. The pedestrian is at time t2 34 has already disappeared from the detection range of the video sensor - but there is still a risk of collision with pedestrians 34 .

5 shows on the left, the camera image with pedestrians 34 and oncoming vehicle 36 from time t1 superimposed on the prediction of the respective object states 38 , 40 for the point in time t2, shown as polygons bordered by dotted lines. In the picture on the left everything is going according to plan as there is no skidding movement of the vehicle.

When this prediction, carried out for time t2, is applied to the associated image of the video sensor from time t2, without performing the compensation for the intrinsic movement, the result is shown in FIG 5 right that the prediction of the object states for the person 38 , the oncoming vehicle 40 and the projection 37 the left lane boundary as well as that of the right 35 and the respective actual object state do not match. The oncoming vehicle 36 Finding it again when surveying the surroundings is associated with a very high search effort. Correct recovery of the pedestrian 34 is not possible at all because it is no longer in the picture. Equally problematic or even more problematic is the position and direction, possibly with the curvature of one's own lane between the boundaries 31 and 33 to find again.

6th contains a sketch with a sensor image at time t2 and with prediction of the position of previously observed objects according to a variant of the inventive compensation for the own movement of the sensor-carrying vehicle.

The prediction of the state 40 for the oncoming vehicle is close to the actual state 36 so that the search space can be kept small. For the pedestrian 38 it is predicted that this is clearly outside the detection range of the video sensor. The search for the pedestrian in the picture is omitted from the start. The same applies to the right lane boundary 33 .

In 7th is shown as the image of the video sensor containing the image of an oncoming vehicle 36 at time t2, after the compensation of the self-motion backwards has been applied to it, in order to match the image from time t1 and the predictions based on it 38 , 40 , and 35 or. 37 to be able to compare. Here, too, it applies that the pedestrian is known to be outside the detection area.

For objects that can no longer be observed that have disappeared from the detection area or have even been covered by other objects, there are still predicted object states 38 and 35 , or. 37 on the basis of which, under certain circumstances, safety functions such as braking, steering, accelerating, activating the horn or headlight flasher, are initiated if the assessment of the situation shows that, without intervention, an accident would occur with a high, predetermined probability.

In 8th is the environment sensing system 5 of a motor vehicle connected to a CAN bus 50 as an interface to the airbag control unit 52 , to the ESP control unit 54 and to another inertial sensor 56 shown. The environment sensing system 5 contains a computing unit to compensate for the vehicle's own movement. According to the invention, inertial measured variables from inertial sensors are used for this purpose. The measurement data are received from the individual inertia sensors via the CAN bus 50 to the respective ports 58 , 60 , 62 the environment sensing system 5 transmitted. In a port 58 for the longitudinal acceleration of the vehicle, the data are sent from the inertia sensor of the airbag control unit 54 , into a second port 60 for yaw and roll movements, the data from the inertia sensors of the ESP control unit are used 54 fed in and the third port 62 receives the measurement data from the inertia sensor 56 for the pitch movement.

Although the present invention has been described above on the basis of preferred exemplary embodiments, it is not restricted thereto, but rather can be modified in many ways.

The lane boundaries mentioned 31 and 33 and thus your own lane in between are only recorded in the area recorded by the respective sensor. Optical sensors are particularly suitable for this. Individual road markings can be described in the parameters of lateral offset, angle relative to the forward direction in the vehicle coordinate system (not to the current direction of travel), curvature and change in curvature, etc. If the vehicle changes its orientation for this purpose, and the sensors used detect this, the parameters can be converted into a forecast projection as described above.

Finally, the features of the subclaims can essentially be freely combined with one another and not by the order present in the claims, provided they are independent of one another.

Claims (9)

A method for detecting the surroundings of a vehicle (30) in current driving situations in traffic, in which objects (31, 33, 34, 36) already detected from the surroundings are tracked, and the tracking of these objects (31, 33, 34, 36) by a Corrected coordinate transformation of these objects is simplified, with the steps: a) Acquisition of inertia measured variables with inertia sensors (3) attached to the vehicle (30), which allow the determination of at least one of the following current movement variables of the vehicle's own movement: a1) pitching movement, a2) yawing movement , a3) rolling motion, b) inclusion (23) of correction data, which result from an evaluation of the inertia measured variables, in calculations when tracking the objects (31, 33, 34, 36) that have already been detected, and c) continuing the control of objects (31, 33) that have already been detected , 34, 36), which have disappeared from the detection area of at least one sensor during a predetermined period of time, safety functions being triggered due to the approach of the vehicle (30) to an object (34, 31) currently not in the detection area of a sensor when a Prediction (38, 35) of the corresponding object state, taking into account the vehicle's own movement (30), makes a collision appear likely. Procedure according to Claim 1 , in which rotation rate sensors and / or acceleration sensors are used as inertia sensors. Procedure according to Claim 1 , in which an object state (35, 37, 38, 40) is predicted from the point of view of the sensor-carrying vehicle (30) taking the correction data into account. Method according to one of the Claims 1 until 3 , containing the further step of evaluating the driving situation detected by sensors for the purpose of initiating measures adapted to the situation. Procedure according to Claim 4 , comprising the step of triggering safety functions of the vehicle (30). Method according to one of the Claims 1 until 5 , whereby lane boundaries (31, 33) are also detected and their projections (35, 37) are formed. Procedure according to Claim 1 , further comprising the step: Intervening in the driving dynamics of the vehicle (30) by actuating predetermined actuators, if the evaluation necessarily shows that an accident would occur with a high, predetermined probability without intervention. Computer-aided driver assistance system (5) containing technical devices (1, 3, 7, 11, 13, 15, 19, 21, 23) for carrying out one of the methods according to one of the preceding claims. Driver assistance system (5) Claim 8 , containing an interface (58, 60, 62) for processing the data acquired by inertial sensors.

Images