DE102009006113A1 - Vehicle's surrounding representation providing method, involves subjecting sensor objects to fusion to generate fusion objects, and fusing existence possibilities of fusion objects based on existence possibilities of sensor objects - Google Patents

Abstract

The method involves providing information about objects detected in a surrounding of a vehicle in the form of sensor objects by using sensor devices (2, 3) e.g. radar and stereo camera. The sensor objects are subjected to an object fusion for generating fusion objects by using a Kalman-filter based on attributes e.g., position and speed, and existence possibilities of the fusion objects are fused based on existence possibilities of the sensor objects. The fusing of the existence possibilities of one of the sensor objects is carried out based on the corresponding sensor device. An independent claim is also included for a device for providing a surrounding representation of a vehicle.

Description

The The invention relates to a method and a device for a vehicle for describing an environment of the vehicle with at least one first sensor device and at least one second Sensor device is detected and their information provided be evaluated in an evaluation to an environment representation provide. In particular, the invention relates to a method for providing an environment representation of a vehicle with at least a first sensor device and at least one second sensor device and an evaluation device. Furthermore, the invention relates to a device for providing an environment representation of a vehicle, comprising at least one first sensor device and at least one second Sensor device that detect the environment, and an evaluation, with the at least one first sensor device and with the at least one second sensor device is coupled.

The Knowledge of an environment of a vehicle is different Vehicle systems, such. B. security systems, of great Importance. With sensor devices, for example, an ultrasonic sensor or include a camera, data about the environment detected. By using information from different sensor devices a representation of the environment is generated, which the Reflect the reality of the environment as well as possible should. There are static objects in the environment, such as Trees, houses, etc., and dynamic objects that to move in a fixed frame of reference, such as Persons, animals, motor vehicles, etc. To provide sufficient information about these moving objects (dynamic objects) in the environment of Vehicle is available at all times continuous monitoring, d. H. a continuous one Acquire measurement data about the environment necessary. In As a rule, the sensor devices continuously work iterative repetitive measuring cycles.

From the DE 10 2006 035 207 a device is known which determines by means of a camera device an approximate value for a position of a possible object. In an evaluation device, two limit values for the position of the object are determined on the basis of the data provided by the camera. A Laser Imaging Detection and Ranging System (Lidar) or a radar system is used to check whether an object is actually in the position range defined by the first limit and the second limit.

Of the Invention is based on the object, a method and an apparatus to create an improved environment representation an environment of a vehicle with static and dynamic objects provide.

The The object is achieved by a method with the features of claim 1 and a device with the features of claim 12 solved. advantageous Embodiments of the invention will become apparent from the dependent claims.

In particular, a method is provided for providing an environment representation of a vehicle having at least one first sensor device and at least one second sensor device and an evaluation device, in which the sensor devices provide information about objects detected in an environment of the vehicle in the form of sensor objects, wherein a sensor object is one of the represents the object detected by the respective sensor device, and the sensor objects comprise as attribute at least one probability of existence of the represented object and the sensor objects detected by the at least one first sensor device and by the at least one second sensor device are subjected to object fusion in which fusion objects are generated which have at least one probability of existence is assigned as an attribute, wherein the existence probabilities of the fusion objects based on the existence probabilities of the sensor objects f is usiertiert, wherein the fusing of the likelihood of existence of one of the sensor objects in each case depends on the respective sensor device from which the corresponding sensor object is provided. A corresponding device for providing an environment representation of a vehicle comprises at least one first sensor device and at least one second sensor device, which detect the environment, and an evaluation device, which is coupled to the at least one first sensor device and the at least one second sensor device, wherein the sensor devices are formed To provide information about each detected objects in the form of sensor objects, which comprise as attribute at least one probability of existence for the detected object, and the evaluation unit is adapted to subject the at least one first sensor device and detected by the at least one second sensor device sensor objects of an object fusion in which fusion objects are generated, which have at least one probability of existence as an attribute is assigned, wherein the fusion of the probability of existence of the fusion object is carried out based on the existence probabilities of the sensor objects and is dependent on the respective sensor device from which the respective sensor object is provided. Although the individual sensor devices already independently recognize objects and provide these detected objects as sensor objects, generally in the form of a list of sensor objects, it is provided according to the invention to subject the sensor objects provided by the different sensor devices to a fusion. Here, fusion objects are generated. Since, in particular, a probability of existence of the recognized objects is of particular importance, the fusion objects comprise at least one probability of existence as an attribute. By merging the sensor objects, fusion objects are created that represent the real existing objects. Due to the fusion of the sensor objects, which are each detected and detected by one of the sensor devices, the probabilities of existence of the fusion objects (as well as other attributes) can be significantly improved by a fusion of the sensor objects to the fusion objects. Since the individual sensor devices differ with regard to their quality of measurement and / or susceptibility to errors, for example due to noise, susceptibility to changing environmental conditions, etc., it is provided that, in the case of the fusion of the probabilities of existence of the individual sensor objects with the probability of existence of the fusion objects, fusion takes place, taking account in each case the sensor device takes place, which has provided the respective sensor object whose sensor object existence probability is fused into the existence probability for the fusion object. By merging the sensor objects into fusion objects, a significantly improved environment representation is achieved. In particular for safety-relevant vehicle systems, which depend on the existence of objects in the surroundings of the motor vehicle, a good knowledge about the actual existence of objects in the environment is necessary. By fusing the probabilities of existence of the sensor objects to a probability of existence of fusion objects that represent the real objects, the statement about the probability of existence of objects in the environment of the motor vehicle is thus significantly improved. This can significantly reduce or completely avoid false tripping of security systems, which, for example, trigger restraint systems before the actual arrival of a collision with an object in the environment, which is recognized as unavoidable.

at a particularly advantageous embodiment is provided that for the fusion objects determined at the time k a prediction is made at a time k + 1 and the sensor objects with the fusion objects determined at time k + 1 whose prediction they match according to association requirements, where the likelihood of existence of the fusion object at the time k + 1 on the basis of the likelihood of the fusion object for Time k and the probability of existence at time k + 1 of the sensor object is fused. As an association requirement can serve a Mahalanobis distance default value. In this embodiment is it possible to do that in previous merging steps generated fusion objects with the currently provided sensor objects to associate and over this the previously obtained Information, in particular regarding the likelihood of existence the existing objects in the environment, optimally evaluate.

wobei P_FusObj(k) die Existenzwahrscheinlichkeit des Fusionsobjekt zum Zeitpunkt k, P_SenObj(k) die Existenzwahrscheinlichkeit des Sensorobjekts zum Zeitpunkt k, γ eine sensoreinrichtungsunabhängige Fusionierungskonstante und δ^a und δ^b sensoreinrichtungsabhängige Fusionierungskonstanten angeben, wobei gilt: δ^a + δ^b = 1. Die angegebene Formel, die der Fuzzy-Logik entlehnt ist, ermöglicht über die festgelegte Wahl der Konstante γ eine Vorwahl, ob die Fusionierung eher einer Und-Verknüpfung oder eher einer Oder-Verknüpfung der beiden Existenzwahrscheinlichkeiten, d. h. der Existenzwahrscheinlichkeit des Fusionsobjekts zum Zeitpunkt k und der Existenzwahrscheinlichkeit des Sensorobjekts zum Zeitpunkt k + 1 entsprechen soll. Darüber hinaus wird über die sensoreinrichtungsabhängigen Fusionierungskonstanten δ^a und δ^b die Möglichkeit geschaffen, die von unterschiedlichen Sensoreinrichtungen gelieferten Existenzwahrscheinlichkeiten unterschiedlich relativ zu der bereits in dem Fusionsobjekt fusionierten Existenzwahrscheinlichkeit zu gewichten. Hierüber kann berücksichtigt werden, dass die Zuverlässigkeit von Existenzwahrscheinlichkeitsaussagen der einzelnen Sensoreinrichtungen eine unterschiedliche Zuverlässigkeit und Güte aufweisen. Ferner kann abhängig von der Güte der Assoziation ebenfalls eine Wichtung der ”alten” Fusionsobjekt-Existenzwahrscheinlichkeit P_FusObj(k) relativ zur Existenzwahrscheinlichkeit P_SenObj(k + 1) des Sensorobjekts erfolgen.A further development provides that the probability of existence of a fusion object that can be associated with a sensor object is calculated according to the following formula:

where P _FusObj (k) _indicates the existence probability of the fusion object at time k, P _SenObj (k) the _{probability of} existence of the sensor object at time k, γ a sensor-independent fusing constant and δ ^a and δ ^b sensor device-dependent fusing constants, where: δ ^a + δ ^b = 1. The given formula, which is borrowed from the fuzzy logic, allows via the fixed choice of the constant γ a preselection, whether the merger rather an AND-link or rather an OR-linking of the two probabilities of existence, ie the likelihood of existence of the fusion object at the time k and the likelihood of existence of the sensor object at time k + 1 should correspond. In addition, via the sensor device-dependent fusing constants δ ^a and δ ^{b it} is possible to weight the existence probabilities provided by different sensor devices differently relative to the probability of existence already fused in the fusion object. This can take into account that the reliability of existence probability statements of the individual sensor devices have a different reliability and quality. Furthermore, depending on the quality of the association, a weighting of the "old" fusion object existence _probability P _FusObj (k) relative to the existence _probability P _SenObj (k + 1) of the sensor object can also be carried out.

In order to be able to take into account new objects occurring in the environment during the merger, is at a Embodiment of the invention provided that for sensor objects that can not be associated with any of the known fusion objects, the fusion includes generating a new fusion object, which is assigned as existence probability, the likelihood of existence of the sensor object.

While an environment representation of sensor objects and fused therefrom Fusion objects especially good for moving, d. H. moving in a fixed frame of reference, objects in the environment of the vehicle appropriate, can provide an improved environment representation can be achieved if of the sensor devices in addition Information is provided by which an occupancy card is created, where the environment is divided into areas and each cell is assigned a cell, the cells together represent the occupancy card, and is provided for To determine the cells occupancy probabilities, which is a measure of an existence of an object in the area assigned to the cell of the environment. In addition to an object recognition in the sensor devices used to derive the sensor objects thus providing information that it provides allow to create an occupancy map of the environment. The creation of the occupancy card is usually carried out in such a way that the individual cells at least one value is assigned, which the likelihood of existence of an object in the cell associated with it Specifies the space area of the environment. An advantage of such a procedure for presentation of the environment or provision of an environment representation lies in the fullest possible use the information of the at least two sensor devices, wherein on the one hand, the good occupancy probability statements in the generated by the cells occupancy maps are generated especially for static objects a high informative value and high dynamic objects Existence probabilities about the merger objects can be obtained.

A Significant further improvement can be achieved, though the results of the object-oriented environment representation with a card-based environment representation are combined. In a preferred Embodiment is therefore intended that for Fusion objects which at a measurement time k + 1 none of the sensor objects can be assigned, the likelihood of existence is merged by replacing the livelihood of the corresponding sensor object determined using the occupancy map Occupancy probability for the area or areas is used, on which or the fusion object according to the Prediction stops. Such situations where no corresponding sensor object for a fusion object is supplied by a measuring device, for example in a radar system, if another static or dynamic Object in the environment corresponding to the corresponding to the fusion object Object hidden in the meantime. This means that the object in the environment corresponding to the fusion object the radar beams can not be detected because another object, for example, a bridge pier or a traffic island mounted road sign, the field of vision of the radar on the corresponding obscured real object in one or more measurement cycles. In a In such a situation, a sensor device is often not able to provide a sensor object for this real object, since the sensor device for this purpose, the real object itself over capture and track for a period of time to make sure that there is a real existing object. The raw data, on the other hand, can often be earlier and more reliable a statement about an occupancy of a deliver specific area around the motor vehicle. Consequently the merging is improved by the fact that the occupancy card known existence probabilities for objects in one certain area corresponding to the area, in which the fusion object according to the prediction has moved.

In one embodiment, it is provided that the fusion takes place according to the following formula: P FusObj (k + 1) = [P FusObj (K) · P Grid (k + 1)] 1-γ · [1 - (1 - P FusObj (k)) · (1 - P Grid (k + 1))] γ . where P _Grid (k + 1) indicates the occupancy probability derived from the occupancy map at the predicted location of the fusion object. In this case, γ again specifies a fusing constant which determines whether the fusing is more similar to an AND combination or an OR combination of the probabilities of existence or occupancy probabilities.

Furthermore, it is advantageous to carry out a classification of the objects. Here, a classification of the sensor objects in terms of their type and preferably a classification of the fusion objects in terms of their type. A classification refers to an object type or an object class. These include pedestrians, cyclists, cars, trees, houses, etc. Particularly advantageously, the classification is made with comparative data, which are either present in the sensor devices and / or stored in the evaluation. For example, reference data can serve as comparison data. For a sensor device which is designed as a stereo camera, for example, a height ratio to egg A width ratio of a detected object within a defined space or a defined area of the environment are determined. On the basis of this ratio can be determined, for example, if it is a pedestrian. The classification based on reference data can be carried out both in the sensor devices and in the evaluation device or alternatively in the sensor devices or the evaluation device. The classification thus represents a possible attribute of the sensor objects and also of the fusion objects. As further attributes, specifications about the geometry of the objects, such as a length, a width and / or a height are determined. It is also advantageous to determine a dynamics of the objects. By the dynamics of the objects is meant a yaw rate, a velocity, an orientation as well as an acceleration etc. of an object. Furthermore, dynamics that characterize a vehicle-related driving behavior, such as an overtaking process, an entry and / or Ausparkvorgang, an emergency braking process, an acceleration process, a deceleration process, a crossing of a lane of pedestrians and / or cyclists, etc., to name a few characterizing the dynamics To call properties of the dynamics of objects.

As well like the existence probabilities of the individual sensor objects can also undergo further attributes of a merger become. A preferred embodiment therefore provides that the sensor objects and fusion objects with regard to further Attributes undergo a fusion process, for example using a Kalman filter. This means that one Dynamics and prediction, for example by means of a Kalman filter can be executed.

to Another embodiment of the invention, it is advantageous that the Information processed in the evaluation unit for a detailed Description of the environment are merged. The evaluation unit can be a computer that receives, processes and processes information put the results together. The joined together Results can also be graphically evaluated by the evaluation unit be prepared so that they on an output unit, z. B. a display device can be displayed. The detailed results are used to advantage by other systems based on environmental information are instructed, for example, to steer the vehicle in the Longitudinal and / or transverse guidance and / or dynamics intervene, take precrash measures, etc. Therefore it is advantageous if the environment representation at least also provided in electronic form.

In The evaluation device is information of an expert system settled or implemented. The expert knowledge, which partly contained in the merger regulations and formulas additionally evaluate further information and usable to improve the information fusion. At a Embodiment of the invention is provided that by means of at least one third sensor device information about the environment and / or the at least one sensor device and / or the at least one second device is detected and the sensor device dependency the merger depending on the information of at least a third sensor device is varied. In the third sensor device For example, it may be a sensor that has rain in it Environment of the motor vehicle detected. Is it a the sensor devices, for example, a stereo camera, so it is known that in heavy rain in terms of object detection is disturbed. Thus, it is advantageous, for example in the fusion of the existence probabilities of the sensor objects, that provided by the designed as a stereo camera sensor device Information weaker weighted in the merger objects to merge, as is the case when dry conditions to rule. Likewise, especially light conditions, For example, a low-level, directed counter to the direction of travel Sunlight plays a crucial role in goodness the data received by the stereo camera. Other sensor devices, who can use other measurement methods and principles different depending on environmental influences deliver reliable measurement results. It is also advantageous an expert system containing information of the sensor devices compares, for example, in terms of their classification, and at least incorporating the data of individual sensor objects only in the fusion, if this through a sensor object of another sensor device plausibility and / or confirmation. If a radar For example, a stationary Gullideckel on one Road captures, then this information is from the expert system in one embodiment only passed if the information confirmed by a camera.

In particular, in order to consistently improve an object-based environment representation, such as provided via the merge objects, with the environment representation provided via the occupancy card, in one embodiment it is provided that tracking information is assigned to the merger objects that is a measure of a goodness of a follow-up of the entity Indicates fusion object over a plurality of measurement times, and for the fusion objects representing moving objects for which the tracking information indicates sufficient tracking quality, the occupancy map cells located in the extension area of the fusion object are designated as occupied and these cells are denoted by Be probability of likelihood corresponds to the likelihood of the merger. This creates an occupancy map representation in which both the static objects and the dynamically moving objects are represented with a high probability of existence at the positions that correspond to the optimal "knowledge level" of Umweisfassung. It is understood that the cells to which assignment and likelihood of existence or occupancy probability are assigned via the fusing objects are identified separately, so that when the occupancy map is updated, the movement of the fusion object is taken into account.

When Particularly advantageous here is an embodiment, where the occupancy probabilities involved in the creation the occupancy card without consideration of the merger objects determined are stored for the cells, if their occupancy probability changed due to a merger object. Has the merging object is from the area assigned to the cell moved away, d. H. then the occupancy state can be removed and / or the occupancy probability of the cell previously stored were reassigned to the cell or into the merger of the newly acquired information of the sensor devices involved become.

The at least one first sensor device preferably comprises one Stereo camera and / or the at least one second sensor device a radar or lidar (Laser Imaging Detection and Ranging System).

The advantages described in connection with the method correspondingly for the corresponding device features the device according to the invention.

following the invention with reference to a drawing based on Embodiments explained in more detail. Hereby show:

1 a schematic block diagram of an apparatus for providing an environment representation of a vehicle;

2 a schematic representation of an explanation of a card-based environment representation;

three a schematic representation for describing the information acquisition with a radar system;

4 a schematic representation for explaining the information acquisition by means of a stereo camera; and

5 a schematic representation of a method for environment representation.

In 1 is schematically an embodiment of a device 1 for providing an environment representation of a vehicle. The device 1 comprises a first sensor device, which is designed for example as a radar or lidar. Such a device emits in different directions electromagnetic radiation which is reflected back to objects in the environment to the vehicle. Based on the transit time and a possible frequency shift, a distance and a relative speed of the object to the vehicle can be determined, at which the first sensor device 2 is arranged. The device 1 further comprises a second sensor device three , which is designed for example as a stereo camera. A stereo camera comprises two image acquisition units arranged at a distance from one another, which capture, for example, a left and a right camera image of the same spatial region of the environment. The stereo camera uses a disparity between the left and right camera images to calculate depth information of objects that are detected in the images. As a result, a 3D reconstruction of the objects detected in the images in the environment of the vehicle is possible on which the second sensor device is arranged. It is understood by those skilled in the art that the device for providing an environment representation may comprise further sensor devices which detect the surroundings of the vehicle according to the same or different measurement principles.

The first sensor device 2 and the second sensor device three are designed so that they make a pre-evaluation of the information collected and identify objects in the environment of the motor vehicle. The information about the individual detected objects is combined into a so-called sensor object, which comprises the attributes that characterize the corresponding object. These attributes can include, for example, the position, a speed, an orientation of the movement, a Expansion in the different spatial directions, etc. include. Likewise, the individual recognized objects can be classified according to their object type. This can be done, for example, by comparing the acquired data with reference data. The sensor devices 2 . three are further designed so that all sensor objects, ie all detected objects, are provided with an existence probability as an attribute. This likelihood of existence is a measure of the fact that in the environment detection of one of the sensor devices 2 . three recognized object actually exists in the environment of the motor vehicle.

The first sensor device 2 and the second sensor device three are with an evaluation device 4 coupled. The information about the detected sensor objects, preferably in the form of an object list, is transmitted to them. In the evaluation device, the detected sensor objects of the various sensor devices 2 . three merged into merger objects. Different methods of fusion can be used here. These can be adapted for the individual attributes. The evaluation device 4 For example, an expert system 5 in which expert knowledge is stored, which makes it possible to classify the fusion objects generated in the fusion more reliably than, for example, the individual sensor devices is possible. For example, while the spatial dimensions of a detected object can often be determined well using a stereo camera, it is not possible to determine dynamic statements, for example a relative speed to the vehicle. However, this information can be determined, for example, from a radar system based on the Doppler effect. If the information about a spatial extent and a detected relative speed are related to the known intrinsic speed of the vehicle, then individual objects that are detected can be classified more reliably. A parked refuse container and a small car, for example, have approximately identical dimensions. On the basis of a determined intrinsic speed of the detected object in a stationary coordinate system, however, it is possible to distinguish such a small car participating in the road from a parked container.

In particular, the evaluation device is designed for the first sensor device 2 and the second sensor device three detected sensor objects to identify fusion objects and in particular for this one existence probability from the existence probabilities of the sensor objects to merge. The fusion is carried out depending on the sensor device. For example, a weight of a likelihood of existence of a sensor object in the fusion is reduced when using a third sensor device 17 , which is formed for example as a rain sensor, rain is detected, which adversely affects a data acquisition of a camera (but not a radar measurement), and the sensor object of the configured as a stereo camera second sensor device three is provided. In some embodiments, it may be provided that the sensor devices 2 . three are designed so that they recognize as objects only those that move in a fixed coordinate system itself. This means that only dynamic objects are detected as sensor objects or to the evaluation device 4 be transmitted.

In order to obtain an improved environment representation, provide the first sensor device 2 and the second sensor device three to the evaluation device 4 In addition, information that enables the evaluation device to create an occupancy map for the environment of the vehicle. For this purpose, the environment of the vehicle, ie its environment, mentally divided into areas. Each area is then assigned a cell. The individual cells then form a so-called occupancy card. Each cell is assigned an occupancy state, which indicates whether an object is in the area of the environment assigned to the cell (state "occupied") or no object is located (state "free"). The cells may moreover assume the status "unknown" if it is not possible, based on the acquired information, to provide information about the corresponding area of the surroundings of the vehicle with regard to an occupancy state.

The person skilled in the art knows methods of how to create such an occupancy card. It is common for each cell to have at least one occupancy probability associated with it, which is a statistical measure of the cell being occupied or free. If the occupancy probability for the cell is limited to a value between zero and one, it is for example possible to assign the value "one" to the state "occupied" and the value zero to the state "free". Since individual measurements indicate only a finite probability for an existence of an object in a corresponding spatial area, it is advantageous to secure or to confirm the occupancy probability of a cell in the occupancy map by fusing the occupancy information of different measurement cycles. Frequently used as fusion methods are the Bayesian theorem, the Dempster-Shafer theory or a fuzzy logic. For example, the occupancy probability P i / slip (k + 1) of a cell i at time k + 1 can be calculated using the following fusion formula:

In this case, P i.sensor / document (k + 1) indicates the occupancy probability for cell i determined by the sensor device at time k + 1. Cells are initialized with an occupancy probability P _slip = 0.5, which means that the occupancy state is unknown.

In 2 is schematic in the upper part of a plan view of a street scene shown in which a motor vehicle 6 on a street 7 in one by an arrow 8th specified direction moves. At the left in the direction of travel roadside 9 are formed as trees and shrubs objects 10 , At the right in the direction of travel roadside 11 there is a guardrail 12 , Further, located on a side strip 13 another object 14 , To facilitate an orientation is a coordinate system 15 located. In the lower part of the representation of the 2 is an occupancy card 16 shown as coming from the motor vehicle 6 while driving has been detected. An occupancy probability is indicated by a hatching. The closer the hatching, the higher the occupancy probability. The motor vehicle itself is based on the occupancy probability 1 Although this is of course not detected by the sensor devices. In the occupancy card, the recognized items with the same reference numerals as in the upper part of 1 marked, but provided with an apostrophe.

Based on three and 4 It will be explained how the various sensor devices acquire and evaluate information for creating an occupancy map and for detecting sensor objects.

In three is a photographic illustration on the right 21 an area in front of a motor vehicle as viewed from the motor vehicle. To recognize is a roadway 22 on which a preceding vehicle 23 located. Further, the lane is on the left 22 a street tree 24 to recognize. At a greater distance laterally to the left of the road 22 are also parked vehicles 25 weak to recognize. On the left side of the three the raw data collected by a radar system is plotted graphically. The ordinate 27 indicates a distance in front of the motor vehicle while the abscissa 28 oriented transversely to the direction of travel. In a detection area 32 The radar uses gray levels to encode a signal strength for those signals that have been reflected back to the vehicle from the corresponding area. The darker an area is, the higher the measured signal strength. Good to see are the strong reflection signals 29 that with the street tree 24 correspond, as well as the reflection signals 30 that with the preceding vehicle 23 correspond. Over a weaker hatching are also reflection signals 31 to recognize the parked vehicles 25 can be assigned. This acquired information can be used directly by the evaluation device to create an occupancy card (cf. 1 ) be used. An occupancy probability is given here by the signal strength of the reflection.

The sensor device is further configured so that individual objects, here the street tree 24 and the preceding vehicle 23 , identified as objects in the environment of the motor vehicle and possibly classified, for example, via a determined airspeed by evaluation of a Doppler shift. A classification can also be done for example via a height-expansion ratio.

In 4 Analogously, the data acquisition for a stereo camera is shown schematically. On the left side is again a photographic image 41 of the area in front of the motor vehicle. To recognize is a roadway 42 , several vehicles in front 43 - 45 , a street lamp 46 , some street trees 47 as well as a hedge or wall 48 , On the basis of a disparity between two recordings taken at a distance from one another, a 3D reconstruction of the graphically recorded elements and objects is undertaken. As a result, both a position of the objects in the plane in which the vehicle moves and a height of the objects above this plane can be determined. On the right side of the 4 is a plan view of the space in front of the vehicle graphed. The ordinate 51 again indicates the distance from the vehicle in the direction of travel and the abscissa 52 an orientation transverse to the direction of travel of the vehicle. About a gray level in each case a height of the reconstructed pixels in the corresponding area is indicated. The heights correspond to heights of the detected objects in relation to the plane. To recognize are both the wall / hedge 48 ' as well as the vehicles in front 43 ' - 45 ' as well as the street lamp 46 ' can be seen in the illustration. A measure of the likelihood of existence is given by how many pixels within a surface area associated with the detected object have been detected. The displayed information can thus also be used to improve an occupancy map. Furthermore, it is possible, due to several consecutively recorded Ste reo images to perform a so-called object tracking. In this case, in one embodiment, the detected objects are selected which are dynamic objects. These are then in the form of an object list of the evaluation device 4 (see 1 ) provided.

In order to determine the probability of existence of the detected objects improved, they are tracked by the individual sensor devices over several measuring cycles (tracked). In this case, based on the frequency with which an object was previously detected, a probability of existence is determined on the basis of its "lifetime", ie acquisition life. The more often an object is detected, the higher its probability of existence and the less likely it is that the detection is due to a measurement noise or the like. In addition, in a camera, for example, the number of 3D points assigned to the object is evaluated in order to obtain a probability of existence. The more 3D pixels are assigned to an object, the greater is its likelihood. However, it should be noted that because of the imaging perspective, objects at a great distance inevitably cover only a smaller image area and a number of pixels that can be assigned to them are forcibly lower than an object located in the immediate vicinity of the vehicle at detection located. Therefore, in one embodiment, the likelihood of existence is determined based on the number of detected and object-assigned measurement points as a function of a distance from the vehicle. For other sensor devices, other criteria can be used to determine a likelihood of existence of the detected objects. For radar or Lidarsensoren this example, as already mentioned above, a reflection signal strength is used. The individual probabilities of existence so determined for the individual objects are preferably fused in the individual sensor devices via an approach originating from the fuzzy logic, which is specified via the following fusing formulas for the probability of existence of individual objects as determined in the sensor devices. The gamma factors are used to select a link that can be selected between an OR link (γ = 1) and an AND link (γ = 0) adapted to the corresponding sensor type. The gamma factors are thus individually adapted for each individual sensor or each individual sensor device. For a stereo camera, the probability of existence of a detected sensor object is thus preferably calculated according to the following formula: P CameraObj (k) = [P lifespan (K) · P 3D points (K)] 1-γ · [1 - (1 - P lifespan (k)) · (1 - P 3D points (K))] γ . where P _lifetime (k) is the probability of existence derived from the number of previous detections and P is _{3D points} that are determined from the number of existence probabilities determined from the number of 3D points associated with the object. Accordingly, the fusion formula for a radar system-detected sensor object is: P RadarObj (k) = [P lifespan (K) · P Signalstäike (K)] 1-γ · [1 - (1 - P lifespan (k)) · (1 - P Signalstäike (K))] γ ,

Here, P _{signal strength (k)} indicates the existence probability of an object determined on the basis of the reflection _{signal strength} .

Even if the existence probabilities determined in the individual sensor devices already have a high quality, they can be significantly improved by fusing the sensor objects to fusion objects in the evaluation device. This is exemplary in 5 shown. A first designed as a radar sensor device 61 and a second sensor device designed as a stereo camera 62 capture the environment. These sensor devices 61 . 62 provide both information 63 for a card-based merger 64 can be used as well as information 65 about detected sensor objects for an object-based sensor fusion 66 , In some embodiments, the information includes 63 for the card-based merger, only information that is not relevant to the object-based merger 66 supplied sensor objects can be assigned. Accordingly, object-based fusion uses only those recognized objects that are recognized as dynamic, ie self-moving, objects.

At the card-based merger 64 the individual information is associated with cells of the occupancy map 67 and then an update 68 the individual cells of the occupancy map. Here, in particular, the previously determined occupancy probabilities of the cells are merged with the newly determined occupancy probabilities. You get an occupancy card 69 with cells to which occupancy probabilities are assigned.

In the object-based merger 66 are the sensor objects, the by the individual Sensoreinrichtun gene 61 . 62 are delivered in the form of object lists, associated with each other 71 and for example via Kalman filter tracking 72 tracked. For the individual merged objects, an initialization is dependent on whether a sensor object can be fused with an object previously detected 73 a merger object or an update 74 of the fusion object to which the sensor object can be associated. For fusion objects, which fall below a given probability of existence and can not be associated with any sensor object, finds a fault 75 instead of.

If an object is detected for the first time, the associated sensor object can not be associated with any of the previously determined fusion objects. Thus, a new fusion object is initialized whose existence _probability P _Fusobj (k) is assigned the existence _{probability of} the corresponding sensor _object P _Sensobj (k): P FusObj (k) = P SenObj (K).

If a sensor object can be assigned to a fusion object, the probability of existence of the fusion object at time k + 1 is calculated according to the following formula:

Here, δ _A and δ _B indicate weights for the sensor object and the fusion object. The following applies: δ _A + δ _B = 1. The weights are determined on the basis of a current sensor confidence level and a deviation between the predicted fusion object and the measured sensor object. Thus, if a large deviation in the tracking of the fusion object is detected, the existence probability of the sensor object is weighted less. The weighting of the sensor object is reduced when external influences for a sensor device affect its measurement reliability. For example, rain adversely affects a detection reliability of a stereo camera. Such information can, for example, via the third sensor device 1 at the device 1 to 1 be won. The weights are thus adjusted to current and sensor-dependent. As with the fusion formulas given above, the gamma factor again indicates whether the two fused existence probabilities are to be fused together rather than OR or as AND.

If a fusion object can not be assigned to any of the detected sensor objects, in a preferred embodiment its probability of existence is updated by using the occupancy probability at the location where the fusion object is to be located according to the prediction instead of a probability of existence provided by a sensor object on the basis of the occupancy map is to derive a likelihood of existence. A fusion of the likelihood of the merger object in such a case takes place according to the following formula: P FusObj (k + 1) = [P FusObj (K) · P Grid (k + 1)] 1-γ · [1 - (1 - P FusObj (k)) · (1 - P Grid (k + 1))] γ ,

γ is again a suitable fusing constant. P _Grid (k + 1) indicates the probability of existence derived from the occupancy map at time k + 1, preferably after updating the occupancy map.

can a fusion object can not be assigned to any of the sensor objects and if its probability of existence is smaller than a default value, For example, less than 0.2, the fusion object is discarded, d. H. not further tracked (tracked).

The object-based merger 66 thus delivers objects, fusion objects, with a very reliable probability of existence 76 ,

at an embodiment, it is provided that the dynamic Objects according to their classification, which is a Indicates extent or only occasionally, d. H. in a cell, in the occupancy card is entered and you there as Occupancy Probability the determined likelihood of existence of the corresponding fusion object is assigned. This gives a very much improved Occupancy map, in which both the static areas and the dynamic objects each with a high reliability in terms of occupancy probability or likelihood of existence are recorded.

In general, the rest of the vehicle systems are provided with an environment representation that optimally characterizes the dynamic objects and the static areas. It is possible that both the Fusion objects are provided with their existence probabilities as well as an occupancy card with and / or without inserted fusion objects.

It it is understood that the illustrated embodiments are merely exemplary embodiments. Especially can more sensor devices in the sensor data fusion be included. It is also noted that the merger the sensor objects are each executed via fusion objects is. This offers the advantage that the individual sensor devices not timed their sensor data available have to ask. Other embodiments may However, very well provide that sensor data individually different Sensor devices first associated with each other and be merged and only subsequently, if necessary additionally with a previously determined fusion object merged into an updated fusion object.

1

contraption for providing an environment representation

2

first sensor device

3

second sensor device

4

evaluation

5

expert system

6

motor vehicle

7

Street

8th

arrow

9

left Roadside (in the direction of travel)

10

objects

11

right Roadside (in the direction of travel)

12

barrier

13

shoulder

14

additional object

15

coordinate system

16

occupancy map

17

third sensor device

21

image

22

roadway

23

driving ahead vehicle

24

street tree

25

parked vehicle

27

ordinate

28

abscissa

29-31

reflection signals

32

detection range

41

Illustration

42

roadway

43-45

driving ahead vehicles

46

street lamp

47

tree

48

Hedge / wall

51

ordinate

52

abscissa

53

edge strips

61

first sensor device

62

second sensor device

63

information for a card-based merger

64

map-based fusion

65

information for a sensor object fusion

66

object-based fusion

67

Associate with cells

68

To update the cells

69

occupancy map

71

association

72

Kalman filter tracking

73

initialization

74

Update

75

rejection

76

Provide of objects with probability of existence

77

Provide an environment representation

QUOTES INCLUDE IN THE DESCRIPTION

This list The documents listed by the applicant have been automated generated and is solely for better information recorded by the reader. The list is not part of the German Patent or utility model application. The DPMA takes over no liability for any errors or omissions.

Cited patent literature

• - DE 102006035207 [0003]

Claims (23)

Method for providing an environment representation of a vehicle with at least one first sensor device ( 2 ) and at least one second sensor device ( three ) and an evaluation device ( 4 ), characterized in that the sensor devices ( 2 . three ) Information about in an environment of the vehicle ( 6 ) recognized objects ( 10 . 14 ) in the form of sensor objects, wherein a sensor object is detected by the respective sensor device ( 2 . three ) recognized object ( 10 . 14 ) and the sensor objects as an attribute at least one probability of existence of the represented object ( 10 . 14 ) and that of the at least one first sensor device ( 2 ) and of the at least one second sensor device ( three ) recognized sensor objects of an object fusion ( 66 ), in which fusion objects are generated to which at least one probability of existence is assigned as an attribute, wherein the existence probabilities of the fusion objects are fused based on the existence probabilities of the sensor objects, the fusing of the likelihood of existence of one of the sensor objects depending on the respective sensor device ( 2 . three ), from which the corresponding sensor object is provided. Method according to claim 1, characterized in that that for the fusion objects determined at the time k a prediction is made at a time k + 1 and the sensor objects with the fusion objects determined at time k + 1 whose prediction they match according to association requirements, where the likelihood of existence of the fusion object at the time k + 1 on the basis of the likelihood of the fusion object for Time k and the probability of existence at time k + 1 of the sensor object is fused. A method according to claim 2, characterized in that the probability of existence of a fusion object that can be associated with a sensor object is calculated according to the following formula:

where P _FusObj (k) the existence probability of the fusion object at time k, P _senObj (k) the existence _{probability of} the sensor object at time k, γ a sensor-independent fusing constant and δ ^a and δ ^b sensor device-dependent fusing constants, where: δ a + δ b = 1. Method according to one of the preceding claims, characterized in that for sensor objects, with can not be associated with any of the known fusion objects, the fusion comprises generating a new fusion object, which as likelihood of existence the likelihood of existence of Assigned to the sensor object. Method according to one of the preceding claims, characterized in that the sensor devices provide information on the basis of which an occupancy card ( 16 . 69 ), wherein the environment is subdivided into areas and a cell is assigned to each area, wherein the cells represent the occupancy map and occupancy probabilities are determined for the cells, which is a measure of an existence of an object in the area of the surrounding area associated with the cell specify. Method according to one of the preceding claims, characterized in that, for fusion objects which can not be assigned to any of the sensor objects at a measurement time k + 1, the probability of existence is fused by replacing the existence probability of a corresponding sensor object with a reference to the occupancy map ( 16 . 69 ) occupancy probability is used for the area or areas where the fusion object is located according to the prediction. Method according to one of the preceding claims, characterized in that the fusion takes place according to the following formula: P FusObj (k + 1) = [P FusObj (K) · P Grid (k + 1)] 1-γ · [1 - (1 - P FusObj (k)) · (1 - P Grid (k + 1))] γ , where P _Grid (k + 1) indicates the occupancy probability derived from the occupancy map at the predicted location of the fusion object. Method according to one of the preceding claims, characterized in that the sensor objects with regard to their Type can be classified. Method according to one of the preceding claims, characterized in that the sensor objects and fusion objects subjected to a fusion process in terms of further attributes using, for example, a Kalman filter. Method according to one of the preceding claims, characterized in that by means of at least one third sensor device ( 17 ) Information about the environment and / or the at least one first sensor device ( 2 ) and / or the at least one second sensor device ( three ) and the sensor device dependency of the fusion depending on the information of the at least one third sensor device ( 17 ) is varied. Method according to one of the preceding claims, characterized in that the merging objects have a tracking information is assigned, which is a measure of a goodness a follow-up of the fusion object over several measurement times indicates, and for the fusion objects, the moving objects represent and for whom the tracking information indicates a sufficient tracking quality in the expansion area occupied cells of the occupancy map as occupied be labeled and these cells the occupancy probability is assigned the probability of existence of the fusion object equivalent. Contraption ( 1 ) for providing an environment representation of a vehicle ( 6 ), comprising at least one first sensor device ( 2 ) and at least one second sensor device ( three ), which capture the environment, and an evaluation device ( 4 ) associated with the at least one first sensor device ( 2 ) and the at least one second sensor device ( three ), characterized in that the sensor devices ( 2 . three ), information about each detected objects ( 9 . 14 ) in the form of sensor objects which, as an attribute, have at least one probability of existence for the detected object ( 10 . 14 ), and the evaluation device ( 4 ) formed by the at least one first sensor device ( 2 ) and of the at least one second sensor device ( three sensor objects to be subjected to an object fusion in which fusion objects are generated to which at least one probability of existence is assigned as an attribute, wherein the fusion of a probability of existence of the fusion object based on the likelihood of the sensor objects is executed and depending on the respective sensor device ( 2 . three ), from which the sensor object is provided. Contraption ( 1 ) according to claim 12, characterized in that the evaluation device ( 4 ) comprises a prediction unit which, for the fusion objects determined at the time k, predicts a time k + 1 which the evaluation device ( 4 ), which associates the sensor objects determined at time k + 1 with the fusion objects whose prediction matches the sensor objects according to association specifications, and the probability of existence of the fusion object at time k + 1 based on the probability of existence of the fusion object at time k and the probability of existence at time k + 1 of the sensor object is fused. Contraption ( 1 ) according to claim 13, characterized in that the evaluation device ( 4 ) is configured to calculate the probability of existence of a fusion object that can be associated with a sensor object according to the following formula:

where P _FusObj (k) the probability of existence of the fusion object at time k, P _SenObj (k) the _{probability of} existence of the sensor object at time k, γ a sensor-independent fusing constant and δ ^a and δ ^b sensor device-dependent fusing constants, where: δ a + δ b = 1. Contraption ( 1 ) according to one of claims 12 to 14, characterized in that the evaluation device ( 4 ) is designed, for sensor objects that can not be associated with any of the known fusion objects, to generate a new fusion object in the fusion, which is considered a probability of existence the probability of existence of the sensor object is assigned. Contraption ( 1 ) according to one of claims 12 to 15, characterized in that the evaluation device ( 4 ) comprises a card generating device and the sensor devices ( 2 . three ), provide information on the basis of which the card-generating device generates an occupancy card ( 16 . 69 ), where the environment is subdivided into areas and each area is assigned a cell and the cells are the occupancy map ( 16 . 69 ), and the card generating device is designed to determine occupancy probabilities for the cells, which are a measure of the existence of an object ( 10 . 14 ) in the region of the environment associated with the cell. Contraption ( 1 ) according to claim 16, characterized in that the evaluation device is designed so that for fusion objects that can be assigned to a measurement time k + 1 none of the sensor objects, the probability of existence is fused by instead of the likelihood of existence of a corresponding sensor object based on the occupancy card ( 16 . 69 ) occupancy probability is used for the area or areas where the fusion object is located according to the prediction. Contraption ( 1 ) according to claim 17, characterized in that the fusion takes place according to the following formula: P FusObj (k + 1) = [P FusObj (K) · P Grid (k + 1)] 1-γ · [1 - (1 - P FusObj (k)) · (1 - P Grid (k + 1))] γ , where P _Grid (k + 1) indicates the occupancy probability derived from the occupancy map at the predicted location of the fusion object. Contraption ( 1 ) according to one of claims 12 to 18, characterized in that in at least one of the sensor devices ( 2 . three ) and or the evaluation device ( 4 ) Comparative information is available, based on which the sensor objects are classifiable in terms of their type. Contraption ( 1 ) according to one of claims 12 to 19, characterized in that the evaluation device ( 4 ) is configured to subject the sensor objects and fusion objects to a fusion method with regard to further attributes, for example using a Kalman filter. Contraption ( 1 ) according to one of claims 12 to 20, characterized in that at least one third sensor device ( 17 ) is coupled to the evaluation device, the information about the environment and / or the at least one first sensor device ( 2 ) and / or the at least one second sensor device ( three ), and the evaluation device ( 4 ), the sensor device dependence of the fusion depending on the information of the at least one third sensor device ( 17 ) to vary. Contraption ( 1 ) according to one of claims 12 to 21, characterized in that the evaluation device ( 4 ) is adapted to associate tracking information with the fusing objects indicative of a merit of tracking the fusion object over multiple measurement times, and for the fusion objects representing the moving objects for which the tracking information indicates sufficient tracking quality in the extent range of the fusion object lying cells of the occupancy card ( 16 . 69 ) and to assign to these cells the occupancy probability corresponding to the likelihood of existence of the fusion object. Contraption ( 1 ) according to one of claims 12 to 22, characterized in that the at least one first sensor device ( 2 ) comprises a stereo camera and / or the at least one second sensor device ( three ) comprises a radar or lidar (Laser Imaging Detection and Ranging System).