DE102022125695A1 - Method and device for efficient object tracking and correspondingly equipped motor vehicle - Google Patents

Abstract

The invention relates to a method (1) and an object tracking device (13) for object tracking. The invention further relates to a motor vehicle (13) equipped accordingly. In the method (1), the state of motion of objects (17) detected using sensor data (3) from a sensor system (12) is determined and, accordingly, each object (17) is classified as a moving or stationary object. Object data (2) is stored for the objects (17), which includes several properties of the respective object (17) and the state of motion determined for it. The object data (2) are updated using the current sensor data (3) for objects (17) already detected in a previous measurement cycle. In this case, only for those detected objects (17) that were classified as a stationary object (17) do their properties stored in the object data (2) become updated independently of one another.

Description

The present invention relates to a method for object tracking based on sensor data. The invention further relates to a corresponding device for the method and a motor vehicle equipped therewith.

In various technical applications and areas - for example, but not exclusively in the vehicle and traffic sector - sensory detection and tracking, i.e. tracking of objects in a particular environment, can be useful, for example to avoid collisions, to plan trajectories, for monitoring and/or the like. However, the sensors and measurement methods used, especially when combined with automatic object recognition, are not always completely exact and precise, which can be undesirable or even safety-relevant depending on the application. For example, an adaptive cruise control (ACC) in a motor vehicle requires the most precise detection of a vehicle in front and the most precise determination of its position. In particular, the distance between the driver's own vehicle and a vehicle in front should not be overestimated if the vehicle in front is currently stationary. The result of such an overestimation could be an unintentional automatic restart of the driver's own vehicle, which in turn could lead to a prescribed minimum distance being undercut or to a collision.

An approach for advanced object tracking using radar is described, for example, by EN 10 2020 123 585 A1 The method therein includes generating one or more clusters based on received return points from a radar sensor of a vehicle. An estimated position and speed are then calculated for each of these clusters. It is also determined whether the clusters are linked to an existing object track. According to the one or more clusters that are linked to the existing object track, the existing object track is then updated using at least the respective position of this one or more clusters.

For effective object tracking, a state of a particular object can typically be determined in some form. For example, the EN 10 2019 109 332 A1 a method for determining an object state at a sequence of points in time. An occupancy grid is determined on the basis of sensor data, which shows evidence for each of a large number of cells that the respective cell is free or occupied by an object at a point in time. Based on this, a subset of the cells that belong to the object at that point in time is determined. Finally, the object state for the point in time is determined on the basis of an object state predicted for that point in time and on the basis of the determined subset of cells of the object. This allows an object to be detected and information relating to the object to be determined in a precise and robust manner.

The object of the present invention is to enable particularly efficient sensor-based object tracking.

This object is achieved by the subject matter of the independent patent claims. Further possible embodiments of the invention are disclosed in the subclaims, the description and the figures. Features, advantages and possible embodiments that are set out in the description for one of the subject matter of the independent claims are to be regarded at least analogously as features, advantages and possible embodiments of the respective subject matter of the other independent claims and of any possible combination of the subject matter of the independent claims, if appropriate in conjunction with one or more of the subclaims.

The method according to the invention can be used for object tracking, i.e. for object tracking over several measurement cycles of a sensor system for environmental or object detection. In particular, the method can be used in a motor vehicle, but is not limited to this application. In a method step of the method according to the invention, objects are detected using sensor data recorded in a measurement cycle of the sensor system. In other words, current sensor data from the sensor system can be recorded in each measurement cycle and evaluated or processed for object detection or object recognition. The sensor system can comprise one or more individual sensors, whereby several such individual sensors of the sensor system can in particular be of different types. Sensor data from a single sensor or several sensors can therefore be recorded in a measurement cycle, for example depending on measurement or recording frequencies or a synchronization of several individual sensors. The sensor system or the individual sensors it comprises can themselves detect and track objects internally. Corresponding objects detected and tracked within the sensor, i.e. managed, can then be referred to as sensor objects. Sensor data provided by the sensor system can also be processed to detect objects outside the sensor system. for example, by an object tracking or tracking device set up to carry out or apply the method according to the invention or coupled to the sensor system, i.e., for example, a correspondingly set up control device or assistance system or the like.

In a further method step of the method according to the invention, the motion state of detected objects is determined based on the sensor data. Each detected object is classified as a stationary object or as a moving object. The motion state of an object can therefore indicate whether this object is in motion or at rest. A stationary object can, for example, be a stopped object that could in principle move, or a stationary object that is naturally stationary and/or has never been detected in motion at least within the scope of the current application of the method. The motion states of the detected objects can, for example, be determined in relation to a predetermined global coordinate system or relative to the sensor system or a device equipped with it, such as a motor vehicle or the like. To determine the motion state, the sensor data or data or quantities derived therefrom can, for example, be evaluated with regard to at least one predetermined criterion. For example, a speed threshold value or statistics can be evaluated across multiple detections of the respective object in successive measurement cycles and/or across multiple individual sensor detections assigned to the respective object, in particular from the current measurement cycle. Such individual sensor detections can be individual data or measurement points in the present sense, for example individual radar echoes or radar detections or lidar detections or lidar reflections and/or the like.

Furthermore, in the method according to the invention, object data associated with each object detected using the sensor data is stored, which includes several properties of the respective object and its determined state of motion. The object data can therefore each be an object data set describing or characterizing the respective object, which is also referred to as an object track. Objects detected in the sensor data or represented by it can therefore each be assigned to such an object data set or object track, in which the respective properties of the object are stored or managed. The object data is therefore each object-specific, so that a corresponding object track is created or managed for each detected object. The objects or their properties can thus be tracked or managed in the object data over several measurement cycles. A new object track can be created for objects newly detected in the current measurement cycle for which no object track with properties determined using sensor data from at least one previous measurement cycle has been created or exists.

The properties stored or specified in the object data or object tracks can be or specify static properties and/or dynamic states or state data of the respective object. Static properties in this sense can be or specify, for example, a type or kind of the respective object. Dynamic states or state data can be or specify - for example from measurement cycle to measurement cycle - at least potentially changeable data or quantities of the respective object, such as a position and/or a speed and/or an acceleration and/or a yaw angle and/or a yaw rate and/or the like. The object tracks can also be referred to as internal tracks because they can be managed, for example, within the sensor or within the sensor or within the object tracking device close to the sensor along a corresponding signal or data processing pipeline. The motion state stored in the sensor objects or as part of the sensor objects and/or in the respective object data can, for example, be derived directly from the movement speed of the object and/or estimated by the sensor system or the respective individual sensor and/or - for example by the object tracking device - determined or estimated from the sensor data or sensor objects or object data originating from the sensor system. In this case, data or estimated motion states from several sensors can be combined - if available. Depending on the sensor, the movement speed of an object can, for example, be measured directly or estimated from available sensor data, for example over several measuring cycles, i.e. based on a change in the sensor data over several measuring cycles.

A stationary object can be recognized as such, for example, directly from the respective movement or object state, i.e. the data or properties for an object, as determined by a sensor and/or based on the sensor data from one or more sensors. An object can therefore be recognized or classified as a stationary object, for example, if the complete object data for this object contains a value below a specified value. speed threshold value is specified. Such a speed threshold value can be specified as 0.5 m/s, for example, although other values may also be possible depending on the application or requirements. The complete object data can be understood here, for example, in contrast to individual sensor detections or raw sensor data assigned to the object. The object data can be or include data or properties of the respective object determined or derived from such sensor detections or raw sensor data.

Depending on the design of the sensor system, the state of motion of detected objects can be determined or classified and/or at least some of the static and/or dynamic properties can be determined or estimated for them by one or more sensors that have detected a specific object, either within the sensor or within the sensor system. This can also be used as a basis for classifying objects as a stationary object or as a moving object. If the sensor system includes several, in particular different, sensors, an object can be classified as a stationary object or as a moving object, for example, if the states of motion determined internally for this object by several sensors match accordingly. This can enable a particularly fast and at the same time particularly reliable classification. In addition, such a classification can also be particularly flexible and robust, since, for example, the same sensors do not always have to be used. This can be particularly useful, for example, if an object cannot be detected by a sensor in the meantime, for example due to obscuration or the like.

In a further method step of the method according to the invention, the object data or at least the dynamic states contained therein of objects already detected in a previous measurement cycle are automatically updated on the basis of the current sensor data, i.e. data recorded in the current measurement cycle. This ensures that current data or information on the respectively assigned object is or will be stored in the object data, i.e. that the stored object data takes into account the current measurement or sensor data. The updating of the object data provided here can be carried out at least for existing objects, i.e. objects already detected in one or more previous measurement cycles. For objects newly contained in the current sensor data or newly, i.e. objects detected for the first time, a new object track can be created in which the current sensor data is also taken into account.

According to the invention, only for those objects that have been classified as stationary objects are their properties stored in the object data updated independently of one another. In other words, for a stationary object, its properties or at least its dynamic states or state data are decoupled from one another, i.e., updated without cross-correlations or mutual influences or dependencies, for example. This updating procedure can also be referred to as a stationary update or stationary update, since it is used for stationary objects. If the stored properties include, for example, a position and a speed of the object, then for a stationary object an updated position that has changed compared to the previous object data has no influence on the determined or stored speed. By updating the properties independently of one another, in this case the position and the speed, for example, a new position that has changed compared to the position previously stored in the object data does not automatically lead to a changed speed of the respective object or a speed that is different from zero, according to the current sensor data. For example, the speed of an object classified as stationary can be automatically kept at zero, unless an independent speed measurement shows a speed of the respective object that is different from zero or above a predetermined threshold. In this case, any corresponding dynamics can be prevented during the update for a stationary object.

This results in a correspondingly stable tracking of stationary objects with regard to dynamic, i.e. potentially changeable properties. This represents an advantage of the method according to the invention over conventional approaches. Such conventional approaches often use data fusion methods for object tracking and updating corresponding object data, which are applied to all detected objects. In conventional approaches, no separate logic or data processing methods are used for stationary and moving objects, but rather a complete recording of the respective object and a correspondingly complete processing of the corresponding sensor or object data is always carried out without adaptation to an actual physical or physically possible behavior of the respective physically real object. In reality, for example, a stationary vehicle cannot change its orientation or build up a yaw rate as long as it is stationary. In addition, measurements show uncertainties or variances in the sensor data, for example over several measurement cycles for an object that is at rest the entire time, do not indicate the real speed of the object.

Conventional approaches therefore fit all measurement points or sensor data for an object with a given model in order to ultimately determine a bounding box for the respective object and thus update the respective object data. If such bounding boxes or bounding frames for an object are redefined in each measurement cycle, they are not always stable from measurement cycle to measurement cycle, i.e. they are not always the same, even though the respective detected object is not actually moving. In such cases, a movement of the respective object - which does not actually exist - can still be falsely detected. For example, with a correspondingly high traffic density or in inner-city environments or the like, stationary measurement points can often occur in the immediate vicinity of a detected foreign vehicle, for example due to sensor detections assigned to the foreign vehicle, which can actually come from traffic infrastructure, such as the ground or a curb or the like, or from an exhaust plume or the like. Such measuring points or sensor detections can be temporarily detectable or detected without movement of the other vehicle, i.e. the object being detected, and can be incorrectly assigned to the respective object or integrated into the boundary box intended for it. This can lead to a speed, rotation or displacement being incorrectly detected even for a stationary object. Such incorrect detections or misjudgments can in turn lead to corresponding incorrect reactions by automated systems and thus potentially to accidents or erratic behavior.

The present invention addresses this problem by handling sensor and object data differently for objects classified as stationary and for moving objects, whereby the object data for stationary objects is updated in such a way that, as a result, an erroneous detection of non-existent movements or dynamic stationary objects is avoided or reduced.

For the remaining, i.e. moving objects, the sensor data can be updated in a different or conventional way, for example based on sensor objects that are already tracked by or in the sensor system and output by it, and a filter-based fusion with the existing, i.e. previous object data. For example, a Kalman filter or the like can be used, dependencies or cross-correlations between different properties of the respective object can be taken into account and/or the like. However, another data filter or fusion mechanism can also be used, especially if it effectively functions as a low-pass filter. Using such a predefined filter or fusion mechanism, current sensor data and the data or properties of the respective object already stored in the respective object data can be combined or fused in a weighted manner. This can contribute to advantageously smooth and consistent tracking, particularly in the case of uniform movements of detected objects. For detected objects that are not classified as stationary, their properties or at least their dynamic states or state data can be updated, for example, taking into account corresponding dependencies between the current sensor data or the corresponding properties or sizes determined from it and the previous object data. This allows for the fact that these properties or dynamic states of moving objects can actually change dynamically in a manner that cannot be precisely predicted but nevertheless with limited dynamics or rate from measurement cycle to measurement cycle.

It may happen that the sensor system or at least a specific sensor, such as a camera or the like used at least as part of the sensor system, does not provide any detections or sensor data in a current measurement cycle for a specific object, in particular an object already detected in a previous measurement cycle. If this object was previously classified as a stationary object, the update described for stationary objects can then also be carried out or applied for this object in the current measurement cycle.

The method according to the invention can be combined or expanded with further measures or method steps. For example, the previous object data from previously detected objects can first be predicted or extrapolated to the current point in time according to a predetermined model, for example the time at which the sensor data of the current measurement cycle was recorded, before they are used as described.

In a possible embodiment of the present invention, the respective movement state of the detected objects - at least for objects already classified as stationary in a previous measurement or update cycle - is determined using radar detections, i.e. radar data or radar echoes. These radar detections can therefore be part of the respective sensor data recorded. An object is classified as a stationary object if a speed of the respective object determined based on radar, i.e. based on the radar detections assigned to the respective object, is smaller in magnitude than a predefined speed threshold. Such a speed threshold can, for example, be based on a respective measurement uncertainty of the radar sensor used, i.e. take this into account. In other words, the radar detections, in particular a speed determined based on them, can be used for a corresponding standstill estimate.

This is based on the knowledge that radar detections can be particularly well suited to determining the state of motion due to the direct measurement of the double speed that they enable. In addition, radar detections can be used over one or more measurement cycles to estimate a stable distance between the sensor and the respective object using a predetermined robust estimation method that can be based, for example, on a median determination or a histogram. This can avoid or reduce a potentially dangerous overestimation or underestimation of the distance, as can occur, for example, with camera-based distance estimation. This means that functions based on the position and/or speed and/or distance of the respective object, such as automatic distance keeping or adaptive cruise control or the like, can be implemented particularly safely and reliably.

If the amount of the radar-based determined speed of the respective object is less than the specified speed threshold, a corresponding flag can then be automatically set in the corresponding data processing, for example in the object data for the respective object, whereby the respective object is classified or defined as a stationary object. For a stationary object, process noise in the respective signal or data processing can also be reduced. If the radar-based determined speed for the respective object indicates movement of this object in a measurement cycle, the process noise can be automatically increased again. In this way, a faster reaction to the respective object starting to move can be achieved, for example by an assistance system or function that uses the object data as input.

In a further possible embodiment of the present invention, an object is only classified as a stationary object, in particular only if at least a predetermined number of measurements and/or at least a predetermined proportion of several measurements carried out result in or indicate a speed of the respective object that is below a predetermined speed threshold value specified elsewhere. If the predetermined proportion of measurements is used as a criterion here, a predetermined minimum number of measurements on the basis of which this proportion is or was determined can also be specified before it is used or evaluated as a criterion for classifying the state of motion of the respective object. In the embodiment of the present invention proposed here, the state of motion of the respective object is not determined based on a single speed measurement, but rather based on several speed measurements. For example, the dynamics described in the object data can only be prevented when updating it for an object if several Doppler measurements for this object over one or more measurement cycles have indicated or confirmed that the respective object is completely at a standstill.

For example, the appropriate criterion for classifying an object as a stationary object can be that at least five measurements, which were carried out consecutively or within a predetermined period of time, result in or indicate a speed of the respective object that is below the predetermined speed threshold value. The speed threshold value can be, for example, 0.1 m/s or 0.5 m/s or the like. However, depending on the application and requirements, other values can also be specified. With regard to the proportion of measurements carried out as a criterion, it can be specified, for example, that of all measurements or sensor detections for the respective object within a predetermined period of time or within a sliding time window of a predetermined length, a maximum of 10% or a maximum of 20% or a maximum of 30% may indicate a speed that is above the speed threshold value in order to still classify the object as a stationary object. Here, too, the corresponding given proportion can depend on a particular application or on particular requirements or be sensor-specific. Overall, the embodiment of the present invention proposed here enables a particularly reliable and robust determination of the respective movement state of the detected objects and a correspondingly robust application of the method according to the invention.

In a further possible embodiment of the present invention, for objects classified as stationary, their object data is updated based on individual sensor detections, whereas for objects classified as moving, their object data is updated based on sensor objects output by the sensor system. Such sensor objects can, for example, be output by the sensor system or by a respective sensor in each measurement cycle, for example in the form of a list or the like. Sensor objects here are therefore objects that are tracked by the sensor system or the respective sensor itself, i.e. internally detected, and also over several measurement cycles. The sensor objects can therefore each have already been formed or reconstructed by the sensor system or the respective sensor from several sensor detections. The sensor objects can include or specify several properties of the respective underlying real object. For example, the sensor objects can each specify a bounding box and/or a position and/or an orientation and/or a speed and/or a yaw rate and/or the like for the respective object as a whole, i.e. not just for the individual sensor detections assigned to the object.

Based on such sensor objects, the respective update can be carried out for the objects classified as moving by applying a predefined filter or fusion mechanism, which can also be referred to here as a full update or complete update.

In contrast, a simplified update can be carried out for objects classified as stationary. In addition to at least one sensor detection assigned to the respective object, properties of the respective object already stored in the associated object data from one or more previous measurement cycles can also be used or taken into account. The individual sensor detections can be processed more easily and/or more quickly, i.e. with less effort overall, to update the respective object data than the sensor objects used only for updating the object data of the objects classified as moving.

The sensor detections associated with an object classified as stationary can be determined here based on the object data, i.e. the respective object track, in particular stored outside the respective sensor itself, in particular the dynamic state stored therein or the dynamic state of the respective object predicted based on this for the respective measurement time. This can be possible in particular because the position and location of a stationary object cannot change by definition and thus sensor detections associated with the respective object can be selected from all available sensor detections using a predetermined position- or location-based criterion. This means that sensor detections actually associated with a stationary object can be selected in a particularly reliable and robust manner and used to update the respective object data. In particular, the fact that the assignment of the sensor detections to a stationary object is not carried out by the respective sensor itself can reduce the probability that outliers, measurement errors, disturbances and/or drifting of the respective sensor lead to incorrect assignments of sensor detections to the respective object or to an ultimately incorrect position update for the respective object.

If the sensor system comprises several different sensors or if respective object sensor detections were recorded by different sensors in the current measurement cycle, these sensor detections can be taken into account in full or only in part, i.e. used for the respective update. This can, for example, depend on detected deviations or inconsistencies between the sensor detections of different sensors and a predefined prioritization list for the different sensors. Such a prioritization list can specify which sensor is to be prioritized for stationary objects in general or for the assignment of sensor detections to a stationary object or for determining or updating a specific property of a stationary object via other sensors or their sensor data. For example, for a stationary object, both lidar detections and radar detections can be present as sensor detections and/or the dynamic state stored in the existing object data, in particular the position of the respective stationary object, can be based on older lidar detections. Since radar detections for stationary objects, for example due to a tendency of radar sensors to drift, may be less suitable, and therefore have a lower priority, than lidar detections, the radar detections to be assigned to the stationary object can then be determined or selected, for example, based on the lidar-based position or a lidar-based bounding box for the respective stationary object. This can be done in particular if the lidar detections from the current measurement cycle continue to be associated with a standstill of the respective object at the known position already stored in the associated object data is consistent. This means that even if the radar detections drift or are inconsistent, at least some of the radar detections that match the current position of the respective stationary object can be used. This means that the sensor detections assigned to the respective stationary object can be determined more precisely and reliably and, if necessary, more sensor detections, i.e. more data, can be used or taken into account for the respective updating of the object data.

In a further possible embodiment of the present invention, for objects classified as stationary, a stable end detection is determined or selected from all current sensor detections that are assigned to the respective object or have currently been assigned in order to determine or set the current position of the respective object to be updated as a property in the object data. This end detection can be one of the sensor detections assigned to the respective object that is used as an end facing the sensor system - or, for example, a motor vehicle or the like equipped with it - or as the corresponding side of the respective object, i.e. is interpreted or viewed as such. The sensor detections are assigned to the respective object according to a predetermined criterion. This predetermined criterion can be distance-based in particular with regard to the previous position of the respective stationary object, i.e. already known or determined from a previous measurement cycle and stored or specified in the respective object data. In other words, the criterion can, for example, query whether a certain sensor detection is within a predetermined distance from the position of the respective stationary object specified in the object data. Only if this is the case is the respective sensor detection then assigned to the respective stationary object. The determined or selected stable end detection can then be viewed or used as the end of the respective object or as a reference for determining the position of the respective object, for example over several measurement cycles.

For example, for a stationary object, the same sensor detection, for example one sensor detection from the same location on the respective object, can be used as a stable end detection over several measurement cycles or as long as the respective object is or remains classified as stationary. Based on this stable end detection, the end or position of the respective object can then be determined unchanged, i.e. stably. This is particularly possible if, for example, other current sensor detections appear to indicate a different position for the respective object.

The stable end detection can also be determined, for example, as a median or mean value of the sensor detections assigned to the object that are closest to the sensor over several measuring cycles. This means that the position of the stable end detection and thus also of the respective stationary object can be kept stable or determined even if not all sensor detections assigned to the respective object are recorded consistently, in particular not in a positionally stable manner, over several measuring cycles.

The embodiment of the present invention proposed here thus makes it possible to track a stationary object particularly reliably, precisely, robustly and stably even in the case of consistent or fluctuating sensor detections and to react to it accordingly in an appropriate and robust manner.

In a possible development of the present invention, if the sensor detections assigned to the respective stationary object include radar detections, the stable final detection is determined or selected taking into account the corresponding radar cross sections (RCS, Radar Cross Section). This can therefore be used if the sensor system is a radar sensor or includes one. In particular, the radar detection closest to the sensor system, whose radar cross section is larger than a predetermined threshold value, can be determined as the final detection. For this purpose, for example, in particular over several measurement cycles, a histogram can be entered in which the radar cross section is plotted as a function of the distance of the radar detections from the sensor system.

By taking into account the radar cross sections or the specified threshold value therefor, the probability that scatter or clutter detections in the area of the respective object are taken into account to determine its position, which could distort the position and location of the bounding box for the respective object, can be reduced.

The threshold value for the radar cross section can be specified depending on an object type. This means that different threshold values for the radar cross section can be considered for different types or kinds of objects. This makes the method robust and reliably applied to different types or kinds of objects.

For a further improvement, it can be provided that several or all of the radar detections assigned to the respective object are taken into account for each measurement cycle, i.e. evaluated with regard to their radar cross-section to determine or select the final detection. This ultimately allows the position of the respective object to be determined particularly reliably and robustly. Since the radar detections can inherently also represent or include measurements or information on the Doppler speed, the speed of the respective object can also be determined or estimated based on these Doppler speeds according to the radar detections assigned to the object. The speed estimated in this way can then also be taken into account when updating the object data, i.e., for example, entered into it or merged.

In a further possible embodiment of the present invention, at least for objects classified as stationary, at least some of the properties stored or to be stored in the respective object data are determined based on lidar detections. The sensor system can therefore be a lidar sensor or include one. The lidar detections here are accordingly sensor detections of such a lidar sensor. The lidar detections from the current measurement cycle are assigned to the boundary frame for the respective object. For this assignment or association, for example, a predetermined position- or location-based criterion can be evaluated. Accordingly, only those lidar detections that are within a predetermined area around the position of the respective object specified in the object data can be assigned to the boundary frame. For this purpose, for example, a corresponding distance and/or an angle and distance range or the like can be specified. Likewise, for example, the distances of the lidar detections can be determined and evaluated by the sensor system, for example in a similar way to what is described elsewhere in connection with the radar detections.

Furthermore, a stable end detection is determined here based on a distribution, in particular a distance histogram, of these lidar detections assigned to the boundary frame of the respective object, which is used as an end or a corresponding side of the respective object facing the sensor system - or, for example, a motor vehicle or the like equipped with it - and is therefore viewed or interpreted as such. In this way, the position of the respective object is determined in particular on a lidar basis as a property of the respective object. The position of the end detection or of the end of the respective object defined by it is used here to determine the position of the object as a corresponding property of the object in the assigned object data. The lidar detections can be entered into the distance histogram, in particular over several measurement cycles, sorted according to their distance - for example to the sensor system or another specified reference point.

The or a lidar detection that is in the significant distance bin with or for the smallest distance or distance range can then be determined or selected as the final detection. A distance bin of the distance histogram can be significant in the present sense if a predetermined threshold value is exceeded by it or in it. Such a threshold value can be specified, for example, for the number of lidar detections in the distance bin or for an intensity, i.e. a corresponding reflectivity value of the lidar detection or lidar detections in the distance bin. The design of the present invention proposed here allows the position of a stationary object to be determined and tracked in a particularly stable, reliable and robust manner. Functions based on this can thus be used particularly reliably and intuitively, i.e. without erratic fluctuations or the like. This is possible thanks to the design of the present invention proposed here, in particular, if, for example, not always the same lidar detections or lidar detections originating from the same places on the object are recorded for a specific stationary object over several measurement cycles. The use of lidar detection proposed here can be particularly useful and effective for determining the position of stationary objects, since lidar sensors for stationary objects do not tend to drift or at least do so less than radar sensors.

In a further embodiment of the present invention, for objects classified as dormant, dependencies between different properties of the respective object stored or specified in the respective object data are removed for updating their object data. In other words, cross-correlations or couplings between different properties of the respective object can be deleted or removed or reset to zero. This can, for example, be dependencies between the position and/or the speed and/or the acceleration and/or the yaw rate. angle and/or the yaw rate of the respective object and/or the like. By removing these dependencies, these properties are decoupled from one another. As a result, for example, a change in the position of the respective object from the last measurement cycle or the last update of the position to the current measurement cycle for a stationary object does not automatically lead to a corresponding speed or acceleration or a changed yaw angle or yaw rate being induced or stored in the object data as part of the update.

Additionally or alternatively, for objects classified as stationary, a diagonal or diagonalized P-matrix is used or switched to for modeling or managing the respective object in the associated object data. A P-matrix can, for example, indicate uncertainties of an object model or deviations from it or its predictions for the properties of the respective object. By using a diagonal or diagonalized P-matrix, it is possible to prevent corresponding dependencies or cross-correlations or the like from building up with one or more other properties of the object as a result of updating a property. This can be seen, for example, in contrast to moving objects, where a changed position or a specific or changed speed or a specific or changed acceleration are each accompanied by changes in the other values compared to the previous object data. Therefore, for moving objects, for example, a P-matrix with non-zero secondary diagonal elements can be used. The embodiment of the present invention proposed here makes it possible to realize or implement the independent updating of the properties of stationary objects in a particularly simple, effective and reliable manner.

The invention also relates to an object tracking device that can be designed or configured in particular for a motor vehicle. For example, the object tracking device can be designed as an assistance system for a motor vehicle or as part of such an assistance system. The object tracking device according to the invention has an interface for recording sensor data and a data processing device for processing the recorded sensor data. For this purpose, the object tracking device, in particular its data processing device, can comprise, for example, a processing device, such as a microchip, microprocessor or microcontroller or the like, and a computer-readable data storage device coupled thereto. A corresponding operating or computer program can then be stored in this data storage device, which codes or implements the method steps, measures or sequences described in connection with the method according to the invention or corresponding control instructions. This operating or computer program can then be executable by means of the processing device in order to effect the execution of the corresponding method. The sensor data can be or comprise, for example, raw data, sensor detections, sensor objects and/or the like. The object tracking device according to the invention can in particular be the object tracking device or tracking device mentioned in connection with the method according to the invention or correspond to this. The object tracking device according to the invention can therefore in particular be set up to manage object data, i.e. object tracks for objects detected by the sensor system or represented in the recorded sensor data. This object data can, for example, be stored or managed in the data storage device mentioned. The object tracking device can, for example, provide the object data via the interface mentioned or another interface or make it retrievable or output it, for example for another assistance system.

The invention also relates to a motor vehicle that has a sensor system for detecting the environment and an object tracking device according to the invention coupled thereto. The motor vehicle according to the invention can in particular be the motor vehicle mentioned in connection with the method according to the invention and/or in connection with the object tracking device according to the invention or correspond to this. Accordingly, the motor vehicle according to the invention can have some or all of the properties or features mentioned in these contexts. For example, the motor vehicle according to the invention can have an assistance system for a predetermined driver assistance function or for assisted or at least partially automated driving of the motor vehicle. This assistance system can then be set up to carry out its function based on the object data managed by the object tracking device according to the invention.

Further features of the invention can be derived from the claims, the figures and the description of the figures. The features and combinations of features mentioned above in the description as well as the features and combinations of features shown below in the description of the figures and/or in the figures alone are not only in the combination specified, but also in other combinations or alone, without departing from the scope of the invention.

• 1 ein beispielhaftes Schemas zur Veranschaulichung eines Verfahrens zur sensorbasierten Objektnachverfolgung;
• 2 eine schematische Darstellung zur Veranschaulichung einer radarbasierten Objektzustandsabschätzung für die Objektnachverfolgung;
• 3 eine beispielhafte Histogrammdarstellung zur weiteren Veranschaulichung der Objektzustandsabschätzung; und
• 4 eine schematische Darstellung zur Veranschaulichung einer lidarbasierten Objektzustandsabschätzung.

The drawing shows in:

• 1 an example scheme to illustrate a method for sensor-based object tracking;
• 2 a schematic diagram illustrating a radar-based object state estimation for object tracking;
• 3 an exemplary histogram representation to further illustrate the object state estimation; and
• 4 a schematic diagram illustrating a lidar-based object state estimation.

In the figures, identical and functionally equivalent elements are provided with the same reference symbols.

1 shows an example of a process diagram 1 for a method for object tracking, such as can be used in a vehicle for tracking, i.e. for tracking objects in a respective environment. Properties of tracked objects are managed in object tracks 2, each of which is assigned to a tracked object. In a measurement cycle or run of the method, sensor data 3 is recorded. A prediction 4 of the object tracks 2 or the data or properties of the tracked, i.e. already known, objects stored therein can then be carried out for the current time or a recording time of the respectively recorded sensor data 3. The object tracks 2 or the corresponding objects as well as objects detected in or based on the current sensor data 3 or specified in the sensor data 3, which can also be referred to here as sensor objects, are then processed in an object association 5. The sensor objects from the current measurement cycle are assigned to one of the object tracks 2 as far as possible. This can be done, for example, based on a - possibly abstract - predetermined distance metric or the like. For sensor objects that cannot be assigned, a new object track 2 can be created if necessary.

Environmental objects classified as moving can then be further processed in a full update 6 based on the object tracks 2 and/or the properties stored or specified in the associated sensor objects. Here, the current sensor data 3 or sensor objects can be fused into the respectively associated object track 2, in particular using a predetermined filter mechanism, such as a Kalman filter or the like. The object tracks 2 are thus updated, which then results in correspondingly updated object tracks 2 as the starting point for the next measurement cycle or run of the method.

The object tracks 2 each refer to a real object detected by a sensor, the properties or state of which are estimated recursively over time in one or by a recursive tracking algorithm. The object tracks 2 can also be referred to as internal tracks and are each stored over several measurement cycles and updated in each case. In each such measurement cycle or frame, i.e. run through the method, the sensor data from one or more sensors can be recorded, which indicate measurement data for sensor objects that are already tracked over time within the respective sensor. Not only directly measured properties, such as position and orientation, but also dynamic states, such as speed or yaw rate or the like, can be estimated over time.

Each object track 2 can also include an entry or a date or data field in which it is stored or indicated whether the respective object is moving or at rest, i.e. stopped or stationary.

The object tracks 2 for stationary objects are described, managed and updated separately, i.e. in a different way than for the moving objects. For this purpose, not the sensor objects but individual sensor detections 18 (see 2 , 4 ) to the object tracks 2 or the objects managed therein. For further explanation, 2 a schematic overview of a motor vehicle 11 in which the method described here is used. For this purpose, the motor vehicle 11 has a sensor system 12, which can in particular comprise a radar sensor and/or a lidar sensor. To carry out the method, the motor vehicle 11 also has an object tracking device 13, which is coupled to the sensor system 12 via an interface 14. The sensor data 3 supplied by the sensor system 12 can be processed by the object tracking device 13, for example by means of a processor 15 and data memory 16, which are indicated schematically here.

Using the sensor system 12, the motor vehicle 11 can scan its surroundings and detect objects there. A foreign vehicle 17 or a corresponding boundary frame is schematically indicated here as such an object. Furthermore, several sensor detections 18 of the sensor system 12 are indicated here. These sensor detections 18 include object detections 19 that actually originate from the foreign vehicle 17 and clutter detections 20 that do not originate from the foreign vehicle 17 and/or are due to measurement errors or disturbances. For the detection association 7, all sensor detections 18 located in a search area 21 assigned to the respective object, in this case the foreign vehicle 17, can now be determined from the current sensor data 3. Only sensor detections 18 located within the search area 21 are then considered for assignment or association with the foreign vehicle 17 or the corresponding boundary frame.

3 shows a corresponding representation to illustrate a possible distribution of sensor detections 18 when using a lidar sensor. When using a lidar sensor, a typically higher distribution of sensor detections 18 can be achieved compared to the 2 In addition to the radar detections shown, a larger number of stable object detections 19 occur at the rear or at the end of the other vehicle 17 facing the sensor system 12. However, additional clutter detections 20 can occur. These can originate, for example, from an exhaust plume of the other vehicle 17 and/or occur inside the other vehicle 17 or the boundary frame for the other vehicle 17, for example due to windows or reflections from parts of an underbody of the other vehicle 17 or the like. As an example, a different form of the search area 21 is shown here for the use of a lidar sensor.

Depending on the sensor used or available or depending on the type of sensor detections 18, the assignment of the sensor detections 18 to the other vehicle 17 can be carried out, for example, based on a predetermined distance criterion and/or a predetermined cluster criterion and/or a predetermined similarity criterion. For example, the associated double speed can be determined for each radar detection and compared with the speed stored in the object track 2 for the other vehicle 17. Only those radar or sensor detections 18 can then be assigned to the other vehicle 17 whose Doppler speed deviates by a maximum of a predetermined amount from the speed stored in the associated object track 2. If, for example, a speed of 0 m/s is stored for a stationary object in the associated object track 2 and a deviation threshold value of, for example, a maximum of 5 m/s is specified, radar or sensor detections 18 whose Doppler speed is, for example, greater than 5 m/s can be discarded, so that they are not assigned to the foreign vehicle 17.

In the event that the sensor system 12 includes a radar sensor or the sensor data 3 includes radar detections, a standstill estimation 8 can be carried out for the respective object, in this case the foreign vehicle 17. Examples are shown in 2 the double speeds of the sensor detections 18 are indicated by corresponding arrows, the length of which correlates with the amount of the respective Doppler speed. A sensor detection 18 that is in the area of the boundary frame for the other vehicle 17, but whose Doppler speed is much greater than the Doppler speeds of the other object detections 19, in particular than the or specified speed threshold value, can be classified as clutter detection 20 on this basis. If the Doppler speeds of the object detections 19 or, for example, their mean or median are smaller in magnitude than the specified speed threshold value, which can take into account or map a measurement uncertainty of the radar sensor, it can be determined from this that the other vehicle 17 is stationary. Accordingly, for example, a corresponding flag can then be set to indicate this standstill. In particular, this standstill flag can be set if the Doppler speeds of several radar detections, for example a predetermined minimum number or a predetermined minimum proportion of the radar or object detections 19 assigned to the respective object, are consistent with a standstill of the respective object. If this is not the case, for example if a corresponding predetermined standstill criterion is not met, the flag cannot be set or reset or deleted.

Furthermore, a state assessment 9 can be carried out based on the object detections 19 assigned to the other vehicle 17. For example, the corresponding radar detections can be evaluated in a histogram. Such a histogram is shown as an example in 4 There, a distance d is plotted on an x-axis and the radar cross section RCS for the radar detections is plotted on a y-axis. In particular, radar detections occurring over several measurement cycles can be entered cumulatively in the histogram. A significance threshold 22 is also indicated here. This can be, for example, a fixed value or a proportion or percentage of a maximum value of the radar cross section. sections RCS in the histogram can be specified. For example, a radar cross-section RCS of 75% of the largest radar cross-section RCS contained in the histogram can be determined as the significance threshold value 22. The smallest distance d or the first distance d or distance bin, viewed from a distance d = 0, at or in which the significance threshold value 22 is exceeded can then be determined. This is identified here as the first significance bin 23. From this first significance bin 23, the radar detection sorted there and assigned to the other vehicle 17 can then be determined as a stable final detection, or a virtual final detection, i.e. not necessarily corresponding to a real sensor detection 18, can be formed or calculated from these radar detections sorted into the first significance bin 23, for example by calculating an average or median or the like.

In a similar way, a histogram can also be constructed for lidar detections assigned to the other vehicle 17, in which, for example, the intensity or a reflectivity value of the lidar detections can be used instead of the radar cross section RCS.

The stable end detection determined in this way based on the object detections 19 can be used as a hold point for the respective foreign vehicle 17. This end detection or this hold point can therefore represent the foreign vehicle 17 or be used to determine its current properties or object data, in particular the position or the distance d from the sensor system 12 b or the motor vehicle 11.

The objects classified as stationary or the corresponding object tracks 2 with the associated current object detections 19 and/or the properties determined from them, such as the distance d or the corresponding position or the radar-based speed or the like, can then be fed into an update 10 or further processed as part of an update 10. The update 10 here therefore refers to the method step for updating the object tracks 2 based on the current sensor data 3 for stationary objects. This update 10 for stationary objects can therefore be carried out separately from the full update 6 for moving objects and can also differ from it.

For the update 10, all dependencies or cross-correlations between different data or properties, such as position, speed, acceleration, yaw angle, yaw rate and/or the like, can be removed in the corresponding object tracks 2. The different properties or states of the respective object can therefore be decoupled from one another. For example, during the update 10, a change in the position of the respective object in the x direction, i.e. along the longitudinal direction of the motor vehicle 11, does not lead to a speed or acceleration of the object being introduced. Likewise, for example, a change in the position of the respective object in the y direction, i.e. in or parallel to the transverse direction of the motor vehicle 11, does not lead to a yaw rate or a changed yaw angle of the respective object being introduced. So as long as the respective object is classified as stationary, i.e. for example the corresponding flag is set, no movement or dynamics of the object are introduced when the object tracks 2 are updated 10. During the update 10, for example, the current, in particular dynamic, properties determined on the basis of the respective current object detections 19 can be written into the respective object track 2. These can be determined or have been determined, for example, on the basis of the stable end detection determined for the respective object, i.e. the stopping point.

Even if the respective object has just started moving, but the radar detections are still consistent with the object being stationary, the position can still be updated accurately and reliably by independently updating it. This enables, for example, a corresponding assistance system to maintain a stable and precise distance d between the motor vehicle 11 and the respective object, in this case the other vehicle 17, even in such a situation.

If, however, the current radar detections indicate a speed of the other vehicle 17 that is different from zero and no longer consistent with a standstill, this speed can be written into the respective object track 2 as part of the update 10. This means that the respective object is then no longer classified as a stationary object in the next measurement cycle or during the next run of the procedure.

As part of the update 10 or the state estimation 9 for stationary objects, in particular only the individual properties of the respective object can be determined and, if necessary, updated, without, for example, estimating or determining a new bounding box for the respective stationary object. This can result in an overall improved stability of the object tracking for stationary objects, for example in comparison to a handling Control or management of stationary objects in the same way as for moving objects, with complete redefinition of the respective bounding box.

If, for example, in a measurement cycle the sensor system 12 or at least one sensor does not provide any sensor detections 18 or object detections 19 for an object that has already been detected, the detection association 7 for this object may not be carried out or may not be carried out sensibly in the current measurement cycle. It may also be the case that under the current circumstances the detection association 7 may be too complex for certain sensor detections, for example of a lidar sensor. However, based on the object track 2 that already exists for the object, the object association 5 can still be carried out. If the object is classified as stationary, the update 10 can then also be carried out for this object. The movement state of the object can be taken from the respective object track 2 or the object data stored therein, for example, and/or a movement state determined internally for the object by at least one sensor that has detected the object can be used.

The procedure described here for stationary objects, for example for estimating standstill 8 and/or for updating 10, can in principle be used for any stationary object. However, it can be particularly useful to apply the described method at least or only for an object immediately ahead, for example a vehicle closest to the motor vehicle 11 in its direction of travel. This vehicle in front can be used or considered to be the vehicle that is in front of, i.e. to the front of, the motor vehicle 11, has a similar orientation to the motor vehicle 11 and is in the same lane as the motor vehicle 11. Such a vehicle in front can be determined, for example, based on the sensor data 3 and/or based on road or lane detection data and/or map data and/or vehicle data received via a Car2Car or Car2X data connection and/or the like. Precise knowledge and updating or tracking of the position and speed of such a vehicle in front can be important, for example, for maintaining a specified target distance between the motor vehicle 11 and the respective vehicle in front as precisely as possible, which is also relevant for safety. For example, an adaptive cruise control or the like can therefore be more dependent on correspondingly precise tracking of the vehicle in front than on precise tracking of other objects that may be located at a greater distance and/or in a different lane.

In summary, the present method can therefore automatically estimate whether or when a detected object is almost or completely stationary. When such a standstill is detected, all properties or states of this object are then estimated separately. For example, there is no influence of the position on the speed. Radar detections in particular can be used to detect standstill - if available. Such radar detections can be particularly well suited for this because they enable direct measurement of the Doppler speed. If, for example, several such Doppler measurements over one or more measurement cycles confirm that the respective object is completely stationary, any dynamics, i.e. any dependencies between different properties or states, can be prevented in the associated object data. Likewise, a stable distance d to the respective object can be estimated based on the radar detections and/or lidar detections over one or more measurement cycles using a predetermined robust estimation method, for example histogram- or median-based. This enables particularly stable and robust tracking of stationary objects. Corresponding data or properties can then be written or fused directly into the respective object data. In this way, overestimation or underestimation of, for example, the distance d to the respective object can be prevented or reduced compared to conventional approaches. The method described here enables particularly stable tracking of stationary objects, prevents the apparent occurrence or detection of movements or rotations due to measurement uncertainties, and enables precise and stable distance estimation or distance monitoring even for objects that were previously stationary but are just starting to move. This in turn enables particularly reliable, robust and safe implementation of position- or distance-based assistance functions.

Overall, the examples described show how an improved object state estimation for stopped or stationary objects, especially vehicles, can be realized.

List of reference symbols

1

Process diagram

2

Object tracks

3

Sensor data

4

Prediction

5

Object association

6

Full update

7

Detection association

8th

Downtime estimation

9

Condition assessment

10

Update

11

Motor vehicle

12

Sensor technology

13

Object tracking facility

14

interface

15

processor

16

Data storage

17

Third-party vehicle

18

Sensor detection

19

Object detection

20

Clutter detection

21

Search area

22

Significance threshold

23

first significance bin

RCS

Radar cross section

d

Distance

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions.

Cited patent literature

• DE 102020123585 A1 [0003]
• DE 102019109332 A1 [0004]

Claims (10)

Method (1) for object tracking over several measuring cycles of a sensor system (12), in which -objects (17) are detected in each case based on sensor data recorded in a measuring cycle of the sensor system (12), - for detected objects (17) the state of motion is determined based on the sensor data (3), wherein each object (17) is classified as a stationary object (17) or as a moving object, - object data (2) are stored for each detected object (17), which include several properties of the respective object (17) and its determined state of motion, -the object data (2) of objects (17) already detected in a previous measuring cycle are updated in each case based on the current sensor data (3), wherein - only for those objects (17) that were classified as a stationary object (17), their properties stored in the object data (2) are updated independently of one another. Procedure (1) according to Claim 1 , characterized in that the state of motion is determined on the basis of radar detections (18, 19), wherein an object (17) is classified as a stationary object (17) if a radar-based determined speed of the object (17) is smaller in magnitude than a predetermined speed threshold value. Method (1) according to one of the preceding claims, characterized in that an object (17) is only classified as a stationary object (17) if at least a predetermined number of measurements and/or at least a predetermined proportion of several measurements carried out result in a speed of the object (17) that is below a predetermined speed threshold value. Method (1) according to one of the preceding claims, characterized in that for objects (17) classified as stationary, their object data (2) are updated based on individual sensor detections (18) and for objects classified as moving, their object data (2) are updated based on sensor objects output by the sensor system (12). Method (1) according to one of the preceding claims, characterized in that for objects (17) classified as stationary, from all current sensor detections (18, 19) which are assigned to the respective object (17) according to a predetermined criterion, in particular a distance-based criterion with respect to the previous position of the respective object (17), a stable end detection is determined in each case for determining the current position of the respective object (17) to be updated as a property in the object data (2), which is used as an end of the respective object (17) facing the sensor system (12). Procedure (1) according to Claim 5 , characterized in that in the case that the sensor detections (19) assigned to the respective object (17) comprise radar detections, the stable final detection is determined taking into account their radar cross sections (RCS), in particular as the radar detection closest to the sensor system (12) whose radar cross section (RCS) is greater than a predetermined threshold value (22). Method (1) according to one of the preceding claims, characterized in that for objects (17) classified as stationary, at least some of the properties, in particular the position of the respective object (17), are determined based on lidar detections, wherein lidar detections from the current measurement cycle are assigned to a boundary frame (17) for the respective object (17) and a stable end detection is determined on the basis of a distribution, in particular a distance histogram, of these lidar detections, which is used as an end of the respective object (17) facing the sensor system (12). Method (1) according to one of the preceding claims, characterized in that for objects (17) classified as dormant, for the updating (10) of their associated object data (2), dependencies between different properties of the respective object (17) stored in the respective object data (2) are canceled and/or a switch is made to a diagonal P-matrix for modeling the respective object in the associated object data (2). Object tracking device (13), in particular for a motor vehicle (11), comprising an interface (14) for detecting sensor data (3) and a data processing device (15, 16) for processing the detected sensor data (3), wherein the object tracking device (13) is set up to carry out a method (1) according to one of the preceding claims. Motor vehicle (11), comprising a sensor system (12) for detecting the environment and an object tracking device (13) coupled thereto according to Claim 9 .

Images