DE102022124192A1 - Method and device for updating the object status of an object - Google Patents

Abstract

A device for updating the object status of an object that is arranged in the surroundings of a vehicle is described. The object state describes one or more properties of a bounding box for the object. The device is set up to assign a detection point of an environment sensor of the vehicle to an edge of the bounding box and to determine the distance of the detection point from the assigned edge of the bounding box. The device is further set up to update the object state, in particular the one or more properties of the bounding box, based on the determined distance.

Description

The invention relates to a method and a corresponding device for updating the object status of an object in the surroundings of a vehicle.

A vehicle can have a driving function that is set up to guide the vehicle longitudinally and/or transversely at least partially or completely automatically. The driving function typically relies on an environment model of the vehicle's surroundings, which displays the object state (such as the position, size, orientation, direction of movement, speed of movement, etc.) in relation to one or more objects in the vehicle's surroundings . The vehicle can be automatically guided longitudinally and/or laterally depending on the surrounding model. The quality of the driving function usually depends on the quality of the environment model.

The environment model can be adapted repeatedly, in particular periodically, based on the sensor data from one or more environment sensors of the vehicle. The environment model determined for the previous time step can be updated at a specific time step. The updating of the environment model includes in particular the updating of the object state for the one or more objects of the environment model determined for the previous time step. Updating the object state for a previously detected object can be referred to as tracking the object. The updating and/or tracking can be effected using a Kalman filter.

This document deals with the technical task of achieving particularly efficient and precise tracking of an object in the vicinity of a vehicle.

The task is solved by each of the independent claims. Advantageous embodiments are described, among other things, in the dependent claims. It should be noted that additional features of a patent claim dependent on an independent patent claim can form a separate invention independent of the combination of all the features of the independent patent claim, without the features of the independent patent claim or only in combination with a subset of the features of the independent patent claim can be made the subject of an independent claim, a division application or a subsequent application. This applies equally to technical teachings described in the description, which may constitute an invention independent of the features of the independent patent claims.

According to one aspect, a device for updating the object status of an object that is arranged in the surroundings of a (motor) vehicle is described. Updating the object state can be done using an (extended and/or unscented) Kalman filter. The object state can describe one or more properties of a (possibly rectangular) bounding box for the object. The bounding box can enclose the object (whereby the bounding box has the smallest possible size to completely enclose the object). The one or more properties of the bounding box can be combined into a state vector that describes the object state.

The object state, in particular the state vector, of the object can include: the (two-dimensional) position of the bounding box (in particular a reference point of the bounding box); the width of the bounding box; the length of the bounding box; the orientation of the bounding box; the (longitudinal) speed of the bounding box; the (longitudinal) acceleration of the bounding box; and/or the rotation rate of the bounding box.

The device is set up to (exactly) assign a detection point of an environment sensor of the vehicle to an edge of the bounding box. The surroundings sensor can be a distance-measuring surroundings sensor, such as a radar sensor, a lidar sensor and/or an ultrasonic sensor. In a corresponding manner, several detection points of the surroundings sensor or possibly detection points of several different surroundings sensors can each be assigned (exactly) to one edge of the bounding box. A detection point can be assigned to the edge that is closest to the detection point.

The device is also set up to determine the distance of the respective detection point from the respectively assigned edge of the bounding box. The orthogonal distance between the respective detection point and the respectively assigned edge of the bounding box, perpendicular to the edge, is preferably determined as the distance.

The (orthogonal) distance to the assigned edge of the bounding box can therefore be determined for at least one detection point. The object state, in particular the one or more properties of the bounding box, can then be updated precisely based on the determined distance. Based on the determined distance of the detection point, a measurement update of a Kalman filter can be effected in order to update the object status.

One or more detection points can therefore be used directly to update the bounding box for an object (as part of a measurement update of a Kalman filter). In this way, the object state of the object can be updated in a particularly precise manner. The detection points can be used directly for the measurement update without the detection points being abstracted into an object box.

The device can be set up to assign each detection point of a plurality of detection points of the environment sensor to exactly one edge of the bounding box in order to determine a set of detection points for one or more edges of the bounding box. A set of detection points that are assigned to the respective edge can therefore be determined for one or more edges of the bounding box. A detection point is typically only assigned to exactly one edge.

For at least a part (or for all detection points) of the set of detection points that have been assigned to an edge, the (orthogonal) distance of the respective detection point from the respectively assigned edge of the bounding box can then be determined. The object state, in particular the one or more properties of the bounding box, can then be updated in a particularly precise manner based on the determined distances. If necessary, several measurement updates could be carried out sequentially (each based on part of the determined distances). Alternatively, the determined distances can be combined into a common innovation vector, which then results in a common measurement update.

The device can be set up to divide an edge of the bounding box into different (possibly the same size) subintervals. For each subinterval, zero, one or more detection points can then be selected from the set of detection points assigned to the edge. Furthermore, the object state, in particular the one or more properties of the bounding box, can be updated based on the distances for the selected detection points.

Individual detection points can therefore be selected along the entire edge of the bounding box in order to carry out the measurement update. This means that the measurement update can be carried out in a particularly precise and efficient manner.

The device can be set up to assign at least one detection point of the surroundings sensor of the vehicle to at least two different edges of the bounding box. The (orthogonal) distance of the respective detection point from the respectively assigned edge of the bounding box can then be determined for each of the detection points. Furthermore, the object state, in particular the one or more properties of the bounding box, can be updated in a particularly precise manner based on the determined distances (to the different edges). By taking several edges of the bounding box into account, the measurement update can be carried out in a particularly precise manner.

The device can be set up to determine a predicted object state for a current time (k) based on the object state for a previous time (k-1). This can be done as part of a Kalman prediction. In particular, a movement and/or prediction model for the object can be used for this purpose. The predicted object state for the current point in time can describe a predicted bounding box and/or predicted properties of the (predicated) bounding box.

The predicted object state can then be updated based on the determined distance for at least one detection point in order to determine the object state for the current point in time. For this purpose, a measurement update of the Kalman filter can be effected based on the distance. The distance between the detection point and the assigned edge of the predicted bounding box can be determined and used as the distance.

The device can in particular be set up to determine a Kalman gain for the current point in time based on the predicted object state. In the case of a one-dimensional measurement update, the Kalman gain typically has a dimension corresponding to the dimension of the object state, in particular the dimension of the state vector.

The determined distance can be multiplied by the Kalman gain to determine an update of the object state (where the update of the object state has the same dimension as the object state). The predicted object state can be updated with the determined update of the object state (e.g. by adding the predicted object state and the update of the object state) in order to determine the object state for the current point in time in a particularly precise manner.

The device can be set up to determine the distance to the edge for N detection points that are assigned to a specific edge of the bounding box and to combine them into an innovation vector, with N>1. A Kalman gain can then be determined for the (entire) innovation vector. Further, the innovation vector can be multiplied by the Kalman gain to determine the update of the object state. This means that within a single measurement update of the Kalman filter, several detection points can be taken into account (possibly also from several different edges). In this way, the accuracy of updating the object status can be further increased.

Alternatively or additionally, the device can be set up to determine a measured speed of at least one detection point of the surroundings sensor (in particular a radar sensor) (e.g. in the form of a Doppler speed). A speed innovation can then be determined based on the speed from the predicted object state and based on the measured speed. If necessary, this can be done for several detection points.

The distance determined in each case and the speed innovation determined in each case for the one or more detection points can be combined into an innovation vector. A Kalman gain can then be determined for the entire innovation vector. Further, the innovation vector can be multiplied by the Kalman gain to determine the update of the object state. By taking into account the speed of one or more detection points, the accuracy of updating the object state can be further increased.

The device can be set up to operate the vehicle, in particular an automated driving function of the vehicle, depending on the updated object state (e.g. to guide the vehicle longitudinally and/or transversely at least partially or completely automatically). In this way, the quality of a driving function can be increased.

According to a further aspect, a (road) motor vehicle (in particular a passenger car or a truck or a bus or a motorcycle) is described which includes the device described in this document.

According to a further aspect, a method for updating the object state of an object that is arranged in an environment of a vehicle is described. The object state can be described by a (multidimensional) state vector. The object state (e.g. as individual variables of the state vector) can describe one or more properties of a bounding box for the object.

The method includes assigning a detection point of an environment sensor (in particular a distance-measuring environment sensor) of the vehicle to an edge of the (rectangular) bounding box. Furthermore, the method includes determining the distance of the detection point from the associated edge of the bounding box, as well as updating the object state, in particular the one or more properties of the bounding box, based on the determined distance.

According to a further aspect, a software (SW) program is described. The SW program can be set up to run on a processor (e.g. on a vehicle control unit) and thereby carry out the method described in this document.

According to a further aspect, a storage medium is described. The storage medium may include a SW program configured to be executed on a processor and thereby carry out the method described in this document.

It should be noted that the methods, devices and systems described in this document can be used both alone and in combination with other methods, devices and systems described in this document. Furthermore, any aspects of the methods, devices and systems described in this document can be combined with one another in a variety of ways. In particular, the features of the claims can be combined with one another in a variety of ways. Furthermore, features listed in brackets are to be understood as optional features.

• 1 beispielhafte Komponenten eines Fahrzeugs;
• 2a beispielhafte Detektionspunkte eines Umfeldsensors;
• 2b eine beispielhafte Nachverfolgung einer Bounding-Box für ein Objekt anhand einer auf Basis der Detektionspunkte ermittelten Objekt-Box;
• 2c eine beispielhafte Aktualisierung einer Bounding-Box für ein Objekt auf Basis einzelner Detektionspunkte eines Umfeldsensors;
• 3a eine beispielhafte Aktualisierung der Bounding-Box unter Berücksichtigung von unterschiedlichen Kanten der Bounding-Box;
• 3b unterschiedliche mathematische Größen, die bei der Aktualisierung der Bounding-Box verwendet werden;
• 4 eine beispielhafte Unterteilung einer Kante einer Bounding-Box in mehrere Teilintervalle; und
• 5 ein Ablaufdiagramm eines beispielhaften Verfahrens zum Nachverfolgen eines Objektes im Umfeld eines Fahrzeugs.

The invention is further described in more detail using exemplary embodiments. Show it

• 1 exemplary components of a vehicle;
• 2a exemplary detection points of an environmental sensor;
• 2 B an exemplary tracking of a bounding box for an object based on an object box determined based on the detection points;
• 2c an exemplary update of a bounding box for an object based on individual detection points of an environment sensor;
• 3a an example update of the bounding box taking into account different edges of the bounding box;
• 3b different mathematical quantities used when updating the bounding box;
• 4 an exemplary subdivision of an edge of a bounding box into several subintervals; and
• 5 a flowchart of an exemplary method for tracking an object in the vicinity of a vehicle.

As explained at the beginning, this document deals with the efficient and precise tracking of objects in the vicinity of a (motor) vehicle. In this context shows 1 an exemplary vehicle 100 that has one or more environment sensors 102, each of which is set up to collect sensor data (which is also referred to as environment data in this document) relating to the environment of the vehicle 100. Exemplary environment sensors 102 are a radar sensor, a lidar sensor, a camera, an ultrasonic sensor, etc. An environment sensor 102 can in particular be a distance-sensing environment sensor that is set up to record the distance of a detection point from the environment sensor 102 as sensor data.

The sensor data of an environment sensor 102 (in particular a radar sensor and/or a lidar sensor) can include a large number of detection points, whereby the individual detection points can each be assigned to an object in the area surrounding the vehicle 100. A detection point can, for example, indicate that a transmission signal emitted by the environment sensor 102 was reflected on an object, and it can therefore be assumed that an object is arranged at the position of the detection point.

A (control and/or processing) device 101 of the vehicle 100 can be set up to evaluate the sensor data of the one or more environment sensors 102 in order to create an environment model for the environment of the vehicle 100, wherein the environment model contains the object state of one or more Displays surrounding objects. The device 101 can further be set up to provide a driving function based on the environment model, which is designed to at least partially automatically guide the vehicle 100 longitudinally and/or transversely. In this context, one or more longitudinal and/or transverse guidance actuators 103 of the vehicle 100 (e.g. a drive motor, a braking device and/or a steering device) can be controlled.

The environment model can be updated iteratively based on the current sensor data from the one or more environment sensors 102. The environment model for a previous point in time k-1 can, for example, display the object state for an object. The object state can, for example, display the object contour of the object. As exemplified in 2a shown, the object contour 200 can correspond to a (rectangular) bounding box that encloses the object. In this case, the object contour 200 can be described by the width 201, by the length 202 and by the orientation 204 of the bounding box. Furthermore, the object state can include the position 203 of the object contour 200, in particular the position 203 of a specific reference point of the object contour 200. The object state for an object can be relative to the coordinate system 210 of an environment sensor 102 of the vehicle 100 and/or relative to the coordinate system 310 of the vehicle 100 (see 3b) to be discribed.

As part of updating the environment model at the current time k, the object state of a previously detected object can be updated. The update can take place on the basis of a current set of detection points 212, which were detected by at least one environment sensor 102 of the vehicle 100 for the current time k. Furthermore, a Kalman filter can be used to update the state of the object. Updating the object state of a (previously detected) object can be referred to as object tracking.

Object tracking includes, for example, the temporal filtering of an object and/or the estimation of the (current) object state (e.g. position 203, speed, orientation 204, acceleration, rotation rate, length 202, width 201, etc.). As explained above, object state estimation can be performed using a Kalman filter for recursive estimation. An estimated or predicted object state for the current time k can be determined based on a prediction of the object state of the previous time k-1. For this purpose, a predefined motion model for the object can be used.

Furthermore, a measurement update of the predicted object state can be carried out based on the current sensor data (in particular based on the current set of detection points 212) in order to determine the object state of the object at the current time k.

As in 2a shown, the object state of an object can be described and/or estimated with a bounding box representation. Based on the bounding box 200 of the object for the previous time k-1, a predicted bounding box 200 for the current time k can be determined using a motion model.

The sensor data, in particular the detection points 212, can, as exemplified in 2 B represented, abstracted into an object box 220 and used for the measurement update of the object state of the object, in particular the predicted bounding box 200. This abstraction usually simplifies object tracking, as the sensor data is reduced to particularly relevant information, which reduces the bandwidth of the data to be transmitted, as well as the complexity and computational requirements of the measurement update of the filtered object state, since the measurement update is directly in the existing object state representation of the Object tracks (ie the bounding box 200) can be done. For example, the position 203 and/or the orientation 204 from the predicted object state of the object can be updated directly with the position and/or the orientation of the object box 220 determined on the basis of the sensor data.

An exemplary object box 220, which was created based on the set of detection points 212 of an environment sensor 102, is shown in 2 B shown.

The abstraction of the sensor data into an object box 220 is typically effective if the detection points 212 can be abstracted reliably and with a relatively high level of accuracy into a bounding box representation and no essential measurement information is lost. However, the abstraction can be error-prone and lead to a relatively inaccurate approximation and thus to a significant loss of information in the sensor data, which makes the measurement update of the object tracking and thus the object state estimate of an object error-prone. In particular, an abstracted object box 220 with an inaccurate or incorrectly selected orientation and/or position can no longer be used to draw retroactive conclusions about the underlying actual measurement information of the individual detection points 212. Furthermore, the uncertainties of the individual state variables of the object box 220 (in particular the covariance matrix for the individual state variables) cannot typically be reliably determined based on the set of detection points 212.

Alternatively or in addition to a measurement update of the object state based on an abstracted object box 220 for the set of detection points 212, a measurement update can take place directly based on the underlying set of detection points 212, i.e. in the state space of the detection points 212, without the need for a box. Abstraction of the set of detection points 212 takes place. As a result, the error-proneness and information loss of sensor data abstraction can be avoided.

The direct use of detection points 212 to carry out a measurement update is exemplified in 2c shown. In this context, a section 205, in particular an edge, the object contour 200 (in particular the bounding box) can be considered. The distance 215 of a detection point 212 from the section 205 of the object contour 200 can then be determined. This can be the perpendicular distance and/or the orthogonal distance of the detection point 212 perpendicular to the section 205 of the object contour 200. This distance 215 can be used for a measurement update of the object state.

The detection points 212 of one or more environmental sensors 102 can thus be used directly for the measurement update of a (possibly multi-sensor) object tracking, without having to first transfer the detection points 212 into the state space of the object track (e.g. by carrying out an object -Box abstraction). Instead, the object state of the object track can be projected onto the measurement space of the individual detection points 212 or an image dependent thereon. The influence of the individual detection points 212 on the measurement update can thus be defined directly and the loss of information resulting from an object box abstraction of the sensor data can be avoided. This procedure increases the robustness of the object tracking, in particular for an environment sensor 102 with a relatively high measurement uncertainty or with a relatively strong noise and / or with a relatively large number of incorrect or false detections (“clutter”).

The measurement update of the predicted object state based on individual detection points 212 described in this document can be used in a particularly advantageous manner when there is partial obscuration of an object and/or when there are multiple reflections.

The measures described in this document can be used for all environment sensors 102 with measured detection points 212 (i.e. in particular for distance-measuring environment sensors), such as radar and lidar sensors. However, the measures described are particularly advantageous in connection with a radar sensor that has a relatively low measurement resolution and/or a relatively high measurement noise (compared to a lidar sensor).

In addition, the measurement data of a radar sensor usually contains a radial Doppler velocity measurement for each individual detection point 121, which can be used as part of the direct detection measurement update. This additional direct integration of Doppler velocity information can further increase the accuracy and robustness of object tracking.

The object tracking described can be used based on a single environmental sensor 102 and/or based on a multi-sensor object fusion approach. Measurement data from other radar, lidar and/or camera sensors can be used to update a predicted object track with the measurement update described in this document.

Furthermore, the described object tracking (i.e. the described object tracking) can be combined with an abstraction of the measurement data to form an object box 220, in particular if such an object box 220 of the measurement data is already provided by an environmental sensor 102. The described object tracking based on the individual detection points 212 can then be carried out selectively for one or more selected objects (e.g. for one or more critical objects). A measurement update based on the respective object box 220 can then be carried out for the one or more other objects.

At a previous point in time k-1 there is already an object track with an object state. The object state may have been predicted (using a motion model) to the current time k. A measurement update of the predicted object state can then be carried out based on a set of detection points 212. The set of detection points 212 may have previously been assigned to the object track. This can be done, for example, by distance gating in Cartesian or polar space, if necessary taking Doppler velocities into account (e.g. gating with a “velocity profile”).

The detection points 212 associated with an object track (eg with a bounding box 200) can, as exemplified in 3a shown, one of the four box edges 205 of the (predicated) bounding box 200 are assigned (eg left/right/front/rear edge 205). The selection of possible edges 205 can be restricted to the edges 205 currently visible in the sensor detection area. Furthermore, only detection points 212 that represent an edge of the object are preferably taken into account (ie only detection points 212 that are not in the middle of an object or in the bounding box 200 of the object, such as background reflections).

As exemplified in 3a shown, at a time k a part of the detection points 212 can be assigned to the rear or front edge 205 and another part of the detection points 212 to the left or right edge of the same object track, ie the same bounding box 200 (which is a typical “L-Shape” corresponds to visibility of an object). If necessary, only a single side edge 205 (left or right) or only a single front/back edge 205 may be visible (which corresponds to a typical “I-shape” visibility of an object).

In the following, for the sake of simplicity, only the right edge 205 of an object track, i.e. a bounding box 200, is considered a visible edge. However, the approach described can be transferred directly to one or more additional edges 205 of the bounding box 200. For this purpose, adjusted difference or offset values can be determined based on the length 202 and/or the width 201 of the bounding box 200.

Furthermore, for simplicity, only a single detection point 212 is considered. The measurement update can be carried out in a corresponding manner for several, in particular for all, detection points 212 (e.g. sequentially and/or one behind the other or by using an appropriately dimensioned measurement update vector). A multi-dimensional measurement update of all detection points 212 of this object track taken into account can therefore be carried out in order to achieve a holistic optimization taking all measurement data into account.

For the measurement update using a detection point 212, the orthogonal distance d 215 between the detection point 212 and the edge 205 of the assigned object track, ie the (predicted) bounding box 200, can be used. The choice of the orthogonal distance 215 causes a resource-efficient one-dimensional pseudo measurement space to be used for the measurement update. Furthermore, the measurement update is limited to a correction of the object track parallel to the edge 205. This is done, as exemplified in 3a shown, only a lateral correction and / or displacement of the object track, whereas an elongated correction (with regard to the mean value and / or the covariance matrix) is deliberately prevented, since the detection point 212 assigned to the lateral edge 205 has no direct information content in relation to the object -Longitudinal position included. In a corresponding manner, for a detection point 212 that has been assigned to the front or rear edge 205, only one measurement update can be carried out along the longitudinal direction of movement of the object track if the measurement update is based on the orthogonal distance 215.

The Kalman filter innovation (i.e. the difference between prediction and measurement) can correspond directly to the orthogonal distance d 215, since the expected or predicted detection point is directly on the edge 205 of the (predicted) object track (i.e. the predicted bounding box 200). For the innovation or measurement update of the covariance matrix, the projection of the object state (e.g. the projection of the approximate position (x _τ , y _τ ) of the center of the rear axle of the object) onto the measurement space can be determined. This is described in this document by the function h _d,τ .

Due to the nonlinear projection h _d,τ of the object state onto the pseudo measurement space, an extended Kalman filter (EKF) is preferably used and the Jakobi matrices J _d,τ with the partial derivatives of the projections are determined accordingly. Alternatively, an unscented Kalman filter (UKF) with a sigma point projection can be used.

Detektionspunkt 212 im Polarraum (Range r_Z, Azimuth-Winkel ϑ_z) der Koordinatensystem 210 des Umfeldsensors 102 $z=\left(r_z,{\vartheta }_z\right)$

Detektionspunkt 212 im kartesischen Koordinatensystem 310 des Fahrzeugs 100; Dynamischer Zustandsvektor des Objekt-Tracks τ: ${\theta }_z={\vartheta }_z+{\vartheta }_s$

$x_z=x_s+r_z\text{cos}\left({\theta }_z\right)$

$y_z=y_s+r_z\text{sin}\left({\theta }_z\right)$

2D-Position, Geschwindigkeit, Beschleunigung, Orientierung, Drehrate (Objekt-Zustandsvektor bezogen auf die Objekt-Position 203 als angenommener Mittelpunkt-Hinterachse) $s_{\tau }=\left[x_{\tau },y_{\tau },v_{\tau },a_{\tau },{\phi }_{\tau },{\omega }_{\tau }\right]$

Geometrie der Bounding Box 200 des Objekt-Tracks τ, Länge l_τ, Breite w_τ $\left[l_{\tau },w_{\tau }\right]$

Positionsdifferenz zwischen dem Detektionspunkt 212 und der Position 203 des Objekt-Tracks im kartesischen Koordinatensystem 310 des Fahrzeugs 100 $\mathrm{\Delta }x=x_{\tau }-x_z$

$\mathrm{\Delta }y=y_{\tau }-y_z$

orthogonale Distanz d (für rechte Kante, mit Δw = -0.5 w_τ), sowie Jacobi-Matrix für das Kovarianzupdate $\begin{array}{l}{h}_{d,z}=d=-\left(-\mathrm{\Delta }x\text{sin}\left({\mathrm{\phi }}_{\mathrm{\tau }}\right)+\mathrm{\Delta }y\text{cos}\left({\mathrm{\phi }}_{\mathrm{\tau }}\right)+\mathrm{\Delta }w\right)\\ =\left(\mathrm{\Delta }x\text{sin}\left({\mathrm{\phi }}_{\mathrm{\tau }}\right)-\mathrm{\Delta }y\text{cos}\left({\mathrm{\phi }}_{\mathrm{\tau }}\right)-\mathrm{\Delta }w\right)\\ =((x_{\tau }-\left(x_s+r_z\text{cos}\left({\theta }_z\right)\right)\text{sin}\left({\mathrm{\phi }}_{\mathrm{\tau }}\right)\\ -\left(y_{\tau }-\left(y_s+r_z\text{sin}\left({\theta }_z\right)\right)\text{cos}\left({\mathrm{\phi }}_{\mathrm{\tau }}\right)+\mathrm{\Delta }w\right)\end{array}$

$\begin{array}{l}{J}_{d,z}=\left[\frac{\partial h_{d,z}}{\partial r_z}\frac{\partial h_{d,z}}{\partial {\vartheta }_z}\right]\hfill \\ =\left[\begin{array}{c}-\text{cos}\left({\vartheta }_z\right)\text{sin}\left({\phi }_{\tau }\right)+\text{sin}\left({\vartheta }_z\right)\text{cos}\left({\phi }_{\tau }\right)\\ r_z\left(\text{sin}\left({\vartheta }_z\right)\text{sin}\left({\phi }_{\tau }\right)+\text{cos}\left({\vartheta }_z\right)\text{cos}\left({\phi }_{\tau }\right)\right)\end{array}\right]\hfill \end{array}$

Erwartete / prädizierte orth. Distanz (Erwartung ist h_d,τ = 0 direkt an der Kante der Bounding-Box für den Erwartungswert) sowie Jacobi-Matrix für das Kovarianzupdate $\begin{array}{c}\widehat{x}=x_z+d\text{ sin}\left({\phi }_{\tau }\right)\\ \widehat{y}=y_z-d\text{ cos}\left({\phi }_{\tau }\right)\end{array}$

$\begin{array}{c}{h}_{d,\tau }=-\left(x_{\tau }-\widehat{x}\right)\text{sin}\left({\mathrm{\phi }}_{\mathrm{\tau }}\right)+\left(y_{\tau }-\widehat{y}\right)\text{cos}\left({\mathrm{\phi }}_{\mathrm{\tau }}\right)\\ \pm \mathrm{\Delta }w\equiv 0\end{array}$

$\begin{array}{l}{J}_{d,\tau }\hfill \\ =\left[\begin{array}{llllll}\frac{\partial h_{d,\tau }}{\partial x_{\tau }}\hfill & \frac{\partial h_{d,\tau }}{\partial y_{\tau }}\hfill & \frac{\partial h_{d,\tau }}{\partial v_{\tau }}\hfill & \frac{\partial h_{d,\tau }}{\partial a_{\tau }}\hfill & \frac{\partial h_{d,\tau }}{\partial {\phi }_{\tau }}\hfill & \frac{\partial h_{d,\tau }}{\partial {\omega }_{\tau }}\hfill \end{array}\right]\hfill \\ =\left[\begin{array}{c}-\text{sin}\left({\phi }_{\tau }\right)\\ \text{cos}\left({\phi }_{\tau }\right)\\ 0\\ 0\\ -\left(x_{\tau }-\widehat{x}\right)\text{cos}\left({\phi }_{\tau }\right)-\left(y_{\tau }-\widehat{y}\right)\text{sin}\left({\phi }_{\tau }\right)\\ 0\end{array}\right]\hfill \end{array}$

The formulas provided in Table 1 are based on a measurement update with a detection assigned to the right edge 205 of the (predicted) bounding box 200 (Δw = -0.5 w _τ ), where w _τ is the width 201 from the predicted object state. Accordingly, this can be transferred to the left edge 205 of the bounding box 200, with Δw = +0.5w _τ . Equivalently, a measurement update can be carried out in the longitudinal direction, taking into account the longitudinal displacement (rear: Δl = -0.25l _τ , front: Δl = +0.75l _τ ), where l _τ corresponds to the length 202 from the predicted object state. It is assumed that the position (x _τ , y _τ ) of the object corresponds to a point within the bounding box 200, which is as in 3b is arranged as shown. The different variables in Table 1 are off 3b refer to. Table 1 Pose of the environment sensor 102 in the coordinate system 310 of the vehicle 100 $\left(x_s,y_s,{\vartheta }_s\right)$

Detection point 212 in polar space (range r _Z , azimuth angle ϑ _z ) of the coordinate system 210 of the environment sensor 102 $\mathrm{e.g}=\left(r_{\mathrm{e.g}},{\vartheta }_{\mathrm{e.g}}\right)$

Detection point 212 in the Cartesian coordinate system 310 of the vehicle 100; Dynamic state vector of the object track τ: ${\theta }_{\mathrm{e.g}}={\vartheta }_{\mathrm{e.g}}+{\vartheta }_s$

$x_{\mathrm{e.g}}=x_s+r_{\mathrm{e.g}}\text{cos}\left({\theta }_{\mathrm{e.g}}\right)$

$y_{\mathrm{e.g}}=y_s+r_{\mathrm{e.g}}\text{sin}\left({\theta }_{\mathrm{e.g}}\right)$

2D position, speed, acceleration, orientation, rotation rate (object state vector related to the object position 203 as the assumed center point rear axis) $s_{\tau }=\left[x_{\tau },y_{\tau },v_{\tau },a_{\tau },{\phi }_{\tau },{\omega }_{\tau }\right]$

Geometry of the bounding box 200 of the object track τ, length l _τ , width w _τ $\left[l_{\tau },w_{\tau }\right]$

Position difference between the detection point 212 and the position 203 of the object track in the Cartesian coordinate system 310 of the vehicle 100 $\mathrm{\Delta }x=x_{\tau }-x_{\mathrm{e.g}}$

$\mathrm{\Delta }y=y_{\tau }-y_{\mathrm{e.g}}$

orthogonal distance d (for right edge, with Δw = -0.5 w _τ ), and Jacobian matrix for the covariance update $\begin{array}{l}{H}_{d,\mathrm{e.g}}=d=-\left(-\mathrm{\Delta }x\text{sin}\left({\mathrm{\phi }}_{\mathrm{\tau }}\right)+\mathrm{\Delta }y\text{cos}\left({\mathrm{\phi }}_{\mathrm{\tau }}\right)+\mathrm{\Delta }w\right)\\ =\left(\mathrm{\Delta }x\text{sin}\left({\mathrm{\phi }}_{\mathrm{\tau }}\right)-\mathrm{\Delta }y\text{cos}\left({\mathrm{\phi }}_{\mathrm{\tau }}\right)-\mathrm{\Delta }w\right)\\ =((x_{\tau }-\left(x_s+r_{\mathrm{e.g}}\text{cos}\left({\theta }_{\mathrm{e.g}}\right)\right)\text{sin}\left({\mathrm{\phi }}_{\mathrm{\tau }}\right)\\ -\left(y_{\tau }-\left(y_s+r_{\mathrm{e.g}}\text{sin}\left({\theta }_{\mathrm{e.g}}\right)\right)\text{cos}\left({\mathrm{\phi }}_{\mathrm{\tau }}\right)+\mathrm{\Delta }w\right)\end{array}$

$\begin{array}{l}{J}_{d,\mathrm{e.g}}=\left[\frac{\partial H_{d,\mathrm{e.g}}}{\partial r_{\mathrm{e.g}}}\frac{\partial H_{d,\mathrm{e.g}}}{\partial {\vartheta }_{\mathrm{e.g}}}\right]\hfill \\ =\left[\begin{array}{c}-\text{cos}\left({\vartheta }_{\mathrm{e.g}}\right)\text{sin}\left({\phi }_{\tau }\right)+\text{sin}\left({\vartheta }_{\mathrm{e.g}}\right)\text{cos}\left({\phi }_{\tau }\right)\\ r_{\mathrm{e.g}}\left(\text{sin}\left({\vartheta }_{\mathrm{e.g}}\right)\text{sin}\left({\phi }_{\tau }\right)+\text{cos}\left({\vartheta }_{\mathrm{e.g}}\right)\text{cos}\left({\phi }_{\tau }\right)\right)\end{array}\right]\hfill \end{array}$

Expected / predicted orth. distance (expectation is h _d,τ = 0 directly on the edge of the bounding box for the expected value) and Jacobi matrix for the covariance update $\begin{array}{c}\widehat{x}=x_{\mathrm{e.g}}+d\text{sin}\left({\phi }_{\tau }\right)\\ \widehat{y}=y_{\mathrm{e.g}}-d\text{cos}\left({\phi }_{\tau }\right)\end{array}$

$\begin{array}{c}{H}_{d,\tau }=-\left(x_{\tau }-\widehat{x}\right)\text{sin}\left({\mathrm{\phi }}_{\mathrm{\tau }}\right)+\left(y_{\tau }-\widehat{y}\right)\text{cos}\left({\mathrm{\phi }}_{\mathrm{\tau }}\right)\\ \pm \mathrm{\Delta }w\equiv 0\end{array}$

$\begin{array}{l}{J}_{d,\tau }\hfill \\ =\left[\begin{array}{llllll}\frac{\partial H_{d,\tau }}{\partial x_{\tau }}\hfill & \frac{\partial H_{d,\tau }}{\partial y_{\tau }}\hfill & \frac{\partial H_{d,\tau }}{\partial v_{\tau }}\hfill & \frac{\partial H_{d,\tau }}{\partial a_{\tau }}\hfill & \frac{\partial H_{d,\tau }}{\partial {\phi }_{\tau }}\hfill & \frac{\partial H_{d,\tau }}{\partial {\omega }_{\tau }}\hfill \end{array}\right]\hfill \\ =\left[\begin{array}{c}-\text{sin}\left({\phi }_{\tau }\right)\\ \text{cos}\left({\phi }_{\tau }\right)\\ 0\\ 0\\ -\left(x_{\tau }-\widehat{x}\right)\text{cos}\left({\phi }_{\tau }\right)-\left(y_{\tau }-\widehat{y}\right)\text{sin}\left({\phi }_{\tau }\right)\\ 0\end{array}\right]\hfill \end{array}$

The object state s _τ of an object track at a previous point in time k-1 can therefore be considered. In a prediction step of the Kalman filter, a predicted object state ŝ _τ,k|k-1 can be determined based on this (using a motion model for the object and/or for the vehicle 100). The updated object state at the current time k can then be determined using the measurement update described in this document.

Die Kalman-Verstärkung K_k weist dabei die gleiche Dimension auf, wie der Zustandsvektor (wenn als Innovation die skalare Größe d verwendet wird). Bei Verwendung eines Innovationsvektors mit mehreren Abständen 215 für mehrere Detektionspunkte 212 kann als Kalman-Verstärkung K_k eine Matrix ermittelt werden, sodass sich aus dem Produkt von Kalman-Verstärkung K_k und Innovationsvektor ein Vektor ergibt, der die gleiche Dimension aufweist wie der Zustandsvektor.The available innovation at time k is the (orthogonal) distance d. The Jacobian matrices described in Table 1 can be used to determine the Kalman gain K _k by which the innovation is to be multiplied in order to update the predicted object state ŝ _τ,k|k-1 based on the innovation, and in order to determine the object state ŝ _τ,k|k at the current time k, for example according to $${\widehat{s}}_{\tau ,k|k}={\widehat{s}}_{\tau ,k|k-1}+K_{k}d$$

The Kalman gain K _k has the same dimension as the state vector (if the scalar size d is used as an innovation). When using an innovation vector with multiple distances 215 for multiple detection points 212, a matrix can be determined as the Kalman gain K _k , so that the product of the Kalman gain K _k and the innovation vector results in a vector that has the same dimension as the state vector.

Alternatively or in addition to the position of an individual detection point 212, in the case of a radar sensor, the existing Doppler velocity measurement can be integrated directly into the measurement update. For this purpose, the predicted radial velocity component h _v,τ (from the predicted object state) can be determined given the azimuth angle θ _z of the measurement.

The innovation of the measurement update results from the difference between the measured absolute radial velocity (of the radar sensor 102) and the expected absolute radial velocity (from the predicted object state). Here, the measured absolute radial speed results from the originally measured relative radial speed, taking into account the compensation of the vehicle's own or sensor speed 100.

$\begin{array}{c}{J}_{d,\tau }=\left[\begin{array}{llllll}\frac{\partial h_{v,\tau }}{\partial x_{\tau }}\hfill & \frac{\partial h_{v,\tau }}{\partial y_{\tau }}\hfill & \frac{\partial h_{v,\tau }}{\partial v_{\tau }}\hfill & \frac{\partial h_{v,\tau }}{\partial a_{\tau }}\hfill & \frac{\partial h_{v,\tau }}{\partial {\phi }_{\tau }}\hfill & \frac{\partial h_{v,\tau }}{\partial {\omega }_{\tau }}\hfill \end{array}\right]\\ =\left[\begin{array}{c}-{\omega }_{\tau }\text{sin}\left({\theta }_z\right)\\ {\omega }_{\tau }\text{cos}\left({\theta }_z\right)\\ \text{cos}\left({\theta }_z-{\phi }_{\tau }\right)\\ 0\\ v_{\tau }\text{sin}\left({\theta }_z-{\phi }_{\tau }\right)\\ \left(x_s-x_{\tau }\right)\text{sin}\left({\theta }_z\right)-\left(y_s-y_{\tau }\right)\text{cos}\left({\theta }_z\right)\end{array}\right]\end{array}$

A measurement update of the object state can therefore also be effected based on the measured radial velocity of a detection point 212 and the corresponding component h _v,τ of the predicted velocity from the predicted object state. Table 2 Expected Doppler velocity of the track given azimuth angle θ _z $\begin{array}{c}{H}_{v,\tau }=v_{\tau }\text{cos}\left({\theta }_{\mathrm{e.g}}-{\phi }_{\tau }\right)+{\omega }_{\tau }(\left(x_s-x_{\tau }\right)\text{sin}\left({\theta }_{\mathrm{e.g}}\right)-(y_s\\ -y_{\tau })\text{cos}\left({\theta }_{\mathrm{e.g}}\right))\end{array}$

$\begin{array}{c}{J}_{d,\tau }=\left[\begin{array}{llllll}\frac{\partial H_{v,\tau }}{\partial x_{\tau }}\hfill & \frac{\partial H_{v,\tau }}{\partial y_{\tau }}\hfill & \frac{\partial H_{v,\tau }}{\partial v_{\tau }}\hfill & \frac{\partial H_{v,\tau }}{\partial a_{\tau }}\hfill & \frac{\partial H_{v,\tau }}{\partial {\phi }_{\tau }}\hfill & \frac{\partial H_{v,\tau }}{\partial {\omega }_{\tau }}\hfill \end{array}\right]\\ =\left[\begin{array}{c}-{\omega }_{\tau }\text{sin}\left({\theta }_{\mathrm{e.g}}\right)\\ {\omega }_{\tau }\text{cos}\left({\theta }_{\mathrm{e.g}}\right)\\ \text{cos}\left({\theta }_{\mathrm{e.g}}-{\phi }_{\tau }\right)\\ 0\\ v_{\tau }\text{sin}\left({\theta }_{\mathrm{e.g}}-{\phi }_{\tau }\right)\\ \left(x_s-x_{\tau }\right)\text{sin}\left({\theta }_{\mathrm{e.g}}\right)-\left(y_s-y_{\tau }\right)\text{cos}\left({\theta }_{\mathrm{e.g}}\right)\end{array}\right]\end{array}$

If necessary, a preselection of detection points 212 can be carried out to reduce the computing effort.

The approach described in this document is based on an existing association of the detection point 212 with a predicted object track and the respective visible box edge 205. All detection points 212 can be combined into a measurement update in order to achieve a holistic optimization of the track state estimation to achieve. For each detection point 212, a one- or two-dimensional measurement update can be carried out, for example by combining a position update and a Doppler velocity update. With N detection points 212, this results in a measurement update vector of 2N, with a corresponding squared matrix dimension (2N) ² . This can lead to a relatively high number N of detection points 212 and thus to a relatively high computing requirement, particularly in the case of relatively elongated objects in the close range (e.g. trucks in the side area).

In order to reduce the calculation requirement, the maximum number of detection points 212 to be taken into account can be limited. Alternatively or additionally, several measurement updates can be carried out sequentially, each with a subset of the detection points 212.

In order to achieve the greatest possible information content with a limited selection of detection points 212, the selected detection points 212 should cover as large an area of the object as possible (and therefore not all be located locally in a partial area). In this context, the total area of the relevant object edge 205 in the measuring range of the environment sensor 102 can be divided into equidistant interval areas 405 (see 4 ). A detection point 412 can then be selected (precisely) from each interval range 405. If no detection point 212 is contained in an interval range 405, further detection points 212 can be selected from one or more other interval ranges 405 until the predefined maximum number of detection points 212 is reached.

5 shows a flowchart of an exemplary (possibly computer-implemented) method 500 for updating the object status of an object (eg another road user) that is arranged in the surroundings of a (motor) vehicle 100. The object state may describe one or more properties (such as the width 201, the length 202 and/or the position 203) of a bounding box 200 for the object. The method 500 can be designed to update the object state along the time axis for a sequence of times k (based on the currently available sensor data from the one or more environment sensors 102 of the vehicle 100).

The method 500 includes assigning 501 a detection point 212 of an environment sensor 102 of the vehicle 100 to an edge 205 of the bounding box 200. The rectangular bounding box 200 typically has four different edges 205. The detection point 212 can be assigned to exactly one of these edges 205, for example the edge 205 that is closest to the detection point 212 (e.g. that has the smallest (orthogonal) distance 215).

The method 500 further includes determining 502 the (orthogonal) distance 215 of the detection point 212 from the associated edge 205 of the bounding box 200.

The method 500 also includes updating 503 the object state, in particular the one or more properties of the bounding box 200, based on the determined distance 215.

In a corresponding manner, the edge distance 215 can be determined for a large number of detection points 212 and combined into an innovation vector. This innovation vector can then be used for the measurement update of an (extended and/or unscented) Kalman filter.

The measure described in this document allows a precise update of the object status of an object, i.e. precise tracking of an object, to be effected directly on the basis of individual detection points 212 that were detected by one or more environmental sensors 102.

The present invention is not limited to the exemplary embodiments shown. In particular, it should be noted that the description and the figures are only intended to illustrate the principle of the proposed methods, devices and systems by way of example.

Claims (14)

Device (101) for updating an object status of an object which is arranged in an environment of a vehicle (100); wherein the object state describes one or more properties of a bounding box (200) for the object; wherein the device (101) is set up, - assign a detection point (212) of an environment sensor (102) of the vehicle (100) to an edge (205) of the bounding box (200); - to determine a distance (215) of the detection point (212) from the assigned edge (205) of the bounding box (200); and - to update the object state, in particular the one or more properties of the bounding box (200), based on the determined distance (215). Device (101) according to Claim 1 , wherein the distance (215) is an orthogonal distance between the detection point (212) and the associated edge (205) of the bounding box (200), perpendicular to the edge (205). Device (101) according to one of the preceding claims, wherein the device (101) is set up to - determine a predicted object state for a current time based on the object state for a previous point in time; and - to update the predicted object state based on the determined distance (215) in order to determine the object state for the current point in time. Device (101) according to Claim 3 , where - the predicted object state for the current point in time describes a predicted bounding box (200); and - the device (101) is set up to determine the distance (215) of the detection point (212) from the assigned edge (205) of the predicted bounding box (200). Device (101) according to one of Claims 3 until 4 , wherein the device (101) is set up to cause a measurement update of a Kalman filter based on the determined distance (215) in order to update the object state. Device (101) according to Claim 5 , wherein the device (101) is set up to - determine a Kalman gain for a current point in time based on the predicted object state; and - multiply the determined distance (215) by the Kalman gain in order to determine an update of the object state; and - update the predicted object state with the determined update of the object state in order to determine the object state for the current point in time. Device (101) according to Claim 6 , wherein the device (101) is set up - to determine a distance (215) for N detection points (212) that are assigned to the edge (205) and to combine them into an innovation vector, with N>1; - determine a Kalman gain for the innovation vector; and - multiply the innovation vector by the Kalman gain to determine the update of the object state. Device (101) according to one of Claims 6 until 7 , wherein the device (101) is set up to - determine a measured speed of the detection point (212) of the environment sensor (102); - to determine a speed innovation based on a speed from the predicted object state and based on the measured speed; - combining the determined distance (215) and the speed innovation for the detection point (212) into an innovation vector; - determine a Kalman gain for the innovation vector; and - multiply the innovation vector by the Kalman gain to determine the update of the object state. Device (101) according to one of the preceding claims, wherein the device (101) is set up - to assign each detection point (212) of a plurality of detection points (212) of the environment sensor (102) to exactly one edge (205) of the bounding box (200) in order to for one or more edges (205) of the bounding box (200) to determine a set of detection points (212); - for at least part of the set of detection points (212) that were assigned to an edge (205), a distance (215) of the respective detection point (212) from the respectively assigned edge (205) of the bounding box (200). determine; and - to update the object state, in particular the one or more properties of the bounding box (200), based on the determined distances (215). Device (101) according to Claim 9 , wherein the device (101) is set up to - divide an edge (205) of the bounding box (200) into different sub-intervals (405); - for each subinterval (405) to select zero, one or more detection points (412) from the set of detection points (212) assigned to the edge (205); and - to update the object state, in particular the one or more properties of the bounding box (200), based on the distances (215) for the selected detection points (212). Device (101) according to one of the preceding claims, wherein the device (101) is set up - at least two different edges (205) of the bounding box (200) each have at least one detection point (212) of the surroundings sensor (102) of the vehicle (100 ) to assign; - for each of the detection points (212) to determine the distance (215) of the respective detection point (212) from the respectively assigned edge (205) of the bounding box (200); and - to update the object state, in particular the one or more properties of the bounding box (200), based on the determined distances (215). Device (101) according to one of the preceding claims, wherein the object state comprises, - a position (203) of the bounding box (200); - a width (201) of the bounding box (200); - a length (202) of the bounding box (200); - an orientation (204) of the bounding box (200); - a bounding box speed (200); - an acceleration of the bounding box (200); and or - a rotation rate of the bounding box (200). Device (101) according to one of the preceding claims, wherein the device (101) is set up to operate the vehicle (100), in particular an automated driving function of the vehicle (100), depending on the updated object state. Method (500) for updating an object state of an object that is arranged in an environment of a vehicle (100); wherein the object state describes one or more properties of a bounding box (200) for the object; wherein the method (500) comprises, - Assigning (501) a detection point (212) of an environment sensor (102) of the vehicle (100) to an edge (205) of the bounding box (200); - Determining (502) a distance (215) of the detection point (212) from the assigned edge (205) of the bounding box (200); and - Updating (503) the object state, in particular the one or more properties of the bounding box (200), based on the determined distance (215).

Images