ES2699501T3 - Procedure for using a processing device for sensor data of a sensor located in a vehicle and vehicle - Google Patents

Abstract

Method for using a processing device (6) for sensor data of at least one sensor (5) installed in a vehicle (1) which captures objects in the environment of the vehicle (1) and its sensor data describe object points (8) with a distance information, wherein from the sensor data the processing device (6) calculates objects and follows the movement of an object classified as a vehicle (3), where the object is monitored by means of a dynamic model which uses a movement information that describes the movement of the object in two directions of coordinates, characterized in that when an allocation criterion used from a movement information and / or a geometry information describing the geometric extension of the object, in the frame of the dynamic model additionally uses orientation information which comprises a Bounding Box (14) with an orientation that describes the angle of rotation of the vehicle (3).

Description

DESCRIPTION

Procedure for using a processing device for sensor data of a sensor located in a vehicle and vehicle

The invention relates to a method for using a processing device for sensor data of at least one sensor installed in a vehicle, a sensor that captures objects located in the vicinity of the vehicle and whose sensor data describe object points with a distance information. , wherein from the sensor data the processing device calculates objects and follows the movement of an object classified as a vehicle, the tracking of the object occurs by means of a dynamic model which uses a movement information that describes the movement of the object in two coordinate directions.

In the generation of vehicle environment models, more and more sensors are used that not only represent in two dimensions, such as cameras, but the captured object points also associate them with distance or depth information. It is already known, by means of a processing device, to process again the sensor data presented by the sensor, where by extracting characteristics from the object points, (logical) objects are calculated. To continue using this data, for example for driver assistance systems, it is also necessary to track the movement of this object (the so-called "tracking") over several sensor measurement cycles. It is also known to apply a dynamic model in which the movement of the object in two directions of coordinates is described, for example by means of a state vector that has a component x and a component y of the speed of movement.

However, it is a disadvantage that a dynamic model such as this one does not present any information about the orientation, ie the angle of rotation of the (real) vehicle represented by the object. While the information acquired about the object should only be used for the driver assistance system, in which only traffic in the transverse or longitudinal direction is considered, the orientation of a vehicle can be determined with the additional acceptance that the Traffic is only presented from one of the addresses mentioned above. In more complex driver assistance systems, for example in those where they must support the driver at street crossings, these additional assumptions can not be clearly taken so that apparent motions of the object can appear or vehicle tracking is interrupted. The article "Simultaneous Tracking and Shape Estimation with Lasers Scanners" (published in Intelligent vehicles Symposium Proceedings 2014, IEEE, pp. 1205-1210, 2014) publishes the preamble of claim 1.

Against this, the invention has as its basis the mission to present a more reliable method of the type mentioned at the beginning which, especially in situations of longitudinal and transversal traffic, makes possible a more robust tracking of the objects.

This mission will be solved by a method of the type mentioned at the beginning in which, according to the invention, it is foreseen that when an allocation criterion used from a movement information and / or a geometry information describing the geometric extension is fulfilled of the object, in the frame of the dynamic model additionally uses guidance information which comprises a Bounding Box with an orientation that describes the angle of rotation of the vehicle.

The invention relates to the reflection that the currently used dynamic model, which uses a movement information describing the movement of the object in two coordinate directions, to complete the orientation information allows a sufficiently accurate prediction about the actual orientation of the object. vehicle represented by the object. Only then is the Bounding Box calculated, that is a specially rectangular envelope that limits the entire object and will be provided with the orientation referring to the angle of rotation of the vehicle. In other words, the guidance information allows a prediction on which longitudinal side or which transverse side of the vehicle corresponds to which side of the Bounding Box.

The method according to the invention achieves the advantage that orientation ambiguities can be solved which typically comprise four possible orientations of the vehicle inside the Bounding Box. Thus, for example, the problem can be found that in a measurement cycle the sensor detects the front side and the driver's side, and in the next measurement cycle detects the front side and the passenger side, thereby Follow-up results in an apparent unwanted movement of the object. By means of the method according to the invention, reliable information on the movement of the object is prepared, which in particular can be used for driver assistance systems in the form of so-called crossing assistants.

For this purpose, the objects are first detected by the sensor in the vicinity of the vehicle, which, in particular by using running time procedures, calculates the distance information which also includes the depth information. Then, from the sensor data, the processing device calculates objects, classifies them as vehicles for example from the classification algorithms known by the state of the art and carries out the tracking or tracking of those objects . The dynamic model used for this describes the movement of the vehicle until completing the award criteria only through the information of movement, that is, in two directions of coordinates. It is suitable for this to use a Euclidean coordinate system whose direction of coordinates x typically corresponds to the transverse axis of the vehicle and whose direction of coordinates and corresponds to the longitudinal axis of the vehicle. With this, the tracking of the object is carried out until the adjudication criterion is completed regardless of the orientation of the object. In other words, the typical four possible orientations of the vehicle represented by the object are treated with equal value.

The method according to the invention is now emphasized by the fact that the processing device now assesses the allocation criterion that according to a first alternative of the invention will be used on the movement information. In this case, preferably on the basis of sensor data of several measuring cycles of the sensor, it will be checked whether it is possible to associate a possible orientation of the vehicle, that is, its angle of rotation, with the direction of movement of the calculated object. Preferably, the movement information describes the speed of the decomposed object in two directions, especially in the x direction and in the y direction. Accordingly, the angle of rotation of the vehicle will be suitably defined as the angular displacement of the longitudinal axis of the vehicle or of the transverse axis of the vehicle towards a coordinate direction. According to a second alternative of the invention, which can however also be used in combination with the one described above, within the framework of the assignment criterion it will be verified whether the geometric extension of the object, that is, the geometric arrangement of the object points allows a sufficiently accurate conclusion about the orientation of the vehicle.

In both alternatives of the invention, when the allocation criteria are met, the Bounding Box is generated for the object and this is assigned an orientation corresponding to the orientation of the vehicle. It is almost commuted to an expanded dynamic model which, together with the usual movement information, additionally uses the orientation information and thereby enables a clear resolution of the orientation ambiguities during the follow-up.

It is especially advantageous if, in several possible orientations, the Bounding Box is assigned those for which the deviation of its orientation angle from the direction of movement of the object is minimal. Thus, several orientation angles 9 of the Bounding Box can be considered, with that orientation angle having the smallest deviation from the direction of movement calculated from the movement information being assigned. If then the movement information comprises the speed of the object in direction x and in direction and then it will be assigned that angle of orientation 9 for which

| atan2 (vy, ^{v%) - q)}

is minimal, where Ox and Oy describe the velocities of the object in the x direction and y direction, and atan2 describes an arc tangent operator that presents two arguments.

Furthermore, in the context of the method according to the invention, it is preferred when the assignment criterion used on the movement information describes that a threshold level has been reached by a measure of an angular uncertainty of the direction of movement of the object. The measure of angular uncertainty describes the accuracy of the estimation of the direction of movement of the object by the processing device. It is advisable that the threshold value be chosen so that sufficient security of the direction of movement can be allowed for the intended use of the procedure. Since the tracking algorithms used to track objects, such as the Kalman filters, simply calculate a measure of this type or as a minimum parameters from which the measurement can be easily deduced, a particularly simple evaluation can be achieved of the allocation criterion.

It is of particular advantage if the measure of the angular uncertainty is calculated depending on the variance of at least one velocity in a coordinate direction and / or of the covariance of the velocities in the coordinate directions. The variance and the covariance are therefore parameters that can be presented when the tracking algorithm is used, since they serve for the statistical calculation of the direction of movement of the object followed. A measure calculated in this way for angular uncertainty can be represented as follows

Here the operators describe Var the variance and Covariate the covariance.

In the method according to the invention, it is particularly suitable if the processing device for determining the object collects several object points in a segment limited by a section or several sections and calculates a reference point. The segmentation procedures used for this purpose are properly known by the state of the art and from a number of object points the segments that are being calculated are calculated. related If the sensor is only oriented on one side of the object, the sensor essentially captures the object points located on a line that are gathered in a segment (form I) that has a section. When, on the other hand, at least two sides of the object are oriented to the sensor, a number of object points that have a corner-shaped path are captured. From this, the segment (form L) that has two sections can be formed. The limits of a segment are then the basis for the definition of the Bounding Box (complete) so that the segment can also be included as a partial Bounding Box.

The reference point therefore forms the link point to determine the position of the object, where it is calculated as the end point of the segment or as the cut-off point of two segments of the segment. The tracking of the movement of the object can then be advantageously carried out on the basis of the reference point. It is also especially advisable if the reference point describes a corner position of the vehicle. This makes possible an especially simple association of the object to be calculated with the captured vehicle.

The assignment criteria used on the geometry information can describe that a threshold level has been reached for a ratio of two span lengths. Preferably, the stretches limited by the reference point are used for this. On the basis of the hypothesis that the front and the rear of a vehicle are essentially shorter than its longitudinal sides, in this way a particularly simple conclusion of the orientation of the vehicle and the orientation information can be determined depending also on this relationship.

In the method according to the invention it is also preferred if, depending on the orientation information and a geometry hypothesis, the size of the Bounding Box is adjusted to the vehicle. During or after the calculation of the Bounding Box and the confirmation of the orientation of the vehicle, based on the geometry hypothesis that for example states that the longitudinal side of the vehicle is greater by a predetermined factor than the transversal side, it can be achieved a correction of the size of the Bounding Box. This is especially advisable when the allocation criterion applied to the movement information is used since a size of the Bounding Box calculated in the first place on the basis of object points captured only fragmentarily, may deviate from the actual size of the captured vehicle. However, if the direction of the object is known, the length of the longitudinal side of the vehicle can be easily determined on the basis of a calculated length of the transverse side, and vice versa.

As already stated, the starting data of the processing device can serve as input data of the driver assistance system. It is therefore preferred if the processing device prepares a vehicle system which, based on the tracking of the information of acquired objects value the movements of vehicles in the transverse direction and longitudinal direction. The vehicle system can be especially a driver assistance system that assesses traffic situations in a street crossing (the so-called crossing assistant). Here, the object data can be immediately sent to the vehicle system or first be merged with the starting data of another vehicle sensor device for an environment model and then be prepared for the vehicle system.

Finally, in the method according to the invention, it is especially preferred if a laser scanner and / or a Time-of-Flight camera and / or a radar sensor are used as the sensor. These types of sensors are all based on the assessment of the running time of the electromagnetic waves sent and are extraordinarily suitable for capturing the object points resolved in depth.

Furthermore, the invention relates to a vehicle comprising a sensor mounted thereon for the detection of objects in the environment of the vehicle and for the preparation of sensor data describing object points having distance information as well as a device processing for the sensor data, which will be operated according to the method according to the invention. All the embodiments for processes according to the invention can be transmitted analogously to the vehicle according to the invention, so that the aforementioned advantages can also be obtained with it.

Other advantages and details of the invention are apparent from the embodiments described below as well as on the basis of the drawings. These are schematic representations and show:

Fig. 1 a schematic diagram of a vehicle according to the invention in a traffic situation by way of example,

Fig. 2 a representation of several possible orientations of a Bounding Box,

Fig. 3. A representation of the calculation of the actual orientation of the Bounding Box, and

Fig. 4 a representation of the size adaptation of the Bounding Box.

Figure 1 shows a principle diagram of a vehicle 1 according to the invention in an example traffic situation. It is located at a junction of streets 2 that has just been passed by a vehicle 3 that runs transversely.

The vehicle 1 has a sensor device 4 with several sensors where, for reasons of clarity, only one sensor 5 located in the front area of the vehicle is shown. The sensor 5 is designed to detect objects in the environment of the vehicle, where its sensor data describe object points with distance information. Therefore, the sensor 5 can be designed, for example, as a laser scanner, Time-of-Light camera or as a radar sensor. The remaining sensors of the sensor device 4 can be located in other mounting positions of the vehicle 1 and / or and can be realized by different types of sensor mentioned above. The sensor data is prepared in a processing device 6. It generates object information representing objects describing objects in the environment of the vehicle 1, for example, the vehicle 3. The object information is evaluated by a vehicle system 7. in the form of a driver assistance system, for example a so-called crossing assistant, which assesses the movements of vehicles in the vicinity of the vehicle 1. The processing device 6 can be operated according to a procedure that will be described in more detail below based on the traffic situation.

As can be seen in figure 1, the sensor 5 detects the vehicle 3 in the area of its rear part and its passenger side, generating sensor data describing object points 8. These are presented to the processing device 6 on the which by means of a segmentation algorithm gathers the object points 8 in a segment 11 (L-shape) limited by two sections 9, 10, which can also be accommodated as a partial Bounding Box. In addition, the processing device 6 calculates the cut point of the sections 9, 10 as a reference point 12 for the next tracking of the movement of an object representing the vehicle 3. With this, the processing device 6 performs a corresponding algorithm of classification that classifies as such the vehicle 3 initially captured only as an object.

The processing device 6 follows the object along other measurement cycles of the sensor 5 using a dynamic model which uses motion information describing the movement of the object in two directions of x, y coordinates. The tracking is therefore carried out independently of the actual orientation of the vehicle 3, that is, of its angle of rotation referred to one of the directions of coordinates x. Y. In other words, four possible orientations 13a-13d of the vehicle 3 shown in FIG. 2 are treated with equal value. During the entire duration of the movement of the vehicle 3 this would, however, be a disadvantage since an erroneous detection of another corner position would be possible. of the vehicle 3 as a reference point 12 in a subsequent measuring cycle would result in an apparent movement of the object or could simply lead to a break in the tracking. In order to solve this problem when an assignment criterion is completed within the framework of the dynamic model, guidance information will be additionally used, which comprises a Bandung Box 14 (complete) (see Figure 4) with an orientation describing the angle of rotation of the vehicle 3. With this, of several possible orientations, corresponding to the orientations 13a-13d, the Bounding Box 14 will be assigned to that for which the deviation of its orientation angle 9 from the movement direction of the object is minimal. Therefore, the movement information describes the speed of the object by means of speed components in the x direction and in the y direction, so that that angle of orientation 9 for which

is minimal, where Ux and Uy describe the velocities of the object in the x direction and in the y direction and atan2 describes a tangent arc operator that presents two arguments.

This assignment, in other words a switching of the dynamic model, is nevertheless achieved only when an assignment criterion has been completed, that is, as soon as the orientation can be determined with sufficient security. Therefore, the assignment criterion is applied to the movement information and describes that a threshold value has been reached by a measure of an angle uncertainty of the direction of movement of the object. The processing device 6 calculates this measurement by means of the following formula: In it the operators describe Var variance and Covarianza. The corresponding values for the variances and the covariance are calculated using a Kalman filter in the framework of object tracking.

Figure 3 again shows the calculation of the actual orientation of the Bounding Box 14, where it represents the displacement of the segment 11 towards a position 15 during a subsequent measuring cycle of the sensor 5. The arrow 16 represents the direction of movement of the object in the form of a velocity vector, which will be calculated during the realization of the tracking algorithm that uses the Kalman filter. Appreciably, the arrow 16 does not point along the longitudinal axis of vehicle 3. This displacement demonstrates that the velocity vector can not be calculated exactly by the tracking algorithm, but that there is a certain angle uncertainty. Only when their measurement within the framework of the allocation criterion is assessed as sufficiently small does the additional use of the guidance information within the framework of the dynamic model occur.

In figure 4 it can also be seen that the size of the Bounding Box 14 of a rectangle theoretically obtained by completing the sections 9, 10 of the segment 11, does not represent any correct envelope of the vehicle 3. Accordingly, an adaptation follows of its size with which the Bounding Box 14 will continue to be used. Size adaptation is made depending on the orientation information and a geometry hypothesis for the vehicle 3, which describes a permissible ratio of the length of the transverse side of the vehicle. a vehicle with the length of its longitudinal side. After it is known from the guidance information that the reference point 12 describes the corner position of the passenger side at the rear of the vehicle 3, the lateral lengths of the Bounding Box 14 are determined according to the length of the vehicle. section 10 and a multiple of the length of section 10 described by the geometry hypothesis determined as the adjusted length of section 9.

The above-described method can also be used fundamentally if, from the sensor data, a segment 11 (I-shape) having only one section is calculated, that is, when the sensor 5 detects only object points 8 on one longitudinal side or a transverse side of the vehicle 3. Also in this case a corresponding reference point 12 can be calculated as the end point of the section, so that depending on the movement information a calculation of a Bounding Box 14 with allocation of the orientation and a size adjustment of the Bounding Box 14.

According to another embodiment of the method that can also be combined with the above described starting example, the assignment criterion is used on a geometry information that describes the geometric extension of the object. This is especially advisable when after one or typically several measurement cycles of the sensor 5 the lengths of the sides of the vehicle 3 can be determined sufficiently accurately, so that, again using a geometry hypothesis for the ratio of these lengths from the sides a prediction can be made about the orientation of the vehicle 3. The orientation angle assignment 9 is produced in a manner analogous to that of the embodiment described above taking into account the movement information.

Claims (12)

Method for using a processing device (6) for sensor data of at least one sensor (5) installed in a vehicle (1) which captures objects in the environment of the vehicle (1) and its sensor data describe points of object (8) with a distance information, wherein from the sensor data the processing device (6) calculates objects and follows the movement of an object classified as vehicle (3), where the tracking of the object is performed by a dynamic model which uses a movement information that describes the movement of the object in two directions of coordinates, characterized in that upon fulfilling an allocation criterion used from a movement information and / or a geometry information that describes the geometric extension of the object, in the frame of the dynamic model additionally uses an orientation information which comprises a Bounding Box (14) with an orientation that which describes the angle of rotation of the vehicle (3). Method according to claim 1, characterized in that of several possible orientations, the Bounding Box (14) is assigned the one for which the deviation of its orientation angle (a) from the direction of movement of the object is minimal . Method according to claim 1 or 2, characterized in that the assignment criterion used on the movement information describes that a threshold value has been reached by a measure of an angle uncertainty of the direction of movement of the object. Method according to claim 3, characterized in that the measurement of the angle uncertainty is calculated depending on the variance of at least one speed in a coordinate direction and / or the covariance of speeds in the coordinate directions. Method according to one of the preceding claims, characterized in that in order to determine the object, the processing device (6) brings together several object points (8) in a segment (11) bounded by a section or several sections (9, 10) and calculates a reference point (12) especially as end point of the section or as crossing point of two sections (9, 10) of segment (11). Method according to claim 5, characterized in that the assignment criterion used on the geometry information describes that a threshold value has been reached for a ratio of two section lengths. Method according to claim 5 or 6, characterized in that the movement of the object is monitored on the basis of the reference point (12). Method according to one of claims 5 to 7, characterized in that the reference point describes a corner position of the vehicle (3). Method according to one of the preceding claims, characterized in that the size of the Bounding Box (14) is adapted depending on the orientation information and a geometry hypothesis for the vehicle (3). Method according to one of the preceding claims, characterized in that from the object information obtained from the tracking the processing device (6) prepares a vehicle system (7) which evaluates vehicle movements (3) in the transverse direction and in the longitudinal direction Method according to one of the preceding claims, characterized in that a laser scanner and / or a Time-of-Flight camera and / or a radar sensor are used as sensor (5). Vehicle comprising a sensor (5) mounted thereon for detecting objects in the vehicle environment (1) and for preparing sensor data describing object points (8) presenting distance information, as well as a device for processing (6) for sensor data, which is designed to perform a method according to claims 1-11.