EP3566104A1

EP3566104A1 - Determining movement information with environment sensors

Info

Publication number: EP3566104A1
Application number: EP17821793.1A
Authority: EP
Inventors: Jean-Francois Bariant; Tino MILSCHEWSKI; Ahmed Kotb; Anto MICHAEL; Markus Heimberger
Original assignee: Valeo Schalter und Sensoren GmbH
Current assignee: Valeo Schalter und Sensoren GmbH
Priority date: 2017-01-03
Filing date: 2017-11-27
Publication date: 2019-11-13
Also published as: WO2018127328A1; JP2020504387A; US20200258379A1; DE102017100060A1

Abstract

The invention relates to a method for determining movement information, in particular for a driver assistance system (20), with a first and a second environment sensor (22, 24), comprising the steps: determining a first key pose (16) of a first environment sensor (22) at a first reference time, wherein the first key pose (16) delivers a feature set of an environment of a position; determining a first feature set (18) with the first environment sensor (22) at a first reference time plus a first time difference relative to the first key pose (16); determining a second key pose (16) of a second environment sensor (24) at a second reference time, wherein the second key pose (16) delivers a feature set of an environment of a position; determining a second feature set (18) with the second environment sensor (24) at a second reference time plus a second time difference relative to the second key pose (16); determining a first relative position change from the features of the first feature set (18) in relation to the first key pose (16) and a second relative position change from the features of the second feature set (18) in relation to the second key pose (16); and estimating the movement information on the basis of the first and second position changes of the first and second environment sensors (22, 24) together with the first and second reference times and the first and second time differences.

Description

Determination of motion information with environmental sensors

The present invention relates to a method for determining a

Motion information, in particular for a vehicle assistance system, with a first and a second environmental sensor.

The present invention also relates to an interface for a

Vehicle assistance system with a first and a second environmental sensor.

Furthermore, the present invention relates to a computer program product for

Implementation of the above method.

The present invention further relates to a vehicle assistance system having a first and a second environmental sensor and a control device, which is connected to the first and second environmental sensor via an interface.

In addition, the present invention relates to a vehicle having an above

Vehicle assistance system.

Various approaches are known in the art to

To gain movement information. For example, approaches to fusion of sensor data in the context of Simultaneous Location and Mapping (SLAM) methods are known in the art. These techniques are based on extracting features and feature-based fusion and localization.

For determining motion information in a real environment, i. For example, a movement by a distance at an angle, these methods are only partially suitable or too expensive. Extracting features is a difficult process.

From US Pat. No. 7,426,449 B2 a measurement data processing system is known, which comprises measurement data from a set of independent, self-validating (SEVA)

Process sensors that monitor the same real-time metric merges to create a combined best estimate of the value, uncertainty and measurement status of the measurand. The system also provides consistency check between the measurements. The measurement data processing system comprises a first process sensor and a second process sensor. Each of the first and second process sensors receives a measurement signal from a transducer and generates an independent process metric. A measurement fusion block is connected to the first and second process sensors, the measurement fusion block being operable to receive the independent process metrics and a measurement analysis process for analyzing the independent ones

Perform process metrics and generate the combined, best estimates of the independent process metrics.

Further, US 8,417,490 B1 discloses a system and method for providing integrated methods for an integrated software development environment for the design, validation and validation of advanced ones

Automotive safety systems known. The system allows automotive software developed on a host computer to provide a collection of

Computer programs simultaneously used as processes and synchronized by a central process. The software uses separate, synchronized

Processes that allow signals from different sources to be passed through a simulation running on the host computer or through actual sensors and

Data bus signals generated by actual vehicle hardware connected to their bus counterparts in the host computer on a real-time basis.

Based on the above-mentioned prior art, the invention is thus based on the object, a method for determining a movement information, in particular for a vehicle assistance system, with a plurality

Environmental sensors, an interface for a vehicle assistance system having a first and a second environmental sensor, a computer program product for carrying out the above method, a vehicle assistance system having a first and a second environmental sensor and a control device connected to the first and second environmental sensor via the above interface, As well as to provide a vehicle with an above vehicle assistance system, which allow a simple and robust determination of movement information. The object is achieved by the features of the invention

independent claims. Advantageous embodiments of the invention are specified in the subclaims.

The invention thus provides a method for determining movement information, in particular for a vehicle assistance system, having a first and a second environmental sensor, comprising the steps of determining a first key pose of a first environmental sensor at a first reference time, wherein the first key pose is a feature set of an environment Position, determining a first feature set with the first environment sensor at a first reference time plus a first time difference relative to the first key pose, determining a second key pose of a second environment sensor at a second reference time, wherein the second key pose a feature set an environment of a position determining, determining a second feature set with the second environment sensor at a second reference time plus a second time difference relative to the second key pose, determining a first relative position change from the features thereof The first set of features related to the first key pose and a second relative position change from the features of the second feature set with respect to the second key pose, and estimating the

Motion information based on the first and second position changes of the first and second environmental sensors along with the first and second reference times and the first and second time differences.

The invention also provides an interface for a vehicle assistance system having a first and a second environmental sensor, the interface having an interface for receiving a first or second key pose from the first or second environmental sensor, and an interface for receiving a first or second environmental sensor; second feature set relative to the first and second key pose of the first and second environmental sensor, respectively.

Furthermore, according to the invention, a computer program product for carrying out the above method is specified. According to the invention, a vehicle assistance system is further provided with a first and a second environmental sensor and a control device, which is connected to the first and second environmental sensor via an interface, wherein the vehicle assistance system is designed to carry out the method specified above.

In addition, according to the invention is a vehicle with an above

Vehicle assistance system specified.

The basic idea of the present invention is thus to create, via the definition of the key poses, reference variables which can be used to compare feature sets generated in subsequent sensor measurements with the characteristics of the key poses. The characteristics of the respective Key Pose and the

corresponding feature set to be mapped to each other

Determine position change of the environmental sensor. From the change in position together with the associated time difference for the two environment sensors, two movement information thus result. The process is simpler than a general feature extraction as in the prior art.

A feature has characteristic properties that can be detected by the respective environmental sensor. A feature thus represents a simple geometric primitive, which is determined directly by the sensor or its ad hoc algorithm. The features may be based on real objects, such as a vehicle, a sign, a mast, a curb, or others.

A feature set accordingly comprises a set of features that an environmental sensor can respectively detect. The feature set is thus a result of a sensor measurement with an environmental sensor, wherein the individual features are extracted from the sensor information. It is desirable that the

Sensor measurements are carried out as unfiltered, i. without previous image processing, for example based on a history.

Accordingly, a key pose and the corresponding feature set do not differ in principle, but only in their function, with one key pose each as a reference is used for more feature sets that match the corresponding one

Environment sensor have been determined.

In principle, the method can also be extended to a large number of environmental sensors, with the processing being carried out as described for the two environmental sensors. In principle, any types of environmental sensors can be used and combined.

In the method, it is irrelevant how the first or second reference time as well as the first and second time difference are selected. Reference times and time difference may have different values for different environmental sensors. The determination of the respective key poses and feature sets can, in principle, be carried out completely asynchronously.

It is also immaterial in principle for the method in which order individual steps are performed. So can position changes for everyone

Environmental sensor can be determined independently. In principle, the estimation of the movement information can take place arbitrarily based on current positional changes.

In estimating the motion information based on the first and second position changes of the first and second environmental sensors, it is

Required that the respective reference times can be assigned to a common time scale. In this case, a preferably continuous

Updating the vehicle movement done. Thus, the movement information can also be estimated if, for example, the first or second environmental sensor has not provided a current feature set.

Determining further first or second feature sets takes place as described above, wherein only the value for the corresponding time difference changes. An estimate of the motion information can be readily performed.

In addition, key poses can be transferred to a card. This can

for example, to define a trajectory via key poses. Similarly, the key poses can be based on your position on the map as References for the trajectory can be used. It is common that the vehicle position is defined by the rear axle, for example, the center of the rear axle.

In an advantageous embodiment of the invention, the method comprises the step of receiving external movement information, and the step of estimating the movement information based on the first and second position changes of the first and second environmental sensors together with the first and second reference times and the first and second time differences comprises estimating the motion information based on the external motion information and the first and second position changes of the first and second environmental sensors together with the first and second reference times and the first and second time differences. By taking into account further information relating to the movement, the estimation of the motion information can be further improved. In addition, the external movement information can be used to initialize individual method steps, whereby the implementation of the method can be accelerated and / or improved. In particular, when estimating the motion information, the external motion information acquired by the first and second position changes of the first and second environmental sensors based on the first and second can be adopted as an initial value

Reference time to be corrected together with the first and second time difference.

In an advantageous embodiment of the invention, the step of estimating the movement information based on the external movement information and the first and second position changes of the first and second environmental sensors together with the first and second reference times and the first and second time differences comprises weighting the external movement information and the first one and second position change of the first and second environmental sensors together with the first and second reference times and the first and second time differences. As a result, a first estimate of the movement information can be corrected, for example as a function of available movement information based on first and second position changes of the first and second

Environment sensor, the first and second reference time and the first and second time difference. Preferably, the weighting is adjusted dynamically during operation. In particular, the weighting may depend on one Evaluating the reliability of the external motion information and / or the first and second position changes of the first and second

Environment sensor along with the first and second reference time and the first and second time difference. For example, the weighting of the first and the second position changes of the first and the second

Environmental sensor along with the first and second reference time and the first and second time difference are increased depending on a number of detected features related to the respective key pose.

In an advantageous embodiment of the invention, the method comprises the step of checking a recognition of a minimum number of features of the first or second feature set with respect to the first or second key pose, and in the event that less than the minimum number of features of the first or second second

Feature set based on the first and second key pose are detected, the method comprises the additional step of determining a further first or second key pose of the first and second environmental sensor. Accordingly, it is checked whether the first or second key pose is still suitable for estimating the motion information. This is the case as long as a sufficient number of features of the respective key pose is contained in the corresponding first or second feature set in order to be able to determine a change in position of these features. As soon as an environmental sensor detects a hitherto unknown part, a new key pose is generated and possibly added to the card. As soon as a key pose is no longer suitable for estimating the movement information, a new key pose is generated for the corresponding sensor. In principle, it is decided independently for each sensor on the generation of key poses. Since the various sensors can sometimes be arranged with different viewing angles and viewing ranges, the recognition of features of the individual

Environmental sensors in principle independent. Also, an environmental sensor, for example, temporarily provide no position changes. In that case, motion information may still be estimated based on the other environment sensor. The further feature sets of the corresponding environmental sensor are subsequently processed with reference to the further key pose.

In an advantageous embodiment of the invention, the method comprises the step of transmitting the first key pose, the second key pose, of the first feature set and the second feature set from the first and second environmental sensors, respectively, to a controller. Accordingly, the method may be performed on a decentralized basis by performing a part of the data processing from the environmental sensors, and performing a part of the data processing in the controller. An interface between the environmental sensors and the control device ensures that all features can be transmitted correctly. Furthermore, key poses can be stored in the control device and based on the current feature sets of the corresponding

Environment sensor to be updated. Here you can merge the key poses with the current feature sets.

In an advantageous embodiment of the invention, the transfer of the first comprises

Feature set and the second feature set from the first and second environmental sensor to a control device, the transmission of a detected with the first and second environmental sensor first and second

Position change and a description of an uncertainty of the first or second position change of the first and second feature set based on the first and second environmental sensor. The uncertainty may in principle depend on the environmental sensor itself, i. one environmental sensor may have a higher accuracy than another. The uncertainty can on the one hand have an accuracy and, on the other hand, a certain type of uncertainty, for example a possible direction information of an uncertainty.

In an advantageous embodiment of the invention, the transmission of a description of an uncertainty of the first or second position change of the first or second feature set based on the first or second environmental sensor comprises the transmission of a covariance matrix. Based on the covariance matrix and the relative position changes, the motion information can thus be estimated. The covariance matrix makes it possible to detect uncertainties in determining the respective position change. For this purpose, the uncertainty of a position change can be represented, for example, as in the form of a three-dimensional ellipsoid.

In an advantageous embodiment of the invention, the steps of determining a first or second feature set relative to the first and second key pose the processing of raw data with an AdHoc algorithm to perform the determination of positions of features related to the first and second key pose, respectively.

In an advantageous embodiment of the invention, the step of determining a first relative change in position comprises the features of the first

Feature set related to the first key pose and a second relative

Position change from the features of the second feature set with respect to the second key pose performing a Kaiman filtering, particle filtering, an information filter or a graph optimization. Such filters are in principle known as such in the prior art and may be used to hereafter the information of the various environmental sensors to the estimated

To be able to process motion information.

In an advantageous embodiment of the invention, the first and the second environmental sensor are designed independently of each other as a laser scanner, radar, ultrasonic sensor or camera. Sensor information from the different sensors can be processed in combination without any fundamental restrictions, as each environmental sensor determines its key pose in its own way and performs a sensor measurement. The recognition of features in the sensor measurements can in principle

done different way. In estimating the motion information, in principle any types of sensors can be combined in arbitrary positions. The features are also individual for each environmental sensor.

In an advantageous embodiment of the invention, the vehicle assistance system to an above-mentioned interface, which is formed between the control unit and the first and the second environmental sensor.

It shows

Fig. 1 is a schematic view of a vehicle with a

Vehicle assistance system according to a first preferred embodiment on a driven trajectory, and Fig. 2 is a flowchart of a method for determining a

Motion information for the vehicle assistance system in accordance with the first embodiment.

1 shows a schematic view of a vehicle 10, which is along a trajectory 12 in this embodiment to a destination point, which is represented here by a garage 14.

The vehicle 10 is designed according to the first, preferred embodiment with a vehicle assistance system 20. The vehicle assistance system 20 comprises a first and a second environmental sensor 22, 24, which are connected via an interface 26 to a control device 28. The first and the second

Ambient sensor 22, 24 are designed independently of each other as a laser scanner, radar, ultrasonic sensor or camera.

A flow diagram of a method according to the invention for determining movement information for the vehicle assistance system 20 in the vehicle 10 according to the first embodiment is shown in FIG. 2. The following general definitions apply to the method.

When moving along the trajectory 12, each environment sensor 22, 24 independently forms key poses 16 as a reference for the determination of

Movement information. In addition, feature sets 18 are formed by each environmental sensor 22, 24 independently, for which a change in position relative to the respective key pose 16 is determined. Accordingly, the trajectory 12 with key poses 16 and feature sets 18 is shown by way of example in FIG. 1, which can be both first key poses 16 and first feature sets 18 or second key poses 16 and second feature sets 18 by way of example. The key poses 16 and feature sets 18 are shown here by way of example as points along the trajectory 12, wherein the points each indicate a position corresponding to the corresponding key pose 16 or the feature set 18.

A feature set 18 is a result of a sensor measurement with a

Environmental sensor 22, 24 and includes a set of features, which detects an environment sensor 22, 24 respectively, wherein the individual features are extracted from the sensor information. The sensor measurements are carried out unfiltered.

In principle, the method can also be extended to a multiplicity of environmental sensors 22, 24, the processing corresponding to that for the two

Environment sensors 22, 24 described.

The method begins with step S100 relating to determining a first key pose 16 of a first environmental sensor 22 at a first reference time. The first key pose 16 provides a feature set of an environment of a position.

Successively, in step S1 10, a first feature set 18 with the first

Environment sensor 22 at a first reference time plus a first time difference relative to the first key pose 16 determined.

According to steps S100 and S110, in step S120, a second key pose 16 of a second environment sensor 24 is determined at a second reference time, the second key pose 16 provides a feature set of an environment of a location, and in step S130, a second feature set 18 with the second environment sensor 24 at a second reference time plus a second one

Time difference relative to the second key pose 16 determined.

In this case, the two specific key poses 16 are additionally transferred to a card in steps S100 and S120, as shown by way of example in FIG. In this case, a vehicle position is defined via a center of the rear axle of the vehicle 10.

The determination of the first and second feature set 18 relative to the first and second key pose 16 in steps S1 10 and S130 respectively includes the processing of raw data with an AdHoc algorithm to perform the determination of positions of features related to the first and second key pose 16, respectively.

In each case, in steps S1 10 and S130, a check is made for a detection of a minimum number of features of the first or second feature set 18 with respect to the first or second key pose 16. In the event that less than the minimum number of features of the first or second key pose 16 Second feature set 18 are detected based on the first and second key pose 16, a further first and second key pose 16 of the first and second environmental sensor 22, 24 in the

Steps S100 and S120 determined. This is done independently for each environmental sensor 22, 24.

In step S140, a determination is made of a first relative position change from the features of the first feature set 18 relative to the first key pose 16 and a second relative position change from the features of the second one

Feature set 18 relative to the second key pose 16. The final fusion involves performing Kalman filtering, particle filtering, a

Information filtering or graph optimization to process the information from the various environmental sensors 22, 24 into the estimated motion information.

In addition, the first key pose 16, the second key pose 16 as well as a first and second detected with the first and second environmental sensor 22, 24, respectively

Position change and a description of an uncertainty of the first or second position change transmitted. The transmission of a description of an uncertainty of the first or second position change of the first and second feature set 18 based on the first and second environmental sensors 22, 24, respectively, comprises the transmission of a covariance matrix. For transferring, the

Interface 26 each have a corresponding interface.

In step S150, external movement information of the vehicle 10 is transmitted to the controller 28 based on odometry information of the vehicle 10. In step S160, estimating the motion information based on the external motion information and the estimated motion information in step S140 is performed by processing the first and second position changes of the first and second environmental sensors 22, 24 to the estimated motion information. When estimating the motion information is possibly a

Updating the vehicle movement.

In addition, the external motion information and the first and second position changes of the first and second environmental sensors 22, 24 are weighted. The weighting is dynamically adjusted during operation, depending on an assessment of the reliability of the external motion information and / or the first and second position changes of the first and second environmental sensors 22, 24 together with the first and second reference time and the first and second time difference. In addition, the weighting of the first and second position changes of the first and second environmental sensors 22, 24 together with the first and second reference times and the first and second time differences from the external motion information is increased depending on a number of detected features related to the respective key pose 16 ,

Determining further first or second feature sets 18 takes place as described above in steps S1 10 and S130, wherein only the value for the corresponding time difference changes. The method thus jumps independently back to the above steps.

LIST OF REFERENCE NUMBERS

Vehicle 10

Trajectory 12

Garage, destination 14

Key pose 16

Feature set 18

Vehicle assistance system 20 first environment sensor 22 second environment sensor 24

Interface 26

Control device 28

Claims

claims

1 . Method for determining movement information, in particular for a vehicle assistance system (20), having a first and a second

Environmental sensor (22, 24) comprising the steps

Determining a first key pose (16) of a first environmental sensor (22) at a first reference time, wherein the first key pose (16) includes a

Provides feature set of an environment of a position,

Determining a first feature set (18) with the first environmental sensor (22) at a first reference time plus a first time difference relative to the first key pose (16),

Determining a second key pose (16) of a second environmental sensor (24) at a second reference time, the second key pose (16) providing a feature set of an environment of a position,

Determining a second feature set (18) with the second one

Environment sensor (24) at a second reference time plus a second time difference relative to the second key pose (16),

Determining a first relative position change from the features of the first feature set (18) relative to the first key pose (16) and a second relative position change from the features of the second one

Feature set (18) related to the second key pose (16), and

Estimating the motion information based on the first and second position changes of the first and second environmental sensors (22, 24) together with the first and second reference times and the first and second time differences.

2. The method according to claim 1, characterized in that

the method is the step of receiving an external one

Includes movement information, and

the step of estimating the motion information based on the first and second position changes of the first and second environmental sensors (22, 24) together with the first and second reference times and the first one and second time difference estimating the motion information based on the external motion information and the first and second

Position change of the first and the second environmental sensor (22, 24) together with the first and second reference time and the first and second time difference.

3. The method according to claim 2, characterized in that

the step of estimating the motion information based on the external motion information and the first and second position changes of the first and second environmental sensors (22, 24) together with the first and second reference times and the first and second time differences

Weighting of the external movement information and the first and second position change of the first and the second environmental sensor (22, 24) together with the first and second reference time and the first and second time difference.

4. The method according to any one of the preceding claims, characterized in that

the method comprises the step of verifying recognition of a minimum number of features of the first or second feature set (18) relative to the first or second key pose (16), respectively, and less than the minimum number of features of the first or second feature set (16) second feature set (18) relative to the first or second key pose (16), the method comprises the additional step of determining a further first or second key pose (16) of the first or second environmental sensor (22, 24). includes.

5. The method according to any one of the preceding claims, characterized in that

the method includes the step of transmitting the first key pose (16), the second key pose (16), the first feature set (18), and the second feature set (18) from the first and second environmental sensors (22, 24), respectively

Control device (28).

6. Method according to the preceding claim 5, characterized in that transmitting the first feature set (18) and the second feature set (18) from the first and second environmental sensors (22, 24), respectively

Control device (28) the transmission of a first or second environmental change detected with the first and second environmental sensor (22, 24) and a description of an uncertainty of the first and second

Position change of the first or second feature set (18) based on the first and second environmental sensor (22, 24).

7. The method according to the preceding claim 6, characterized in that the transmission of a description of an uncertainty of the first or

second position change of the first or second feature set (18) based on the first and second environmental sensor (22, 24) the

Transferring a covariance matrix includes.

8. The method according to any one of the preceding claims, characterized in that

the steps of determining first and second feature sets (18) relative to the first and second key-pots (16), respectively, comprise processing raw data with an AdHoc algorithm to determine positions of features relative to the first and second, respectively Key pose (16).

9. The method according to any one of the preceding claims, characterized in that

the step of determining a first relative position change from the features of the first feature set (18) relative to the first key pose (16) and a second relative position change from the features of the second feature set (18) relative to the second key pose (16) performing a Kalman filtering, particle filtering, an information filter or a

Graphene optimization includes.

10. Interface (26) for a vehicle assistance system (20) with a first and

a second environmental sensor (22, 24), the interface (26) having an interface for receiving a first and second key pose (16) from the first and second environmental sensors (22, 24), respectively an interface for receiving first and second feature sets (18) relative to the first and second key poses (16), respectively, from the first and second environmental sensors (22, 24).

1 1. Computer program product for carrying out the method according to one of the preceding claims.

12. vehicle assistance system (20) having a first and a second

Environmental sensor (22, 24) and a control device (28), which is connected to the first and second environmental sensor (22, 24) via an interface (26), wherein the vehicle assistance system (20) is executed, the method according to any one of the preceding claims 1 to 10.

13. Vehicle assistance system (20) according to the preceding claim 12, characterized in that

the first and the second environmental sensor (22, 24) are designed independently of each other as a laser scanner, radar, ultrasonic sensor or camera.

14. Vehicle assistance system (10) according to any one of the preceding claims 12 or 13, characterized in that

the vehicle assistance system (20) has an interface (26) after the

preceding claim 10, which is formed between the control unit (28) and the first and the second environmental sensor (22, 24).

15. Vehicle (10) with a vehicle assistance system (20) according to one of

preceding claims 12 to 14.