DE102021101717A1 - Method for providing merged data, assistance system and motor vehicle - Google Patents

Abstract

The invention relates to a method for providing data (26) for an assistance system (20) of a motor vehicle (30). The invention further relates to such an assistance system (20) and a motor vehicle (30) equipped with it. In the method, respective raw data from (12) several sensors (10) are recorded. A performance value (24) is predicted for each of these raw data (12) by means of a trained evaluation model (18) assigned to the respective sensor (10). These performance values (24) indicate how well the assistance system (20) can probably carry out its intended function based on the respective raw data (12). The raw data (12) from the multiple sensors (10) are then fused together, weighted according to their performance values (24), in order to provide the data (26) for the assistance system (20).

Description

The present invention relates to a method for providing data for an assistance system of a motor vehicle. The invention further relates to a corresponding assistance system and a motor vehicle equipped therewith.

With the attempt to improve road safety and to enable autonomous driving of motor vehicles, a large number of sensors and increasingly complex data processing methods are often used in today's motor vehicles. In principle, this can lead to an improvement, but it can also cause problems. For example, low-quality data from a sensor can lead to a suboptimal result when combined with other data. In addition, as the complexity of the systems increases, it is not always understandable or comprehensible how the system arrived at a specific decision or a specific result.

A method for explaining driving behavior actions of an autonomous vehicle is described in US 2020/0 356 828 A1, for example. Therein, information collected from a variety of information sources of the vehicle is combined in a scene understanding module. Based on a state representation generated from this, one or more weighted behavioral guidelines are determined. Based on these, a driving behavior action for the vehicle is selected and implemented. Finally, information is provided that indicates why this driving behavior action was selected in a particular driving context of the vehicle. In this way, an easy-to-explain learning system for an autonomous vehicle is to be provided, regardless of the number and type of sensor modules and data processing methods of the vehicle. However, not all influences and optimization possibilities are taken into account.

A fusion or pre-processing of sensor data is also in the DE 10 2018 101 505 A1 described in connection with an efficient situational awareness of streams of perception in autonomous driving systems. In the method proposed there, a combined coordinate system is generated in response to a first coordinate system centered on a moving object and a second coordinate system centered on a fixed location. The combined coordinate system is segmented into a first cell and a second cell, with the first cell being associated with an object located within the cell. The first cell is then assigned a first level of risk in response to the first object, while the second cell is assigned a second level of risk. A control signal is then generated in response to the first level of risk. Within the framework of this method, an object classification, image classification, traffic light classification, object segmentation, ground segmentation and an object tracking process can be carried out, for example, based on a sensor output output by a sensor fusion and upstream processing module. Ultimately, the aim is to address the problem that sensors provide a large amount of information by reducing a quantity of perceptual data while retaining the information required for autonomous driving.

The object of the present invention is to enable a particularly reliable and robust functioning of an assistance system based on sensor data in different operating conditions.

According to the invention, this object is achieved by the subject matter of the independent patent claims. Possible refinements and developments of the present invention are disclosed in the dependent patent claims, in the description and in the figures.

The method according to the invention serves to provide data for or to an assistance system of a motor vehicle. The method can be used for different types of data and for different assistance systems, also referred to as ADAS (Advanced Driver Assistant System). The assistance system can therefore perform different functions or outputs based on the data provided, depending on the implementation or application, for example object recognition, object localization, detection of a free space that can be driven on in an area surrounding the motor vehicle, lateral and/or longitudinal guidance of the motor vehicle and/or more like that.

In a method step of the method according to the invention, respective raw data, that is to say measured values, sensor data or sensor signals, are recorded by a number of sensors, in particular different sensors. In other words, the respective raw data is recorded by means of the multiple sensors or corresponding raw data originating from multiple sensors is received via one or more data interfaces provided for connecting the sensors of a device provided for executing the method according to the invention.

In a further method step of the method according to the invention, a power value is predicted for this raw data in each case by means of a predefined, trained evaluation model assigned to the respective sensor or raw data stream originating from a specific sensor. The evaluation models can be, for example, in particular deep, that is to say multi-layer, artificial neural networks or can each include such a network. The performance values indicate how well, i.e. how precisely, how reliably and/or how effectively, the assistance system can probably exercise or execute its intended function based on the respective raw data, which is specified depending on the implementation or use case. The performance values can ultimately be almost any metric that is suitable for evaluating a performance or effectiveness of the assistance system with regard to its intended function, that is to say is correlated with it. For example, if the intended function of the assistance system is object detection or includes object detection, a so-called intersectionover union (loU) can be used as a metric, i.e. as a performance value or as a basis for determining the respective performance value.

In a practical application or implementation, for example as part of the assistance system, the evaluation models can be downstream of the sensors or integrated into the sensors.

In a further method step of the method according to the invention, the raw data from the multiple sensors are weighted relative to one another according to their performance values predicted or determined by the evaluation models, fused with one another, i.e. combined with one another, in order to form or provide the data for the assistance system. This merging of the raw data can be carried out by the device mentioned—possibly completely or partially separate from the assistance system—for executing the method according to the invention, or by the assistance system itself. In the latter case, the assistance system can be set up for at least partially executing the method according to the invention, that is to say it can correspond to the device mentioned or at least partially include it.

The assistance system can then further process the data provided, ie the weighted fused raw data, depending on its intended function, for example to generate a corresponding output or control signal for the motor vehicle.

With the weighted merging of the raw data, the power values can be used directly as corresponding weights or weighting factors. Likewise, the power values can be normalized in a predetermined manner, for example, in order to form the weights. Likewise, the weights can be calculated from the power values according to a predetermined calculation rule. For example, this can depend on or take place on whether the power values determined or their formats are compatible with a particular implementation of the data fusion and/or whether other influencing factors are to be taken into account therein. Ultimately, almost any fusion strategy can be applied here, as long as it can process different inputs, i.e. the raw data from the multiple sensors, and allows individual weighting of these inputs in order to influence the respective influence of the inputs on a result of the fusion.

By weighting the raw data according to the predicted performance values, the influence of raw data of inferior quality, ie less robust, less accurate, less reliable or generally less suitable for the intended function of the assistance system, on the final processing results of the assistance system can be reduced. Accordingly, better, more robust, more reliable results of the assistance system can ultimately be achieved by overweighting raw data with greater performance values, ie raw data that is more robust, more accurate, more reliable or better suited for the intended function of the assistance system.

The present invention can therefore simultaneously use a particularly broad database for the respective function of the assistance system and dynamically limit the influence of raw data of inferior quality. This can be advantageous based on the knowledge that a combination of high-quality raw data from a first sensor with low-quality raw data from a second sensor for a database of the assistance system can deliver a better result or improve functionality despite the relatively poorer quality of the raw data from the second sensor of the assistance system than would be possible individually based on the raw data of each of the sensors. Depending on the type or function of the respective assistance system, for example, improved safety in traffic, a greater degree of autonomy for the respective motor vehicle and/or the like can be achieved compared to conventional systems in which an assistance system only uses raw data from a single sensor or data that is independent of quality , i.e. combined with each other on an equal footing.

By means of the evaluation models, the present invention enables individual observation or monitoring of the individual sensors in order to predict whether the raw data provided by the individual sensors will lead to a deterioration in performance when they are used exclusively or are of equal priority to the other raw data or even overweight in comparison to them , i.e. the performance or effectiveness, of the assistance system. This in turn makes it possible to automatically adjust the corresponding weightings for the individual raw data or raw data streams from the individual sensors for the subsequent data fusion, for example in order to meet or maximize a predetermined criterion. Such a criterion can be, for example, a maximization of a confidence value of an output from the assistance system. Since each sensor or the raw data or raw data streams of the individual sensors is individually monitored within the scope of the present invention, ie evaluated by the evaluation models, these can be regarded as a so-called watchdog system, ie as a guard system. In contrast to conventional guard systems, not only can the functionality of the sensors be monitored here, but it can also be checked or monitored whether the raw data provided by the individual sensors is useful, i.e. can be used by the assistance system to perform its intended function in a beneficial or targeted manner be able.

The present invention takes into account the knowledge that, in principle, the use of the broadest possible data basis, i.e. the use of raw data from several, in particular different, sensors can often lead to an improved result, but different sensors can have different strengths and weaknesses or different advantages and disadvantages . For example, different types of sensors can deliver raw data of different quality or usefulness depending on the environmental or application conditions. Appropriate weighting of the raw data can at least partially prevent weaknesses or disadvantages of one or more individual sensors that exist under the respective environmental or operational conditions from having a negative effect on the overall performance of the assistance system.

For example, one or more cameras, a radar device, a lidar device, an ultrasound device and/or the like can be used as sensors. As part of their training, the evaluation models can then learn, for example, that object detection based on images recorded with low ambient brightness can typically lead to an error, for example to a so-called disengagement of an autonomous driving function of the motor vehicle. In use, the evaluation model assigned to a corresponding camera can then recognize such images, so that the data fusion weights raw data from other sensors more heavily due to their underweighting or underweighting, so that ultimately the assistance system also relies more heavily on the raw data from the other sensors in order to carry out its intended function . Such other sensors can be or include a radar or lidar device, for example, which is not affected by reduced ambient brightness or is at least less affected than a camera that works in the optical range.

For each sensor or for each raw data stream of the sensors, the respective power value can be determined or predicted exclusively based on the respective raw data. As a result, a particularly simple, robust and real-time capable implementation of the present invention is possible.

Since the raw data is weighted before it is used or processed by the assistance system to perform its intended function, the correspondingly low performance of the assistance system can be prevented by underweighting sensors or raw data with a relatively low performance value. There are already a number of approaches to determine whether a system for data-based decision-making, in this case the assistance system, is secure, i.e. reliable or confident enough to make trustworthy decisions that are also accepted and automatically executed in a security-relevant application be able. The present invention is based on the finding that corresponding uncertainties in such systems are often directly attributable to a respective input, ie in this case the raw data provided by the sensors. Instead of allowing the respective system to process correspondingly problematic inputs, which can result in correspondingly unsafe or unreliable output, the approach proposed here is to process correspondingly problematic, i.e. low-quality or unreliable input even before processing for actually exercising the intended assistance function of the assistance system, for example, already on or immediately after the respective sensor, and to limit its influence on a resulting later output of the assistance system in advance.

As a further example, an optical camera or the raw images recorded by it can be compromised by direct sunlight. In such a situation, other sensors, such as radar or lids, could represent more suitable raw data sources to enable the assistance system to perform its intended function safely and reliably. By detecting such problems or circumstances during or immediately after recording the respective camera image, the data provided to the assistance system can be automatically adjusted by overweighting the sensors different from the camera or the corresponding raw data until the respective problem has been resolved, i.e. no longer exists, in In this case, for example, the camera is no longer affected by solar radiation. In this way, the present invention thus allows imminent or imminent uncertainties in the assistance system to be detected solely based on the raw data before they occur and to prevent the assistance system from actually experiencing or developing this uncertainty.

In one possible development of the present invention, the power values are predicted at least essentially with the same frequency as the acquisition of the raw data. The trained evaluation models can therefore process and evaluate all of the raw data or measured values recorded by the respective assigned sensor in order to determine the respective power value or to predict the respective raw data for subsequent further processing. The raw data can be bundled, for example into individual images, scans, sweeps and/or the like. An individual power value can then be determined for each such bundle of raw data. In other words, an individual power value is not determined for each pixel or individual measurement point, but for each bundle of raw data, ie for a complete individual image. This provides a robust basis for reliably predicting performance values.

The raw data or the bundles of raw data can be processed serially for each sensor by the assigned evaluation model. If this is not fast enough, for example due to hardware or resource restrictions, i.e. not possible in real time, several identical evaluation models can also be used for each sensor, which can then be operated at least partially in parallel. For example, the raw data or bundles of raw data from a sensor can be supplied to the parallel evaluation models in accordance with a predetermined, repetitive sequence. In this way, one of the evaluation models can still process the penultimate raw data, while the current raw data is already being fed to another parallel evaluation model. In this way, the performance values can be determined comprehensively, i.e. completely for all raw data and also particularly precisely and reliably, since the evaluation models do not have to be compressed to the maximum in order to individually enable sufficient processing speed for real-time operation on the respective sensor, which typically involves a loss associated with accuracy. Improved performance, reliability and/or robustness of the assistance system can thus ultimately be achieved. Depending on the type of sensors, the raw data or the bundles of raw data can be recorded or recorded with a frequency of the order of 10 Hz, for example 10 Hz, 25 Hz, 30 Hz, 60 Hz or the like. This enables the assistance system to have a response time that is sufficiently short for practical applications in road traffic and can at the same time limit the amount of data processing required for this to practicable values.

In a further possible embodiment of the present invention, the raw data from the multiple sensors are fused using a low-level data fusion method. In principle, almost any method can be used for the data fusion, but a low-level fusion offers the advantage that raw data can be handled or processed particularly early in a data processing pipeline according to their respective predicted performance values or possibly discarded. This enables a particularly effective and efficient allocation or use of available hardware or calculation resources with regard to the intended function of the assistance system. In the technical sense, a low-level fusion means here that at least essentially unprocessed raw data is fused, i.e. the individual raw data are not or only rudimentarily pre-processed before the fusion.

In a further possible embodiment of the present invention, the raw data is recorded by different types of sensors, in particular by sensors based on different physical measurement principles. The sensors can thus include, for example, an optical camera, a lidar device, a radar device, an ultrasound device and/or the like. Accordingly, the raw data or raw data streams can also be of different types. By using such different sensors or raw data, it can be achieved that the assistance system performs its intended function in different environmental conditions can be maintained or run particularly reliably and robustly. Since under such different environmental conditions the different types of sensors often offer significantly different performance or reliability and this is inherently taken into account in the present invention by the respective performance values dynamically determined according to the situation, the present invention can be used in a particularly simple and flexible way to achieve optimal functionality of the assistance system are ensured.

In a further possible embodiment of the present invention, the power values are each compared with a predetermined power threshold value. Raw data whose power value is less than the power threshold value or the respective power threshold value is then discarded so that it does not flow into the fusion with the remaining raw data. In particular, if different types of raw data or different types or models of sensors are used, individual performance threshold values can be specified. However, a common power threshold value can also be specified for all raw data or sensors. For example, the discarded raw data can be deleted directly or assigned a weight of zero for data fusion, for example. In the latter case, the corresponding raw data may also be available for other purposes, i.e. used. Deterioration in the performance of the assistance system or generation of above-average uncertainties in the assistance system can be particularly effectively avoided by the predefined performance threshold value, which represents an acceptable performance of the assistance system in the respective application. If necessary, complete failures of the respective sensor can also be predicted or recognized. In general, a combination of raw data from multiple sensors can lead to a better result than would be achievable on the basis of the raw data from each individual sensor. Raw data from an individual sensor that is of poor quality can, however, lead to a deterioration in the overall result, i.e. the performance or performance of the assistance system, when taken into account in the data fusion. This is prevented in a particularly simple and effective manner by the embodiment of the present invention proposed here.

In a further possible embodiment of the present invention, the power values are compared with a predetermined threshold value. If the power values of all raw data recorded at least essentially simultaneously are less than the specified threshold value or at most correspond to it, a warning signal or a request to take over a manual control is automatically output - for example by the device mentioned for carrying out the method according to the invention. In other words, in the event that none of the sensors provides sufficiently good raw data according to the specified threshold value, i.e. the assistance system in this sense cannot perform its intended function or cannot perform it reliably enough, this intended function or the control or Control of the respective motor vehicle is taken over by an operator. A corresponding request or a corresponding signal can be output or sent, for example, to a vehicle occupant or driver of the respective motor vehicle equipped with the assistance system or to an operator external to the vehicle, ie to a teleoperator. The raw data recorded at least essentially simultaneously can be or include all raw data recorded in a respective clock or measurement cycle of the sensors, ie at times corresponding to one another in terms of time. The embodiment of the present invention proposed here can thus be safeguarded against the case that at a specific point in time or during a specific time window for all sensors or all currently available raw data power values below the specified threshold value are predicted, for example due to a highly complex traffic situation, more demanding or unfavorable weather conditions such as heavy snowfall and/or the like. Because in this case the request to take over manual control can be issued immediately after the performance values have been determined, a respective operator can possibly already take over the corresponding control before the assistance system actually processes the corresponding raw data, i.e. in the exercise of its intended tasks function has actually failed. This enables further improved safety when operating the assistance system or the motor vehicle.

In a further possible embodiment of the present invention, the assistance system includes a predefined function model for the automated execution of its intended function. This functional model is set up, ie possibly trained, to process data originating from a plurality of, in particular different, sensors. The individual evaluation models are based on raw data and based on processing results of one only with the raw data of the respective Sensor assigned to the evaluation model, but otherwise trained to the training function model corresponding to the predetermined function model. In the application of the present invention, the evaluation models effectively predict what would happen, that is, how the assistance system would behave, if only the raw data of the sensor assigned to the respective evaluation model were used. As a result, precisely those sensors or raw data that make the least positive contribution or even make a negative contribution to the performance of the assistance system can be underweighted or disregarded. Almost any known training or regression prediction method can be used to train the scoring models. Because training function models are used when training the evaluation models, which at least essentially correspond to the function model ultimately used in the application, the evaluation models can be effectively trained introspectively. The evaluation models can therefore be or have been trained using previous processing results or a previous performance of the respective training function model that resulted from the processing of raw data from the respective sensor. However, instead of only predicting that the assistance system is likely to fail in the case of low-quality raw data, the present invention can prevent the assistance system from getting into a corresponding unsafe state or error state by underweighting or discarding the corresponding raw data according to the prediction of the evaluation models. When the training of the evaluation models is complete, the training models can be discarded, since they are not set up or trained for exercising the intended function of the assistance system based on a number of different raw data streams. Using the finally trained evaluation models, each of the multiple sensors can be individually monitored and a continuous stream of predicted performance values can be provided for the corresponding continuously recorded raw data.

In a further possible embodiment of the present invention, the evaluation models are or will be trained with raw data from sensors that are functioning as intended or according to specifications and with raw data from damaged and/or soiled sensors. Processing results from the processing of raw data from such sensors can be used or have been used as a training reference. Due to the fact that the evaluation models are also trained for damaged and/or soiled sensors, that is to say correspondingly changed raw data, the performance values can be predicted particularly reliably and robustly in practical use in a large number of different situations.

The training procedures or training methods mentioned can be part of the method according to the invention or can be carried out independently of it, ie before practical use in a motor vehicle. Likewise, the training procedures or training methods mentioned can each form a separate aspect of the present invention.

Another aspect of the present invention is an assistance system for a motor vehicle. The assistance system according to the invention has several evaluation models for predicting performance values for raw data recorded by means of several sensors, a fusion unit for automatically merging the raw data, a functional model for exercising an intended assistance function of the assistance system and a data interface for outputting a result signal resulting from the processing of the fused raw data. The assistance system according to the invention is set up for the automatic or semi-automatic execution of at least one variant of the method according to the invention. The assistance system according to the invention and its components can accordingly correspond in particular to the assistance system mentioned in connection with the method according to the invention or the components mentioned there. The number of sensors mentioned can likewise be part of the assistance system or devices separate from it, which are or can be connected to the assistance system in practical use of the assistance system in the respective motor vehicle. In particular if the sensors are separate from the assistance system, the assistance system can have one or more corresponding input interfaces for acquiring the raw data provided by the sensors.

To carry out the method according to the invention, the assistance system can be equipped with appropriate data processing means. For this purpose, the assistance system can in particular have a computer-readable data memory and a processor device connected thereto, for example a microchip, a microprocessor or a microcontroller. An operating or computer program can then be stored in the data memory, which represents, ie encodes or implements, the method steps or sequences of the method according to the invention and can be executed by means of the processor device practicable to carry out the relevant method or cause it to be carried out.

Another aspect of the present invention is a motor vehicle that has multiple sensors, in particular of different types, for recording raw data and an assistance system that is connected to them or includes them, according to the present invention. The motor vehicle according to the invention can in particular be the motor vehicle mentioned in connection with the method according to the invention and/or in connection with the assistance system according to the invention.

Further features of the invention can result from the claims, the figures and the description of the figures. The features and feature combinations mentioned above in the description and the features and feature combinations shown below in the description of the figures and/or in the figures alone can be used not only in the combination specified in each case, but also in other combinations or on their own, without going beyond the scope of the invention to leave.

• 1 eine schematische Darstellung zum Veranschaulichen eines Trainingsprozesses eines Bewertungsmodells;
• 2 eine schematische Darstellung zum Veranschaulichen eines Verfahrens zum Bereitstellen von fusionierten Daten für ein Assistenzsystem; und
• 3 eine schematische Darstellung eines Kraftfahrzeugs mit einem solchen Assistenzsystem.

The drawing shows in:

• 1 a schematic representation for illustrating a training process of an evaluation model;
• 2 a schematic representation to illustrate a method for providing merged data for an assistance system; and
• 3 a schematic representation of a motor vehicle with such an assistance system.

In the figures, identical and functionally identical elements are provided with the same reference symbols.

1 shows a schematic representation to illustrate a training process of a model for supporting at least partially automated operation, in particular of a vehicle. A sensor 10 is used therein to receive signals from a particular environment and provide them as recorded raw data 12 . These raw data 12 are supplied to a training function model 14 as input or input data. Depending on the specific application, the respective training function model 14 can be set up or trained for different intended functions. For example, the training function model 14 may be or include an object detection model. The training function model 14 is or is trained here specifically for the respective sensor 10, that is to say with the raw data 12 provided by this sensor or a functionally corresponding sensor 10.

After the training function model 14 has been trained, the raw data 12 recorded by means of the respective sensor 10 is fed to it for processing. The processing results 16 resulting from this, which indicate a performance, ie an accuracy, effectiveness, reliability, confidence and/or the like, of the training function model 14 when the respective raw data 12 is processed are recorded or made available. The corresponding raw data 12 and the respectively associated processing result 16 are then supplied as training data to an evaluation model 18 assigned to the respective sensor 10 . Based on this training data, the evaluation model 18 is trained to generate a performance value 24 (see 2 ) that indicates the performance of the training function model 14 in processing this raw data 12 .

This training process is carried out for a plurality of, in particular different, sensors 10 with a respectively assigned training function model 14 and a respectively assigned evaluation model 18 .

2 shows a schematic representation to illustrate a practical application of the evaluation models 18 trained in this way. An assistance system 20, in particular for a vehicle, is represented schematically for this purpose. The assistance system 20 can include a plurality of sensors 10 or have corresponding interfaces for connecting a plurality of sensors 10 or for acquiring the raw data 12 provided by them. For each sensor 10, each sensor type or each corresponding interface, the assistance system 20 can be trained for the respective sensor 10 or a corresponding sensor type, ie using the raw data 12 provided by this or such a sensor 10. By way of example, the assistance system 20 here includes two separate sensors 10 each with an associated evaluation model 18. However, further sensors 10 or further evaluation models 18 can also be provided. If several sensors 10 of the same type are provided, a separate evaluation model 18 or a common evaluation model 18 can be provided for each of them, to which the raw data 12 of several or all sensors 10 of the same type can then be fed, for example.

Next includes the assistance system 20 a fusion unit 22. This fusion unit 22 is set up to the raw data 12 of the various Sensors 10 to merge together. However, not only the raw data 12 but also the power values 24 predicted for them by the evaluation models 18 are not fed to the fusion unit 22 as input data. The fusion unit 22 then carries out a weighted fusion of the raw data 12, the individual weightings of the raw data 12 being given or determined by the respective associated power values 24. The fusion unit 22 then supplies correspondingly fused data 26 as output.

Assistance system 20 also includes a functional model 28 that is set up or trained to exercise an intended assistance function of assistance system 20 . The functional model 28 can also be or include an artificial neural network, in particular a deep artificial neural network. The functional model 28 can in particular be of the same type, i.e. it can be set up or trained for the same function as the training function models 14. In contrast to these, however, the functional model 28 is trained to use this function based on the merged data 26, ultimately based on Data from several, in particular different, sensors 10 exercise. Accordingly, the merged data 26 is supplied to the functional model 28 as input data. The functional model 28 then processes this merged data 26 and can generate a corresponding result signal, which can then be output by the assistance system 20, for example, as indicated here schematically by a corresponding arrow.

3 shows a schematic representation of a motor vehicle 30, which is equipped with such an assistance system 20. Motor vehicle 30 includes three sensors 10 here, for example, which can be or include, for example, an optical camera, a lidar or radar device, and an ultrasonic device. Here, these sensors 10 are connected to the assistance system 20 via an input interface 32 . For processing the raw data 12 of the sensors 10 recorded via the input interface 32, the assistance system 20 comprises a computer-readable data memory 34 and a processor 36 connected thereto. or computer program can be stored, which can be executed by the processor 36 in order to implement the intended function of the assistance system 20.

The assistance system 20 can be set up, for example, to automatically control the motor vehicle 30 or at least one vehicle device 38 of the motor vehicle 30 based on the merged data 26 .

In the event that none of the sensors 10 supplies raw data 12 that can be used for this purpose, i.e. the performance values 24 for all sensors 10 or their raw data 12 are, for example, below a predefined threshold value, the assistance system 20 can be set up to use an output interface 40 to safeguard Request signal 42 to issue. The assistance system 20 can use the request signal 42 to request manual control of the control of its intended function or of the motor vehicle 30 as a whole by an operator. This can be, for example, a driver 44 of motor vehicle 30 or a vehicle-external teleoperator 46, which is also indicated schematically here.

Overall, the examples described show how problematic sensor data can be detected using a sensor-specific performance prediction in order to enable a particularly reliable and robust assistance functionality in different operating conditions.

Reference List

10

sensor

12

raw data

14

training function model

16

processing result

18

scoring model

20

assistance system

22

fusion unit

24

performance values

26

merged data

28

functional model

30

motor vehicle

32

input interface

34

data storage

36

processor

38

in-vehicle equipment

40

output interface

42

request signal

44

driver

46

teleoperator

QUOTES INCLUDED IN DESCRIPTION

This list of documents cited by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent Literature Cited

• DE 102018101505 A1 [0004]

Claims (10)

Method for providing data (26) for an assistance system (20) of a motor vehicle (30), in which - respective raw data (12) are recorded by several sensors (10), - a performance value (24) is predicted for each of these raw data (12) by means of a predetermined, trained evaluation model (18) assigned to the respective sensor (10), the performance values (24) indicating how well the assistance system (20) is functioning as intended can probably exercise based on the respective raw data (12), and - the raw data (12) from the plurality of sensors (10) are weighted according to their performance values (24) and merged with one another in order to provide the data (26) for the assistance system (20). procedure after claim 1 , characterized in that the power values (24) are predicted at least essentially at the same frequency as the acquisition of the raw data (12). Method according to one of the preceding claims, characterized in that the raw data (12) from the plurality of sensors (10) are fused using a low-level data fusion method. Method according to one of the preceding claims, characterized in that the raw data (12) are recorded by different types of sensors (10), in particular by sensors (10) based on different physical measurement principles. Method according to one of the preceding claims, characterized in that the power values (24) are each compared with a predetermined power threshold value and raw data (12) whose power value (24) is less than the power threshold value is discarded so that they are not included in the fusion with flow into the remaining raw data (12). Method according to one of the preceding claims, characterized in that the power values (24) are compared with a specified threshold value and if the power values (24) of all raw data (12) recorded at least substantially simultaneously are smaller than the specified threshold value, a request is automatically made Taking over a manual control is issued. Method according to one of the preceding claims, characterized in that the assistance system (20) comprises a predefined functional model (28) for the automated exercise of its intended function, which is set up to process data (26) originating from a plurality of sensors (10), and the individual evaluation models (18) based on raw data (12) and based on processing results (16) of a training function model that is only loaded with raw data (12) from the sensor (10) assigned to the respective evaluation model (18) but otherwise corresponds to the specified function model (28). (14) are trained. Method according to one of the preceding claims, characterized in that the evaluation models (18) are trained with raw data (12) from sensors (10) functioning as intended and from damaged and/or soiled sensors (10). Assistance system (20) for a motor vehicle (30), having several evaluation models (18) for predicting performance values (24) for raw data (12) recorded by means of several sensors (10), a fusion unit (22) for automatically merging the raw data (12) , a functional model (28) for performing a specific assistance function of the assistance system (20) and a data interface (40) for outputting a result signal resulting from the processing of the merged raw data (12), the assistance system (20) for automatically executing a method according to one of the preceding claims is set up. Motor vehicle (30), having a plurality of sensors (10) for recording raw data (12) and an assistance system (20) connected to them claim 9 .

Images