DE102022204355A1 - Method for operating a partially automated or automated vehicle and device for a partially automated or automated vehicle - Google Patents

Abstract

The invention relates to a method for operating a partially automated or automated vehicle (50), wherein a partially automated or automated driving function of the vehicle (50) is provided by means of at least one component (51-x) which has an output (52-x). provides a corresponding uncertainty value (10) or reliability value (11), wherein the provided uncertainty value (10) or reliability value (11) is compared with a predetermined threshold value (12), and at least one warning action (20) is carried out based on a comparison result. The invention further relates to a device (1) for a partially automated or automated vehicle (50).

Description

The invention relates to a method for operating a partially automated or automated vehicle and a device for a partially automated or automated vehicle.

Partially automated or automated vehicles record an environment using sensors, process environmental data and plan a future path of the vehicle using a trajectory planner. It is important to take other road users in the area into account and protect them, especially pedestrians.

From the CN 110329152 A is an early warning system for the blind spot. The system includes an angle detection module, an early warning module, an image target detection module, a video auxiliary module and a single-chip microcomputer. The angle detection module includes two angle sensors and detects the actual turning angle of the front wheels, transmits the actual turning angle to a single-chip microcomputer host, and calculates a danger zone; a light early warning module reserves the light of the danger area for early warning; Cameras of a vehicle body are started, and the image target detection module captures the classification of an obstacle, and the video auxiliary module projects the state of the danger area on a screen through the vehicle camera and outputs the state of the danger area to a sound early warning module at the same time. The sound early warning module sends a warning tone to pedestrians in the danger area.

From J. Gawlikowski et al., A Survey of Uncertainty in Deep Neural Networks, arXiv:2107.03342v3 [cs.LG], arXiv, 2021, https://doi.org/10.48550/arXiv.2107.03342, are various uncertainty measures for neurons networks known.

The invention is based on the object of improving a method for operating a partially automated or automated vehicle and a device for a partially automated or automated vehicle, in particular with regard to protecting other road users.

The object is achieved according to the invention by a method with the features of patent claim 1 and a device with the features of patent claim 9. Advantageous embodiments of the invention result from the subclaims.

In particular, a method for operating a partially automated or automated driving vehicle is provided, wherein a partially automated or automated driving function of the vehicle is provided by means of at least one component that provides an uncertainty value or reliability value corresponding to an output, the provided uncertainty value or reliability value being linked to a predetermined threshold value is compared, and at least one warning action is carried out based on a comparison result.

Furthermore, in particular, a device is created for a partially automated or automated driving vehicle, wherein the vehicle comprises at least one component for providing a partially automated or automated driving function, comprising a monitoring device and a warning device, the monitoring device being set up to provide an output of the at least one Component of the vehicle to receive corresponding uncertainty value or reliability value, to compare the received uncertainty value or reliability value with a predetermined threshold value, and based on a comparison result to control the warning device to carry out at least one warning action.

The method and the device make it possible to carry out a warning action based on an uncertainty or reliability corresponding to an output of a component of the vehicle used in partially automated or automated driving. Such a warning action serves in particular to warn other road users in the vicinity of the vehicle. The idea behind this is that if an uncertainty associated with the output is too large, that is, if an uncertainty value corresponding to the output exceeds the predetermined threshold (for the uncertainty), at least one warning action, in particular to warn other road users in the area, is carried out. As an alternative to uncertainty, reliability associated with the output can also be considered. If the reliability drops, that is, if the reliability value falls below the threshold value specified (for reliability), at least one warning action is carried out, in particular to warn other road users in the area. The uncertainty and the reliability are fundamentally interchangeable and can be selected depending on the specific embodiment. The uncertainty value or reliability value corresponding to the output is provided by the at least one component together with the output. The monitoring device then compares the one provided Uncertainty value or reliability value with the specified threshold value and carries out at least one warning action based on a comparison result.

An advantage of the method and the device is that an environment can be warned depending on an uncertainty or reliability of outputs from components of the vehicle that provide the driving functions in partially automated or automated driving. As a result, uncertainty or reliability when generating the outputs or the functionality of the partially automated or automated driving function can be continuously monitored and an environment can be warned if necessary.

In principle, a component can be a component at any position in the entire signal chain of a partially automated or automated driving function. The signal chain can include an environment detection, an environment recognition (detection), a prediction (prediction) and / or planning of actions of the vehicle (planning).

The uncertainty or reliability corresponds to the output of the at least one component. In particular, the component functions error-free, that is, the uncertainty or reliability does not concern the operation of the at least one component itself, but rather the uncertainty or reliability of information or instructions contained in the output. For example, if an object is detected in the environment, an uncertainty value or reliability value can be assigned to the detected object, for example in the form of a probability (e.g. for uncertainty: “the object is not a pedestrian with a probability of 5%” or correspondingly for reliability: “The object has a 95% probability of being a pedestrian” etc.).

The predetermined threshold value can be determined and/or specified in particular based on empirical tests and/or simulations.

In particular, it is provided that the method is repeated continuously, so that the uncertainty or reliability of the output of the at least one component of the vehicle can be continuously monitored.

Parts of the device, in particular the monitoring device and the warning device, can be designed individually or collectively as a combination of hardware and software, for example as program code that is executed on a microcontroller or microprocessor. The monitoring device can in particular comprise a computing device and a memory. However, it can also be provided that parts are designed individually or combined as an application-specific integrated circuit (ASIC) and/or field-programmable gate array (FPGA).

In one embodiment it is provided that the at least one warning action includes outputting an acoustic signal and/or optical signal. This means that other road users, especially pedestrians, in the area around the vehicle can be warned particularly effectively. The warning device can in particular comprise at least one acoustic signal generator, for example a horn, a horn and/or a loudspeaker, for outputting the acoustic signal. Furthermore, the warning device can additionally or alternatively comprise at least one optical signal generator, for example a warning light, a signal light, a flashing light, a display, a display device, a headlight and/or a flash light, for outputting the optical signal. The warning device is arranged in particular on the outside of the vehicle.

In one embodiment it is provided that the at least one warning action includes communication with at least one infrastructure device in the area surrounding the vehicle. This means that the warning campaign can also include infrastructure facilities and use them to warn other road users in the area. For example, loudspeakers on traffic signals and/or warning lights provided for this purpose on infrastructure elements can be used. The monitoring device communicates in particular by means of a communication interface with the at least one infrastructure device (e.g. via a V2X interface).

In one embodiment, it is provided that the at least one component provides and executes at least one artificial intelligence method, wherein the uncertainty value or reliability value for an output of the at least one artificial intelligence method is estimated and/or provided. The artificial intelligence method can in particular be a machine learning method, in particular a trained neural network, in particular a trained deep neural network. The artificial intelligence processes provide uncertainty values or reliability values for outputs (for example in the form of assigned probability values or confidence values). From J. Gawlikowski et al., A Survey of Uncertainty in Deep Neural Networks, arXiv:2107.03342v3 [cs.LG], arXiv, 2021, https://doi.org/10.48550/arXiv.2107.03342, Various uncertainty measures are known for neural networks. Examples include the Monte Carlo dropout method and the ensemble method. When object recognition or object classification in the environment using neural networks, an uncertainty or an uncertainty value can also be estimated, for example, via a frequency of a reclassification that occurs. If a neural network reclassifies an object in the environment several times within a short period of time (e.g. pedestrian, tree, pedestrian,...), this is a sign of greater uncertainty or lower reliability of the classification. If, on the other hand, a neural network continuously sticks to one object class for the object in the environment, this is a sign of lower uncertainty or greater reliability of the classification. An uncertainty value or reliability value can be derived from this, which can then be compared with the specified threshold value.

In one embodiment it is provided that the at least one component is a sensor system, with the sensor system providing the uncertainty value or reliability value for output sensor data. This also allows the uncertainty or reliability of recorded and output sensor data to be taken into account. For example, it is known from aviation that sensors calculate and provide a trust value, which is reduced if a sensor has provided incorrect values. Such a trust value can be provided and used, for example, as a reliability value or, after appropriate conversion, as an uncertainty value.

In one embodiment it is provided that the at least one component is a planning device (e.g. a trajectory planner), the planning device providing the uncertainty value or reliability value for a planned or selected trajectory. In this way, uncertainty or reliability that occurs when planning actions of the vehicle can also be taken into account.

In one embodiment it is provided that the uncertainty values or reliability values corresponding to outputs from several components are combined and compared together with the predetermined threshold value. This allows a common or aggregated uncertainty value or reliability value, in which the uncertainty or reliability of the information provided by the multiple components is taken into account, to be generated and monitored. When summarizing, the uncertainty values or reliability values can also be weighted.

In one embodiment it is provided that the uncertainty values or reliability values corresponding to outputs from several components are each compared individually with a respective predetermined threshold value, with the at least one warning action being carried out based on an x-out-of-N decision of the respective comparison results. This allows the sensitivity of the method to be gradated. In a 1-out-of-N decision, it is enough for a single component to exceed or fall below a predetermined threshold to trigger the warning action. With x-out-of-N decisions (with x > 1), however, it is only enough to exceed or fall below the threshold value for a number x of components, so that the method reacts less sensitively.

In one embodiment it is provided that the predetermined threshold value or the respective predetermined threshold values are selected depending on a position of the vehicle and/or a current context and/or a current scenario. As a result, the threshold value or the respective threshold values can be varied depending on the location and/or the context and/or the scenario. For example, if a component for detecting pedestrians is monitored, the predetermined threshold value can be selected differently on a highway, where the appearance of pedestrians is less likely, than, for example, on a play street, where the appearance of pedestrians is much more likely. The selection can then be location-dependent (e.g. based on a geographical position) and/or context-dependent (e.g. many other road users detected in the area, only other vehicles detected in the area, pedestrians detected in the area, sunny weather, rain, vehicle in front yes or no, etc. ) and/or depending on the scenario (motorway, country road, city street, side street, play street, etc.). The respective predetermined threshold values are then retrieved by the monitoring device depending on the location and/or context and/or scenario, for example from a table or database.

Further features for the design of the device result from the description of embodiments of the method. The advantages of the device are the same as in the embodiments of the method.

Furthermore, in particular, an assistance system for a partially automated or automated driving vehicle is created, comprising at least one component for providing a partially automated or automated driving function, and at least one device according to one of the described embodiments.

Furthermore, a vehicle is also created in particular, comprising at least one assistance system according to one of the described embodiments and/or at least one device according to one of the described embodiments. The vehicle is in particular a motor vehicle. In principle, the vehicle can also be another land, rail, water, air or space vehicle, for example an air taxi or a drone.

• 1 eine schematische Darstellung einer Ausführungsform der Vorrichtung für ein teilautomatisiert oder automatisiert fahrendes Fahrzeug;
• 2 ein schematisches Ablaufdiagramm einer Ausführungsform des Verfahrens zum Betreiben eines teilautomatisiert oder automatisiert fahrenden Fahrzeugs.

The invention is explained in more detail below using preferred exemplary embodiments with reference to the figures. Show here:

• 1 a schematic representation of an embodiment of the device for a partially automated or automated vehicle;
• 2 a schematic flowchart of an embodiment of the method for operating a partially automated or automated vehicle.

1 shows a schematic representation of an embodiment of the device 1 for a partially automated or automated driving vehicle 50. The vehicle 50 includes several components 51-x for providing functionality for partially automated or automated driving. The components 51-x include a sensor system 51-1 for detecting the surroundings, a planning device 51-2 for evaluating recorded sensor data from the sensor system 51-1 and for planning actions of the vehicle 50, and an actuator system 51-3 for longitudinal and lateral guidance of the vehicle 50, by means of which the actions planned by the planning device 51-2 are implemented.

The device 1 is in the vehicle 50 arranged. The device 1 comprises a monitoring device 2 and a warning device 3. The device 1 is in particular set up to carry out the method described in this disclosure. The device 1 and the components 51-x together form in particular an assistance system 200 of the vehicle 50.

The monitoring device 2 includes, for example, a computing device (not shown) and a memory (not shown). The computing device carries out the computing operations necessary in particular to carry out measures of the method. The monitoring device 2 is set up to receive an uncertainty value 10 or reliability value 11 corresponding to an output 52-x from at least one component 51-x of the vehicle 50. In the example shown, the uncertainty value 10 or reliability value 11 is supplied to the monitoring device 2 by the planning device 51-2. The monitoring device 2 compares the received uncertainty value 10 or reliability value 11 with a predetermined threshold value 12 and, based on a comparison result, controls the warning device 3 to carry out at least one warning action 20. If it is an uncertainty value 10, the specified threshold value 12 indicates a maximum value that should not be exceeded. If the uncertainty value 10 exceeds the predetermined threshold value 12, the warning action 20 is carried out. Otherwise, no warning action 20 is carried out. Alternatively, if it is a reliability value 11, the predetermined threshold value 12 indicates a minimum value that should not be fallen below. If the reliability value 11 falls below the predetermined threshold value 12, the warning action 20 is carried out. Otherwise, no warning action 20 is carried out.

It can be provided that the at least one warning action 20 includes outputting an acoustic signal and/or optical signal. For this purpose, the warning device 3 includes in particular at least one acoustic signal generator 4-1 and/or at least one optical signal generator 4-2.

It can be provided that the at least one warning action 20 includes communication with an infrastructure device 60 in the vicinity of the vehicle 50. For this purpose, the warning device 3 includes, for example, a suitable communication interface 4-3, with which the warning device 3 can communicate with the infrastructure device 60 in the environment. The infrastructure device 60 can, for example, be controlled in such a way that it emits an acoustic warning signal and/or a visual warning signal.

It can be provided that the at least one component 51-x provides and executes at least one artificial intelligence method, with the uncertainty value 10 or reliability value 11 being estimated for an output 52-2 of the at least one artificial intelligence method 53. From J. Gawlikowski et al., A Survey of Uncertainty in Deep Neural Networks, arXiv:2107.03342v3 [cs.LG], arXiv, 2021, https://doi.org/10.48550/arXiv.2107.03342, are various uncertainty measures for neurons Networks are known which can provide suitable uncertainty values 10.

It can be provided that the at least one component 51-x is a sensor system 51-1, with the sensor system 51-1 providing the uncertainty value 10 or reliability value 11 for output sensor data (output 52-1).

It can be provided that the at least one component 51-x is a planning device 51-2, with the planning device 51-2 providing the uncertainty value 10 or reliability value 11 for a planned or selected trajectory (output 52-2).

It can be provided that the uncertainty values 10 or reliability values 11 corresponding to outputs 52-x from several components 51-x are combined and compared together with the predetermined threshold value 12. This is done in particular by means of the monitoring device 2.

It can be provided that the uncertainty values 10 or reliability values 11 corresponding to outputs 52-x from several components 51-x are each compared individually with a predetermined threshold value 12, with the at least one warning action 20 starting from an x-out-of-N- Decision of the respective comparison results is carried out. This is done in particular by means of the monitoring device 2.

It can be provided that the predetermined threshold value 12 or the respective predetermined threshold values 12 are selected depending on a position 30 of the vehicle 50 and/or a current context 31 and/or a current scenario 32. For this purpose, the corresponding information is supplied to the monitoring device 3. The monitoring device 3 then selects the respective threshold values 12, for example from a table or a database in which the threshold values 12, linked to different positions 30 and/or contexts 31 and/or scenarios 32, are stored.

The 2 shows a schematic flowchart of an embodiment of the method for operating a partially automated or automated vehicle. The method is carried out in particular by means of a device according to one of the embodiments described above.

In a measure 100, an uncertainty value or a reliability value that corresponds to an output of at least one component is received by the monitoring device. Depending on how many components are monitored, multiple uncertainty values and/or reliability values may also be received.

In a measure 101, the uncertainty value or reliability value is compared with a predetermined threshold value. This is done using the monitoring device. If there is an uncertainty value, it is checked whether the specified threshold value has been exceeded or not. If this is the case, at least one warning action is carried out in a measure 102. Otherwise, action 100 continues for the next time step. A reliability value is used to check whether the specified threshold value is undershot or not. If this is the case, at least one warning action is carried out in a measure 102. Otherwise, action 100 continues for the next time step.

After measure 102, the measures are repeated.

Further embodiments of the method have already been described above with reference to the device.

Reference symbol list

1

contraption

2

Monitoring facility

3

Warning device

4-1

acoustic signal generator

4-2

optical signal transmitter

4-3

Communication interface

10

Uncertainty value

11

Reliability score

12

specified threshold value

20

Warning action

30

position

31

context

32

current scenario

50

vehicle

51-x

component

51-1

Sensors

51-2

Planning facility

51-3

Actuators

52-x

output

52-1

Sensor data

53

Artificial intelligence processes

60

Infrastructure facility

100-102

Measures of the procedure

200

Assistance system

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• CN 110329152 A [0003]

Claims (10)

Method for operating a partially automated or automated vehicle (50), wherein a partially automated or automated driving function of the vehicle (50) is provided by means of at least one component (51-x) which has an uncertainty value (10) corresponding to an output (52-x). ) or reliability value (11), wherein the provided uncertainty value (10) or reliability value (11) is compared with a predetermined threshold value (12), and at least one warning action (20) is carried out based on a comparison result. Procedure according to Claim 1 , characterized in that the at least one warning action (20) comprises issuing an acoustic signal and / or optical signal. Procedure according to Claim 1 or 2 , characterized in that the at least one warning action (20) includes communication with at least one infrastructure device (60) in the vicinity of the vehicle (50). Method according to one of the preceding claims, characterized in that the at least one component (51-x) provides and executes at least one artificial intelligence method (53), wherein the uncertainty value (10) or reliability value (11) for an output (52- x) the at least one artificial intelligence method (53) is estimated and/or provided. Method according to one of the preceding claims, characterized in that the at least one component (51-x) is a sensor system (51-x), the sensor system (51-1) determining the uncertainty value (10) or reliability value (11) for output sensor data (52-1) provides. Method according to one of the preceding claims, characterized in that the uncertainty values (10) or reliability values (11) corresponding to outputs (52-x) of several components (51-x) are combined and compared together with the predetermined threshold value (12). . Procedure according to one of the Claims 1 until 5 , characterized in that the uncertainty values (10) or reliability values (11) corresponding to outputs (52-x) of several components (51-x) are each compared individually with a predetermined threshold value (12), the at least one warning action ( 20) is carried out based on an x-out-of-N decision of the respective comparison results. Method according to one of the preceding claims, characterized in that the predetermined threshold value (12) or the respective predetermined threshold values (12) depend on a position (30) of the vehicle (50) and/or a current context (31) and/or a current scenario (32). Device (1) for a partially automated or automated driving vehicle (50), the vehicle (50) comprising at least one component (51-x) for providing a partially automated or automated driving function, comprising: a monitoring device (2), and a warning device (3), wherein the monitoring device (2) is set up to receive an uncertainty value (10) or reliability value (11) corresponding to an output (52-x) of the at least one component (51-x) of the vehicle (50), the received uncertainty value ( 10) or reliability value (11) to compare with a predetermined threshold value (12), and based on a comparison result, to control the warning device (3) to carry out at least one warning action (20). Assistance system (200) for a partially automated or automated vehicle (50), comprising at least one component (51-x) for providing a partially automated or automated driving function, and at least one device (1). Claim 9 .

Images