DE102022209457A1 - Method for controlling an artificial neural network when providing a vehicle function - Google Patents

Abstract

The invention relates to a method for controlling an artificial neural network (ANN) when providing a vehicle function (F), comprising: - starting the artificial neural network (ANN), - starting the vehicle function (F), - monitoring the vehicle function (F). Deviations (A) when performing the vehicle function (F), - Analyzing functional data (D) when performing the vehicle function (F), if a deviation (A) was detected during monitoring, - Determining, depending on the analysis, whether a decision by the artificial neural network (ANN) is at least partially responsible for the deviation (A), - starting at least one method for adapting the artificial neural network (ANN) if it was recognized during the determination that a decision by the artificial neural network (ANN) is at least partially responsible for the deviation (A).

Description

The invention relates to a method for controlling an artificial neural network when providing a vehicle function, which can be carried out in particular when the artificial neural network is being trained and/or during normal operation of the artificial neural network. The invention further relates to a corresponding computer program product. The invention further relates to a corresponding control device for carrying out a corresponding method. The invention also relates to a corresponding vehicle with a corresponding control device.

In the operation of modern vehicles, artificial neural networks are increasingly being used to automatically provide certain vehicle functions without the involvement of a driver. The use of artificial neural networks is often crucial, particularly in the area of autonomous vehicles. When operating vehicles, rare situations can arise that can adversely affect the learning behavior and/or operating behavior of artificial neural networks. The following situation can serve as an example. An artificial neural network learns that an unavoidable accident will result from a wrong-way driver on a highway. The artificial neural network can learn the following behavior from this situation. The vehicle is no longer driven on motorways, with routes with one direction of travel being preferred, for example. The error may be due to the fact that the neural network has learned that a wrong-way driver on a highway, albeit a potential one, can cause an inevitable accident.

The object of the invention is therefore to provide an improved method for controlling an artificial neural network when providing a vehicle function, which can be carried out during training of the artificial neural network and / or during normal operation of the artificial neural network. In particular, it is the object of the invention to provide an improved method for controlling an artificial neural network when providing a vehicle function, which avoids impairment of vehicle functions due to rare or unlikely deviations, which provides improved vehicle functions, and which improves customer comfort and increases the safety of operating the vehicle. A further object of the invention is to provide a corresponding computer program product for carrying out a corresponding method. It is also an object of the invention to provide a corresponding control device. It is also an object of the invention to provide a corresponding vehicle with a corresponding control device.

The object according to the invention is solved by a method for controlling an artificial neural network when providing a vehicle function with the features of the independent method claim. The object according to the invention is solved by a corresponding computer program product for carrying out a corresponding method with the features of the independent product claim. The object according to the invention is achieved by a corresponding control device for carrying out a corresponding method with the features of the independent device claim. The object according to the invention is achieved by a corresponding vehicle with a corresponding control device with the features of the second independent device claim. Further features, advantages and details of the invention emerge from the subclaims, the description and the drawings. Features and details that are described in connection with individual aspects of the invention naturally also apply in connection with the other aspects of the invention and vice versa, so that reference is or can always be made to each other with regard to the disclosure of the individual aspects of the invention.

• - Starten des künstlichen neuronalen Netzes,
• - Starten der Fahrzeugfunktion,
• - Überwachen der Fahrzeugfunktion auf Abweichungen beim Durchführen der Fahrzeugfunktion (insbesondere im Vergleich zum Normalbetrieb der Fahrzeugfunktion im Betrieb des Fahrzeuges, wie z. B. auf Fehler, unerwartete und/oder abwegige Entscheidungen des künstlichen neuronalen Netzes, nicht optimale Abläufe beim Durchführen der Fahrzeugfunktion, nicht optimale Abläufe im Betrieb des Fahrzeuges, sogar Unfälle, usw.),
• - Analysieren von Funktionsdaten beim Durchführen der Fahrzeugfunktion, wenn beim Überwachen eine Abweichung erkannt wurde,
• - Bestimmen in Abhängigkeit von dem Analysieren, ob (gemeint ist mindestens) eine Entscheidung des künstlichen neuronalen Netzes für die Abweichung zumindest zum Teil ursächlich ist,
• - Starten von mindestens einem Verfahren zum Anpassen (bzw. Korrigieren und/oder Anlernen) des künstlichen neuronalen Netzes, wenn beim Bestimmen erkannt wurde, dass eine Entscheidung des künstlichen neuronalen Netzes für die Abweichung zumindest zum Teil ursächlich ist, um vorzugsweise die (erkannte) Abweichung (zukünftig) möglichst zu vermeiden.

The invention provides: a method for controlling (meaning at least) an artificial neural network (can also be referred to as an artificial intelligence) when providing (meaning at least) a vehicle function, which in particular when training the artificial neural network and / or can be carried out during normal operation of the artificial neural network, comprising:

• - starting the artificial neural network,
• - Starting the vehicle function,
• - Monitoring the vehicle function for deviations when carrying out the vehicle function (particularly in comparison to the normal operation of the vehicle function during operation of the vehicle, such as errors, unexpected and/or erroneous decisions by the artificial neural network, not opti mal processes when carrying out the vehicle function, sub-optimal processes in the operation of the vehicle, even accidents, etc.),
• - Analyzing functional data when performing the vehicle function if a deviation was detected during monitoring,
• - Determine, depending on the analysis, whether (meaning at least) a decision by the artificial neural network is at least partially responsible for the deviation,
• - Starting at least one method for adapting (or correcting and/or training) the artificial neural network if, during the determination, it was recognized that a decision by the artificial neural network is at least partially responsible for the deviation, in order preferably to achieve the (recognized) Deviations should be avoided (in the future) if possible.

An artificial neural network in the context of the invention can be understood to mean all methods, structures and/or devices that enable the operation of an artificial intelligence. An artificial neural network within the scope of the invention can also be referred to for simplicity as an artificial intelligence. An artificial neural network within the scope of the invention can comprise one or more identical and/or different networks of artificial neurons. A network within the scope of the invention can be designed as single-layer, multi-layer and/or recurrent.

• - Auswertung von Sensordaten, insbesondere von Ultraschallsensoren, Radarsensoren, Lidarsensoren, Kameras,
• - Verkehrszeichenerkennung,
• - Fahrerassistenzfunktionen,
• - Berechnung von Fahrmanövern,
• - Abstandsregelung,
• - Notbremsassistent,
• - Spurhalteassistent,
• - Spurwechselassistent,
• - Totwinkel-Überwachung,
• - Nachtsichtassistent,
• - Einparkhilfe,
• - Pre-Crash-Assistent,
• - Pre-Brake-Assistent,
• - Notbremsassistent,
• - Navigation,
• - Infotainment,
• - Entertainment,
• - On-Board-Diagnostik,
• - Telematik,
• - Voice-Control,
• - Gestenerkennung,
• - Positionserkennung, usw.

A vehicle function can include, for example:

• - Evaluation of sensor data, especially from ultrasonic sensors, radar sensors, lidar sensors, cameras,
• - traffic sign recognition,
• - driver assistance functions,
• - Calculation of driving maneuvers,
• - distance control,
• - emergency brake assistant,
• - lane departure warning,
• - lane change assistant,
• - blind spot monitoring,
• - night vision assistant,
• - parking assistance,
• - Pre-crash assistant,
• - Pre-Brake Assistant,
• - emergency brake assistant,
• - navigation,
• - Infotainment,
• - entertainment,
• - On-board diagnostics,
• - telematics,
• - voice control,
• - gesture recognition,
• - Position detection, etc.

Functional data can include all inputs, such as: B. Sensor data, for the artificial neural network as well as all outputs or decisions of the artificial neural network.

The method can preferably be used for automated, highly automated or autonomous vehicles that use (at least) an artificial neural network when providing (at least) one vehicle function.

The method can be carried out when training the artificial neural network in order to train the artificial neural network. The method can be carried out during normal operation of the artificial neural network in order to adapt the artificial neural network.

A supervised, partially supervised, active, independent or even unsupervised training method can be used to train the artificial neural network. To train the artificial neural network, different driving situations can be trained gradually.

During normal operation of the artificial neural network, the artificial neural network can already support the operation of the vehicle, and after deviations have been detected, an adjustment of the artificial neural network can be initiated in order to avoid the deviation in the future.

Adjusting can include retraining the artificial neural network, for example through backpropagation, and/or further measures.

The invention recognizes that certain situations or events in the operation of the vehicle can be a hindrance to properly training and/or safely operating artificial neural networks. One reason for this may be that these situations rarely occur during vehicle operation. The invention further recognizes that these situations must not influence the learning process of the artificial neural networks to a certain extent.

The invention proposes that after detecting deviations that can occur, for example, as a result of rarely occurring and/or unlikely situations in the operation of the vehicle, at least one method for adapting the artificial neural network is initiated in order to correct the same deviation to avoid future operation of the artificial neural network.

On the one hand, in a method for adapting the artificial neural network, it can be provided that rarely occurring events are not specified as a source of training data for training and / or adapting the artificial neural network or another artificial neural network for the same vehicle function. In this way the training data can be filtered.

On the other hand, in a method for adapting the artificial neural network, it can be provided that an evaluation of experiences is carried out according to their importance for the customer and their probability of occurrence, for example with a key figure in each case. In this way, the influence of deviations on training can be reduced and the influence of normal situations can be increased, for example by training many (in number) cases without the detected deviation, for example without a wrong-way driver.

Furthermore, in a method for adapting the artificial neural network, provision can be made for a, in particular selective, learning of the special situation in the operation of the vehicle in which the deviation occurs to be carried out. In the situation with a wrong-way driver, learning, particularly selective learning, can carry out different learning scenarios, e.g. B. as follows: First learning scenario: - Information about wrong-way drivers in the planned motorway section is available: Answer: Do not drive on the endangered section of the highway → correct, Answer: Driving on the endangered section of the highway → wrong. Second learning scenario: - There is no information about wrong-way drivers in the planned motorway section: Answer: Do not drive on the highway section → wrong, Answer: Driving on the stretch of highway → correct.

Furthermore, in a method for adapting the artificial neural network, it can be provided that a targeted deletion of experiences (so-called forgetting, for example training data and/or results that the artificial neural network provides) is carried out. If, for example, an artificial intelligence or an artificial neural network is saved in versions, then the deviation can be traced back until a version without the deviation has been found. In this case, all subsequent experiences can be deleted and, if necessary, retrained, preferably with improved training data. However, deleting experiences can also be advantageous during normal operation of the vehicle function in order to enable further operation of the vehicle, albeit with a reduced version of the artificial neural network. Further operation of the vehicle can serve to safely stop the vehicle or, preferably, to drive it to a workshop and/or training platform where the artificial neural network can be adjusted.

A deviation can be judged based on an advantage from the customer's perspective. In the event of a deviation, the process can provide effective mechanisms to detect the cause and to avoid the same deviation in the future by adapting the artificial intelligence.

With the help of the method, it can be made possible in an advantageous manner that deviations in the implementation of the vehicle function (in particular in comparison to the normal operation of the vehicle function in the operation of the vehicle, such as errors, unexpected and/or erroneous decisions of the artificial neural network, are not optimal Processes when carrying out the vehicle function, sub-optimal processes in the operation of the vehicle, even accidents, etc.) should be avoided as far as possible. The method can advantageously be used to avoid deviations when performing the vehicle function, which can be a hindrance to properly training and/or safely operating artificial neural networks. With the help of the invention, an improved method for controlling an artificial neural network when providing a vehicle function can be made possible, which avoids impairment of vehicle functions due to rare or unlikely incorrect decisions by the artificial neural network, which provides improved vehicle functions, and which improves customer comfort and the Safety in the operation of the vehicle is significantly increased.

Furthermore, it can be provided that user input is taken into account when monitoring the vehicle function for deviations. In this way, the user's view can be taken into account, which is viewed as a deviation in the performance of the vehicle function, in particular in comparison to the normal operation of the vehicle function, and/or an impairment in the operation of the vehicle. This means customer comfort can be significantly increased.

Furthermore, it can be provided that when monitoring the vehicle function for deviations, reports from at least one assistance system of the vehicle are taken into account. In this way, when monitoring the vehicle function for deviations, the effects on the vehicle's assistance systems can be taken into account. This means that the safety of operating the vehicle can be significantly increased.

It can also be provided that at least one further artificial neural network is used when monitoring the vehicle function for deviations. This makes it possible to automatically detect deviations.

Furthermore, it can be provided that traffic data, navigation data, environmental data and/or weather data are taken into account when analyzing functional data. In this way, an improved analysis can be made as to whether a decision by the artificial neural network is at least partially responsible for the deviation. This also allows the decisions of the artificial neural network to be analyzed in an improved way. This enables improved analysis of connections between deviations and decisions of the artificial neural network and improved error analysis.

In addition, it can be provided that sensor data from the vehicle, a road user and/or an infrastructure element are taken into account when analyzing functional data. In this way, the analysis can be improved as to whether a decision by the artificial neural network is at least partially responsible for the deviation. This allows the decisions of the artificial neural network to be analyzed in an improved way. Consequently, improved analysis of connections and improved error analysis can be made possible.

Furthermore, it can be provided that at least one external computer is used when analyzing functional data and / or when determining whether a decision by the artificial neural network is at least partially responsible for the deviation (or in a so-called error analysis). The computer can be deployed in the cloud, in a workshop and/or on an artificial neural network training station. In this way, maintenance services for the artificial neural network can be provided.

Furthermore, it can be provided that at least one further artificial neural network is used when analyzing functional data and/or when determining whether a decision by the artificial neural network is at least partially responsible for the deviation. This enables automated analysis of connections and improved error analysis.

• - Bereitstellen von Trainingsdaten zum Anpassen des künstlichen neuronalen Netzes und/oder zum Trainieren dieses und/oder eines weiteren künstlichen neuronalen Netzes für die Fahrzeugfunktion, bei denen die Funktionsdaten gelöscht wurden, zu denen eine Abweichung erkannt wurde.

Furthermore, the at least one method for adapting the artificial neural network can include at least a first method in order to avoid the deviation:

• - Providing training data for adapting the artificial neural network and/or for training this and/or another artificial neural network for the vehicle function, in which the functional data for which a deviation was detected was deleted.

In this way, a computationally simple solution can be provided to clean up the training data and thus hide situations or events in the operation of the vehicle that can affect the training and/or the safe operation of artificial neural networks.

• - Bewerten der Abweichung in Bezug auf Kundenkomfort, Sicherheit, Auftretenswahrscheinlichkeit und/oder Entdeckungswahrscheinlichkeit,
• - Kennzeichnen von Funktionsdaten beim Durchführen der Fahrzeugfunktion, bei denen eine Abweichung erkannt wurde, mit einer Kennzahl in Abhängigkeit vom Bewerten, und/oder
• - Bereitstellen von Trainingsdaten zum Anpassen des künstlichen neuronalen Netzes und/oder zum Trainieren dieses und/oder eines weiteren künstlichen neuronalen Netzes für die Fahrzeugfunktion, bei denen die Funktionsdaten beim Durchführen der Fahrzeugfunktion mit einer Kennzahl gewichtet wurden.

In addition, the at least one method for adapting the artificial neural network can include at least a second method in order to avoid the deviation:

• - Evaluate the deviation in terms of customer comfort, safety, probability of occurrence and/or probability of detection,
• - Marking functional data when performing the vehicle function in which a deviation was detected with a key figure depending on the evaluation, and / or
• - Providing training data for adapting the artificial neural network and/or for training this and/or another artificial neural network for the vehicle function, in which the functional data was weighted with a key figure when performing the vehicle function.

In this way, an advantageous solution can be provided to reduce the influence of training data and thus of situations or events in the operation of the vehicle that can affect the training and/or the safe operation of artificial neural networks.

• - Bereitstellen von einem ersten Set von Trainingsdaten zum Anpassen des künstlichen neuronalen Netzes und/oder zum Trainieren dieses und/oder eines weiteren künstlichen neuronalen Netzes für die Fahrzeugfunktion aus den Funktionsdaten beim Durchführen der Fahrzeugfunktion, bei denen eine Abweichung erkannt wurde,
• - Bereitstellen von einem zweiten Set von Trainingsdaten zum Anpassen des künstlichen neuronalen Netzes und/oder zum Trainieren dieses und/oder eines weiteren künstlichen neuronalen Netzes für die Fahrzeugfunktion aus Funktionsdaten beim Durchführen der Fahrzeugfunktion, bei denen keine Abweichung auftritt, und/oder
• - Anlernen des künstlichen neuronalen Netzes zum Bereitstellen der Fahrzeugfunktion mithilfe vom ersten Set von Trainingsdaten und mithilfe vom zweiten Set von Trainingsdaten,

wobei insbesondre beim Anlernen des künstlichen neuronalen Netzes zum Bereitstellen der Fahrzeugfunktion mithilfe vom ersten Set von Trainingsdaten und mithilfe vom zweiten Set von Trainingsdaten die Sets von Trainingsdaten mit unterschiedlichen Kennzahlen gewichtet werden.In addition, the at least one method for adapting the artificial neural network may include at least a third method in order to avoid the deviation:

• - Providing a first set of training data for adapting the artificial neural network and/or for training this and/or a further artificial neural network for the vehicle function from the functional data when performing the vehicle function in which a deviation was detected,
• - Providing a second set of training data for adapting the artificial neural network and/or for training this and/or a further artificial neural network for the vehicle function from functional data when performing the vehicle function in which no deviation occurs, and/or
• - training the artificial neural network to provide the vehicle function using the first set of training data and using the second set of training data,

in particular when training the artificial neural network to provide the vehicle function using the first set of training data and using the second set of training data, the sets of training data are weighted with different key figures.

In this way, selective learning of a specific situation or event during operation of the vehicle can be provided in order to improve the situation or event to be able to control and enable improved operation of the vehicle if the situation or event does not occur.

• - Bestimmen einer Version des künstlichen neuronalen Netzes, bei welcher die Abweichung erkannt wurde,
• - Löschen aller Erfahrungsdaten des künstlichen neuronalen Netzes ab der Version, bei welcher die Abweichung erkannt wurde,
• - Verwenden einer vorläufigen Version des künstlichen neuronalen Netzes beim Bereitstellen der Fahrzeugfunktion, und/oder
• - erneutes Anlernen des künstlichen neuronalen Netzes angefangen von der Version des künstlichen neuronalen Netzes, bei welcher die Abweichung erkannt wurde.

Furthermore, it can be provided that the at least one method for adapting the artificial neural network includes at least a fourth method in order to avoid the deviation:

• - determining a version of the artificial neural network in which the deviation was detected,
• - Deleting all experience data of the artificial neural network from the version in which the deviation was detected,
• - Using a preliminary version of the artificial neural network when providing the vehicle function, and/or
• - re-training the artificial neural network starting from the version of the artificial neural network in which the deviation was detected.

Like the functional data, the experience data can also contain all inputs, such as: B. Sensor data, for the artificial neural network as well as all outputs or decisions of the artificial neural network.

In this way, targeted adaptation and/or improved training of an artificial neural network can be made possible. This also makes it possible that if the artificial neural network is already in operation, it can continue to function, albeit with a reduced version or capacity.

• - Überprüfen, ob nach dem Anpassen des künstlichen neuronalen Netzes, beim Durchführen der Fahrzeugfunktion die Abweichung auftritt,
• - Zurückkehren zu einem Verfahren nach einem der vorhergehenden Ansprüche, wenn beim Überprüfen die Abweichung wieder erkannt wurde.

Furthermore, it can be provided that the method includes at least one further step:

• - Check whether the deviation occurs after adjusting the artificial neural network when performing the vehicle function,
• - Returning to a method according to one of the preceding claims if the deviation was recognized again during checking.

In this way, an iterative adjustment of the artificial neural network can be made possible.

The invention further provides: a computer program product comprising instructions which, when the computer program is executed by a computer, cause the computer to carry out a method which can run as described above. With the help of the computer program product according to the invention, the same advantages can be achieved that were described in connection with the method described above. These advantages are fully referenced here.

In addition, the invention provides: a control device for controlling an artificial neural network when providing a vehicle function, comprising: a memory unit and a computing unit, a code being stored in the memory unit, and a method being carried out when the code is executed by the computing unit, which can proceed as described above. With the help of the control device according to the invention, the same advantages can be achieved that were explained in connection with the method described above. These advantages are fully referenced here.

The control device can be implemented, for example, in a control device of the vehicle and/or in a control device on an external computer, for example in the cloud, in a workshop and/or on a training station for artificial neural networks.

The invention also provides: a vehicle, in particular a partially automated, highly automated, fully automated or autonomously driving vehicle, having a corresponding control device. With the help of the vehicle according to the invention, the same advantages can be achieved that were explained in connection with the method described above. These advantages are fully referenced here.

• 1 einen beispielhaften Ablauf eines Verfahrens zum Steuern eines künstlichen neuronalen Netzes beim Bereitstellen einer Fahrzeugfunktion, und
• 2 ein beispielhaftes Verfahren zum Anpassen des künstlichen neuronalen Netzes.

Further advantages, features and details of the invention emerge from the following description, in which exemplary embodiments of the invention are detailed with reference to the drawings are described. The features mentioned in the claims and in the description can be essential to the invention individually or in any combination. Show it:

• 1 an exemplary sequence of a method for controlling an artificial neural network when providing a vehicle function, and
• 2 an exemplary method for adapting the artificial neural network.

The 1 and 2 serve to explain a method which was developed for controlling an artificial neural network KNN when providing a vehicle function F, in particular when training the artificial neural network KNN and / or during normal operation of the artificial neural network KNN.

101

Starten des künstlichen neuronalen Netzes KNN,

102

Starten der Fahrzeugfunktion F,

103

Überwachen der Fahrzeugfunktion F auf Abweichungen A beim Durchführen der Fahrzeugfunktion F, die insbesondere im Vergleich zum Normalbetrieb der Fahrzeugfunktion F im Betrieb des Fahrzeuges 100 auftreten können, wie z. B. auf fehlerhafte, unerwartete und/oder abwegige Entscheidungen des künstlichen neuronalen Netzes KNN, nicht optimale Abläufe beim Durchführen der Fahrzeugfunktion F, nicht optimale Abläufe im Betrieb des Fahrzeuges 100, oder sogar Unfälle,

104

Analysieren von Funktionsdaten D beim Durchführen der Fahrzeugfunktion F, die sämtliche Eingaben, wie z. B. Sensordaten, für das künstliche neuronale Netz KNN sowie sämtliche Ausgaben bzw. Entscheidungen des künstlichen neuronalen Netzes KNN umfassen können, wenn beim Überwachen eine Abweichung A erkannt wurde,

105

Bestimmen in Abhängigkeit von dem Analysieren, ob eine Entscheidung des künstlichen neuronalen Netzes KNN für die Abweichung A zumindest zum Teil ursächlich ist,

106

Starten von mindestens einem Verfahren zum Anpassen, was eine Korrektur und/oder ein erneutes Anlernen umfassen kann, des künstlichen neuronalen Netzes KNN, wenn beim Bestimmen erkannt wurde, dass eine Entscheidung des künstlichen neuronalen Netzes KNN für die Abweichung A zumindest zum Teil ursächlich ist, um vorzugsweise die erkannte Abweichung A zukünftig möglichst zu vermeiden.

The procedure has the following actions/procedural steps:

101

Starting the artificial neural network KNN,

102

Starting the vehicle function F,

103

Monitoring the vehicle function F for deviations A when carrying out the vehicle function F, which can occur in particular in comparison to the normal operation of the vehicle function F during operation of the vehicle 100, such as. B. due to incorrect, unexpected and / or absurd decisions of the artificial neural network ANN, non-optimal processes when carrying out the vehicle function F, non-optimal processes in the operation of the vehicle 100, or even accidents,

104

Analyzing functional data D when carrying out the vehicle function F, which includes all inputs, such as. B. Sensor data for the artificial neural network ANN as well as all outputs or decisions of the artificial neural network ANN can include if a deviation A was detected during monitoring,

105

Determining, depending on the analysis, whether a decision by the artificial neural network ANN is at least partially responsible for the deviation A,

106

Starting at least one method for adapting, which can include a correction and/or re-learning, of the artificial neural network KNN if it was recognized during the determination that a decision of the artificial neural network KNN is at least partially responsible for the deviation A, in order to preferably avoid the detected deviation A in the future as far as possible.

An artificial neural network ANN within the scope of the present disclosure can include all methods, structures and/or devices that enable the operation of an artificial intelligence Kl. An artificial neural network ANN can therefore be referred to as an artificial intelligence Kl. An artificial neural network ANN within the scope of the invention can comprise one or more identical and/or different networks of artificial neurons, which can be designed as single-layer, multi-layer and/or recurrent.

• - Auswertung von Sensordaten, insbesondere von Ultraschallsensoren, Radarsensoren, Lidarsensoren, Kameras,
• - Verkehrszeichenerkennung,
• - Fahrerassistenzfunktionen,
• - Berechnung von Fahrmanövern,
• - Abstandsregelung,
• - Notbremsassistent,
• - Spurhalteassistent,
• - Spurwechselassistent,
• - Totwinkel-Überwachung,
• - Nachtsichtassistent,
• - Einparkhilfe,
• - Pre-Crash-Assistent,
• - Pre-Brake-Assistent,
• - Notbremsassistent,
• - Navigation,
• - Infotainment,
• - Entertainment,
• - On-Board-Diagnostik,
• - Telematik,
• - Voice-Control,
• - Gestenerkennung,
• - Positionserkennung, usw.

An artificial neural network ANN within the scope of the present disclosure can provide and/or support at least one of the following vehicle functions F:

• - Evaluation of sensor data, especially from ultrasonic sensors, radar sensors, lidar sensors, cameras,
• - traffic sign recognition,
• - driver assistance functions,
• - Calculation of driving maneuvers,
• - distance control,
• - emergency brake assistant,
• - lane departure warning,
• - lane change assistant,
• - blind spot monitoring,
• - night vision assistant,
• - parking assistance,
• - Pre-crash assistant,
• - Pre-Brake Assistant,
• - emergency brake assistant,
• - navigation,
• - Infotainment,
• - entertainment,
• - On-board diagnostics,
• - telematics,
• - voice control,
• - gesture recognition,
• - Position detection, etc.

An artificial neural network ANN within the scope of the present disclosure can be used in an automated, highly automated or autonomously driving vehicle 100.

On the one hand, the method can be carried out when training the artificial neural network KNN in order to train the artificial neural network in an improved manner.

On the other hand, the method can be carried out during normal operation of the artificial neural network ANN in order to adapt the artificial neural network ANN, which can include correction.

A supervised, partially supervised, active, independent or even unsupervised training procedure can be used to train the artificial neural network ANN. How it is 2 indicates, different driving situations FS1, FS2, ..., FSi can be gradually trained to train the artificial neural network KNN.

During normal operation of the artificial neural network KNN, the artificial neural network KNN can already support the operation of the vehicle 100, whereby after deviations A have been detected, an adjustment/correction of the artificial neural network KNN can be initiated in order to reduce the deviation A as much as possible in the future avoid.

Adjusting can include retraining the artificial neural network ANN, for example through backpropagation, and/or further measures.

With the help of the method, such situations or events can be recognized and taken into account in the operation of the vehicle 100, which can excessively affect the training and / or the operation of artificial neural networks ANN if they only occur rarely in the operation of the vehicle 100, but the learning process of the artificial neural networks have a noticeable impact.

In step 103, the vehicle function F is monitored for deviations A, which can occur, for example, as a result of rarely occurring and/or unlikely situations in the operation of the vehicle 100.

If such a deviation A was detected in step 103, then in step 104 the functional data D is analyzed and an error analysis is carried out to determine whether the artificial neural network ANN may have contributed to the deviation A.

If it has been determined using the analysis in step 105 that a decision by the artificial neural network ANN is at least partially responsible for the deviation A, then in step 106 at least one method for adapting the artificial neural network ANN is initiated in order to achieve the same deviation A to be avoided in the future operation of the artificial neural network ANN.

1. 1 dass selten eintretende Abweichungen A nicht als Quelle für Trainingsdaten zum Trainieren und/oder zum Anpassen des künstlichen neuronalen Netzes KNN oder eines weiteren künstlichen neuronalen Netzes KNN für die gleiche Fahrzeugfunktion F vorgegeben werden. Auf diese Weise können die Trainingsdaten bereinigt werden.

How it is 1 indicates, can be provided in step 106:

1. 1 that rarely occurring deviations A are not specified as a source of training data for training and/or for adapting the artificial neural network KNN or another artificial neural network KNN for the same vehicle function F. In this way, the training data can be cleaned.

• - Bereitstellen von Trainingsdaten zum Anpassen des künstlichen neuronalen Netzes KNN und/oder zum Trainieren dieses und/oder eines weiteren künstlichen neuronalen Netzes KNN für die Fahrzeugfunktion F, bei denen die Funktionsdaten D gelöscht werden, zu denen eine Abweichung A erkannt wurde.

Solution 1 can generally provide:

• - Providing training data for adapting the artificial neural network KNN and/or for training this and/or another artificial neural network KNN for the vehicle function F, in which the functional data D for which a deviation A was detected is deleted.

• 2 dass eine Bewertung von Abweichungen A im Hinblick auf Kundenkomfort und nach ihrer Auftretenswahrscheinlichkeit, bspw. mit jeweils einer Kennzahl, durchgeführt wird. Auf diese Weise kann der Einfluss von Abweichungen A auf des Training reduziert werden. Ebenfalls kann auf diese Weise der Einfluss von öfter vorkommenden Situationen im Betrieb des Fahrzeuges 100 erhöht werden, bspw. durch Anlernen von vergleichsweise mehreren öfter vorkommenden Situationen ohne die erkannte Abweichung, bspw. ohne einen Geisterfahrer.

How it is 1 indicates, can be provided in step 106:

• 2 that an evaluation of deviations A is carried out with regard to customer comfort and their probability of occurrence, for example with a key figure in each case. In this way, the influence of deviations A on training can be reduced. In this way, the influence of more frequently occurring situations in the operation of the vehicle 100 can also be increased, for example by learning comparatively several more frequently occurring situations without the detected deviation, for example without a wrong-way driver.

• - Bewerten der Abweichung A in Bezug auf Kundenkomfort, Sicherheit, Auftretenswahrscheinlichkeit und/oder Entdeckungswahrscheinlichkeit,
• - Kennzeichnen von Funktionsdaten D beim Durchführen der Fahrzeugfunktion F, bei denen eine Abweichung A erkannt wurde, mit einer Kennzahl in Abhängigkeit vom Bewerten, und/oder
• - Bereitstellen von Trainingsdaten zum Anpassen des künstlichen neuronalen Netzes KNN und/oder zum Trainieren dieses und/oder eines weiteren künstlichen neuronalen Netzes KNN für die Fahrzeugfunktion F, bei denen die Funktionsdaten D beim Durchführen der Fahrzeugfunktion F mit einer Kennzahl gewichtet wurden.

Solution 2 can generally provide:

• - Evaluate deviation A in terms of customer comfort, safety, probability of occurrence and/or probability of detection,
• - Marking functional data D when carrying out the vehicle function F, in which a deviation A was detected, with a key figure depending on the evaluation, and / or
• - Providing training data for adapting the artificial neural network ANN and/or for training this and/or another artificial neural network ANN for the vehicle function F, in which the functional data D was weighted with a key figure when carrying out the vehicle function F.

• 3 dass ein, insbesondere selektives, Lernen der speziellen Situation im Betrieb des Fahrzeuges 100 durchgeführt wird, bei welcher die Abweichung A auftritt. In der Situation mit einem Geisterfahrer kann ein, insbesondere selektives, Lernen unterschiedliche Lernszenarien bereitstellen, z. B. wie folgt:

Erstes Lernszenario: - Inforamtion über Geisterfahrer in dem geplanten Autobahnabschnitt liegt vor: Antwort: Nicht auf dem gefährdeten Autobahnabschnitt fahren → richtig, Antwort: Fahren auf dem gefährdeten Autobahnabschnitt → falsch. Zweites Lernszenario: - Information über Geisterfahrer in dem geplanten Autobahnabschnitt liegt nicht vor: Antwort: Nicht auf dem Autobahnabschnitt fahren → falsch, Antwort: Fahren auf dem Autobahnabschnitt → richtig. How it is 1 indicates, can be provided in step 106:

• 3 that a, in particular selective, learning of the special situation in the operation of the vehicle 100 is carried out in which the deviation A occurs. In the situation with a wrong-way driver, learning, particularly selective learning, can provide different learning scenarios, e.g. B. as follows:

First learning scenario: - Information about wrong-way drivers in the planned motorway section is available: Answer: Do not drive on the endangered section of the highway → correct, Answer: Driving on the endangered section of the highway → wrong. Second learning scenario: - There is no information about wrong-way drivers in the planned motorway section: Answer: Do not drive on the highway section → wrong, Answer: Driving on the stretch of highway → correct.

• - Bereitstellen von einem ersten Set S1 von Trainingsdaten zum Anpassen des künstlichen neuronalen Netzes KNN und/oder zum Trainieren dieses und/oder eines weiteren künstlichen neuronalen Netzes KNN für die Fahrzeugfunktion F aus den Funktionsdaten D beim Durchführen der Fahrzeugfunktion F, bei denen eine Abweichung A erkannt wurde,
• - Bereitstellen von einem zweiten Set S2 von Trainingsdaten zum Anpassen des künstlichen neuronalen Netzes KNN und/oder zum Trainieren dieses und/oder eines weiteren künstlichen neuronalen Netzes KNN für die Fahrzeugfunktion F aus Funktionsdaten Dn beim Durchführen der Fahrzeugfunktion F, bei denen keine Abweichung A auftritt, und/oder
• - Anlernen des künstlichen neuronalen Netzes KNN zum Bereitstellen der Fahrzeugfunktion F mithilfe vom ersten Set S1 von Trainingsdaten und mithilfe vom zweiten Set S2 von Trainingsdaten,

wobei insbesondere beim Anlernen des künstlichen neuronalen Netzes KNN zum Bereitstellen der Fahrzeugfunktion F mithilfe vom ersten Set S1 von Trainingsdaten und mithilfe vom zweiten Set S2 von Trainingsdaten die Sets S1, S2 von Trainingsdaten mit unterschiedlichen Kennzahlen gewichtet werden.Solution 3 can generally provide:

• - Providing a first set S1 of training data for adapting the artificial neural network KNN and/or for training this and/or a further artificial neural network KNN for the vehicle function F from the functional data D when carrying out the vehicle function F, in which a deviation A was recognized,
• - Providing a second set S2 of training data for adapting the artificial neural network ANN and/or for training this and/or another artificial neural network KNN for the vehicle function F from function data Dn when performing the vehicle function F, in which no deviation A occurs, and / or
• - training the artificial neural network KNN to provide the vehicle function F using the first set S1 of training data and using the second set S2 of training data,

in particular when training the artificial neural network KNN to provide the vehicle function F using the first set S1 of training data and using the second set S2 of training data, the sets S1, S2 of training data are weighted with different key figures.

• 4 dass ein gezieltes Löschen von Erfahrungen (sog. Vergessen, bspw. von Trainingsdaten und/oder Ergebnissen, die das künstliche neuronale Netz dazu liefert) durchgeführt wird. Wenn das künstliche neuronale Netz KNN in Versionen V1, V2, ..., Vi abgespeichert wird, dann kann die Abweichung A rückverfolgt werden, bis eine Version V ohne die Abweichung gefunden wurde. In diesem Fall können alle danach dazugekommenen Erfahrungen gelöscht werden. Das künstliche neuronale Netz KNN kann anschließend neu trainiert werden, vorzugsweise mit verbesserten Trainingsdaten. Das Löschen von Erfahrungen kann aber auch im Normalbetrieb der Fahrzeugfunktion F vorteilhaft sein, um einen weiteren Betrieb des Fahrzeuges 100 zu ermöglichen, wenn auch mit einer reduzierten Version des künstlichen neuronalen Netzes KNN. Der weitere Betrieb des Fahrzeuges 100 kann dazu genutzt werden, das Fahrzeug 100 sicher anzuhalten oder vorzugsweise zu einer Werkstatt und/oder Trainingsplattform zu fahren, wo das künstliche neuronale Netz KNN angepasst werden kann.

How it is 1 indicates, can be provided in step 106:

• 4 that a targeted deletion of experiences (so-called forgetting, for example training data and/or results that the artificial neural network provides) is carried out. If the artificial neural network KNN is saved in versions V1, V2, ..., Vi, then the deviation A can be traced back until a version V without the deviation was found. In this case, all experiences added afterwards can be deleted. The artificial neural network ANN can then be retrained, preferably with improved training data. However, deleting experiences can also be advantageous during normal operation of the vehicle function F in order to enable further operation of the vehicle 100, albeit with a reduced version of the artificial neural network KNN. The further operation of the vehicle 100 can be used to safely stop the vehicle 100 or, preferably, to drive it to a workshop and/or training platform where the artificial neural network ANN can be adapted.

• - Bestimmen einer Version V des künstlichen neuronalen Netzes KNN, bei welcher die Abweichung A erkannt wurde,
• - Löschen aller Erfahrungsdaten des künstlichen neuronalen Netzes KNN ab der Version V, bei welcher die Abweichung A erkannt wurde,
• - Verwenden einer vorläufigen Version V des künstlichen neuronalen Netzes KNN beim Bereitstellen der Fahrzeugfunktion F, und/oder
• - erneutes Anlernen des künstlichen neuronalen Netzes KNN angefangen von der Version V des künstlichen neuronalen Netzes KNN, bei welcher die Abweichung A erkannt wurde.

Solution 4 can generally provide:

• - determining a version V of the artificial neural network ANN in which the deviation A was detected,
• - Deleting all experience data of the artificial neural network KNN from version V, in which deviation A was detected,
• - Using a preliminary version V of the artificial neural network KNN when providing the vehicle function F, and / or
• - Re-learning of the artificial neural network KNN, starting from version V of the artificial neural network KNN, in which deviation A was detected.

Like the functional data D, the experience data can also contain all inputs, such as. B. Sensor data, for the artificial neural network ANN as well as all outputs or decisions of the artificial neural network ANN include.

• - Benutzereingaben,
• - Meldungen von mindestens einem Assistenzsystem des Fahrzeuges 100, usw.

The following parameters can be taken into account in step 103:

• - user input,
• - Messages from at least one assistance system of vehicle 100, etc.

At least one further artificial neural network ANN can be used to carry out step 103.

• - Verkehrsdaten,
• - Navigationsdaten,
• - Umgebungsdaten,
• - Wetterdaten, usw.

In step 104 the following data can be taken into account:

• - traffic data,
• - navigation data,
• - environmental data,
• - Weather data, etc.

In this way, improved analysis can be provided.

In step 104, sensor data from the vehicle 100, a road user and/or an infrastructure element can be taken into account.

At least one external computer can be used to perform steps 104 and/or 105. The computer can be deployed in the cloud, in a workshop and/or on an artificial neural network training station.

At least one further artificial neural network ANN can be used to carry out steps 104 and/or 105.

• 107 Überprüfen, ob nach dem Anpassen des künstlichen neuronalen Netzes KNN, beim Durchführen der Fahrzeugfunktion F die Abweichung A auftritt,
• 108 Zurückkehren zu einem Verfahren nach einem der vorhergehenden Ansprüche, wenn beim Überprüfen die Abweichung A wieder erkannt wurde.

How it is 1 further suggests, the process can include at least one further step:

• 107 Check whether the deviation A occurs after adapting the artificial neural network KNN when carrying out the vehicle function F,
• 108 Returning to a method according to one of the preceding claims if deviation A was recognized again during checking.

Otherwise, the artificial neural network KNN can be switched off in step 109 and the vehicle function F can be ended.

A corresponding control device 110 for carrying out the method also represents an aspect of the invention. The control device 110 can be implemented, for example, in a control device 110 of the vehicle 100 and/or in a control device 110 on an external computer, for example in the cloud, in a workshop and/or at a training station for artificial neural networks.

A corresponding vehicle 100, in particular a partially automated, highly automated, fully automated or autonomous vehicle 100, having a corresponding control device 110 also represents an aspect of the invention.

The foregoing explanation of the embodiments describes the present invention solely in terms of examples. Of course, individual features of the embodiments can, if technically sensible, be freely combined with one another without departing from the scope of the present invention.

Reference symbol list

1

first procedure

2

second procedure

3

third procedure

4

fourth procedure

100

vehicle

110

Control device

A

Deviations

D

Functional data

F

Vehicle function

FS1

Driving situation

FS2

Driving situation

FSi

Driving situation

KNN

artificial neural network

Cl

artificial intelligence

S1

set

S2

set

v

version

V1

version

V2

version

Vi

version

Claims (11)

Method for controlling an artificial neural network (ANN) when providing a vehicle function (F), in particular when training the artificial neural network (ANN) and / or during normal operation of the artificial neural network (ANN), comprising: - Starting the artificial neural network (ANN), - Starting the vehicle function (F), - Monitoring the vehicle function (F) for deviations (A) when carrying out the vehicle function (F), - Analyzing functional data (D) when performing the vehicle function (F) if a deviation (A) was detected during monitoring, - Determine, depending on the analysis, whether a decision of the artificial neural network (ANN) is at least partially responsible for the deviation (A), - Starting at least one method for adapting the artificial neural network (ANN) if it was recognized during the determination that a decision by the artificial neural network (ANN) is at least partially responsible for the deviation (A). Procedure according to Claim 1 , wherein when monitoring the vehicle function (F) for deviations (A) user inputs are taken into account, and / or wherein when monitoring the vehicle function (F) for deviations (A) messages from at least one assistance system of the vehicle (100) are taken into account, and / or wherein at least one further artificial neural network (ANN) is used when monitoring the vehicle function (F) for deviations (A). Procedure according to Claim 1 or 2 , wherein when analyzing functional data (D) traffic data, navigation data, environmental data and / or weather data are taken into account, and / or wherein when analyzing functional data (D) sensor data of the vehicle (100), a road user and / or an infrastructure element are taken into account, and /or wherein at least one external computer is used when analyzing functional data (D) and/or when determining whether a decision of the artificial neural network (ANN) is at least partially responsible for the deviation (A), and/or wherein at Analyzing functional data (D) and/or when determining whether a decision of the artificial neural network (ANN) is at least partially responsible for the deviation (A), at least one further artificial neural network (ANN) is used. Method according to one of the preceding claims, wherein the at least one method for adapting the artificial neural network (ANN) to avoid the deviation (A) comprises at least a first method (1): - Providing training data for adapting and/or training an artificial neural network (ANN) for the vehicle function (F), in which the functional data (D) to which a deviation (A) was detected was deleted. Method according to one of the preceding claims, wherein the at least one method for adapting the artificial neural network (ANN) to avoid the deviation (A) comprises at least one second method (2): - Evaluate the deviation (A) in terms of customer comfort, safety, probability of occurrence and/or probability of detection, - Marking functional data (D) when performing the vehicle function (F), in which a deviation (A) was detected, with a key figure depending on the evaluation, and / or - Providing training data for adapting and/or training an artificial neural network (ANN) for the vehicle function (F), in which the functional data (D) was weighted with a key figure when performing the vehicle function (F). Method according to one of the preceding claims, wherein the at least one method for adapting the artificial neural network (ANN) to avoid the deviation (A) comprises at least a third method (3): - Providing a first set (S1) of training data for adapting and/or training an artificial neural network (ANN) for the vehicle function (F) from the functional data (D) when performing the vehicle function (F), in which a deviation ( A) was detected, - providing a second set (S2) of training data for adapting and/or training an artificial neural network (ANN) for the vehicle function (F) from functional data (D) when performing the vehicle function (F). in which no deviation (A) occurs, and/or - training the artificial neural network (ANN) to provide the vehicle function (F) using the first set (S1) of training data and using the second set (S2) of training data, in particular when Training the artificial neural network (ANN) to provide the vehicle function (F) using the first set (S1) of training data and using the second set (S2) of training data, the sets (S1, S2) of training data are weighted with different key figures. Method according to one of the preceding claims, wherein the at least one method for adapting the artificial neural network (ANN) to avoid the deviation (A) comprises at least a fourth method (4): - determining a version (V) of the artificial neural network (ANN) in which the deviation (A) was detected, - Deleting all experience data of the artificial neural network (ANN) from version (V) in which the deviation (A) was detected, - Using a preliminary version (V) of the artificial neural network (ANN) when providing the vehicle function (F), and / or - Re-learning of the artificial neural network (ANN) starting from version (V) of the artificial neural network (ANN) in which the deviation (A) was detected. Method according to one of the preceding claims, wherein the method comprises at least one further step: - Check whether the deviation (A) occurs after adapting the artificial neural network (ANN) when performing the vehicle function (F), - Returning to a method according to one of the preceding claims if the deviation (A) was recognized again during checking. Computer program product, comprising instructions which, when the computer program is executed by a computer, cause the computer to carry out a method according to one of the preceding claims. Control device (110) for controlling an artificial neural network (ANN) when providing a vehicle function (F), comprising: a storage device and a computing device, wherein a code is stored in the storage device, and wherein when the code is executed by the computing device, a method according to one of the preceding claims is carried out. Vehicle (100) comprising a control device (110) according to the preceding claim.

Images