DE102019105389A1 - Safety device for a self-learning measuring system in the vehicle for the detection and classification of objects in the area around the vehicle with the help of a deep learning process - Google Patents

Abstract

The invention relates to a safety-relevant measuring system for use in vehicles and / or motor vehicles, with a first safety device (142) and one or more neural network models (16, 136, 138) for evaluating the measurement data of at least one sensor arrangement (11, 131) of the measuring system and wherein the first safety device (142) monitors at least the neural network model (138) that processes this measurement data or data derived therefrom.

Description

Generic term

The invention relates to an ultrasonic measuring system for use in motor vehicles which, with the aid of a neural network model and a deep learning method based thereon, detects objects in the vicinity of the vehicle, determines their position and carries out a classification.

This invention thus relates to safety-relevant ultrasound systems for automotive purposes and in particular to ultrasound systems with deep learning networks (deep learning network models) which preferably provide real-time image identification and a system structure.

General introduction

Assistance systems to support the driver when parking, as well as systems for autonomous parking, are often based on ultrasonic measurement systems, which inexpensively and reliably detect obstacles in front of or behind the vehicle. The use of an artificial neural network model and a deep learning process allows the received signals of the ultrasonic measurement system to be evaluated beyond the previous determination of distances to obstacles and enables object classification.

Currently available automotive ultrasound systems enable vehicle drivers to control their vehicles with the help of ultrasound signals, especially when parking. In addition, the acquisition of images, the implementation of measurements and the application of algorithms integrated in the overall system as well as reporting software will be dealt with here. These functionalities should help to better understand the traffic situation and the dangers of the surroundings of a vehicle. The system presented here is intended to take over part of the driver's environmental analysis activity and to report the results of these analyzes to the driver or other subsystems of the vehicle. Before starting the journey, the driver of the vehicle must set up the vehicle and thus the system by selecting the settings and functions that are to be used and carried out during the journey. While driving, the vehicle driver can preferably carry out modifications. This usually starts with choosing which sensors to use. Depending on the weather, it can make sense to switch off certain sensors and to strengthen others in their importance. Possibly. can e.g. The vehicle manufacturer can store a different measurement protocol in a memory of the vehicle depending on the type of driver. Other operating settings of the ultrasonic measuring device can also be set. In the past, the basic configuration of prototypical ultrasonic sensor systems has taken a long time to develop because an automotive ultrasonic device typically has various control parameters. It is unacceptable to burden the driver with a reconfiguration of the ultrasonic measurement systems in his vehicle. It is therefore sensible if ultrasonic measuring devices in the motor vehicle store the parameters that have been optimized fully automatically or semi-automatically during driving operation in an operating parameter memory. By means of suitable presettings of the ultrasonic devices, it can then be ensured that they begin their learning process at a sensible starting point.

It is desirable to further advance such automation of setting up an ultrasound system for a particular traffic situation. One area of evolving artificial intelligence that holds great promise is “deep learning”. Deep learning is a rapidly evolving branch of machine learning algorithms that mimics how the human brain works when analyzing problems. The human brain remembers what was learned from solving a similar problem in the past and applied that knowledge to solve a new problem. Research is underway to identify potential applications of this technology in a number of areas such as pattern recognition, natural language processing, and computer vision. Deep learning algorithms have a distinct advantage over traditional forms of computer programming algorithms in that they can be generalized and trained to recognize image features by analyzing sample images rather than writing custom computer code. Another application is the recognition of structures or groups or tendencies in multidimensional data sets. However, the situation surrounding a vehicle visualized in an ultrasound system does not seem to be suitable for automated image recognition without further ado. Every situation is different and the shapes, sizes, positions and functions of the objects vary widely. This is especially true for pedestrians and / or cyclists. In addition, the quality and clarity of the ultrasound images vary even when the same ultrasound system is used. This is because the body habit of pedestrians, the wind, the air circulation, the humidity and the air temperature all influence the ultrasonic signals that are returned from the surroundings of the vehicle and that are used to generate the images. Scanning an object in the vicinity of the vehicle at a greater distance leads to strongly attenuated ultrasonic signals and a poorly defined geometry in the pictures. Accordingly, it is desirable to apply deep learning in an ultrasound system in a manner that is effective for a wide variety of environmental situations in order to expedite the creation of meaningful and ultrasonic environmental maps with properly classified abstracted objects through ultrasound image recognition.

State of the art

Ultrasonic measurement systems are currently used in vehicles as part of parking assistance systems. The precise localization of the obstacles is necessary, for example in the publication, in particular for vehicles that park or drive completely autonomously DE 10 2008 036 009 B4 described. Here, however, the obstacles are only located and not classified and assigned to object lists, such as other vehicles, hedges, curbs, etc. This is necessary in order to be able to derive further possible driving maneuvers. One way of classifying objects is by measuring the external dimensions and comparing them with the dimensions of reference objects in a database while taking into account the object speed, as in DE 103 36 638 A1 described. This method has the disadvantage that it is generally not possible to scan any volume of space with any precision. The reliable detection of objects that are rotated in their position relative to the vehicle by angles that are unknown to the database is difficult.

This disadvantage can be overcome by neural network models, as shown in [Dror 1995]. Here, a three-layer, forward-facing neural network model was used to reliably detect objects at different angles relative to the ultrasonic sensors under laboratory conditions.

Since the objects in the vicinity of vehicles have more complex geometries than the objects used in [Dror 1995] and the signal propagation conditions are worse than under laboratory conditions, a multi-layered neural network model is used here, on which a deep learning process based, suggested. The term “deep learning” does not yet have a uniform definition [Zhang 2018]. In this publication, the use of the term “deep learning” follows the definition given in [LeCun 2015]. According to this, deep learning methods are representation learning methods in which a system, e.g. a neural network model, raw data are given as input information and the system independently learns which representation obtained from the raw data is necessary for a classification. Deep learning processes use several layers of representation, which correspond to intermediate layers in a neural network model.

task

The proposal is therefore based on the task of developing a system that not only locates detected objects in the vicinity of the vehicle, but also carries out a classification and is able to reliably carry out this classification for different orientations of the objects relative to the vehicle .

Another object of the present invention is to use deep learning technology in order to more reliably recognize the geometry in the surroundings of a vehicle in ultrasound images and maps of the surroundings and / or ultrasound measurement data.

Another task is to recognize not only the geometry of the arrangement of objects in the vicinity of the vehicle, but also the view of this geometry in ultrasound images and environment maps and / or ultrasound measurement data.

It is a further task to use a deep learning recognition of this geometry anatomy in order to automate the optimization of the ultrasound system for an ultrasound measurement of the surroundings of a vehicle.

Solution of the task

In accordance with the principles of the present invention, an ultrasound system and a method are described which enable an ultrasound system to identify the geometry in an ultrasound map of the surroundings by processing using a neural network model. The neural network model is first trained by presenting several maps of the environment or ultrasonic measurement signals that can be traced back to a known geometric arrangement of objects. Once trained, live images recorded by a user are analyzed in real time by the neural network model, whereby the geometry of the arrangement is identified in the environment maps and / or ultrasonic measurement signals. The identification of the geometry and / or the objects and / or the object arrangements is used to comment on or improve the environment map image, which is typically already present, and / or the user control elements on the ultrasound system to carry out an in-depth analysis of the identified geometry of the objects in the environment of the Set up vehicle.

In a system of the type described at the outset, the object is proposed in that a deep learning method based on a neural network model uses the received signals of an ultrasonic measurement system in the vehicle as input to classify objects and determine the positions of the objects.

The proposal is first explained using the example of an ultrasonic measuring system, then transferred to a measuring system. In the following, the combination of such a measuring system with an ultrasonic measuring system and a self-learning method is described that includes a self-learning voter based on a neural network. This principle is then transferred to the combination of two measuring systems. A generalized self-learning network made up of neural networks is then represented as a node.

We first refer to 1 . In 1 An ultrasonic vehicle environment analysis system is shown schematically simplified in the form of a block diagram, which is constructed in accordance with the principles of the present invention. An ultrasonic transducer arrangement ( 1012 ) is in a vehicle part ( 1010 ), for example a bumper, for sending ultrasonic waves and receiving echo information. The ultrasonic transducer arrangement ( 1012 ) can be a one- or two-dimensional arrangement of ultrasonic transducer elements which are able to scan, for example, in two or preferably three dimensions, both in height (in 3D) and in azimuth. The ultrasonic transducer arrangement ( 1012 ) is preferred with a microbeamformer ( 1014 ) preferably already within the vehicle part ( 1010 ) coupled. The microbeamformer ( 1014 ) controls the transmission and reception of ultrasonic signals by the ultrasonic transducer elements of the ultrasonic transducer arrangement ( 1012 ). Microbeamformer ( 1014 ) within the meaning of this disclosure are at least partially able to at least partially align the sensitivity lobe of the signals received from groups or "spots" of ultrasonic transducer elements. The microbeamformer ( 1014 ) is equipped with a send / receive switch ( 1016 ) coupled, which switches between sending and receiving and the main beam shaper ( 1020 ) protects against high energy signals in the transmission phase. The emission of in this way under the control of the microbeamformer ( 1014 ) formed ultrasonic beams through the ultrasonic transducer array of the ultrasonic transducer arrangement ( 1012 ) is controlled by a transmission control ( 1018 ) controlled with the send / receive switch ( 1016 ) and the main jet shaper ( 1020 ) is coupled. The main jet shaper ( 1020 ) receives input from the user interface or the control panel ( 1038 ) of the user. To the transmission control ( 1018 ) controlled transmission characteristics include amplitude, phase and polarity of transmission waveforms. Beams formed in the direction of the impulse transmission can be triggered by a different phase control of the individual ultrasonic transducers from the ultrasonic transducer arrangement ( 1012 ) can be steered directly forwards or orthogonally to this or be emitted at different angles for a wider field of view.

The echoes that are received by an adjacent group of ultrasonic transducer elements are beam-formed by specifically delaying them in a suitable manner for each ultrasonic transducer element and then combining them. In this way, a reception lobe can be generated. The partially beam-shaped signals generated by the microbeamformer ( 1014 ) are generated by each ultrasonic transducer group, are generated with the main beamformer 1020 ) coupled. The main jet shaper ( 1020 ) combines the partially beam-shaped signals from individual ultrasonic transducer groups into a fully beam-shaped coherent echo signal. For example, the main jet shaper ( 1020 ) Have 128 channels, each of which receives a partially beam-shaped signal from a respective group of 12 ultrasonic transducer elements. In this way, the more than 1500 ultrasonic transducer elements of a two-dimensional ultrasonic transducer array ( 1012 ) efficiently contribute to a single beamform signal.

The coherent echo signals are subjected to signal processing by a signal processor ( 1026 ), which includes filtering through a digital filter and preferably noise reduction. The signal processor ( 1026 ) can shift the frequency band to a lower frequency range or baseband range. The processed echo signals are then preferably demodulated into quadrature components (I and Q) which provide signal phase information.

The beam-formed and processed coherent echo signals are sent to a video processor ( 1052 ), which generates a map of the surroundings of the vehicle. The video processor ( 1052 ) preferably performs amplitude (envelope) detection of the demodulated quadrature I and Q signal components by calculating the echo signal amplitude in the form of (I ² + Q ² ) ^1/2 . The quadrature echo signal components are also available with a Doppler processor ( 1054 ), which stores ensembles of echo signals from discrete points in an image field which are then used to estimate the Doppler shift at points in the image with a fast Fourier transform (FFT) processor. For a color Doppler image, the estimated Doppler flux values at each point are the Environment map filtered and preferably converted into color values using a look-up table or a heuristic formula. The image signals of the environment map and the Doppler flow values are sent to a scanning converter ( 1032 ) which converts the environment map and Doppler samples from their typically sensed R-θ coordinates to Cartesian (x, y) coordinates for display in a desired display format, e.g. In a straight line display format or a sector display format. Either the environment map image or the Doppler image alone can preferably be displayed in the user display. It is also advantageous to display both images together in one color Doppler image.

The data from the scan converter ( 1032 ) generated ultrasound images are with an image processor ( 1030 ) and a multiplanar reformer ( 1044 ) coupled. The multiplanar reformatter ( 1044 ) converts echoes received from points in a common plane in a volumetric area of space in the vicinity of the vehicle into an ultrasound image of this plane. A volume renderer ( 1042 ) converts the echo signals of a 3D data set into a projected 3D image as seen from a given reference point. The 2D or 3D images are processed with an image processor ( 1030 ) for further improvement, buffering and temporary storage for display on an image display ( 1040 ) coupled. A graphic display overlay that contains text and other graphic information, such as: B. the data of recognized objects, e.g. from geographical map databases, is preferred by a graphics processor ( 1036 ) for display with the ultrasound images.

In accordance with the principles of the present invention, the ultrasound system comprises a neural network model ( 1080 ) whose software is stored in digital memory. Deep learning neural network models include software that can be written by a software designer and that is also publicly available from a number of sources. An application that can be used to create a neural network model called NVidia Digits is available at https://developer.nvidia.com/digits. NVidia Digits is a high-level user interface that includes a deep learning framework called "Caffe," developed by the Berkley Vision and Learning Center
(http://caffe.berkeleyvision.org/) was developed. A list of popular deep learning frameworks suitable for use in an implementation of the present invention can be found at https://developer.nvidia.com/deep-learning-frameworks. With the neural network model ( 1080 ) is a training image memory ( 1034 ), in which ultrasound images of known anatomy of a patient are stored and used to create the neural network model ( 1080 ) to identify ultrasound images of the geometry of the vehicle environment. Live images from the ultrasound system of 1 are generated with known vehicle surroundings geometries, the neural network model ( 1080 ) after training to identify the geometry in the images using the neural network model ( 1080 ) presents. In a designed implementation of the present invention, the neural network model ( 1080 ) trained to identify not only the respective vehicle surroundings geometry in an ultrasound image, but also the view of the vehicle surroundings geometry that is seen in the ultrasound image, e.g. B. Another car or a curb or a hydrant. The trained neural network model generates an object recognition result for the identified objects in the vehicle environment and also prefers a confidence factor of what the neural network model ( 1080 ) than the accuracy of its identification (e.g. 80%, 100% or some other confidence factor). If the neural network model ( 1080 ) is 100% convinced of its identification, the image ID is controlled with a user control setting control ( 1048 ) of the ultrasound system, where settings of the system for an analysis of the identified geometry type can be specified. These settings are preferred to a touchscreen display control ( 1046 ) where the graphics of a touchscreen user control display are modified accordingly and processed with the graphics processor ( 1036 ) can be coupled. The graphics processor ( 1036 ) generates the corresponding control graphics, which are shown on a touchscreen display ( 1060 ) on the user control panel ( 1038 ) of the ultrasound system is applied.

One method of training and using a neural network model to identify anatomy in ultrasound images is disclosed in US Pat 2 shown. In step A ( 1090 ) a neural network model is installed on a computer-based system. The neural network model can be one designed by a software architect or it can be one that is created using one of the deep learning frameworks described above. In step B ( 1092 ), images of the same vehicle environment situation geometry, which were preferably obtained from a large number of test drives for similar situations, are presented to the neural network model. In order to train it to develop the geometry, this presentation takes place together with the known type of situation geometry and its view in the images in step C ( 1094 ). The number of training images or training data sets used is preferably hundreds or thousands in order to provide the neural network model in the variations of a to train such vehicle environment situation geometry. If the trained neural network model is then installed in an ultrasound system in a vehicle, in step D ( 1096 ) the environment maps recorded by the ultrasound system are presented to the neural network model for identification. In step E ( 1098 ) the identification generated by the neural network model is used to annotate the images, e.g. B. as an obstacle or curb. The information is also used to set up the ultrasound system and train it in a self-learning manner.

Variations on the system and method described above will readily occur to those skilled in the art. An ultrasound system can use deep learning models to assess the image quality and provide feedback to the driver so that the driver can receive better-resolution images if possible. Support for decisions by the vehicle driver can be implemented by combining information on the previous history with the identification of the geometry of the vehicle's surroundings. This data can be processed by deep learning algorithms to provide decision support outcomes.

It should be noted here that an ultrasound system suitable for use in an implementation of the present invention, and in particular the component structure of the ultrasound system of FIG 1, e.g. . B. can be implemented in hardware, software, or a combination thereof. The various embodiments and / or components of an ultrasound system, for example the deep learning software modules or components and controls therein, can also be implemented as part of one or more computers or microprocessors. The computer or processor can for example comprise a computing device, an input device, a display unit and an interface for accessing the Internet, as in FIG 1 shown. The computer or processor can include a microprocessor. The microprocessor can, for example, be connected to a communication bus in order to access a data network of the vehicle and / or a data network external to the vehicle in order to import training images and / or training data sets. The computer or processor can also include memory. The storage devices such as the image memory ( 1028 ) may include random access memory (RAM) and read only memory (ROM). The computer or processor can also include a storage device, which can be a hard disk drive or removable media such as a floppy disk drive, optical drive, solid state drive, and the like. The storage device can also be some other similar device for loading computer programs or other instructions into the computer or processor.

As used herein, the term “computer” or “module” or “processor” or “workstation” can include any processor-based or microprocessor-based system, including systems using microcontrollers, reduced instruction set computers (RISC), ASICs. Logic circuits and any other circuit or processor that can perform the functions described herein. The above examples are only exemplary and are not intended to limit the definition and / or meaning of these terms in any way.

The computer or processor executes a set of instructions stored in one or more storage elements to process input data. The storage elements can also store data or other information as desired or required. The storage element may be in the form of a source of information or a physical storage element in a processing machine.

The instruction set of an ultrasound system, including that which controls the acquisition, processing and transmission of ultrasound images as described above, may contain various instructions that direct a computer or processor as the processing machine, certain operations such as the methods and methods to be performed of the various embodiments of the invention . The set of instructions can be in the form of a software program. The software can be in various forms, such as system software or application software, and it can be implemented as a tangible and non-transitory computer readable medium. A complete or partial implementation as a finite automaton is also possible. Furthermore, the software can be in the form of a collection of separate programs or modules, such as e.g. B. a neural network model module, a program module within a larger program or part of a program module. The software can also include modular programming in the form of object-oriented programming. The processing of input data by the processing machine may be in response to operator commands or in response to results of previous processing or in response to a request made by another processing machine.

Ultrasonic measuring system with deep learning

A proposed ultrasonic measuring system ( 164 ) for use in motor vehicles to detect obstacles and / or objects in the The environment of a vehicle using deep learning methods preferably includes an ultrasonic transducer arrangement ( 11 ). This is set up to record the first ultrasonic measured values. This detection takes place by sending out ultrasonic signals that are sent out as burst, pulse or coded. The ultrasonic transducer arrangement preferably consists of several ultrasonic transmitters and ultrasonic receivers. Ultrasonic transducers which can work as ultrasonic transmitters and ultrasonic receivers can also be considered as components of the ultrasonic transducer arrangement. The ultrasonic transducer arrangement ( 11 ) generates the first ultrasonic measured values depending on the received ultrasonic signals. These are preferably ultrasonic echo signals of the ultrasonic signals transmitted by the ultrasonic arrangement. An image processor is preferred to generate a user-friendly representation ( 18th ) is used, which is connected to the ultrasonic transducer arrangement ( 11 ) is coupled and the two- or three-dimensional representations depending on the first ultrasonic measured values of the ultrasonic transducer arrangement ( 11 ) can generate. The proposed ultrasonic measuring system ( 164 ) furthermore preferably comprises a first memory ( 16 ) with a first neural network model ( 16 ), which is in the first memory ( 16 ) is saved. The first neural network model ( 16 ) can be implemented in hardware or as software on a processor. The first neural network model is preferred ( 16 ) adapted and provided so that the ultrasonic measured values from the ultrasonic transducer arrangement ( 11 ) if necessary after passing through a suitable processing by amplifiers, filters etc. and identify the objects in the area around the vehicle and / or their geometric arrangement in the area around the vehicle in the first ultrasonic measured values by means of a deep learning technique by assigning an identification and can thus generate a first object recognition result. Furthermore, the proposed device comprises a two- or three-dimensional display ( 19th ) used to display the two- or three-dimensional representations generated by the image processor ( 18th ) were generated, trained or suitable or intended. The two- or three-dimensional display ( 19th ) is preferably suitable or designed or provided for at least temporarily displaying one of the identifications and / or first object recognition results and / or information derived therefrom. It is therefore an ultrasonic measuring system ( 164 ) for use in motor vehicles to recognize obstacles and / or objects in the vicinity of a vehicle using deep learning methods with an ultrasonic transducer arrangement ( 11 ), which is set up to record the first ultrasound measurement values and an image processor ( 18th ) which is connected to the ultrasonic transducer arrangement ( 11 ) is coupled and the two- or three-dimensional representations depending on the first ultrasonic measured values of the ultrasonic transducer arrangement ( 11 ), as well as a first memory ( 16 ) includes a first neural network model ( 16 ), which is in the first memory ( 16 ) is stored and is adapted to receive the ultrasonic measured values and to identify the objects and their geometric arrangement in the surroundings of the vehicle in the first ultrasonic measured values by means of a deep learning technique by assigning an identification and thus to generate a first object recognition result, and with a two- or three-dimensional display ( 19th ) used to display the two- or three-dimensional representations generated by the image processor ( 18th ) was generated, designed or suitable or provided, said two- or three-dimensional display ( 19th ) is suitable or designed or provided for at least temporarily displaying one of the identifications and / or first object recognition results and / or information derived therefrom.

In a first refinement of this ultrasonic measuring system ( 164 ) the first neural network model is also designed to identify the view of the arrangement of objects and / or object components in the first ultrasonic measurement values and / or in the two- or three-dimensional representations. The first neural network model thus has the function of an image recognition system. In the basic version, which was described above, the detection takes place alternatively on the basis of the raw sensor data. In such a device, it is conceivable, on the one hand, to recognize arrangements of objects and / or objects in the vicinity of the vehicle on the basis of the raw sensor data, i.e. in this case the ultrasound measurement values, with the aid of a first neural network model or on the basis of an environment map extracted thereon Carry out the aid of a second neural network model and then use a third neural network model to differentiate between the output data streams of the first neural network model and the second neural network model. First parts of the output data streams of the first and second neural network model preferably have a first weighting and second parts of the output data streams of the first and second neural network model have a second weighting. The first weighting is preferably higher than the second weighting. The purpose of this is to ensure that safety-relevant, potentially critical constellations that were detected by a neural network model of the first and second neural network model prevail in any case. The same applies to signals in the raw sensor data, which also indicate a high probability of a critical event.

The ultrasonic measuring system described above ( 164 ) preferably further comprises a plurality of first user controls ( 10 ), whereby setting one or more of the first user controls ( 10 ) takes place as a reaction to the identification of an arrangement of objects and / or object components or of objects in the first ultrasonic measurement values and / or in the two- or three-dimensional representations and / or as a function of the first object recognition results.

With the ultrasonic measuring system described above ( 164 ) preferably the setting or the adjustment of control parameters of one or more of the controls takes place in response to the identification of the arrangement of objects and / or object components in the first ultrasonic measured values and / or in the two- or three-dimensional representations and / or as a function of the first Object recognition results. One or more of the controls are preferably set here ( 13th ) of the ultrasonic measuring system or the vehicle as a reaction to the identification of objects and / or object components and / or the arrangement of objects and / or objects and / or object components in the first ultrasound measurement values and / or in the two- or three-dimensional representations and / or in Dependence on the first object recognition results.

The ultrasonic measurement system described in this section also includes ( 164 ) prefers a user control display ( 9 ), which displays settings made in response to the identification of objects and / or object components and / or the arrangement of objects and / or object components in the first ultrasound measurement values and / or in the two- or three-dimensional representations and / or as a function of the first Object recognition result can be made.

With the ultrasonic measuring system described here ( 164 ) preferably activates the user control display ( 9 ) one control options in response to the identification of objects and / or object components and / or the arrangement of objects and / or object components in the first ultrasound measurement values and / or in the two- or three-dimensional representations and / or in the first object recognition results and displays them .

The ultrasonic measuring system described in this section ( 164 ) includes for the purpose of training the first neural network model ( 16 ) also prefers a first training image memory ( 17th ), the training images for the first neural network model ( 16 ) saves.

With the ultrasonic measuring system described in this section ( 164 ) the display is ( 19th ) preferably also designed to display the two- or three-dimensional representations, which at least at times also contain the information provided by the first neural network model ( 16 ) certain identification and / or a part of the first object recognition result are provided.

Procedure for operating an ultrasonic measuring system with deep learning

1. a) das Installieren (21) eines ersten neuronalen Netzwerkmodells (16) auf einem computerbasierten System. Hierbei kann es sich beispielsweise um ein räumlich separiertes Rechnersystem oder ein Mikrorechnersystem handeln, das beispielsweise mit Teilen der Ultraschallsensoranordung (11) eine Einheit bildet und
2. b) das Trainieren (22) des ersten neuronalen Netzwerkmodells (16), um die Objekte und/oder Objektkomponenten und/oder Anordnungen von Objekten und/oder Objektkomponenten in den ersten Ultraschallmesswerten und/oder in den zwei- oder dreidimensionalen Darstellungen, die von ersten Ultraschallmesswerten abhängen, zu identifizieren und ein erstes Objekterkennungsergebnis zu erzeugen. Hierbei werden dem ersten neuronalen Netzwerkmodell vorbekannte Ultraschallmesswerte und/oder in den zwei- oder dreidimensionalen Darstellungen, die von ersten Ultraschallmesswerten abhängen, als Eingangssignale des ersten neuronalen Netzwerkmodells (16) präsentiert. Die vom ersten neuronalen Netzwerkmodell (16) erzeugten ersten Ausgangssignale werden beispielsweise durch eine erste Parametermodifikationsvorrichtung (140) für das erste neuronale Netzwerkmodell (16) bewertet. Die erste Parametermodifikationsvorrichtung (140) führt somit eine Bewertung der ersten Ausgangssignale des ersten Netzwerkmodells (16) durch. Je nachdem, ob die Bewertung der ersten Ausgangssignale des ersten Netzwerkmodells (16) durch die erste Parametermodifikationsvorrichtung (140) als positiv oder negativ bewertet wird, führt die erste Parametermodifikationsvorrichtung (140) eine Modifikation der Parameter des ersten neuronalen Netzwerksmodells (16) so durch, dass die eine vorausgegangene Veränderung der Parameter des ersten neuronalen Netzwerkmodells (16) mit vorzugsweise gleichem Vorzeichen erneut durchgeführt wird, wenn der aktuelle Vergleich mit einem Zielwert, insbesondere einem Zielwert für das Bewertungsergebnis der Bewertung durch die erste Parametermodifikationsvorrichtung (140), ein positives Ergebnis liefert, was einer Verbesserung des Erkennungsergebnisses entspricht, und dass die eine vorausgegangene Veränderung der Parameter des ersten neuronalen Netzwerkmodells (16) mit vorzugsweise entgegengesetztem Vorzeichen erneut durchgeführt wird, wenn der aktuelle Vergleich mit einem Zielwert, insbesondere einem Zielwert für das Bewertungsergebnis der Bewertung durch die erste Parametermodifikationsvorrichtung (140), ein negatives Ergebnis liefert, was einer Verschlechterung des Erkennungsergebnisses entspricht. Der jeweilige Vergleich wird bevorzugt auch durch die Parametermodifikationsvorrichtung (140) vorgenommen. Solche Lernalgorithmen können auch abgewandelt werden in der Form, dass nicht nur die unmittelbar vorausgehende Parameteränderung zur Bestimmung der nächsten Änderung im Trainingsprozess herangezogen wird, sondern auch weitere noch früher vorausgehende Änderungen. Auch können die vorausgehenden Änderungen selbst als weitere Eingabeparameter der Parametermodifikationsvorrichtung (140) verwendet werden. Des Weiteren kann der Lernprozess je nach Vorgeschichte und auf die einzelnen Parameter des ersten neuronalen Netzwerkmodells (16) sowie auf einzelne Ausgangssignale des ersten neuronalen Netzwerksmodells (16) bezogen unterschiedlich stark ausgeführt werden. Auch ist eine lineare und/oder nichtlineare Verknüpfung zwischen den Ausgangssignalen des ersten neuronalen Netzwerkmodells (16) und ggf. weiteren Eingangssignalen der Parametermodifikationsvorrichtung (140) auf der einen Seite inkl. der ggf. vorausgehenden Werte dieser Signale, die ggf. zwischengespeichert werden, und dem Bewertungsergebnis und der daraus resultierenden Veränderung der Parameter des ersten neuronalen Netzwerkmodells (16) denkbar und in Abhängigkeit von der jeweiligen Anwendung sinnvoll. Auch kann das Bewertungsergebnis ein Vektor-Signal mehrerer Einzelbewertungsergebnisse sein, was eine Verbesserung des Lernprozesses erlaubt. Beispielsweise kann so sichergestellt werden, dass sicherheitsrelevante Abweichungen von Ausgangssignalen des ersten neuronalen Netzwerkmodells (16) eine wesentlich stärkere Anpassung der Parameter des ersten neuronalen Netzwerkmodells (16) zur Folge haben, als nicht sicherheitsrelevante Abweichungen von Ausgangssignalen des ersten neuronalen Netzwerkmodells (16). Das erste neuronale Netzwerkmodell (16) ist dann fertig trainiert, wenn alle vorgegebenen Eingangstrainingsdatensätze eine positive Bewertung, insbesondere durch die Parametermodifikationsvorrichtung (140), zur Folge haben. D.h. deren Antwort liegt dann innerhalb eines vorgegebenen spezifizierten Bereiches;
3. c) das Verwenden (25) des trainierten, ersten neuronalen Netzwerkmodells (16) zur Identifizierung von Objekten und/oder Objektkomponenten und/oder von Anordnungen von Objekten und/oder Objektkomponenten in den ersten Live-Ultraschallmesswerten und/oder in den zwei- oder dreidimensionalen Live-Darstellungen, die von den ersten Live-Ultraschallmesswerten abhängen, wobei ein erstes Objekterkennungsergebnis erzeugt wird. Hierbei wird die Ultraschallmessvorrichtung, deren Teil das erste neuronale Netzwerkmodell (116) ist, bestimmungsgemäß zum Identifizieren von Objekten und/oder Objektkomponenten und/oder von Anordnungen von Objekten und/oder Objektkomponenten im Umfeld des beispielhaften Fahrzeugs eingesetzt. Die Ultraschallsensoranordnung (11) sendet dabei Ultraschallsignale aus und empfängt die Echos dieser Ultraschallsignale als Ultraschallmesssignale. Da diese im eigentlichen Betrieb empfangen werden, werden sie hier als Live- Ultraschallmesssignale bezeichnet. Die zugehörigen Messwerte werden hier als Live-Ultraschallmesswerte bezeichnet. Auf Basis dieser Live-Ultraschallmesssignale oder von daraus abgeleiteten Signalen und/oder Informationen führt das erste neuronale Netzwerkmodell (16) die besagte Identifikation von Objekten und/oder Objektkomponenten und/oder von Anordnungen von Objekten und/oder Objektkomponenten im Umfeld des Fahrzeugs dann durch.

As part of the drafting of the proposal, a method for recognizing objects and / or object components and / or arrangements of objects and / or object components in first ultrasonic measured values and / or in the two- or three-dimensional representations that depend on first ultrasonic measured values and / or worked out to determine a first object recognition result on the basis of first ultrasonic measured values. It includes

1. a) installing ( 21st ) of a first neural network model ( 16 ) on a computer-based system. This can be, for example, a spatially separated computer system or a microcomputer system that, for example, with parts of the ultrasonic sensor arrangement ( 11 ) forms a unit and
2. b) training ( 22nd ) of the first neural network model ( 16 ) in order to identify the objects and / or object components and / or arrangements of objects and / or object components in the first ultrasound measurement values and / or in the two- or three-dimensional representations that depend on first ultrasound measurement values and to generate a first object recognition result. In this case, ultrasound measurement values known to the first neural network model and / or in the two- or three-dimensional representations that depend on first ultrasound measurement values are used as input signals of the first neural network model ( 16 ) presents. Those from the first neural network model ( 16 ) generated first output signals are for example by a first parameter modification device ( 140 ) for the first neural network model ( 16 ) rated. The first parameter modification device ( 140 ) thus leads to an evaluation of the first output signals of the first network model ( 16 ) by. Depending on whether the evaluation of the first output signals of the first network model ( 16 ) by the first parameter modification device ( 140 ) is evaluated as positive or negative, the first parameter modification device ( 140 ) a modification of the parameters of the first neural Network model ( 16 ) so that the previous change in the parameters of the first neural network model ( 16 ) is carried out again with preferably the same sign if the current comparison with a target value, in particular a target value for the evaluation result of the evaluation by the first parameter modification device ( 140 ), supplies a positive result, which corresponds to an improvement in the recognition result, and that the previous change in the parameters of the first neural network model ( 16 ) is carried out again with preferably the opposite sign if the current comparison with a target value, in particular a target value for the evaluation result of the evaluation by the first parameter modification device ( 140 ), delivers a negative result, which corresponds to a deterioration in the recognition result. The respective comparison is preferably also made by the parameter modification device ( 140 ) performed. Such learning algorithms can also be modified in such a way that not only the immediately preceding parameter change is used to determine the next change in the training process, but also other changes that precede even earlier. The preceding changes themselves can also be used as further input parameters of the parameter modification device ( 140 ) be used. Furthermore, the learning process can be based on the previous history and on the individual parameters of the first neural network model ( 16 ) as well as individual output signals of the first neural network model ( 16 ) based on different strengths. A linear and / or non-linear link between the output signals of the first neural network model ( 16 ) and possibly further input signals of the parameter modification device ( 140 ) on the one hand, including the possibly preceding values of these signals, which may be temporarily stored, and the evaluation result and the resulting change in the parameters of the first neural network model ( 16 ) conceivable and useful depending on the respective application. The evaluation result can also be a vector signal of several individual evaluation results, which allows the learning process to be improved. For example, it can be ensured in this way that safety-relevant deviations from the output signals of the first neural network model ( 16 ) a much stronger adjustment of the parameters of the first neural network model ( 16 ) as non-safety-relevant deviations from the output signals of the first neural network model ( 16 ). The first neural network model ( 16 ) The training is finished when all the specified input training data sets have been evaluated positively, in particular by the parameter modification device ( 140 ), have as a consequence. In other words, their answer is then within a given specified range;
3. c) using ( 25 ) of the trained, first neural network model ( 16 ) to identify objects and / or object components and / or arrangements of objects and / or object components in the first live ultrasound measurement values and / or in the two- or three-dimensional live representations that depend on the first live ultrasound measurement values, with a first object recognition result is generated. The ultrasonic measuring device, part of which is the first neural network model ( 116 ) is used as intended for identifying objects and / or object components and / or arrangements of objects and / or object components in the vicinity of the exemplary vehicle. The ultrasonic sensor array ( 11 ) sends out ultrasonic signals and receives the echoes of these ultrasonic signals as ultrasonic measurement signals. Since these are received during actual operation, they are referred to here as live ultrasonic measurement signals. The associated measured values are referred to here as live ultrasonic measured values. On the basis of these live ultrasonic measurement signals or signals and / or information derived from them, the first neural network model ( 16 ) the said identification of objects and / or object components and / or of arrangements of objects and / or object components in the vicinity of the vehicle.

The training expediently includes ( 22nd ) of the first neural network model ( 16 ) also a training ( 22nd ) with previously recorded ultrasound live training values (data from live ultrasound measured values) of objects and / or object components and / or of arrangements of objects and / or object components in the vicinity of the vehicle and / or in the vicinity of a prototypical vehicle and / or from a Laboratory scenery of such objects and / or object components and / or such arrangements of objects and / or object components. Of course, the calculation of such recorded ultrasound live training values is also conceivable, for example to provide a good starting point for training the first neural network model ( 16 ) to achieve.

The workout ( 22nd ) of the first neural network model ( 16 ) can also be a training ( 22nd ) with two- or three-dimensional Include training plots. In this case, the ultrasonic measurement signals of the ultrasonic transducer arrangement ( 11 ), but typically derived, preferably pictorial, two- or three-dimensional representations.

The workout ( 22nd ) of the first neural network model ( 16 ) a training ( 22nd ) with training images of a common arrangement of a plurality of subjects.

These exemplary three evaluation methods (training ( 22nd ) with ultrasonic measurement signals and, if necessary, further measurement signals, training ( 22nd ) with two- or three-dimensional training displays, training ( 22nd ) with training images) can, as already described above, be mixed with one another and with other training data (eg simulated user inputs) as training data.

The workout ( 22nd ) preferably includes the identification of objects and / or object components and / or arrangements of objects and / or object components in the two- or three-dimensional training representations. This then generally enables objects and / or object components and / or arrangements of objects and / or object components in the vicinity of the vehicle to be identified during later operation.

The same applies to alternative or supplementary training ( 22nd ), which enables the identification of objects and / or object components and / or arrangements of objects and / or object components on the basis of the first ultrasonic measured values of the ultrasonic transducer arrangement ( 11 ) includes. This, too, usually enables objects and / or object components and / or arrangements of objects and / or object components in the vehicle's surroundings to be identified during later operation. The workout ( 22nd ) thus includes the determination of a first object recognition result by the first neural network model ( 16 ). This can be done on the basis of the two- or three-dimensional training representations and / or on the basis of the first ultrasonic measured values of the ultrasonic transducer arrangement ( 11 ) and possibly on the basis of additional, further measured values. Instead or in parallel, this can also be done on the basis of derived measured values and / or signals and / or information obtained from the two- or three-dimensional training representations and / or from the first ultrasonic measured values of the ultrasonic transducer arrangement ( 11 ) and possibly derived from the additional, further measured values. The two- or three-dimensional training representations within the meaning of this disclosure can actually be displayed on a display ( 19th ) or are only available as a virtual data record, for example in pixel form and / or vectorial and / or in compressed, for example object-oriented form. An object-oriented form typically includes a data record for each object including, for example, a date that specifies an object class and, for example, a set of coordinates, for example for position and possibly orientation in space and possibly speed and possibly speed direction, etc. Data record, for example, specify the size of the object and / or deformations of the object compared to a reference object model.

In this sense, the first object recognition result is a very general result of the first neural network model ( 16 ), which in particular is typically the essential input signal of the preferably present first parameter modification device ( 140 ), which is preferred especially during exercise ( 22nd ) the parameters of the first neural network model ( 16 ) adjusts depending on the result of the assessment and thus the training ( 22nd ) performs. For this purpose, the first parameter modification device ( 140 ) transmitted reference values matching the input data presented to the first neural network model. The parameter modification device ( 140 ) compares the respective reference value to a set of input data for the first neural network model ( 16 ) with the first object recognition result of the first neural network model ( 16 ) and generates a modification of the parameters of the first neural network model based on the comparison result ( 16 ). The workout ( 22nd ) can be carried out in the target system. In practice, however, it also makes sense to do the training ( 22nd ) by means of another device which is structurally identical or sufficiently similar in essential parts. Such another device can be, for example, a laboratory setup of the ultrasonic measuring device or parts thereof that is not suitable for series production, which fulfills the intended purpose of performing the training 22nd ) and the determination of the parameter data for the first neural network model ( 16 ) is fulfilled and / or intended. The through training ( 22nd ) determined parameters of the first neural network model ( 16 ) after completion of the training ( 22nd ) then stored in a memory or on another storage medium and at a given time, for example shortly before the end of production of another target ultrasonic measuring device of the same or similar design, then in its first neural network model 16 ) of this target ultrasonic measuring device. It is also conceivable to use the determined parameters of the first neural network model ( 16 ) to take into account when designing such a target ultrasonic measuring device and to design the construction of the target ultrasonic measuring device in such a way that the determined parameters, for example, by hard wiring of circuit parts of an exemplary target Ultrasonic measuring device are fixed. Not all parameters have to be either programmable or adjustable on the one hand, or fixed on the other hand. Rather, for example, some parameters can be hard-wired and other parameters can be designed to be programmable. This has the advantage that programmability and / or adjustability that is more complex in terms of circuitry can be limited to the absolutely necessary parts.

Said training ( 22nd ) can in the proposal presented here, in the case of programmable or adjustable parameters, possibly also during operation (use ( 25 )) under certain conditions, which will be discussed later, which is why the device described here is then self-learning. In this respect, the training ( 22nd ) in self-learning mode also with the use already described above ( 25 ) alternate. Preferably, but not necessarily, a method according to the invention therefore has phases of training ( 22nd ) and phases of use ( 25 ) which, for example, can alternate if necessary. Before using for the first time ( 25 ) but typically a training ( 22nd ) or preferably at least one programming with or a setting according to previously determined parameter sets of the first neural network model ( 16 ), which were determined by means of an identical or sufficiently similar ultrasonic measuring device or a sufficiently matching simulation device. The recording of such ultrasonic measured values and the necessary training data with test vehicles in real traffic in the target markets is particularly preferred. The necessary parameter sets for the first neural network model ( 16 ) calculated or otherwise determined with which the initial programming or initial setting of the target ultrasound systems is carried out or from which the design data for the hard-wired parts of the first neural network ( 16 ) be determined.

The first neural network model ( 16 ) the objects and / or object components and / or arrangements of objects and / or object components and / or the first object recognition results of the first neural network model ( 16 ) is used to create the two- or three-dimensional representations generated by ultrasonic readings from the ultrasonic sensor array ( 11 ) and possibly further measured values of further sensors or signals or information derived therefrom, to be commented on in the display on a screen, for example in a displayed map of the surroundings of a vehicle for use by the driver. In particular, it is desirable that particularly dangerous and / or otherwise relevant objects are shown on a display ( 19th ) for the driver.

Thus, in one variant, the proposed method also includes the use of the identification of the objects and / or object components and / or arrangements of objects and / or object components and / or the first object recognition results in such a map of the surroundings of the vehicle and / or for informing the driver by means of a two- or three-dimensional representation and / or for informing the driver by a control system and / or control device for controlling and / or regulating autonomous driving and / or autonomous parking.

Second measurement system with deep learning

The ultrasonic measuring system described in Section A) above ( 164 ) can be connected to a second measuring system ( 161 ) for use in motor vehicles for the detection of obstacles and / or objects and / or object components and / or for the detection of arrangements of obstacles and / or objects and / or object components in the vicinity of a vehicle using deep learning methods. The characteristic structure of the second measuring system described below ( 161 ) agrees with the ultrasonic measuring system described in section A) ( 164 ) match. Instead of an ultrasonic sensor arrangement ( 11 ), for example of transducers, a sensor arrangement of further sensors ( 131 ) used. This can be a single sensor or a plurality of sensors. For example, it can be a LIDAR system or a RADAR. Instead of the first neural network model ( 16 ) includes the second measuring system ( 161 ) a second neural network model ( 136 ). The structure of the second measuring system ( 161 ) is analogous to the ultrasonic measuring system described in Section A) ( 164 ). For the sake of completeness, the structure of this second measuring system ( 161 ) described again below. If the difference between the "ultrasonic" measuring principle of the ultrasonic measuring system described in section A) ( 164 ) and the measuring principle of the second measuring system described below ( 161 ) does not necessarily require a deviation in the structural component described in each case, the same applies in Section A) via the ultrasonic measuring system ( 164 ) Written notes, at least in the figurative sense, for this second measuring system ( 161 ).

The proposed second measuring system modified in this way ( 161 ) then comprises said second sensor arrangement ( 131 ), which is set up to record second sensor readings, as well as an image processor ( 18th ) that includes the second sensor arrangement ( 131 ) is coupled and the two- or three-dimensional representations depending on the second sensor measured values of the second sensor arrangement ( 131 ) can generate a second memory ( 136 ) with a second neural network model ( 136 ) which is stored in the second memory ( 136 ) is stored and adapted to receive the second sensor measured values and to identify the objects and their geometric arrangement in the surroundings of the vehicle in the second sensor measured values by means of a deep learning technique by assigning a second identification and thus to generate a second object recognition result, and a two- or three-dimensional display ( 19th ) used to display the two- or three-dimensional representations generated by the image processor ( 18th ) were generated, trained or suitable or intended. The two- or three-dimensional display ( 19th ) preferably suitable and / or designed and / or provided to display at least one of the second identifications and / or second object recognition results and / or information and / or signals derived therefrom.

The second neural network model ( 136 ) is also preferably designed to display the arrangement of objects and / or object components in the second measured values of the second sensor arrangement ( 131 ) and / or to be identified in the two- or three-dimensional representations. Since in many cases it is not the nature of the objects that is a problem, but their arrangement to one another or the change in this arrangement over time, this is an advantage. As a rule, this should therefore also be recorded by the methods presented here.

The proposed second measuring system ( 161 ) preferably further comprises a multiplicity of second user controls ( 130 ), whereby setting one or more of the second user controls ( 130 ) takes place as a reaction to the identification of an arrangement of objects and / or object components in the second sensor measurement values and / or in the two- or three-dimensional representations and / or as a function of the second object recognition results. For example, the vehicle driver can mark objects that he considers particularly dangerous (for example children playing or objects that are easy to overlook such as posts, etc.) in his representation. The measuring system (or the ultrasonic measuring system ( 164 ) In the case of device A), first of all, the movement of these objects identified as critical relative to the vehicle, for example during a parking maneuver, can be tracked. Second, this user input can be used as a training signal for the first neural network model ( 16 ) in the case of an ultrasonic measuring system ( 164 ) or for the second neural network model ( 136 ) in the case of a second measuring system ( 161 ) be used.

With the proposed second measuring system ( 161 ) the setting or adjustment of control parameters of a control of the controls or several controls of the controls of the vehicle or of vehicle components, for example lighting and / or a warning signal generator, is preferably carried out in response to the identification of the arrangement of objects and / or object components in the second Sensor measured values of the second sensor arrangement ( 131 ) and / or in the two- or three-dimensional representations and / or depending on the second object recognition results of the second neural network model ( 136 ).

With the proposed second measuring system ( 161 ) one or more of the controls of the second measuring system are preferably set ( 161 ) or the vehicle or vehicle components as a reaction to the identification of objects and / or object components and / or the arrangement of objects and / or object components in the second sensor measured values of the second sensor arrangement ( 131 ) and / or in the two- or three-dimensional representations and / or depending on the second object recognition results of the second neural network model ( 136 ).

The proposed second measuring system ( 161 ) further preferably includes a user control display ( 9 ), which displays settings that are made in response to the identification of objects and / or object components and / or the arrangement of objects and / or object components in the second sensor measured values of the second sensor arrangement ( 131 ) and / or in the two- or three-dimensional representations and / or as a function of the second object recognition result of the second neural network model ( 136 ) be made.

The user control display ( 9 ) further activated in response to the identification of objects and / or object components and / or the arrangement of objects and / or object components in the second sensor measured values of the second sensor arrangement ( 131 ) and / or in the two- or three-dimensional representations and / or depending on the first object recognition results of the second neural network model ( 136 ) Control options for the measuring device and / or the ultrasonic measuring device and / or the vehicle and / or vehicle components and parts. The user control display ( 9 ) preferably also or alternatively shows these control options.

The proposed second measuring system ( 161 ) furthermore preferably comprises a second training image memory ( 137 ), the training images for the second neural network model ( 136 ) saves. These training images can, for example, as already described above, be present in the form of pixel data or as vector data or as compressed object-oriented data.

The preferred display is ( 19th ) in the case of the second measuring system ( 161 ) is also designed to display the two- or three-dimensional representations, which are at least temporarily additionally linked to the second neural ( 136 ) Network model-specific identification and / or part of the second object recognition result of the second neural ( 136 ) Network model.

Second method for the second measuring system

• • Das Verfahren umfasst den Schritt des Installierens (51) des zweiten neuronalen Netzwerkmodells (136) auf einem computerbasierten System, beispielsweise einem Laborrechner.
• • Das Verfahren umfasst den Schritt des Trainierens (52) des zuvor installierten zweiten neuronalen Netzwerkmodells (136) mit synthetisch per Simulation ermittelten und/oder zuvor aufgezeichneten Trainingsdaten zweiter Sensormesswerte der zweiten Sensoranordnung (131), die bevorzugt manuell oder sonst wie zuvor bereits durch Bewertung mit um Bewertungsdaten ergänzt wurden. Dies hat das Ziel, die Objekte und/oder Objektkomponenten und/oder Anordnungen von Objekten und/oder Objektkomponenten in den zweiten Sensormesswerten der zweiten Sensoranordnung (131) und/oder in den zwei- oder dreidimensionalen Darstellungen, die von zweiten Sensormesswerten der zweiten Sensoranordnung (131) abhängen, zu identifizieren und somit ein zweites Objekterkennungsergebnis des zweiten neuronalen Netzwerkmodells (136) zu erzeugen, das mit den betreffenden Bewertungsdaten verglichen werden kann. Auf Basis des jeweiligen Vergleichsergebnisses kann dann eine Anpassung der Parameter des zweiten neuronalen Netzwerks (136) erfolgen.
• • Das Verfahren umfasst den Schritt des Verwendens (54) des trainierten zweiten neuronalen Netzwerkmodells (136) zur Identifizierung von Objekten und/oder Objektkomponenten und/oder von Anordnungen von Objekten und/oder Objektkomponenten in den zweiten Sensormesswerten und/oder in den zwei- oder dreidimensionalen Darstellungen, die von zweiten Sensormesswerten abhängen, wobei ein zweites Objekterkennungsergebnis erzeugt wird. Dabei werden die im Training (52) ermittelten Parameter in das zweite neuronale Netzwerkmodell (136) eines Ziel-Messsystems geladen und/oder eingestellt und/oder diese Parameter werden bei der Konstruktion des Ziel-Messsystems dazu verwendet, zumindest Teile dieser Parameter beispielsweise durch feste Verdrahtung innerhalb der Vorrichtung, die das zweite neuronale Netzwerkmodell realisiert, fest einzustellen.

The proposed method for recognizing objects and / or object components and / or arrangements of objects and / or object components in second sensor measured values of a second sensor arrangement ( 131 ) and / or in the two- or three-dimensional representations that are derived from second sensor measured values of the second sensor arrangement ( 131 ) depend, and / or to determine a second object recognition result on the basis of second sensor measured values of the second sensor arrangement ( 131 ) includes the following steps:

• • The procedure includes the step of installing ( 51 ) of the second neural network model ( 136 ) on a computer-based system, for example a laboratory computer.
• • The procedure includes the step of training ( 52 ) of the previously installed second neural network model ( 136 ) with synthetic training data determined by simulation and / or previously recorded from second sensor measurement values of the second sensor arrangement ( 131 ), which were preferably supplemented manually or otherwise, as before, with evaluation data. The aim of this is to find the objects and / or object components and / or arrangements of objects and / or object components in the second sensor measured values of the second sensor arrangement ( 131 ) and / or in the two- or three-dimensional representations that are derived from second sensor measured values of the second sensor arrangement ( 131 ) depend, to identify and thus a second object recognition result of the second neural network model ( 136 ) that can be compared with the relevant evaluation data. On the basis of the respective comparison result, an adjustment of the parameters of the second neural network ( 136 ) respectively.
• • The procedure includes the step of using ( 54 ) of the trained second neural network model ( 136 ) for identifying objects and / or object components and / or arrangements of objects and / or object components in the second sensor measurement values and / or in the two- or three-dimensional representations that depend on second sensor measurement values, a second object recognition result being generated. The training ( 52 ) determined parameters into the second neural network model ( 136 ) a target measuring system is loaded and / or set and / or these parameters are used in the construction of the target measuring system to permanently set at least parts of these parameters, for example by hard wiring within the device that realizes the second neural network model.

The workout ( 52 ) of the second neural network model ( 136 ) thus preferably includes training ( 52 ) with sensor live training values of the second sensor arrangement ( 131 ) of objects and / or object components and / or of arrangements of objects and / or object components and / or with two- or three-dimensional training representations, with training images of a common arrangement of a large number of subjects.

The workout ( 52 ) also preferably includes the identification of objects and / or object components and / or arrangements of objects and / or object components in the two- or three-dimensional training representations and preferably the assessment of these identifications with regard to any previously manually or automatically ascertained assessment data, which these are to be assigned to two- or three-dimensional training representations. The training ( 52 ) the identification of objects and / or object components and / or arrangements of objects and / or object components on the basis of the second sensor measured values of the second sensor arrangement ( 131 ) with the preferred use of existing evaluation data after this identification has been evaluated, possibly with the help of evaluation data previously determined manually or automatically, which these second sensor measured values of the second sensor arrangement ( 131 ) are to be assigned. Furthermore, the training ( 52 ) include the determination of a second object recognition result on the basis of the two- or three-dimensional training representations and its evaluation, if necessary with the aid of evaluation data previously determined manually or automatically. The training includes ( 52 ) the determination of a second object recognition result on the basis of the second measured values of the second sensor arrangement ( 131 ).

The method preferably includes using the identification of the objects and / or object components and / or arrangements of objects and / or object components and / or the second object recognition results in order to comment on the two- or three-dimensional representations that depend on sensor measured values.

In a further refinement of this variant, the method includes the use of the identification of the objects and / or object components and / or arrangements of objects and / or object components and / or the second object recognition results in a map of the surroundings of the vehicle and / or for informing the driver a two- or three-dimensional representation and / or by a control system and / or control device for controlling and / or regulating autonomous driving and / or parking.

First combination method using the first ultrasonic measuring system and the second measuring system

• Es erfolgt die Durchführung (90a) eines Verfahrens entsprechend Abschnitt A) für das Ultraschallmesssystem (164) mit Ermittlung der Ultraschallmesswerte der Ultraschallwandleranordnung (11) und die Ermittlung eines ersten Objekterkennungsergebnisses des ersten neuronalen Netzwerkes (16). In dieser Schrift umfasst das erste Objekterkennungsergebnis Identifizierungsinformationen und/oder sonstige Erkennungsinformationen von Objekten und/oder Objektkomponenten und/oder von Anordnungen von Objekten und/oder Objektkomponenten in den ersten Ultraschallmesswerten der Ultraschallwandleranordnung (11).
• Des Weiteren erfolgen die Durchführung (190b) eines Verfahrens gemäß Abschnitt D) für das zweite Messsystem (161) mit Ermittlung der zweiten Sensormesswerte der zweiten Sensoranordnung (131) und die Ermittlung eines zweiten Objekterkennungsergebnisses. In dieser Schrift umfasst das zweite Objekterkennungsergebnis Identifizierungsinformationen und/oder sonstige Erkennungsinformationen von Objekten und/oder Objektkomponenten und/oder von Anordnungen von Objekten und/oder Objektkomponenten in den zweiten Sensormesswerten der Sensoranordnung (131).
• Des Weiteren umfasst das in diesem Abschnitt E) beschriebene erste Kombinationsverfahren das Bereitstellen (91) eines dritten neuronalen Netzwerkmodells (138). Dieses dritte neuronale Netzwerkmodell (138) hat im Folgenden die Funktion eines selbstlernfähigen Entscheiders (englisch Voter). Das dritte neuronale Netzwerkmodell (138) erzeugt aus dem ersten Objekterkennungsergebnis des ersten neuronalen Netzwerkmodells (16) und dem zweiten Objekterkennungsergebnis des zweiten neuronalen Netzwerkmodells (136) das dritte neuronale Objekterkennungsergebnis. In dieser Schrift umfasst die gemeinsame Objekterkennungshypothese Identifizierungsinformationen und/oder sonstige Erkennungsinformationen von Objekten und/oder Objektkomponenten und/oder von Anordnungen von Objekten und/oder Objektkomponenten in den zweiten Sensormesswerten der Sensoranordnung (131) und in den ersten Ultraschallmessergebnissen der ersten Ultraschallwandleranordnung (11). Darüber hinaus kann die gemeinsame Objekterkennungshypothese erste Trainingssignale für das erste Neuronale Netzwerk (16) und zweite Trainingssignale für das zweite neuronale Netzwerk (136) umfassen. Diese ersten und zweiten Trainingssignale sind bevorzugt zwar separat, sie können aber auch für das erste und
• zweite neuronale Netzwerkmodell (16, 136) gemeinsam sein, was typischerweise nicht bevorzugt aber möglich ist.
• In dem hier in Abschnitt E) vorgestellten ersten Kombinationsverfahren ist bevorzugt wie zuvor ein Trainieren (92) des dritten neuronalen Netzwerkmodells (138) als weiterer Schritt vorgesehen. Dieses Training erfolgt bevorzugt mit bekannten ersten Objekterkennungsergebnissen des ersten neuronalen Netzwerkmodells (16) und mit bekannten zweiten Objekterkennungsergebnissen des zweiten neuronalen Netzwerkmodells (136), die um zugeordnete Bewertungsdaten ergänzt sind. Die Stimulierung des dritten neuronalen Netzwerkmodells erfolgt während des Trainings (92), um die Objekte und/oder Objektkomponenten und/oder Anordnungen von Objekten und/oder Objektkomponenten in den zweiten Sensormesswerten und in den Ultraschallmesswerten und/oder in den zwei- oder dreidimensionalen Darstellungen, die von Sensormesswerten und/oder Ultraschallmesswerten abhängen, zu identifizieren und eine gemeinsame Objekterkennungshypothese zu erzeugen. Diese gemeinsame Objekterkennungshypothese kann dann mit Hilfe der zugeordneten Bewertungsdaten bewertet werden. Die Parameter des dritten neuronalen Netzwerkmodells (138) werden dann beispielsweise durch eine erste Sicherheitsvorrichtung (142) in Abhängigkeit vom Ergebnis des Vergleichs der gemeinsamen Objekterkennungshypothese mit den zugeordneten Bewertungsdaten während des Trainings (92) modifiziert. Die erste Sicherheitsvorrichtung (142) kann hochpriorisierte Signale (H11, H15, H131, H135) von Komponenten des ersten Ultraschallmesssystems (164) und von Komponenten des zweiten Messsystems (161) erhalten. Stellen nämlich die Ultraschallwandleranordnung (11) und/oder die Sensoranordnung (131) auf welchem Wege auch immer eine potenziell sicherheitsgefährdende Situation oder eine erhöhte Wahrscheinlichkeit für den zukünftigen Eintritt einer solchen Situation auf welchem Wege auch immer fest, so ist es möglich auf diesem Weg die gemeinsame Objekterkennungshypothese des dritten neuronalen Netzwerkmodells (138) zu überschreiben, wenn diese nicht die Erkennung dieser Situation zeigt und so einen sicheren Betrieb gewährleistet. Ebenso soll der Benutzer bevorzugt in gewissen Fällen, falls dies erforderlich ist, durch eine Benutzereingabe (H142) die gemeinsame Objekterkennungshypothese überschreiben können. Des Weiteren soll ein höher priorisiertes System bevorzugt in gewissen Fällen, falls dies erforderlich ist, durch eine Benutzereingabe (H142) die gemeinsame Objekterkennungshypothese überschreiben können.

Both methods B and D can be used when using a combined device from a first ultrasonic measuring system ( 164 ) according to Section A) and a second measuring system ( 161 ) according to section C). The question here is how the data from the first object recognition result of the first ultrasonic measuring system ( 164 ) and the second object recognition result of the second measuring system ( 161 ) a common object recognition hypothesis is generated. This generation of a common object recognition hypothesis should preferably be self-learning. The method described below thus comprises a method according to section B) and a method according to section D) for recognizing objects and / or object components and / or arrangements of objects and / or object components in the second sensor measured values of the second sensor arrangement ( 131 ) and in the first ultrasonic measured values of the ultrasonic transducer arrangement ( 11 ) and / or on the basis of the first object recognition results of the first neural network model ( 16 ) and on the basis of the second object recognition results of the second neural network model ( 136 ). In order to generate the common object recognition hypothesis, the following additional steps are preferably carried out:

• The implementation takes place ( 90a) a procedure according to Section A) for the ultrasonic measuring system ( 164 ) with determination of the ultrasonic measured values of the ultrasonic transducer arrangement ( 11 ) and the determination of a first object recognition result of the first neural network ( 16 ). In this document, the first object recognition result includes identification information and / or other identification information of objects and / or object components and / or of arrangements of objects and / or object components in the first ultrasonic measured values of the ultrasonic transducer arrangement ( 11 ).
• Furthermore, the implementation takes place ( 190b) of a method according to Section D) for the second measuring system ( 161 ) with determination of the second measured values of the second sensor arrangement ( 131 ) and the determination of a second object recognition result. In this document, the second object recognition result includes identification information and / or other identification information of objects and / or object components and / or of arrangements of objects and / or object components in the second sensor measured values of the sensor arrangement ( 131 ).
• Furthermore, the first combination method described in this section E) includes the provision ( 91 ) of a third neural network model ( 138 ). This third neural network model ( 138 ) has the function of a self-learning decision maker (Voter). The third neural network model ( 138 ) generated from the first object recognition result of the first neural network model ( 16 ) and the second object recognition result of the second neural network model ( 136 ) the third neural object recognition result. In this document, the common object detection hypothesis comprises identification information and / or other identification information of objects and / or object components and / or of arrangements of objects and / or object components in the second sensor measured values of the sensor arrangement ( 131 ) and in the first ultrasonic measurement results of the first ultrasonic transducer arrangement ( 11 ). In addition, the common object recognition hypothesis can provide first training signals for the first neural network ( 16 ) and second training signals for the second neural network ( 136 ) include. Although these first and second training signals are preferably separate, they can also be used for the first and
• second neural network model ( 16 , 136 ) be common, which is typically not preferred but possible.
• In the first combination method presented here in Section E), training is preferred as before ( 92 ) of the third neural network model ( 138 ) planned as a further step. This training is preferably carried out with known first object recognition results of the first neural network model ( 16 ) and with known second Object recognition results of the second neural network model ( 136 ), which are supplemented by assigned evaluation data. The third neural network model is stimulated during training ( 92 ) in order to identify the objects and / or object components and / or arrangements of objects and / or object components in the second sensor measurement values and in the ultrasound measurement values and / or in the two- or three-dimensional representations that depend on sensor measurement values and / or ultrasound measurement values to generate a common object recognition hypothesis. This common object recognition hypothesis can then be evaluated with the aid of the assigned evaluation data. The parameters of the third neural network model ( 138 ) are then, for example, through a first safety device ( 142 ) depending on the result of the comparison of the common object recognition hypothesis with the assigned evaluation data during the training ( 92 ) modified. The first safety device ( 142 ) can receive high-priority signals (H11, H15, H131, H135) from components of the first ultrasonic measuring system ( 164 ) and of components of the second measuring system ( 161 ) receive. Namely, place the ultrasonic transducer arrangement ( 11 ) and / or the sensor arrangement ( 131 ) in whatever way a potentially security-endangering situation or an increased probability of the future occurrence of such a situation in whatever way is determined, it is possible in this way to use the common object detection hypothesis of the third neural network model ( 138 ) to overwrite, if this does not show the detection of this situation and thus ensures safe operation. Likewise, in certain cases, if necessary, the user should preferably be able to overwrite the common object recognition hypothesis by means of a user input (H142). Furthermore, a system with a higher priority should preferably in certain cases, if this is necessary, be able to overwrite the common object recognition hypothesis by means of a user input (H142).

In the case of such overwriting, it makes sense if the first safety device ( 142 ) the parameters of the third neural network model ( 138 ) modified and such a training ( 92 ) of the third neural network model ( 138 ) executes. The special thing is that this can be done in the company.

The workout ( 92 ) of the third neural network model ( 138 ) is preferably carried out, as before, first with the help of a suitable laboratory system that is sufficiently similar to the target system for the correct implementation of the training. The parameters determined in this way for the third neural network model ( 138 ) can then be transferred to the target system later. This can be done again as before, for example by programming and / or setting and / or already in the construction phase of the target system.

As before, the combination method presented here in this Section E) involves using ( 95 ) of the trained third neural network model ( 138 ), preferably within a target system from an ultrasonic measuring system ( 164 ) according to Section A) and a measuring system according to Section C), for the identification of objects and / or object components and / or arrangements of objects and / or object components in the second measured values of a sensor arrangement ( 131 ) and in the ultrasonic measured values of an ultrasonic transducer arrangement ( 11 ) and / or on the basis of the first object recognition results of the first neural network model ( 16 ) and on the basis of the second object recognition results of the second neural network model ( 136 ).

Here, too, the training preferably includes ( 92 ) of the third neural network model ( 138 ) also training with previously recorded or simulated first live object recognition results of the first network model ( 16 ) of objects and / or object components and / or of arrangements of objects and / or object components, and / or with second live object recognition results of the second network model ( 136 ) of objects and / or object components and / or of arrangements of objects and / or object components and / or with two- or three-dimensional training representations and / or with training images of a common arrangement of a large number of subjects. These training data, typically determined beforehand by simulation or measurement / experiment, are preferably manually and / or automatically evaluation data before the start of the training ( 92 ) added. A common object recognition hypothesis of the third neural network model determined on the basis of this training data ( 138 ) can then, for example, by the first safety device ( 142 ) or another training device can be compared with evaluation data assigned to the respective training data. Depending on the comparison result, the first safety device then modifies ( 142 ) or the other training device the parameters of the third neural network model ( 138 ) corresponding.

Preferably the training includes ( 92 ) of the third neural network model ( 138 ) the identification of objects and / or object components and / or of arrangements of objects and / or object components in the two- or three-dimensional training representations and / or the identification of objects and / or object components and / or of arrangements of objects and / or object components Based on the first and second object recognition results and / or the determination of a common object recognition hypothesis based on the two- or three-dimensional training representations and / or the determination of a common object recognition hypothesis based on the first ultrasonic measured values of the first ultrasonic transducer arrangement ( 11 ) and the second measured values of the second sensor arrangement ( 131 ). As already explained above, in the course of the training these training data are assigned evaluation data which serve as reference data.

As previously in the methods described in Sections B) and D), the first combination method described here in Section E) also includes using ( 92 ) the identification of the objects and / or object components and / or arrangements of objects and / or object components according to the common object recognition hypothesis in order to comment on the two- or three-dimensional representations, these being dependent on the second sensor measured values of the second sensor arrangement ( 131 ) and the ultrasonic measured values of the first ultrasonic transducer arrangement ( 11 ) and / or from the first object recognition result of the first neural network model ( 16 ) and from the second object recognition result of the second neural network model ( 136 ) and / or the common object recognition hypothesis.

Preferably, the first combination method described here in section E) further comprises using ( 95 ) the identification of the objects and / or object components and / or arrangements of objects and / or object components according to the common object recognition hypothesis and / or the common object recognition hypothesis in a map of the surroundings of the vehicle and / or to inform the driver through a two- or three-dimensional representation and / or for informing the driver by a control system and / or control device for controlling and / or regulating autonomous driving and / or parking.

To get the training ( 22nd ) of the first neural network model ( 16 ), a comparison of the common object recognition hypothesis of the third neural network model is preferably carried out in this first combination method ( 138 ) with the first object recognition result of the first neural network model ( 16 ) and, if necessary, changing the parameterization of the first neural network model ( 16 ) depending on the result of this comparison. This comparison and this change are preferably made by a first parameter modification device ( 140 ) for the first neural network model ( 16 ).

To get the training ( 52 ) of the second neural network model ( 136 ), a comparison of the common object recognition hypothesis of the third neural network model is preferably carried out in this first combination method ( 138 ) with the second object recognition result of the second neural network model ( 136 ) and, if necessary, changing the parameterization of the second neural network model ( 136 ) depending on the result of this comparison. This comparison and this change are preferably made by a second parameter modification device ( 141 ) for the second neural network model ( 136 ).

As previously described, a training ( 92 ) of the third neural network model ( 138 ) preferably by the first safety device ( 142 ) carried out. The higher-priority signals (H131, H135, H11, H15 and H142) are expressly pointed out again in this context, as these are essentially either self-induced training ( 92 ) by means of internal higher-priority signals (H131, H135, H11 and H15) or a user-induced training ( 92 ) enable higher prioritized signals (H142) generated by the user. As mentioned above, a technical device with a higher priority can also carry out training ( 92 ) of the third neural network model ( 138 ) initiate.

Of course, changing the parameterization of the first neural network model ( 16 ) and / or the parameterization of the second neural network model ( 136 ) and / or the parameterization of the third neural network model ( 138 ) depending on a user input ( 142 ) with the help of a training step ( 22nd , 52 , 92 ) respectively. The user input can be sent to the first parameter modification device (e.g. 140 ) for the first neural network model ( 16 ) and / or the second parameter modification device ( 141 ) for the second neural network model ( 136 ) and / or the first safety device ( 142 ) and / or the second safety device to be described later ( 143 ) and / or the fourth parameter modification device to be described later ( 142 ) for the fourth neural network model described later ( 156 ) respectively.

Such a user input for generating such a higher-priority external signal (for example H142) can, for example, by means of a touch gesture on a touch pad or as a Touch-screen display ( 9 ) or by means of a three-dimensional gesture. Such a three-dimensional preferably in the space in front of the display ( 9 ) executed spatial gesture or such a preferably two-dimensional touch gesture is preferably recognized by a correspondingly designed gesture recognition system and converted into the said higher-priority signal (eg H142). In order to reduce the probability of false triggering, these gestures are preferably sequences of individual gestures that are combined to form the relevant gestures and that must be given in the correct order and speed.

To do the training described above ( 92 ) of the third neural network model ( 138 ), it is advantageous if the first combination method E) described here comprises a step of monitoring one or more of the high-priority signals (H131, H135, H11, H15, H142). As described above, these high-priority signals (H131, H135, H11, H15, H142) can then be used via the first parameter modification device ( 140 ) for the first neural network model ( 16 ) and / or via the second parameter modification device ( 141 ) for the second neural network model ( 136 ) and / or via the first safety device ( 142 ) and / or via the second safety device ( 143 ), which will be described later, and / or the fourth parameter modification device for the fourth neural network model ( 156 ), which will be described later, a change in the parameterization of the third neural network model ( 138 ) depending on the status of these high-priority signals (H131, H135, H11, H15, H142). Instead of or in addition to this change, a change in the representation on the display ( 19th ) depending on the status of the high-priority signal (H131, H135, H11, H15, H142) and / or a change in the representation on the display ( 19th ) take place depending on the common object recognition hypothesis.

In addition to these high-priority signals (H11, H15, H131, H135, H142), the first neural network model ( 16 ) and / or the second neural network model ( 136 ) and / or the third neural network model ( 138 ) have further input signals that represent the first object recognition result of the first neural network model ( 16 ) and / or the second object recognition result of the second neural network model ( 136 ) and / or the common object recognition hypothesis of the third neural network model ( 138 ) influence. As a result, as a potential method step of the first combination method described here, there is a possible change in the common object recognition hypothesis and / or the first object recognition result and / or the second object recognition result as a function of one or more other external parameters that are determined by such a further input signal transmitted. Such external parameters can include, for example, the vehicle speed, the temperature, the air pressure, the humidity, the on or off status of the headlights and / or the parking lights and / or the indicators and / or the fog lights and / or the auxiliary lights etc. Ice sensor detected icing of the road or the route, rain detected by a rain sensor and / or the on or off status of the windshield wipers, the leveling of the vehicle, the rate of rotation of the vehicle about its axes, the translational and / or rotational acceleration and speed values, the steering angle, the steering angle speed, the tire pressure, the engine speed, the speed of the wheels, the braking status, etc.

The workout ( 92 ) of the neural network models ( 16 , 136 , 138 ) preferably takes place at least always in the form of a change in the parameterization of the first neural network model ( 16 ) and / or the second neural network model ( 136 ) and / or the third neural network model ( 138 ) when the vehicle reaches a first position and when the relevant neural network model has recognized or recognizes an object incorrectly at this first position. For example, if one of these neural network models detects an obstacle and if the system now determines that the vehicle is driving through the previously determined obstacle, then this obstacle was obviously not present. Thus the previous recognition was wrong, which leads to training ( 22nd , 52 , 92 ) of the neural network model in question can be used.

In the same way, a training ( 22nd , 52 , 92 ) by changing the parameterization of the first neural network model ( 16 ) and / or the second neural network model ( 136 ) and / or the third neural network model ( 138 ) occurs when a higher-priority signal (H11, H15, H131, H135) and / or a higher-priority signal (H142) or a user input signals a collision or collision warning or other danger.

A signal is then preferably sent to the user and / or a fault database of the vehicle and / or another computer system in the event of a change in the parameterization of the first neural network model ( 16 ) and / or the second neural network model ( 136 ) and / or the third neural network model ( 138 ), that a critical learning step has been carried out and / or that an error has occurred.

Second combination method using the second measuring system and a third measuring system

The combination method presented in accordance with Section E) can be used for the use of a combined device and a second measuring system ( 161 ) according to Section C) and a third measuring system ( 162 ) according to section C). Here again the question is how from the second object recognition result of the second measuring system ( 161 ) and the third object recognition result of the third measuring system ( 162 ) a common object recognition hypothesis is generated. This generation of a common object recognition hypothesis should again preferably be self-learning. The method described in the following thus comprises the parallel application of two methods according to Section D) for recognizing objects and / or object components and / or arrangements of objects and / or object components in the second measured values of a second sensor arrangement ( 131 ) of the second measuring system ( 161 ) and in third sensor measured values of a third sensor arrangement ( 151 ) of the third measuring system ( 162 ) and / or on the basis of the second object recognition results of a second neural network model ( 136 ) of the second measuring system ( 161 ) and / or on the basis of third object recognition results of a fourth neural network model ( 156 ) of the third measuring system ( 162 ). The combination process presented in this section preferably includes the additional step of implementation ( 171 ) a procedure according to section C) for the second measuring system ( 161 ), where the second measuring system ( 161 ) said second neural network model ( 136 ) comprises, and the step of determining the second sensor measured values of the second sensor arrangement ( 131 ) and the step of determining a second object recognition result of the second neural network model ( 136 ). The combination method presented in this section F) preferably also includes the step of performing ( 172 ) a method according to Section C) for the third measuring system ( 162 ), whereby the third measuring system ( 162 ) a fourth neural network model ( 156 ), and the step of determining the third sensor measured values of a third sensor arrangement ( 151 ) and the step of determining a third object recognition result of the fourth neural network model ( 156 ). Furthermore, the second combination method presented here includes the provision ( 173 ) of a fifth neural network model ( 158 ) and training ( 174 ) this fifth neural network model ( 158 ) in order to determine the objects and / or object components and / or arrangements of objects and / or object components in the second sensor measured values of the second sensor arrangement ( 131 ) and in the third sensor readings of the third sensor arrangement ( 151 ) and / or in the two- or three-dimensional representations, which depend on second and / or third sensor measured values, and to identify a common object recognition hypothesis of the fifth neural network model ( 158 ) to create. Analogous to the first combination method from Section E), the second combination method preferably comprises using ( 175 ) of the trained fifth neural network model ( 158 ) to identify objects and / or object components and / or arrangements of objects and / or object components in the second measured values of the second sensor arrangement ( 131 ) and in the third sensor readings of the third sensor arrangement ( 151 ) on the one hand and / or on the basis of the first object recognition results of the first neural network model ( 136 ) and on the basis of the second object recognition results of the second neural network model ( 156 ) on the other hand and / or to generate a common object recognition hypothesis of the fifth neural network model ( 158 ).

Preferably the training includes ( 174 ) of the fifth neural network model ( 158 ) training with second live object recognition results of the second network model ( 136 ) of objects and / or object components and / or of arrangements of objects and / or object components and / or training with third live object recognition results of the third network model ( 156 ) of objects and / or object components and / or of arrangements of objects and / or object components and / or training with two- or three-dimensional training representations and / or with training images of a common arrangement of a large number of subjects. The workout ( 174 ) runs in the same way as the training ( 92 ) in the first combination procedure from Section E).

Likewise analogous to the training ( 92 ) of the first combination procedure from Section E) preferably includes training ( 174 ) of the fifth neural network model ( 158 ) the identification of objects and / or object components and / or of arrangements of objects and / or object components in the two- or three-dimensional training representations and / or the identification of objects and / or object components and / or of arrangements of objects and / or object components Based on the second and third object recognition results and / or the determination of a common object recognition hypothesis on the basis of the two- or three-dimensional training representations includes the determination of a common object detection hypothesis based on the second measured values of the second sensor arrangement ( 131 ) and the third sensor readings of the third sensor arrangement ( 151 ).

The second combination method described here comprises, in analogy to the first combination method of section E), preferably using ( 175 ) the identification of the objects and / or object components and / or arrangements of objects and / or object components according to the common object recognition hypothesis in order to comment on the two- or three-dimensional representations, these being dependent on the second sensor measured values and the third sensor measured values on the one hand and / or where this from the second object recognition result of the second neural network model ( 136 ) and from the third object recognition result of the third neural network model ( 156 ) and / or the common object recognition hypothesis.

The second combination method described here also comprises, in analogy to the first combination method of section E), preferably using ( 175 ) the identification of the objects and / or object components and / or arrangements of objects and / or object components according to the common object recognition hypothesis and / or the common object recognition hypothesis in a map of the surroundings of the vehicle and / or to inform the driver through a two- or three-dimensional representation and / or for informing the driver by a control system and / or control device for controlling and / or regulating autonomous driving and / or parking.

The second combination method described here also comprises, in analogy to the first combination method of section E), preferably the comparison of the common object recognition hypothesis with the second object recognition result of the second neural network model ( 136 ) and changing the parameterization of the second neural network model ( 136 ) depending on the result of this comparison. This comparison is preferred by said second parameter modification device ( 141 ) for the second neural network model ( 136 ) carried out.

The second combination method described here also includes, in analogy to the first combination method of section E), preferably the comparison of the common object recognition hypothesis with the third object recognition result of the fourth neural network model ( 156 ) and changing the parameterization of the fourth neural network model ( 156 ) depending on the result of this comparison. This comparison is preferred by the said fourth parameter modification device ( 144 ) for the fourth neural network model ( 156 ) carried out.

The second combination method described here also comprises, in analogy to the first combination method of section E), preferably changing the parameterization of the second neural network model ( 136 ) and / or the parameterization of the fourth neural network model ( 156 ) and / or the parameterization of the fifth neural network model ( 158 ) depending on a user input (H142) with the help of a training step ( 52 , 174 Again, the user input takes place, for example, by means of a touch gesture on a touch pad or a display designed as a touch screen ( 9 ) or by means of a three-dimensional gesture.

The second combination method described here also comprises, in analogy to the first combination method of section E), monitoring one or more high-priority signals (H131, H135, H151, H155) and changing the parameterization of the fifth neural network model ( 158 ) depending on the state of the high-priority signal (H131, H135, H151, H155). This again enables independent training of the fifth neural network model ( 158 ) especially during operation.

The second combination method described here, in analogy to the first combination method of section E), preferably includes monitoring a high-priority signal (H131, H135, H151, H155) and changing the representation on the display ( 19th ) depending on the status of the high-priority signal (H131, H135, H151, H155) and / or the change in the representation on the display ( 19th ) depending on the common object recognition hypothesis of the fifth neural network model ( 158 ).

The second combination method described here also comprises, in analogy to the first combination method of section E), preferably changing the common object recognition hypothesis and / or the first object recognition result and / or the second object recognition result as a function of one or more other external parameters, which are again via another input signal transmitted. Such external parameters can, for example, again be the vehicle speed, the temperature, the air pressure, the humidity, the on or off status of the headlights and / or the parking lights and / or the indicators and / or the fog lights and / or the auxiliary lights etc. an ice sensor detected icing of the road or the driveway, one by one Rain sensor recorded rain and / or the on or off status of the windshield wipers, the leveling of the vehicle, the rate of rotation of the vehicle about its axes, the translational and / or rotational acceleration and speed values, the steering angle, the steering angle speed, the tire pressure, the engine speed, the speed of the wheels, the braking condition, etc.

The second combination method described here also comprises, in analogy to the first combination method of section E), preferably changing the parameterization of the second neural network model ( 136 ) and / or the fourth neural network model ( 156 ) and / or the fifth neural network model ( 158 ) when the vehicle has reached a first position and when the neural network in question detects or detects an object in an incorrect manner at this first position. What was written in Section E) for the first combination procedure applies accordingly.

The second combination method described here also comprises, in analogy to the first combination method of section E), preferably changing the parameterization of the second neural network ( 136 ) and / or the fourth neural network ( 156 ) and / or the fifth neural network ( 158 ), if a higher-priority signal (H151, H155, H131, H135) and / or another high-priority signal (H152) or a user input signals a collision or collision warning or other danger. What was written in Section E) for the first combination procedure applies accordingly.

The second combination method described here also comprises, in analogy to the first combination method of section E), preferably a signaling to the user and / or an error database and / or a computer system in the event of a change in the parameterization of the second neural network ( 136 ) and / or the fourth neural network ( 156 ) and / or the fifth neural network ( 158 ) that a critical learning step has been carried out and / or that an error has occurred.

Simplest self-learning system or measuring system

The combination of the measuring systems described above in Sections A) and C) together with the device structure required in the combination methods E) and F) can be characterized as a self-learning system and / or measuring system as follows: Such an overall measuring system represents a network of neural network models ( 16 , 136 , 138 , 156 , 158 ) in the following called neural network. The neural network models ( 16 , 136 , 138 , 156 , 158 ) represent the nodes of the neural network. The neural network proposed here comprises at least one first neural network model ( 16 ) with a first set of parameters and a second neural network model ( 136 ) with a second set of parameters and a third neural network model ( 138 ) with a third set of parameters. In relation to the previously presented second combination method from Section F), the neural network proposed here comprises at least one second neural network model ( 136 ) with a second set of parameters and a fourth neural network model ( 156 ) with a fourth set of parameters and a fifth neural network model ( 158 ) with a fifth parameter set. So there are again at least three neural network models that are distributed over at least two node layers of the neural network. Each neural network model of the neural network has at least a first input data stream and a first output data stream. Since the basic structure of the neural network for carrying out the second combination method of section F), as just discussed, is the same as the basic structure of the neural network for carrying out the first combination method of section E), we will now limit ourselves to the neural network for carrying out the first combination method of Section E). As is evident to the person skilled in the art, the results can be transferred to the neural network for carrying out the second combination method of section F). This restriction therefore expressly includes the application of the following in this section to a neural network for carrying out the second combination method from section F).

The third neural network model ( 138 ) has a second input data stream. The output data stream of the first neural network model ( 16 ) is the first input data stream of the third neural network model ( 138 ). The output data stream of the second neural network model ( 136 ) is the second input data stream of the third neural network model ( 138 ). The output data stream of the third neural network ( 138 ) depends at least on its first and second input data stream. An output data stream of the third neural network ( 138 ) can then, as described in Section E, the first parameter set of the first neural network ( 16 ) and / or the second parameter set of the second neural network ( 136 ) change. This is preferably done by means of said first parameter modification device ( 140 ) for the first neural network model ( 16 ) and / or by means of the second parameter modification device ( 141 ) for the second neural network model ( 136 ).

More complex self-learning system or measuring system

Since the basic structure of the neural network for carrying out the second combination method of section F), as discussed above, is the same as the basic structure of the neural network for carrying out the first combination method of section E), we will now limit ourselves to the neural network for Carrying out the first combination procedure of section E). As is evident to the person skilled in the art, the results can be transferred to the neural network for carrying out the second combination method of section F). This restriction therefore expressly includes the application of the following in this section to a neural network for carrying out the second combination method from section F).

The proposed self-learning system and / or measurement system comprises a network of neural network models ( 16 , 136 , 138 ). This network of neural network models ( 16 , 136 , 138 ) is again referred to below as a neural network as in Section G). Again, the neural network models ( 16 , 136 , 138 ) represent the nodes of the neural network. The neural network comprises at least one first neural network model ( 16 ) with a first set of parameters and a second neural network model ( 136 ) with a second set of parameters and a third neural network model ( 138 ) with a third set of parameters. Furthermore, the neural network comprises a first parameter modification device ( 140 ) for the first neural network model ( 16 ) and a second parameter modification device ( 141 ) for the second neural network model ( 136 ). Any neural network model ( 16 , 136 , 138 ) of the neural network has at least a first input data stream and a first output data stream. The third neural network model ( 138 ) has a second input data stream. The output data stream of the first neural network model ( 16 ) is the first input data stream of the third neural network model ( 138 ). The output data stream of the second neural network model ( 136 ) is the second input data stream of the third neural network model ( 138 ). The output data stream of the third neural network ( 138 ) depends at least on its first and second input data stream. An output data stream from the third neural network ( 138 ) the first parameter set of the first neural network ( 16 ) by means of the first parameter modification device ( 140 ) and / or the second parameter set of the second neural network ( 136 ) by means of the second parameter modification device ( 141 ) and / or the said output data stream of the third neural network ( 138 ) is intended to influence these changes.

The change in the parameter sets is preferably influenced by user inputs using input means such as touch screens, switches, etc. Said input means preferably comprise gesture recognition means.

Parts of the third output data stream of the third neural network model preferably correspond ( 138 ) either with parts of the first output data stream of the first network model ( 16 ) or with parts of the second output data stream of the second network model ( 136 ) selectable. This has the advantage that these parts, for example, through the third network model ( 138 ) can be exchanged for each other. This selection is preferably made by the third neural network model ( 138 ) depending on the first output data stream of the first network model ( 16 ) and from the second output data stream of the second network model ( 136 ).

The first parameters of the first network model are preferred ( 16 ) through the third network model ( 138 ) changed if the selection was made in such a way that parts of the third output data stream of the third neural network model ( 138 ) not with parts of the first output data stream of the first network model ( 16 ) correspond. In this case it is assumed that the recognition by the first neural network model ( 16 ) was not optimal and this effect should be reduced.

The first parameters of the first network model are preferred ( 16 ) through the third network model ( 138 ) changed if the selection was made in such a way that parts of the third output data stream of the third neural network model ( 138 ) with parts of the first output data stream of the first network model ( 16 ) correspond. In this case it is assumed that the recognition by the first neural network model ( 16 ) was optimal and this effect should be reinforced.

Second parameters of the second network model are preferred ( 136 ) through the third network model ( 138 ) changed if the selection was made in such a way that parts of the third output data stream of the third neural network model ( 138 ) not with parts of the second output data stream of the second network model ( 136 ) correspond. In this case it is assumed that the recognition by the second neural network model ( 136 ) was not optimal and this effect should be reduced.

Second parameters of the second network model are preferred ( 136 ) through the third network model ( 138 ) changed if the selection was made in such a way that parts of the third output data stream of the third neural network model ( 138 ) with parts of the second output data stream of the second network model ( 136 ) correspond. In this case, it is assumed that the recognition by the second neural network model ( 136 ) was optimal and this effect should be reinforced.

General summary regarding an automotive ultrasonic measuring device with deep learning

In this disclosure, an ultrasonic measuring system ( 164 ) for use in motor vehicles for the detection of obstacles and / or objects in the vicinity of a vehicle using deep learning methods, which is characterized in that it comprises a device, in particular a computer, in which a neural network model ( 16 , 136 , 138 ), which is adapted by a deep learning technique to receive ultrasonic measured values and to identify the objects and their geometric arrangement in the surroundings of the vehicle in the ultrasonic measured values by assigning an identification and thus to generate a first object recognition result. The device has the means ( 141 , 140 ) by at least part ( 16 , 136 ) of the neural network model ( 16 , 136 , 138 ) to continue training while the vehicle is in operation. In order to exclude the training of safety-relevant malfunctions, the device preferably has further means ( 142 ), for example by means of high-priority signals (H11, H15, H131, H135), user inputs (H142) and / or external signals and by checking the object recognition results and / or common object recognition hypotheses, to identify incorrect and security-relevant recognition results and / or potentially incorrect and / or potentially security-relevant To prevent and / or suppress recognition results and, in the event of such undesired recognition results, the relevant neural network models ( 16 , 136 , 138 ) to identify and retrain.

Automobile ultrasonic measuring system with beam forming

In this disclosure, an ultrasonic measuring system for use in motor vehicles for the detection of obstacles and / or objects in the vicinity of a vehicle using deep learning methods was disclosed, characterized in that it is a microbeamformer ( 1014 ) and several ultrasonic transducers within an ultrasonic transducer arrangement ( 1012 ), wherein the alignment and / or shape of the receiving and / or transmitting lobe of the multiple ultrasonic transducers within an ultrasonic transducer arrangement ( 1012 ) of signals from the microbeamformer ( 1014 ) depends. The microbeamformer generates ( 1014 ) In the transmission mode, each of the multiple ultrasonic transducers has its own specially modulated control signal so that the ultrasonic wave produced in the free space in front of the ultrasonic transducers has the desired cross-sectional shape, spherical amplitude distribution and main direction of propagation. A simple example would be the generation of similar, but phase-shifted control signals by the microbeamformer ( 1014 ) i, which are then converted into sound signals by the ultrasonic transducers. In reception mode, the microbeamformer ( 1014 ) for a space-time inverse transformation of the vector received signal, which is composed of the received signals of the various ultrasonic transducers. In the simplest case, the received signals of the various ultrasonic transducers are specifically in the respective phase by the microbeamformer ( 1014 ) delayed so that the superposition of the delayed received signals of the various ultrasonic transducers results in an aligned sensitivity lobe.

It is conceivable to use a separate transmission microbeamformer for the transmission channel ( 1014 ) and for the receiving channel a separate receiving microbeamformer ( 1014 ) to be provided.

24 shows the alignment of transmitted and received signals by means of constructive interference of the ultrasonic waves. The more ultrasonic transducers an ultrasonic transducer arrangement ( 1012 ), the better the daughter maxima can be suppressed and thus the signal focusing improved. It is therefore recommended to use more than 10 ultrasonic transducers, preferably more than 20, even better than 50 ultrasonic transducers. In order to achieve a height resolution, it is recommended that the ultrasonic transducer arrangement ( 1012 ) Comprises ultrasonic transducers which are arranged in at least two rows essentially parallel to the roadway and at least 5 columns perpendicular to the roadway. Of course, this position can only be achieved with ideally equally sized wheels, exactly parallel bumper mounts, etc. The term “essentially” should therefore include a deviation of +/- 10 ° from these positions.

Automobile ultrasonic measuring system with Doppler processor

In the above description, an ultrasound measuring system for use in motor vehicles for recognizing obstacles and / or objects in the vicinity of a vehicle using deep learning methods was also described proposed, which is characterized in that it has a Doppler processor ( 1054 ) that stores ensembles of echo signals from discrete points and / or spatial directions, in particular in an image field, which are then used to estimate the Doppler shift, particularly at points in the image and / or in spatial directions, using a fast Fourier transform (FFT) .

A multispectral image resolved according to spatial directions is preferably generated by means of a fast discrete Fourier transformation. Since the transmission frequency is known, the following values result for each spatial direction: amplitude of the reflection, time delay or distance of the reflection, frequency shift of the reflection or positive or negative speed in the sensor direction (towards the ultrasonic transducer). If past images are also evaluated, a movement vector can be assigned to distinctive points in the respective image by tracking features over time. The situation is particularly preferred when the distance between the ultrasonic transducers e.g. The Doppler processor on the bumper carries out the Doppler analysis specifically for the measurement signal from each ultrasonic transducer. Since the angle between the reflecting object and the respective ultrasonic transducer is different when it is sufficiently close, the results of the respective Doppler analyzes of the received signals from different ultrasonic transducers differ slightly from one another. In this way, it is also possible to deduce speeds perpendicular to the line of sight between the ultrasonic transducers and the respective object directly from the ultrasonic transducer reception signals. This can then be correlated with the speeds determined by image processing of the ultrasound image sequences.

Automobile ultrasonic measuring system with multiplanar reformatter

In the above description, an ultrasonic measuring system for use in motor vehicles for the detection of obstacles and / or objects in the vicinity of a vehicle using deep learning methods was proposed, which is characterized in that it uses a multiplanar reformatter ( 1044 ), which converts echoes received from points in a common plane in a volumetric area of the room in the vicinity of the vehicle into an ultrasound image of this plane.

The function of the multiplanar reformatter ( 1044 ) is typically closely related to the function and interaction of the microbeamformer ( 1014 ) and several ultrasonic transducers within an ultrasonic transducer arrangement ( 1012 ) interwoven. While the microbeamformer ( 1014 ) the alignment and / or shape of the receiving and / or transmitting lobe of the multiple ultrasonic transducers within an ultrasonic transducer arrangement ( 1012 ) of signals from the microbeamformer ( 1014 ) controls and controls, selects the multiplanar reformatter ( 1044 ) from the incoming data stream of the received signals of the ultrasonic transducers, such signal components in a plane that is typically parallel to a vehicle plane and / or the road plane. To do this, it combines the received signals from at least two ultrasonic transducers, which are mounted at different heights, for example on the bumper, and evaluates the phase shift of the signals in such a way that only those signals are processed which include reflections in one plane. Here, a plane is defined as a limited solid angle area with a minimum sensitivity angle φ _min (see 25 ) between the vehicle and perpendicular to the roadway and a maximum angle φ _max (see 25 ) between the vehicle and perpendicular to the road.

Here, too, it is recommended to use more than 10 ultrasonic transducers, better than 20, even better than 50 ultrasonic transducers. In order to achieve a height resolution, it is recommended that the ultrasonic transducer arrangement ( 1012 ) Comprises ultrasonic transducers which are arranged in at least two rows, better than 5 rows, essentially parallel to the roadway. Of course, this position can only be achieved with ideally equally sized wheels, exactly parallel bumper mounts, etc. The term “essentially” should therefore also include a deviation of +/- 10 ° from these positions.

Automobile ultrasonic measurement system with volume renderer

In the above description, an ultrasonic measuring system for use in motor vehicles for recognizing obstacles and / or objects in the vicinity of a vehicle using deep learning methods was proposed, which is characterized in that it uses a volume renderer ( 1042 ) that converts the echo signals of a 3D data set into a projected 3D image as viewed from a given reference point.

For this purpose, a computer system calculates a three-dimensional vehicle environment model from the received data. This can then be presented to the vehicle driver depending on the hazard situation in such a way that he can grasp the situation with maximum probability. For this it makes sense if a neural network model, for example the third neural network model ( 138 ), the optimal observer position and the optimal viewing direction to the vehicle surroundings model, for example determined as a function of the common object detection hypothesis and a corresponding representation of the determined vehicle environment model on the screen ( 19th ). What is a good perspective can be very driver-specific and also culture-dependent. It is therefore advisable to use so-called focus groups consisting of a representative cross-section of the vehicle drivers in the target market to determine a starting parameter set for the neural network model in question and then to adapt the driver-specific perspective using user inputs (H142).

Automobile ultrasonic measuring system with safety device

In the above description, a measuring system for use in motor vehicles for recognizing obstacles and / or objects in the vicinity of a vehicle using deep learning methods was proposed, which is characterized in that it has a first safety device ( 142 ) and one or more neural network models ( 16 , 136 , 138 ) to evaluate the measurement data of at least one sensor arrangement ( 11 , 131 ) of the measuring system and that the first safety device ( 142 ) at least the neural network model ( 138 ) that processes this measurement data or data derived from it.

The safety device can preferably use signals (H11, H131) of the at least one sensor arrangement ( 11 , 131 ) of the measuring system can be influenced. In addition, the safety device ( 142 ) preferably the parameters of at least one neural network model ( 16 , 136 , 138 ) influence. Furthermore, the safety device ( 142 ) can be influenced by a user signal (H142) or an external control signal (H142). In this way, in the event of incidents endangering security, the user can initiate negative signaling of a learning process that rules out such incidents in the future.

Conversely, the safety device ( 142 ) prefers the property that it contains safety-endangering and / or potentially safety-endangering signaling of the at least one neural network model ( 16 , 136 , 138 ) can safely prevent. This can be done by plausibility checks of the determined signals depending on high-priority further signals (H11, H15, H131, H135, H155, H151) within the device and / or by external control signals (H142), these signals being a or multiple signals in time and / or space multiplex and / or information at predetermined storage locations of a computer network.

advantage

The proposed system enables the determination of the geometric arrangement and the classification of objects in the vicinity of the vehicle by means of an overall measurement system with self-learning measurement systems ( 161 , 162 ) and a self-learning decision maker ( 138 , 158 ). This learning process can also take place during normal operation, which means that the system is continuously improved. User inputs and safety signals (H11, H15, H131, H135, H142) secure the potential space of possible behavior in such a way that critical states are avoided.

Figure list

• 1 In 1 An ultrasonic vehicle environment analysis system is shown schematically simplified in the form of a block diagram, which is constructed in accordance with the principles of the present invention.
• 2 Figure 11 shows a method of training and using a neural network model to identify anatomy in ultrasound images.
• 3 shows an alternative structure of the proposed ultrasonic measuring system ( 164 ) with a first neural network model ( 16 ) for the corresponding procedure according to section B).
• 4th shows a model for training and using the first neural network model ( 16 ).
• 5 shows a model for training and storing the parameters of the first neural network model ( 16 ).
• 6 shows a model for loading the stored parameters of the first neural network model ( 16 ) and then using the first neural network model ( 16 ).
• 7th shows a model for training and using the second neural network model ( 136 ).
• 8th shows a model for training and storing the parameters of the second neural network model ( 136 ).
• 9 shows a model for loading the stored parameters of the second neural network model ( 136 ) and then using the second neural network model ( 136 ).
• 10 shows schematically the first combination method from section E).
• 11 shows a model for training and using the third neural network model ( 138 ).
• 12th shows a model for training and storing the parameters of the third neural network model ( 138 ).
• 13th shows a model for loading the stored parameters of the third neural network model ( 138 ) and then using the third neural network model ( 138 ).
• 14th shows a model for training and using the fifth neural network model ( 158 ).
• 15th shows a model for training and storing the parameters of the fifth neural network model ( 158 ).
• 16 shows a model for loading the stored parameters of the fifth neural network model ( 158 ) and then using the fifth neural network model ( 158 ).
• 17th shows schematically the second combination method from section F).
• 18th shows the structure of the proposed combined overall system from an ultrasonic measuring system ( 164 ) and a second measuring system ( 161 ) for the corresponding first combination procedure according to Section E).
• 19th shows the structure of the proposed combined overall system from a second measuring system ( 161 ) and a third measuring system ( 162 ) for the corresponding first combination procedure according to Section F).
• 20th shows a model for training and using an overall system 19th . (Can refer to an overall system 18th easily transferred.)
• 21st shows a model for training and storing the parameters of the neural network models of an overall system 19th . (Can refer to an overall system 18th easily transferred.)
• 22nd Fig. 12 shows a model for loading the parameters of the neural network models of an overall system 19th and using the entire system prepared in this way. (Can refer to an overall system 18th easily transferred.)
• 23 shows the beam forming through constructive and destructive interference based on the phase information in the echo signal for ultrasonic transducers of the ultrasonic transducer arrangement ( 1012 ) with different positions along the width of the vehicle.
• 24 Shows the limitation of the reception area to an angular range by a multiplanar reformatter using triangulation based on the phase information in the echo signal for the ultrasonic transducers of the ultrasonic transducer arrangement ( 1012 ) with different height positions.

Description of the figures

Figures 1 and 2

The 1 and 2 are described above in the text.

Figure 3

3 shows the structure of the proposed ultrasonic measuring system ( 164 ) for use in a vehicle. The vehicle can be, for example, a motor vehicle, a motorcycle, an aircraft or a ship or the like.

A proposed ultrasonic transducer arrangement ( 11 ) records ultrasonic measured values by sending out ultrasonic waves and receiving the echo signals. The ultrasonic transducer arrangement ( 11 ) can be a one or two-dimensional arrangement of one or more ultrasonic transducers in order to record measured values in one, two or three dimensions. An ultrasound transducer of the ultrasound transducer arrangement can in the sense of this proposal consist of an ultrasound transmitter and an ultrasound receiver separate therefrom in space division multiplex or an ultrasound transducer which transmits and receives in time division multiplex. The ultrasonic transducer arrangement ( 11 ) is equipped with control electronics ( 12th ) connected, optionally for beam shaping, for example by means of Phase shift of the transmitted ultrasonic signals of the individual ultrasonic transducers of the ultrasonic transducer arrangement ( 11 ) can be capable. The control electronics ( 12th ) controls the corresponding ultrasonic transducers of the ultrasonic transducer arrangement ( 11 ) for preferably directed transmission or reception of the ultrasonic signals. This alignment is desirable, but not mandatory. The emission of ultrasonic signals by the ultrasonic transducer arrangement ( 11 ) is controlled by a transmitter control ( 13th ). This is with the control electronics ( 12th ) and sets the transmission parameters of the ultrasonic transducers of the ultrasonic transducer arrangement ( 11 ), depending on the settings made via the optional user settings stored in an optional memory ( 14th ), or by a higher-level control unit. Send parameters, which are from the transmitter control ( 13th ) can be controlled, for example, burst length, frequency, frequency curve, coding, polarity, phase or amplitude.

The ultrasonic transducers of the ultrasonic transducer arrangement ( 11 ) received echo signals are processed by the signal processor ( 15th ), including filtering using digital filters and noise reduction. With the signal processor ( 15th ) it can also be wholly or partially a digital signal processing device, for example a signal processor in the sense of processor technology or a device with the same effect, and / or an analog circuit and / or a mixture of these two circuit variants. The signals from the signal processor ( 15th ) processed received signals are the input signals of the first neural network model ( 16 ), which is typically stored on a digital memory of a computer. The first neural network model ( 16 ) can also be implemented, for example, as a special microelectronic circuit or other functionally equivalent device. The neural network model ( 16 ) is the implementation of a deep learning process, as described above. The training of the first neural network model ( 16 ) is carried out using training data from various objects in different positions. These training data can be determined by means of measurement and / or simulation. Before the training, evaluation data are assigned to the data sets of the training data, which the expected evaluation by the first neural network model ( 16 ) and which are used by the training device to decide whether and how the parameters of the first neural network model ( 16 ) have to be changed. These training data are in a digital memory ( 17th ) saved. To the training ( 22nd ) methods of supervised or unsupervised learning can be used. After completing the training ( 22nd ) the trained data can be used to parameterize the connections of the first neural network model ( 16 ) are extracted (= stored) (35) in a database of neural network data. These can then be integrated into an identical ultrasonic measuring system ( 164 ) or essentially identical ultrasonic measuring system without separate training in the production of identical or essentially identical additional ultrasonic measuring systems ( 164 ) can be loaded without a training session ( 22nd ) of the other identical ultrasonic measuring systems ( 164 ) must be done. An ultrasonic measuring system ( 164 ) is considered to be structurally identical for the purposes of this disclosure if the ultrasonic measuring system 164 ) fulfills its intended purpose when using the network data determined in this way. The network data are essentially the parameters of the neural network model. It makes sense to determine this network data with a laboratory setup and then to feed the network data into a series product suitable for series production. This has the advantage that the time-consuming initial training phase in production can be saved. The identical other ultrasonic measuring systems ( 164 ) then continue to optimize themselves during operation from this specified starting point. As an identical ultrasonic measuring system ( 164 ) in the sense of this disclosure is therefore an ultrasonic measuring system ( 164 ) considered that it is suitable or intended to store the determined training data in the database of neural network data of the ultrasonic measurement system ( 164 ) to use. With the ultrasonic measuring system ( 164 ), with the help of which the training data was determined in the database of neural network data, it can be, for example, a laboratory system with considerable mechanical quality defects, while the other structurally identical ultrasonic measuring systems ( 164 ) to produce mechanically stable ultrasonic measuring systems ( 164 ), which can differ significantly in appearance and form from the laboratory system.

The output values of the first neural network model ( 16 ) are preferably the positions of the recognized objects and preferably their classification, as well as, for example, a corresponding confidence value that can be assigned to the recognized recognition result as a whole, and / or confidence values that are assigned to the individual objects. Further data can be part of the output values. Hypothesis lists that are alternative to one another are also possible, the hypothesis list including alternative recognition results with different confidence values for these object recognition results. Each object recognition result includes one or more objects. The preferred image processor used ( 18th ) converts this information into a suitable two- or three-dimensional representation, which is provided by the display ( 19th ) is issued to the driver. The information about object positions and classifications are also given preference to the user control ( 110 ) to hand over. The selection of a saved user control or the adjustment of control parameters of a control in response to the classification is carried out by the user control ( 110 ) and transferred to the settings memory ( 14th ) or from the user control display ( 111 ) is displayed. The settings memory ( 14th ) also transmits parameters dependent on the classification to the image processor ( 18th ) so that the display ( 19th ) the displayed representation of the objects and their identification can be automatically adapted accordingly.

Figure 4

4th shows a model for training and using the described neural network model for determining the position and classifying objects in the vicinity of the vehicle. In the installation step ( 21st ) a first neural network model ( 16 ) installed as the basis of the deep learning process. The first neural network model ( 16 ) are used in the training step ( 22nd ) the corresponding training received signals from measured or simulated objects are transferred as input values in as many different arrangements and positions as possible. In one identification step ( 23 ) the objects for the neural network model are identified. During the training step ( 22nd ) and the identification step ( 23 ) the actual learning happens. The responses of the first neural network model ( 16 ) are compared with evaluation data and the parameters of the first neural network model ( 16 ) appropriately modified. The largest possible amount of training data should be used for this.

The trained first neural network model ( 16 ) is preferably on a digital memory in the vehicle, see chap. 3 . Here, the trained first neural network model in the identification step ( 24 ) the live ultrasonic reception signals from the operation of the ultrasonic transducer arrangement ( 11 ) in order to determine the position and classify the objects. The output values of the first neural network model ( 16 ), the item and the classification, are then used in the subsequent usage step ( 25 ) used by the described system.

Figure 5

shows a model for training and storing the parameters of the first neural network model ( 16 ).

In the installation step ( 21st ) a first neural network model ( 16 ) installed as the basis of the deep learning process. The first neural network model ( 16 ) are used in the training step ( 22nd ) the corresponding training received signals from measured or simulated objects are transferred as input values in as many different arrangements and positions as possible. In one identification step ( 23 ) the objects for the neural network model are identified. During the training step ( 22nd ) and the identification step ( 23 ) the actual learning happens. The responses of the first neural network model ( 16 ) are compared with evaluation data and the parameters of the first neural network model ( 16 ) appropriately modified. The largest possible amount of training data should be used for this.

The network data (parameter data) of the trained first neural network model ( 16 ) are in contrast to 4th but now in one backup step ( 35 ) saved.

Figure 6

shows a model for loading the backup step ( 35 ) stored parameters of the first neural network model ( 16 ) and then using the first neural network model ( 16 ).

In one loading step ( 41 ) the previously stored data on a first ultrasonic measuring system ( 164 ) stored parameters (network data) of the first neural network model ( 16 ) back into the memory of the first neural network model ( 16 ) a second ultrasonic measuring system ( 164 ), which is essentially the same as the first ultrasonic measuring system ( 164 ) is loaded. In this loading step ( 41 ) it can also be a design step in which the structure and construction of the second ultrasonic measuring system ( 164 ) is defined in such a way that it functions as an ultrasonic measuring system 164 ) with such a memory.

That by means of this loading step ( 41 ) trained first neural network model ( 16 ) of the second ultrasonic measuring system ( 164 ) is then finally located in the vehicle at a typically later point in time, preferably on said digital memory, s. 3 , the second ultrasonic measuring system ( 164 ). Here the trained first neural network model ( 16 ) in the identification step ( 24 ) the live ultrasonic reception signals from the operation of the ultrasonic transducer arrangement ( 11 ) in vehicle operation, for example to determine the position and classify the objects. The output values of the first neural network model ( 16 ), which may include the position and the classification, are then used in the following usage step ( 25 ) from the system described as before then preferably used further.

Figure 7

7th shows a model for training and using the described second neural network model ( 136 ) to determine the position and classify objects in the vicinity of the vehicle using a second measuring system ( 161 ). In the installation step ( 51 ) a second neural network model ( 136 ) installed as the basis of the deep learning process. The second neural network model ( 136 ) are in a training step ( 52 ) the corresponding training received signals from measured or simulated objects are transferred as input values in as many different arrangements and positions as possible. In one identification step ( 53 ) the objects for the second neural network model ( 136 ) identified. During the training step ( 52 ) and the identification step ( 53 ) the actual learning happens. The responses of the second neural network model ( 136 ) are compared with evaluation data and the parameters of the second neural network model ( 136 ) appropriately modified. The largest possible amount of training data should also be used for this.

The trained second neural network model ( 136 ) is preferably located on a digital memory in the vehicle. Here the trained second neural network model ( 136 ) in the identification step ( 54 ) the live measurement signals from the operation of the sensor arrangement ( 131 ), for example to carry out an alternative determination of the position and an alternative classification of the objects. The output values of the second neural network model ( 136 ), which can include the alternative item and the alternative classification, for example, are then used in the subsequent use step ( 55 ) used by the described system.

Figure 8

shows a model for training and storing the parameters of the second neural network model ( 136 ).

In the installation step ( 51 ) a second neural network model ( 136 ) installed as the basis of the deep learning process. The second neural network model ( 136 ) are in training step ( 52 ) the corresponding training received signals from measured or simulated objects are transferred as input values in as many different arrangements and positions as possible. In one identification step ( 53 ) the objects for the second neural network model ( 136 ) identified. During the training step ( 52 ) and the identification step ( 53 ) the actual learning happens. The responses of the second neural network model ( 136 ) are compared with evaluation data and the parameters of the second neural network model ( 136 ) appropriately modified. Again, the largest possible amount of training data should be used for this.

The network data (parameter data) of the trained second neural network model ( 136 ) are in contrast to 7th but now in one backup step ( 65 ) saved.

Figure 9

shows a model for loading the backup step ( 65 ) stored parameters of the second neural network model ( 136 ) and then using the second neural network model ( 136 ).

In one loading step ( 71 ) are saved in the backup step ( 65 ) parameters (network data) of the second neural network model stored on a first overall measuring system ( 136 ) back into the memory of the second neural network model ( 136 ) a second overall measuring system, which is essentially the same as the first overall measuring system, is loaded. In this loading step ( 71 ) it can also be a design step in which the structure and construction of the second overall measuring system is determined in such a way that it is functionally equivalent to an overall measuring system with such a memory.

That by means of this loading step ( 71 ) trained second neural network model ( 136 ) of the second overall measuring system is then finally located at a typically later point in time in the vehicle, preferably on said digital memory of the second overall measuring system. Here the trained second neural network model ( 136 ) in the identification step ( 54 ) the live measurement signals from the operation of the sensor arrangement ( 131 ) in vehicle operation, for example to carry out an alternative determination of the alternative position and an alternative classification of the objects. The output values of the second neural network model ( 136 ), which can include the alternative item and the alternative classification, are then used in the subsequent usage step ( 55 ) from the system described as before then preferably used further.

Figure 10

shows schematically the first combination method from section E). Two processes are preferably carried out in parallel.

On the one hand there is an installation ( 51 ) of the second neural network model ( 136 ) for the second measuring system ( 161 ) on a second computer system. On the other hand, the installation is carried out in the same way ( 21st ) of the first neural network model ( 16 ) for the ultrasonic measuring system ( 164 ) on a first computer system. The first computer system and the second computer system can be identical and / or identical in terms of hardware. One or both computer systems can also be an analog circuit that carries out the arithmetic operations of the neural network models and does not necessarily have to be programmable. In the sense of this disclosure, such an installation can rather always take place in such a way that the structure and essential parameters of the respective neural network model are fixed in the construction phase, for example in the form of a suitable electronic circuit. It can also be, for example, an analog or hybrid computer which, in the form of an electrical circuit, firmly specifies the structure and / or parameterization of the respective neural network model in parts. A joint installation of the neural network models ( 16 , 136 , 138 ) in the form of pure software on a fully programmable computer system in Harvard and / or Von-Neumann architecture is preferable and preferred.

Typically this is followed by loading ( 71 ) of previously saved parameters of the second neural network model ( 136 ) for the second measuring system ( 161 ) from the second training data memory ( 137 ) as current parameters of the second neural network model ( 136 ) for the second measuring system ( 161 ) and loading ( 41 ) of preferably also previously saved parameters of the first neural network model ( 16 ) for the ultrasonic measuring system ( 164 ) from the first training data memory ( 17th ) as the current parameters of the first neural network model ( 16 ). In this way, the entire measuring system is now prepared for processing signals. To process the signals, transfer now takes place ( 54 ) of live sensor signals to the trained second neural network model ( 136 ) for the second measuring system ( 161 ) and implementation of an object identification by the trained second neural network model ( 136 ) to generate a second object recognition result and the transfer ( 24 ) of live ultrasound signals to the trained first neural network model for the ultrasound measurement system ( 164 ) in the vehicle and carrying out an object identification using the trained first neural network model to generate a first object recognition result.

The live ultrasound signals differ from the ultrasound signals with which the parameters of the first neural network model ( 16 ) were determined by the fact that they are now in the actual application by the ultrasonic transducer arrangement ( 11 ) can be determined in the target operation, for example in the consumer's vehicle while the vehicle is being used, and not in the laboratory or during test drives.

The live sensor signals differ from the sensor signals with which the parameters of the second neural network model ( 136 ) were determined by the fact that they are now in the actual application by the sensor arrangement ( 131 ) can be determined in the target operation, for example in the consumer's vehicle while the vehicle is being used, and not in the laboratory or during test drives.

Finally a comparison is made ( 84 ) the first object recognition results generated by the first neural network model ( 16 ) for the ultrasonic measuring system ( 164 ) carried out first object identification with the alternative second object recognition results of the second neural network model ( 136 ) for the second measuring system ( 161 ) carried out alternative second object identification and the determination ( 84 ) a common object recognition hypothesis, which represents the recognition result of the overall measurement system. The common object recognition hypothesis can be a list of objects with recognition probabilities and arrangements of such objects with recognition probabilities. It can also contain only one object and / or only one arrangement. A probability specification does not necessarily have to be contained in the common object recognition hypothesis.

• Es erfolgt eine erste Parameteranpassung (86) des ersten neuronalen Netzwerkmodells (16) für das Ultraschallmesssystem (164) in Abhängigkeit vom Ergebnis des Vergleichs (84) der ersten Objekterkennungsergebnisse der durch das erste neuronale Netzwerkmodell (16) für das Ultraschallmesssystem (164) durchgeführten ersten Objektidentifizierung mit den alternativen zweiten Objekterkennungsergebnissen der durch das zweite neuronale Netzwerkmodell (136) für das zweite Messsystem (161) durchgeführten alternativen zweiten Objektidentifizierung. Hierbei wird die Parameteranpassung in der Regel so durchgeführt, dass eine Nichtberücksichtigung von Teilen der ersten Objekterkennungsergebnisse des ersten neuronalen Netzwerkmodells (16) in der gemeinsamen Objekterkennungshypothese durch das dritte neuronale Netzwerkmodell (138), das den Vergleich (84) ausführt, zu einer Dämpfung der entsprechenden Parameter des ersten neuronalen Netzwerkmodells (16) führt und dass eine Berücksichtigung von Teilen der ersten Objekterkennungsergebnisse des ersten neuronalen Netzwerkmodells (16) in der gemeinsamen Objekterkennungshypothese durch das dritte neuronale Netzwerkmodell (138), das den Vergleich (84) ausführt, zu einer Verstärkung der entsprechenden Parameter des ersten neuronalen Netzwerkmodells (16) führt.

The learning process is now carried out as follows:

• A first parameter adjustment takes place ( 86 ) of the first neural network model ( 16 ) for the ultrasonic measuring system ( 164 ) depending on the result of the comparison ( 84 ) the first object recognition results generated by the first neural network model ( 16 ) for the ultrasonic measuring system ( 164 ) carried out first object identification with the alternative second object recognition results of the second neural network model ( 136 ) for the second measuring system ( 161 ) performed alternative second object identification. The parameter adjustment is usually carried out in such a way that it is not taken into account of parts of the first object recognition results of the first neural network model ( 16 ) in the common object recognition hypothesis by the third neural network model ( 138 ) that compares ( 84 ), leads to a damping of the corresponding parameters of the first neural network model ( 16 ) and that a consideration of parts of the first object recognition results of the first neural network model ( 16 ) in the common object recognition hypothesis by the third neural network model ( 138 ) that compares ( 84 ) performs, to an amplification of the corresponding parameters of the first neural network model ( 16 ) leads.

A second parameter adjustment takes place ( 87 ) of the second neural network model ( 136 ) for the second measuring system ( 161 ) depending on the result of the comparison ( 84 ) the first object recognition results generated by the first neural network model ( 16 ) for the ultrasonic measuring system ( 164 ) carried out first object identification with the alternative second object recognition results of the second neural network model ( 136 ) for the second measuring system ( 161 ) performed alternative second object identification. Here, the parameter adjustment is usually carried out in such a way that parts of the second object recognition results of the second neural network model ( 136 ) in the common object recognition hypothesis by the third neural network model ( 138 ) that compares ( 84 ), leads to a damping of the corresponding parameters of the second neural network model ( 136 ) and that taking into account parts of the second object recognition results of the second neural network model ( 136 ) in the common object recognition hypothesis by the third neural network model ( 138 ) that compares ( 84 ) performs, to an amplification of the corresponding parameters of the second neural network model ( 136 ) leads.

The third neural network model ( 138 ) that compares ( 84 ), thus makes a decision.

The last step is to use ( 85 ) the determined common object recognition hypothesis.

Figure 11

shows a model for training and using the third neural network model ( 138 ).

The procedure starts with the installation ( 91 ) of the third neural network model ( 138 ) for the comparison of the first object recognition results of the first neural network model ( 16 ) for the ultrasonic measuring system ( 164 ) carried out first object identification with the alternative second object recognition results of the second neural network model ( 136 ) for the second measuring system ( 161 ) carried out alternative second object identification and for the determination of a common object recognition hypothesis on a third computer system. The third computer system can with the first and second computer system of 10 be identical.

As already mentioned above, the computer systems can also be one or more analog circuits that carry out the arithmetic operations of the neural network models and do not necessarily have to be programmable. For the purposes of this disclosure, such an installation ( 91 ) rather always take place in such a way that the structure and essential parameters of the respective neural network model, for example in the form of a suitable electronic circuit, are firmly specified in the construction phase. It can also be, for example, an analog or hybrid computer which, in the form of an electrical circuit, firmly specifies the structure and / or parameterization of the respective neural network model in parts. An installation ( 91 ) of the neural network models ( 16 , 136 , 138 ) in the form of pure software on a fully programmable computer system in Harvard and / or Von-Neumann architecture is preferable and preferred.

Training follows ( 92 ) of the third neural network model ( 138 ) by transferring correct and incorrect first object recognition results and correct and incorrect alternative second object recognition results to the third neural network model ( 138 ) together with data about their correctness and / or incorrectness and determination of the parameters of the third neural network model ( 138 ) depending on this data and / or changing the parameters of the third neural network model ( 138 ) depending on this data. This can be done, for example, in such a way that only the first neural network model ( 16 ) is fully trained with laboratory data and the second neural network model ( 136 ) is fully trained with laboratory data. Then, laboratory data sets that correspond to one another are sent as training data sets to the first neural network model ( 16 ) and the second neural network model ( 136 ) presents. These generate a first object recognition result and a second object recognition result for each of these pairs of training data sets. A target object recognition hypothesis is then calculated for each of these, either manually or automatically, or otherwise determined. This then results in a Training data set for the third neural network model ( 138 ), consisting of the respective first object recognition result and the respective second object recognition result and the respective target object recognition hypothesis, which represents the corresponding evaluation date.

With this data, the third neural network model ( 138 ) trained ( 92 ) become.

Is the training ( 92 ) has been successfully completed, the system (eg a laboratory device) or a sufficiently functionally similar device (eg a series product) can be used in a vehicle, for example. There the transfer takes place ( 93 ) of first live object recognition results and of alternative second live object recognition results to the trained third neural network model ( 138 ).

First live object recognition results differ from first object recognition results in that they were generated with first live ultrasound signals, while first object recognition results can be generated, for example, with laboratory and / or simulation data.

Second live object recognition results differ from second object recognition results in that they were generated with live sensor signals, while second object recognition results can be generated, for example, with laboratory and / or simulation data.

The live ultrasound signals differ from the ultrasound signals with which the parameters of the first neural network model ( 16 ) were determined by the fact that they are now in the actual application by the ultrasonic transducer arrangement ( 11 ) can be determined in the target operation, for example in the consumer's vehicle while the vehicle is being used, and not in the laboratory or during test drives.

The live sensor signals differ from the sensor signals with which the parameters of the second neural network model ( 136 ) were determined by the fact that they are now in the actual application by the sensor arrangement ( 131 ) can be determined in the target operation, for example in the consumer's vehicle while the vehicle is being used, and not in the laboratory or during test drives.

The third neural network model ( 138 ) then the determination takes place ( 94 ) a common object recognition hypothesis based on the first live object recognition results and the alternative second live object recognition results by the third neural network model ( 138 ).

Finally follows the usage ( 95 ) through the third neural network model ( 138 ) common object recognition hypothesis determined for the overall measurement system. This can be done, for example, by warning messages to the vehicle driver, etc.

Figure 12

shows a model for training and storing the parameters of the third neural network model ( 138 ).

The procedure starts with the installation ( 91 ) of the third neural network model ( 138 ) for the comparison of the first object recognition results of the first neural network model ( 16 ) for the ultrasonic measuring system ( 164 ) carried out first object identification with the alternative second object recognition results of the second neural network model ( 136 ) for the second measuring system ( 161 ) carried out alternative second object identification and for the determination of a common object recognition hypothesis on a third computer system. The third computer system can with the first and second computer system of 10 be identical.

As already mentioned above, the computer systems can also be one or more analog circuits that carry out the arithmetic operations of the neural network models and do not necessarily have to be programmable. For the purposes of this disclosure, such an installation ( 91 ) rather always take place in such a way that the structure and essential parameters of the respective neural network model, for example in the form of a suitable electronic circuit, are firmly specified in the construction phase. It can also be, for example, an analog or hybrid computer which, in the form of an electrical circuit, firmly specifies the structure and / or parameterization of the respective neural network model in parts. An installation ( 91 ) in the form of pure software on a fully programmable computer system in Harvard and / or Von-Neumann architecture is preferable and preferred.

Training follows ( 91 ) of the third neural network model ( 138 ) by transferring correct and incorrect first object recognition results and correct and incorrect alternative second object recognition results to the third neural network model ( 138 ) together with data about their correctness and / or incorrectness and determination of the parameters of the third neural network model ( 138 ) depending on this data and / or changing the parameters of the third neural network model ( 138 ) depending on this data. This can for example, so that only the first neural network model ( 16 ) is fully trained with laboratory data and the second neural network model ( 136 ) is fully trained with laboratory data. Then, laboratory data sets that correspond to one another are sent as training data sets to the first neural network model ( 169 and the second neural network model ( 136 ) presents. These generate a first object recognition result and a second object recognition result for each of these pairs of training data sets. A target object recognition hypothesis is then calculated manually or automatically from these or otherwise determined. This then results in a training data set for the third neural network model ( 136 ) consisting of the respective first object recognition result and the respective second object recognition result and the respective target object recognition hypothesis, which represents the corresponding evaluation date.

With this data, the third neural network model ( 138 ) trained ( 92 ) become.

If the training has been successfully completed, the system (eg a laboratory device) or a sufficiently functionally similar device (eg a series product) can be used in a vehicle, for example. There the transfer takes place ( 93 ) of first live object recognition results and of alternative second live object recognition results to the trained third neural network model ( 138 ).

First live object recognition results differ from first object recognition results in that they were generated with first live ultrasonic signals, while first object recognition results can be generated, for example, with laboratory and / or simulation data.

Second live object recognition results differ from second object recognition results in that they were generated with live sensor signals, while second object recognition results can be generated, for example, with laboratory and / or simulation data.

The live ultrasound signals differ from the ultrasound signals with which the parameters of the first neural network model ( 16 ) were determined by the fact that they are now in the actual application by the ultrasonic transducer arrangement ( 11 ) can be determined in the target operation, for example in the consumer's vehicle while the vehicle is being used, and not in the laboratory or during test drives.

The live sensor signals differ from the sensor signals with which the parameters of the second neural network model ( 136 ) were determined by the fact that they are now in the actual application by the sensor arrangement ( 131 ) can be determined in the target operation, for example in the consumer's vehicle while the vehicle is being used, and not in the laboratory or during test drives.

In contrast to 11 closes the procedure of 12th now with the backup ( 105 ) the parameter data of the trained third neural network ( 138 ) in a third training data memory ( 139 ) from. This allows the parameters of the third neural network model ( 138 ) are used on functionally similar systems by programming and / or construction of these systems or parts of them.

Figure 13

shows a model for loading the stored parameters of the third neural network model ( 138 ) and then using the third neural network model ( 138 ).

There are those in the process of 12th create data used. Thus the procedure begins with loading ( 111 ) the previously saved parameters of the third neural network model ( 138 ) from the third training data memory ( 139 ) or another data storage device that currently holds this data in the third neural network model ( 138 ). This third neural network model ( 138 ) are in a completely different, functionally similar overall system than the overall system with which the data to be loaded was created.

The third neural network model ( 138 ) then the determination takes place ( 94 ) a common object recognition hypothesis based on the first live object recognition results and the alternative second live object recognition results by the third neural network model ( 138 ).

Finally follows the usage ( 95 ) through the third neural network model ( 138 ) common object recognition hypothesis determined for the overall measurement system. This can be done, for example, by warning messages to the vehicle driver, etc.

Figure 14

shows a model for training and using the fifth neural network model ( 158 ).

The procedure starts with the installation ( 201 ) of the fifth neural network model ( 158 ) for comparing the second Object recognition results obtained by the second neural network model ( 136 ) for the second measuring system ( 161 ) carried out second object identification with the third object recognition results of the fourth neural network model ( 156 ) for the third measuring system ( 162 ) carried out alternative third object identification and for the determination of a common object recognition hypothesis on a fourth computer system. The fourth computer system can with the first and / or second and / or third computer system of 10 be identical.

As already mentioned above, the computer systems can also be one or more analog circuits that carry out the arithmetic operations of the neural network models and do not necessarily have to be programmable. For the purposes of this disclosure, such an installation ( 201 ) rather always take place in such a way that the structure and essential parameters of the respective neural network model, for example in the form of a suitable electronic circuit, are firmly specified in the construction phase. It can also be, for example, an analog or hybrid computer which, in the form of an electrical circuit, firmly specifies the structure and / or parameterization of the respective neural network model in parts. An installation ( 201 ) of the neural network models ( 16 , 136 , 138 , 156 , 158 ) in the form of pure software on a fully programmable computer system in Harvard and / or Von-Neumann architecture is preferable and preferred.

Training follows ( 202 ) of the fifth neural network model ( 158 ) by transferring correct and incorrect second object recognition results and correct and incorrect alternative third object recognition results to the fifth neural network model ( 158 ) together with data about their correctness and / or incorrectness and determination of the parameters of the fifth neural network model ( 158 ) depending on this data and / or change in the parameters of the fifth neural network model ( 158 ) depending on this data. This can happen, for example, that only the second neural network model ( 136 ) is fully trained with laboratory data and only the fourth neural network model ( 156 ) is fully trained with laboratory data. Then corresponding laboratory data sets are sent to the second neural network model as training data sets ( 136 ) and the fourth neural network model ( 156 ) presents. These generate a second object recognition result and a third object recognition result for each of these pairs of training data sets. A target object recognition hypothesis is then calculated for each of these, either manually or automatically, or otherwise determined. This then results in a training data set for the fifth neural network model ( 158 ) consisting of the respective first object recognition result and the respective second object recognition result and the respective target object recognition hypothesis, which represents the corresponding evaluation date.

With this data, the fifth neural network model ( 158 ) trained ( 202 ) become.

Is the training ( 202 ) has been successfully completed, the system (eg a laboratory device) or a sufficiently functionally similar device (eg a series product) can be used in a vehicle, for example. There the transfer takes place ( 203 ) of second live object recognition results and of alternative third live object recognition results to the trained fifth neural network model ( 158 ).

Second live object detection results differ from second object detection results in that they are based on live sensor signals from the second sensor arrangement ( 131 ) were generated, while second object recognition results can be generated for example with laboratory and / or simulation data.

Third live object detection results differ from third object detection results in that they are linked to third live sensor signals from the third sensor arrangement ( 151 ) were generated, while third object recognition results can be generated with laboratory and / or simulation data, for example.

The second live sensor signals of the second sensor arrangement ( 131 ) differ from the second sensor signals of the second sensor arrangement ( 131 ), with which the parameters of the second neural network model ( 136 ) were determined by the fact that they are now in the actual application by the second sensor arrangement ( 131 ) can be determined in the target operation, for example in the consumer's vehicle while the vehicle is being used, and not in the laboratory or during test drives.

The third live sensor signals differ from the third sensor signals of the third sensor arrangement ( 151 ), with which the parameters of the fourth neural network model ( 136 ) were determined by the fact that they are now in the actual application by the third sensor arrangement ( 151 ) can be determined in the target operation, for example in the consumer's vehicle while the vehicle is being used, and not in the laboratory or during test drives.

The fifth neural network model ( 158 ) then the determination takes place ( 204 ) a common object recognition hypothesis based on the second live object recognition results and the third live object recognition results by the fifth neural network model ( 158 ).

Finally follows the usage ( 205 ) which by the fifth neural network model ( 158 ) common object recognition hypothesis determined for the overall measurement system. This can be done, for example, by warning messages to the vehicle driver, etc.

Figure 15

shows a model for training and storing the parameters of the fifth neural network model ( 158 ).

The procedure starts with the installation ( 201 ) of the fifth neural network model ( 158 ) for the comparison of the second object recognition results of the second neural network model ( 136 ) for the second measuring system ( 161 ) carried out second object identification with the third object recognition results of the fourth neural network model ( 156 ) for the third measuring system ( 162 ) carried out alternative third object identification and for the determination of a common object recognition hypothesis on a fourth computer system. The fourth computer system can with the first and / or second and / or third computer system of 10 be identical.

As already mentioned above, the computer systems can also be one or more analog circuits that carry out the arithmetic operations of the neural network models and do not necessarily have to be programmable. For the purposes of this disclosure, such an installation ( 201 ) rather always take place in such a way that the structure and essential parameters of the respective neural network model, for example in the form of a suitable electronic circuit, are firmly specified in the construction phase. It can also be, for example, an analog or hybrid computer which, in the form of an electrical circuit, firmly specifies the structure and / or parameterization of the respective neural network model in parts. An installation ( 201 ) of the neural network models ( 16 , 136 , 138 , 156 , 158 ) in the form of pure software on a fully programmable computer system in Harvard and / or Von-Neumann architecture is preferable and preferred.

Training follows ( 202 ) of the fifth neural network model ( 158 ) by transferring correct and incorrect second object recognition results and correct and incorrect alternative third object recognition results to the fifth neural network model ( 158 ) together with data about their correctness and / or incorrectness and determination of the parameters of the fifth neural network model ( 158 ) depending on this data and / or change in the parameters of the fifth neural network model ( 158 ) depending on this data. This can happen, for example, that only the second neural network model ( 136 ) is fully trained with laboratory data and only the fourth neural network model ( 156 ) is fully trained with laboratory data. Then corresponding laboratory data sets are sent to the second neural network model as training data sets ( 136 ) and the fourth neural network model ( 156 ) presents. These generate a second object recognition result and a third object recognition result for each of these pairs of training data sets. A target object recognition hypothesis is then calculated for each of these, either manually or automatically, or otherwise determined. This then results in a training data set for the fifth neural network model ( 158 ) consisting of the respective first object recognition result and the respective second object recognition result and the respective target object recognition hypothesis, which represents the corresponding evaluation date.

With this data, the fifth neural network model ( 158 ) trained ( 202 ) become.

Is the training ( 202 ) has been successfully completed, the system (eg a laboratory device) or a sufficiently functionally similar device (eg a series product) can be used in a vehicle, for example. The transfer, which is only optional here, then takes place there ( 203 ) of second live object recognition results and of alternative third live object recognition results to the trained fifth neural network model ( 158 ). This optional step is used here for learning in the real application.

Second live object detection results differ from second object detection results in that they are based on live sensor signals from the second sensor arrangement ( 131 ) were generated while second object recognition results for example, can be generated with laboratory and / or simulation data.

Third live object detection results differ from third object detection results in that they are linked to third live sensor signals from the third sensor arrangement ( 151 ) were generated, while third object recognition results can be generated with laboratory and / or simulation data, for example.

The second live sensor signals of the second sensor arrangement ( 131 ) differ from the second sensor signals of the second sensor arrangement ( 131 ), with which the parameters of the second neural network model ( 136 ) were determined by the fact that they are now in the actual application by the second sensor arrangement ( 131 ) can be determined in the target operation, for example in the consumer's vehicle while the vehicle is being used, and not in the laboratory or during test drives.

The third live sensor signals differ from the third sensor signals of the third sensor arrangement ( 151 ), with which the parameters of the fourth neural network model ( 136 ) were determined by the fact that they are now in the actual application by the third sensor arrangement ( 151 ) can be determined in the target operation, for example in the consumer's vehicle while the vehicle is being used, and not in the laboratory or during test drives.

In contrast to 14th closes the procedure of 15th now with the backup ( 215 ) the parameter data of the trained fifth neural network ( 158 ) in a fifth training data memory ( 159 ) from. This allows the parameters of the fifth neural network model ( 158 ) are used in functionally similar systems by programming and / or construction of these systems or parts of them.

Figure 16

shows a model for loading the stored parameters of the fifth neural network model ( 158 ) and then using the fifth neural network model ( 158 ).

There are those in the process of 15th create data used. Thus the procedure begins with loading ( 211 ) the previously saved parameters of the fifth neural network model ( 158 ) from the fifth training data memory ( 159 ) or another data storage device that currently holds this data in the fifth neural network model ( 158 ). This fifth neural network model ( 158 ) are in a completely different, functionally similar overall system than the overall system with which the data to be loaded was created.

The fifth neural network model ( 158 ) then the determination takes place ( 204 ) a common object recognition hypothesis based on the second live object recognition results and the third live object recognition results by the fifth neural network model ( 158 ).

Finally follows the usage ( 205 ) which by the fifth neural network model ( 158 ) common object recognition hypothesis determined for the overall measurement system. This can be done, for example, by warning messages to the vehicle driver, etc.

Figure 17

shows schematically the second combination method from section F).

First, there is an installation ( 51 ) a second neural network model ( 136 ) for the second measuring system ( 161 ) on a second computer system.

Second, there is an installation ( 61 ) a fourth neural network model ( 156 ) for the third measuring system ( 162 ) on a fourth computer system.

Thirdly, there is an installation ( 201 ) of a fifth neural network model ( 158 ) on a fifth computer system.

The second computer system and / or the fourth computer system and / or the fifth computer system can be identical and / or identical in terms of hardware. One, several or all computer systems can also be an analog circuit that carries out the arithmetic operations of the neural network models ( 136 , 156 , 158 ) and does not necessarily have to be programmable. In the sense of this disclosure, such an installation can rather always take place in such a way that the structure and essential parameters of the respective neural network model are fixed in the construction phase, for example in the form of a suitable electronic circuit. It can also be, for example, an analog or hybrid computer which, in the form of an electrical circuit, firmly specifies the structure and / or parameterization of the respective neural network model in parts. An installation in the form of pure software on a fully programmable computer system in Harvard and / or Von Neumann architecture is preferable and preferred.

Typically this is followed by loading ( 71 ) of previously saved parameters of the second neural network model ( 136 ) for the second measuring system ( 161 ) from the second training data memory ( 137 ) as current parameters of the second neural network model ( 136 ) for the second measuring system ( 161 ).

Typically this is followed by loading ( 72 ) of previously saved parameters of the fourth neural network model ( 156 ) for the third measuring system ( 162 ) from the fourth training data memory ( 157 ) as the current parameters of the fourth neural network model ( 156 ) for the third measuring system ( 162 ).

Typically this is followed by loading ( 211 ) of previously saved parameters of the fifth neural network model ( 158 ) from the fifth training data memory ( 159 ) as the current parameters of the fifth neural network model ( 158 ).

In this way, the entire measuring system is now prepared for processing signals.

In order to process the signals, the first step is a transfer ( 54 ) of second live sensor signals of the second sensor arrangement ( 131 ) to the trained second neural network model ( 136 ) for the second measuring system ( 161 ) and secondly a handover ( 64 ) of third live sensor signals of the third sensor arrangement ( 151 ) to the trained fourth neural network model ( 1356 ) for the third measuring system ( 162 )

A second object identification is carried out by the trained second neural network model ( 136 ) to generate a second object recognition result and to carry out a third object identification by the trained fourth neural network model ( 156 ) to generate a third object recognition result.

The second live sensor signals of the second sensor arrangement ( 131 ) differ from the second sensor signals of the second sensor arrangement ( 131 ) with which the parameters of the second neural network model ( 136 ) were determined by the fact that they are now in the actual application by the second sensor arrangement ( 131 ) can be determined in the target operation, for example in the consumer's vehicle while the vehicle is being used, and not in the laboratory or during test drives.

The third live sensor signals of the third sensor arrangement ( 151 ) differ from the third sensor signals of the third sensor arrangement ( 151 ) with which the parameters of the fourth neural network model ( 156 ) were determined by the fact that they are now in the actual application by the third sensor arrangement ( 151 ) can be determined in the target operation, for example in the consumer's vehicle while the vehicle is being used, and not in the laboratory or during test drives.

The handover follows ( 203 ) of first live object recognition results and of alternative second live object recognition results to the trained third neural network model ( 138 )

Finally a comparison is made ( 204 ) the second object recognition results obtained by the second neural network model ( 136 ) for the second measuring system ( 161 ) carried out second object identification with the third object recognition results of the fourth neural network model ( 156 ) for the third measuring system ( 162 ) carried out third object identification and the determination ( 204 ) a common object recognition hypothesis, which represents the recognition result of the overall measurement system. The common object recognition hypothesis can be a list of objects with recognition probabilities and arrangements of such objects with recognition probabilities. It can also contain only one object and / or only one arrangement. A probability specification does not necessarily have to be contained in the common object recognition hypothesis.

• Es erfolgt eine zweite Parameteranpassung (87) des zweiten neuronalen Netzwerkmodells (136) für das zweite Messsystem (161) in Abhängigkeit vom Ergebnis des Vergleichs (204) der zweiten Objekterkennungsergebnisse der durch das zweite neuronale Netzwerkmodell (136) für das zweite Messsystem (161) durchgeführten zweiten Objektidentifizierung mit den dritten Objekterkennungsergebnissen der durch das vierte neuronale Netzwerkmodell (156) für das dritte Messsystem (162) durchgeführten alternativen dritten Objektidentifizierung. Hierbei wird die Parameteranpassung in der Regel so durchgeführt, dass eine Nichtberücksichtigung von Teilen der zweiten Objekterkennungsergebnisse des zweiten neuronalen Netzwerkmodells (136) in der gemeinsamen Objekterkennungshypothese durch das fünfte neuronale Netzwerkmodell (158), das den Vergleich (204) ausführt, zu einer Dämpfung der entsprechenden Parameter des zweiten neuronalen Netzwerkmodells (136) führt und dass eine Berücksichtigung von Teilen der zweiten Objekterkennungsergebnisse des zweiten neuronalen Netzwerkmodells (136) in der gemeinsamen Objekterkennungshypothese durch das fünfte neuronale Netzwerkmodell (158), das den Vergleich (204) ausführt, zu einer Verstärkung der entsprechenden Parameter des zweiten neuronalen Netzwerkmodells (136) führt.

The learning process is now carried out as follows:

• A second parameter adjustment takes place ( 87 ) of the second neural network model ( 136 ) for the second measuring system ( 161 ) depending on the result of the comparison ( 204 ) The second Object recognition results obtained by the second neural network model ( 136 ) for the second measuring system ( 161 ) carried out second object identification with the third object recognition results of the fourth neural network model ( 156 ) for the third measuring system ( 162 ) carried out alternative third object identification. Here, the parameter adjustment is usually carried out in such a way that parts of the second object recognition results of the second neural network model ( 136 ) in the common object recognition hypothesis by the fifth neural network model ( 158 ) that compares ( 204 ), leads to a damping of the corresponding parameters of the second neural network model ( 136 ) and that taking into account parts of the second object recognition results of the second neural network model ( 136 ) in the common object recognition hypothesis by the fifth neural network model ( 158 ) that compares ( 204 ) performs, to an amplification of the corresponding parameters of the second neural network model ( 136 ) leads.

A fourth parameter adjustment takes place ( 88 ) of the fourth neural network model ( 156 ) for the third measuring system ( 162 ) depending on the result of the comparison ( 204 ) the second object recognition results obtained by the second neural network model ( 136 ) for the first measuring system ( 161 ) performed second object identification with the alternative third object recognition results of the fourth neural network model ( 156 ) for the third measuring system ( 162 ) carried out alternative third object identification. Here, the parameter adjustment is usually carried out in such a way that parts of the third object recognition results of the fourth neural network model ( 156 ) in the common object recognition hypothesis by the fifth neural network model ( 158 ) that compares ( 204 ), leads to a damping of the corresponding parameters of the fourth neural network model ( 1356 ) and that consideration of parts of the third object recognition results of the fourth neural network model ( 156 ) in the common object recognition hypothesis by the fifth neural network model ( 158 ) that compares ( 204 ), to an amplification of the corresponding parameters of the fourth neural network model ( 156 ) leads.

The fifth neural network model ( 158 ) that compares ( 204 ), thus makes a decision.

The usage ( 205 ) which by the fifth neural network model ( 158 ) the common object recognition hypothesis determined for the overall measurement system concludes the process.

Figure 18

shows the structure of the proposed combined overall system from an ultrasonic measuring system ( 164 ) and a second measuring system ( 161 ) for the corresponding first combination procedure according to Section E).

• Die Ultraschallwandleranordnung (11) sendet die akustischen Ultraschallsignale aus und empfängt die Echos der zu detektierenden Objekte im Umfeld des Fahrzeugs. Die Ultraschallwandleranordnung wandelt diese empfangenen Echosignale und sonstigen empfangenen Ultraschallsignale in Ultraschallmesswerte um und stellt diese einem Signalprozessor (15) zur Verfügung, der diese weiterverarbeitet. Der Signalprozessor (15) kann ein digitaler programmierbarer Prozessor in von Neumann oder Havard-Architektur oder in einer anderen geeigneten Architektur sein. Es kann sich aber auch um eine geeignete analoge Schaltung zur Vorverarbeitung der Signale handeln. Im Falle analoger neuronale Netze kann der Prozessor ggf. auch weggelassen werden, was sich in der Praxis aber bisher nicht bewährt hat.

The first group of components is the ultrasonic measuring system ( 164 ):

• The ultrasonic transducer arrangement ( 11 ) sends out the acoustic ultrasonic signals and receives the echoes of the objects to be detected in the vicinity of the vehicle. The ultrasonic transducer arrangement converts these received echo signals and other received ultrasonic signals into ultrasonic measured values and presents them to a signal processor ( 15th ) is available to process them further. The signal processor ( 15th ) can be a digital programmable processor in von Neumann or Havard architecture or in another suitable architecture. However, it can also be a suitable analog circuit for preprocessing the signals. In the case of analog neural networks, the processor can optionally also be omitted, but this has so far not proven itself in practice.

The signals from the signal processor ( 15th ) processed received signals are the input signals of the first neural network model ( 16 ), which is typically stored on a digital memory of a computer. The first neural network model ( 16 ) can also be implemented, for example, as a special microelectronic circuit or other functionally equivalent device. The neural network model ( 16 ) is the implementation of a deep learning process, as described above.

The parameters of the first neural network model ( 16 ) can from a first training data memory ( 17th ) Loading. The training data are preferably determined in the laboratory by a method as described above.

The ultrasonic transducer arrangement ( 11 ) is equipped with control electronics ( 12th ), which are optionally used for beam shaping, for example by means of phase shifting of the transmitted ultrasonic signals of the individual ultrasonic transducers of the ultrasonic transducer arrangement ( 11 ) can be capable. The control electronics ( 12th ) controls the corresponding ultrasonic transducers of the ultrasonic transducer arrangement ( 11 ) for preferably directed transmission or reception of the ultrasonic signals. This alignment is desirable, but not mandatory. The emission of ultrasonic signals by the ultrasonic transducer arrangement ( 11 ) is controlled by a transmitter control ( 13th ). This is with the control electronics ( 12th ) and sets the transmission parameters of the ultrasonic transducers of the ultrasonic transducer arrangement ( 11 ), depending on the settings made via the optional user settings stored in an optional memory ( 14th ), or by a higher-level control unit. Send parameters, which are from the transmitter control ( 13th ) can be controlled, for example, burst length, frequency, frequency curve, coding, polarity, phase or amplitude.

The output values of the first neural network model ( 16 ) are preferably the positions of the recognized objects and preferably their classification, as well as, for example, a corresponding confidence value that can be assigned to the recognized recognition result as a whole, and / or confidence values that are assigned to the individual objects. These data form the first object recognition result. Further data can be part of the output values. Hypothesis lists that are alternative to one another are also possible, the hypothesis list having alternative recognition results with different confidence values for these first ones Includes object recognition results. Each first object recognition result comprises one or more objects.

The ultrasonic measuring system described above ( 164 ) preferably further comprises a plurality of first user controls ( 10 ), whereby setting one or more of the first user controls ( 10 ) takes place as a reaction to the identification of an arrangement of objects and / or object components or of objects in the first ultrasonic measurement values and / or in the two- or three-dimensional representations and / or as a function of the first object recognition results.

With the ultrasonic measuring system described above ( 164 ) preferably the setting or the adjustment of control parameters of one or more of the controls takes place in response to the identification of the arrangement of objects and / or object components in the first ultrasonic measured values and / or in the two- or three-dimensional representations and / or as a function of the first Object recognition results. One or more of the controls are preferably set here ( 13th ) of the ultrasonic measuring system or the vehicle as a reaction to the identification of objects and / or object components and / or the arrangement of objects and / or objects and / or object components in the first ultrasound measurement values and / or in the two- or three-dimensional representations and / or in Dependence on the first object recognition results.

The ultrasonic transducer arrangement ( 11 ) is equipped with control electronics ( 12th ), which are optionally used for beam shaping, for example by means of phase shifting of the transmitted ultrasonic signals of the individual ultrasonic transducers of the ultrasonic transducer arrangement ( 11 ) can be capable. The control electronics ( 12th ) controls the corresponding ultrasonic transducers of the ultrasonic transducer arrangement ( 11 ) for preferably directed transmission or reception of the ultrasonic signals. This alignment is desirable, but not mandatory. The emission of ultrasonic signals by the ultrasonic transducer arrangement ( 11 ) is controlled by a transmitter control ( 13th ). This is with the control electronics ( 12th ) and sets the transmission parameters of the ultrasonic transducers of the ultrasonic transducer arrangement ( 11 ), depending on the settings made via the optional user settings stored in an optional memory ( 14th ), or by a higher-level control unit. Send parameters, which are from the transmitter control ( 13th ) can be controlled, for example, burst length, frequency, frequency curve, coding, polarity, phase or amplitude.

• Hier wird davon ausgegangen, dass das zweite Messsystem ebenfalls Messsignale aussendet, deren Echos empfangen werden. Dabei kann es sich beispielsweise um eine LIDAR- oder RADAR-System handeln. Dies ist jedoch nur ein Beispiel. Es kann sich auch um ein anderes Messsystem, wie beispielsweise ein PIR-Infrarotmesssystem handeln, das auf ohnehin im Umfeld des Fahrzeugs befindliche Signale zurückgreift. PIR steht hierbei für „passiv infra red“ (Deutsch: passives Infrarot).

A second group of components forms the second measuring system ( 161 ):

• It is assumed here that the second measuring system also sends out measuring signals whose echoes are received. This can be a LIDAR or RADAR system, for example. However, this is just an example. It can also be a different measuring system, such as a PIR infrared measuring system, which makes use of signals that are already in the vicinity of the vehicle. PIR stands for “passive infra red” (German: passive infrared).

If applicable, the second sensor arrangement sends ( 131 ) sends the measurement signals into the surroundings of the vehicle and receives the response from the surroundings, specifically in particular the signal response of the objects to be detected or characterized in the surroundings of the vehicle to these emitted measurement signals. The sensor arrangement ( 131 ) converts these received signal responses and other received interference and useful signal signals into measured signal values and presents them to a second signal processor ( 135 ) is available to process them further. The second signal processor ( 135 ) can again be a digital programmable processor in von Neumann or Havard architecture or in another suitable architecture. However, it can also be a suitable analog circuit for preprocessing the signals. In the case of analog neural networks, the processor can optionally also be omitted, but this has so far not proven itself in practice. In the case of a programmable processor, the second signal processor ( 135 ) for example with the first signal processor ( 15th ) and / or the said computer form a unit.

The signals from the second signal processor ( 135 ) processed received signals of the sensor arrangement ( 131 ) are the input signals of the second neural network model ( 136 ), which is typically on a digital memory of a second computer that is connected to the first computer and / or the signal processor ( 15th ) and / or the second signal processor ( 135 ) can be identical, is stored. The second neural network model ( 136 ) can also be implemented, for example, as a special microelectronic circuit or other functionally equivalent device. The second neural network model ( 136 ) It is preferred to implement a deep learning process, as described above.

The parameters of the second neural network model ( 136 ) can from a second training data memory ( 137 ) are given preference. The training data are preferably determined in the laboratory by a method as described above.

The second sensor arrangement ( 131 ) is equipped with a second control electronics ( 132 ), which in the case of a LIDAR or radar system can optionally be used for beam shaping, for example by means of phase shifting of the measurement signals transmitted by the individual sensor elements of the sensor arrangement ( 131 ) can be capable. The second control electronics ( 132 ) controls the corresponding sensor elements of the sensor arrangement ( 11 ) for preferably directed transmission or reception of the measurement signals. In the case of a RADAR system, this can be done using phase antennas, for example. This alignment is desirable, but not mandatory. The transmission of measurement signals by the sensor arrangement ( 131 ) is controlled by a second sensor control ( 133 ). This is connected to the second control electronics ( 132 ) and provides the transmission parameters of the sensor elements of the sensor arrangement ( 131 ), depending on the settings that are stored in an optional second settings memory ( 134 ), or by a higher-level control unit. Send parameters that are sent by the second sensor control ( 133 ) can be controlled, in the case of a RADAR system as a second measuring system ( 161 ) For example burst length, frequency, frequency curve, coding, polarity, phase or amplitude.

The output values of the second neural network model ( 136 ) are preferably the positions of the recognized objects and preferably their classification, as well as, for example, a corresponding confidence value that can be assigned to the recognized recognition result as a whole, and / or confidence values that are assigned to the individual objects. These data form the second object recognition result. Further data can be part of the output values. Alternative hypothesis lists are also possible, the hypothesis list including alternative recognition results with different confidence values for these second object recognition results. Every second object recognition result includes one or more objects.

The previously described second measuring system ( 161 ) preferably further comprises a multiplicity of second user controls ( 140a) setting one or more of the second user controls ( 140a) in response to the identification of an arrangement of objects and / or object components or of objects in the second measured values of the second sensor arrangement ( 131 ) and / or takes place in the two- or three-dimensional representations and / or as a function of the second object recognition results.

With the previously described second measuring system ( 161 ) the setting or adjustment of control parameters of one or more of the controls is preferably carried out in response to the identification of the arrangement of objects and / or object components in the second measured values of the second sensor arrangement ( 131 ) and / or in the two- or three-dimensional representations and / or as a function of the second object recognition results. One or more of the second sensor controls ( 133 ) of the second measuring system ( 161 ) or of the vehicle in response to the identification of objects and / or object components and / or the arrangement of objects and / or objects and / or object components in the second measured values of the second sensor arrangement ( 131 ) and / or in the two- or three-dimensional representations and / or as a function of the second object recognition results.

The second sensor arrangement ( 131 ) is equipped with a second control electronics ( 132 ), which are optionally used for beam shaping, for example by means of phase shifting of the transmitted measurement signals of the individual sensor elements of the sensor arrangement 131 ) can be capable. The second control electronics ( 132 ) controls the corresponding sensor elements of the sensor arrangement ( 131 ) for preferably directed transmission or reception of the measurement signals. This alignment is desirable, but not mandatory. The transmission of measurement signals by the sensor arrangement ( 131 ) is controlled by the sensor control ( 133 ). This is connected to the second control electronics ( 132 ) and provides the transmission parameters of the sensor elements of the sensor arrangement ( 131 ), depending on the settings that are stored in an optional second settings memory ( 134 ), or by a higher-level control unit. Send parameters that are used by the sensor control ( 133 ) can be controlled, for example, burst length, frequency, frequency curve, coding, polarity, phase or amplitude.

The proposed overall sensor system also includes a third neural network model ( 138 ), which is preferably implemented on a third computer system. The third computer system can be combined with the first computer system of the first neural network model ( 16 ), and / or with the second computer system of the second neural network model ( 136 ) and / or the first signal processor ( 15th ) and / or the second signal processor ( 135 ) form a unit.

The first neural network model ( 16 ) forms a first object recognition result.

The second neural network model ( 136 ) forms a second object recognition result.

The third neural network model ( 136 ) now forms a common object recognition hypothesis as a function of the first object recognition result and the second object recognition result. The third neural network model controls depending on the determined common object recognition hypothesis and the first object recognition result and the second object recognition result ( 138 ) a first parameter modification device ( 140 ) for the first neural network model ( 16 ) and a second parameter modification device ( 141 ) for the second neural network model ( 136 ).

Both the first parameter modification device ( 140 ) and the second parameter modification device ( 141 ) can be implemented on another computer system or one of the aforementioned computer systems and / or signal processors. However, they can also be special devices, for example special circuits.

The first parameter modification device ( 140 ) modified depending on the signals of the third neural network model ( 138 ) and / or of the first object recognition result and / or of the second object recognition result, the parameters of the first neural network model ( 16 ).

The second parameter modification device ( 141 ) modified depending on the signals of the third neural network model ( 138 ) and / or of the first object recognition result and / or of the second object recognition result, the parameters of the second neural network model ( 136 ). This makes the device self-learning.

For safety-relevant applications, it is necessary to prevent the third neural network model ( 138 ) and / or the other neural network models. Therefore, the proposed overall system includes a first safety device ( 142 ). The task of the first safety device ( 142 ) is to monitor the work of the overall system and to detect obviously incorrect results of the three neural networks. To do this, the first safety device ( 142 ) High-priority direct signals from the ultrasonic transducer array ( 11 ) in the form of a high-priority signal (H11) from the ultrasonic transducer arrangement ( 11 ), high-priority signals (H15) of the first signal processor ( 15th ), high-priority direct signals from the second sensor arrangement ( 131 ) in the form of a high-priority signal (H131) from the second sensor arrangement ( 131 ), high-priority signals (H135) of the second signal processor ( 135 ) and user inputs (H142) or other external high-priority signals. All of these high-priority signals are only exemplary. Each of these high-priority signals can be a single signal or a bundle of several signals in time and / or space multiplex. These high-priority signals are preferred but not absolutely necessary. They enable the system to train the decision maker in the form of the third neural network model ( 138 ).

Furthermore, the overall measurement system described in this section preferably includes a user control display ( 9 ), which displays settings made in response to the identification of objects and / or object components and / or the arrangement of objects and / or object components in the first ultrasound measurement values and / or in the two- or three-dimensional representations and / or as a function of the first Object recognition result can be made. The user control display ( 9 ) can basically with the display ( 19th ) form a unit.

With the ultrasonic measuring system described here ( 164 ) preferably activates the user control display ( 9 ) in response to the identification of objects and / or object components and / or the arrangement of objects and / or object components in the first ultrasound measurement values and / or in the two- or three-dimensional representations and / or in the first object recognition results control options and displays them.

The ultrasonic measuring system described in this section ( 164 ) includes for the purpose of training the first neural network model ( 16 ) also prefers a first training image memory ( 17th ), the training images for the first neural network model ( 16 ) saves.

With the ultrasonic measuring system described in this section ( 164 ) the display is ( 19th ) preferably also designed to display the two- or three-dimensional representations, which at least at times also contain the information provided by the first neural network model ( 16 ) certain identification and / or a part of the first object recognition result are provided.

The overall measurement system described above preferably also includes a plurality of third user controls ( 145 ), whereby setting one or more of the third user controls ( 145 ) in response to the identification of an arrangement of objects and / or object components or of objects in the first object recognition results and / or in the second Object recognition results and / or takes place in the two- or three-dimensional representations and / or as a function of the common object recognition hypothesis.

The preferred image processor used ( 18th ) converts this information into a suitable two- or three-dimensional representation, which is provided by the display ( 19th ) is issued to the driver. The information about object positions and classifications is also given preference to the user controls ( 10 , 140a , 145 ) to hand over. The selection of a saved user control or the adjustment of control parameters of a control in response to the classification is carried out by the user control ( 10 , 140a , 145 ) and transferred to the settings memory ( 14th ) or from the user control display ( 19th ) is displayed. The settings memory ( 14th ) also transmits parameters dependent on the classification to the image processor ( 18th ) so that the display ( 19th ) the displayed representation of the objects and their identification can be automatically adapted accordingly.

Finally, it should be expressly pointed out here that the transmission control of the second measuring system ( 161 ) is not absolutely necessary if the second measuring system ( 161 ) no transmission of measuring signals is required for the applied measuring principle. In this case it may even be that the second control electronics ( 134 ), the second sensor control ( 133 ) and the second setting memory ( 134 ) even omitted. As a rule, however, these are still receiving parameters of the second sensor arrangement ( 131 ) determine.

Figure 19

shows the structure of the proposed combined overall system from a second measuring system ( 161 ) and a third measuring system ( 162 ) for the corresponding first combination procedure according to Section F). The basic structure corresponds to that of 18th with the difference that the ultrasonic measuring system ( 164 ) through the third measuring system ( 162 ) is replaced.

• Die dritte Sensoranordnung (151) sendet bevorzugt (aber nicht notwendigerweise) dritte Messsignale aus und empfängt die Echos der zu detektierenden Objekte im Umfeld des Fahrzeugs. Die dritte Sensoranordnung (151) wandelt diese empfangenen Echosignale und sonstigen empfangenen Messsignale in dritte Messwerte um und stellt diese einem dritten Signalprozessor (155) zur Verfügung, der diese weiterverarbeitet. Der dritte Signalprozessor (155) kann ein digitaler programmierbarer Prozessor in von Neumann oder Havard-Architektur oder in einer anderen geeigneten Architektur sein. Es kann sich aber auch um eine geeignete analoge Schaltung zur Vorverarbeitung der Signale handeln. Im Falle analoger neuronale Netze kann der dritte Prozessor ggf. auch weggelassen werden, was sich in der Praxis aber bisher nicht bewährt hat.

A first group of components forms the third measuring system ( 162 ):

• The third sensor arrangement ( 151 ) preferably (but not necessarily) sends out third measurement signals and receives the echoes of the objects to be detected in the vicinity of the vehicle. The third sensor arrangement ( 151 ) converts these received echo signals and other received measurement signals into third measurement values and presents them to a third signal processor ( 155 ) is available to process them further. The third signal processor ( 155 ) can be a digital programmable processor in von Neumann or Havard architecture or in another suitable architecture. However, it can also be a suitable analog circuit for preprocessing the signals. In the case of analog neural networks, the third processor can optionally also be omitted, but this has so far not proven itself in practice.

The signals from the third signal processor ( 155 ) processed received signals are the input signals of the fourth neural network model ( 156 ), which is typically stored on a digital memory of a computer. The fourth neural network model ( 156 ) can also be implemented, for example, as a special microelectronic circuit or other functionally equivalent device. The fourth neural network model ( 156 ) is the implementation of a deep learning process, as described above.

The parameters of the fourth neural network model ( 156 ) can from a fourth training data memory ( 157 ) Loading. The training data are preferably determined in the laboratory by a method as described above.

The third sensor arrangement ( 151 ) is with a third control electronics ( 152 ), which are optionally used for beam shaping, for example by means of phase shifting of the optionally transmitted measurement signals of the individual third sensor elements of the third sensor arrangement ( 151 ) can be capable. The third control electronics ( 152 ) controls the corresponding sensor elements of the third sensor arrangement ( 151 ) for preferably directed transmission or reception of the measurement signals. This alignment is desirable, but not mandatory. The transmission of measurement signals by the third sensor arrangement ( 151 ) is controlled by a third sensor control ( 153 ). This is connected to the third control electronics ( 152 ) and provides the transmission parameters of the sensor elements of the sensor arrangement ( 151 ), depending on the settings made via the optional third user settings stored in an optional memory ( 154 ), or by a higher-level control unit.

Send parameters that are used by the sensor control ( 153 ) can be controlled, in the case of a RADAR or LIDAR system, for example, burst length, frequency, frequency curve, coding, polarity, phase or amplitude.

The output values of the fourth neural network model ( 156 ) are preferably the positions of the recognized objects and preferably their classification, as well as, for example, a corresponding confidence value of the recognized Recognition result can be assigned as a whole, and / or confidence values that are assigned to the individual objects. These data form the third object recognition result. Further data can be part of the output values. Alternative hypothesis lists are also possible, the hypothesis list comprising alternative recognition results with different confidence values for these third object recognition results. Every third object recognition result includes one or more objects.

The third measuring system described above ( 162 ) preferably further comprises a plurality of third user controls ( 163 ), whereby setting one or more of the third user controls ( 163 ) takes place as a reaction to the identification of an arrangement of objects and / or object components or of objects in the first ultrasonic measurement values and / or in the two- or three-dimensional representations and / or as a function of the first object recognition results.

With the third measuring system described above ( 162 ) the setting or the adaptation of control parameters of one or more of the controls takes place in response to the identification of the arrangement of objects and / or object components in the third measured values of the third sensor arrangement ( 151 ) and / or in the two- or three-dimensional representations and / or as a function of the third object recognition results. One or more of the controls are preferably set here ( 153 ) of the third measuring system ( 162 ) or the vehicle as a reaction to the identification of objects and / or object components and / or the arrangement of objects and / or objects and / or object components in the third measured values of the third sensor arrangement ( 151 ) and / or in the two- or three-dimensional representations and / or as a function of the third object recognition results.

The third sensor arrangement ( 151 ) is with a third control electronics ( 152 ), which, in the case of a LIDAR or radar system, optionally for beam shaping, for example by means of phase shifting of the transmitted ultrasonic signals of the individual sensor elements of the sensor arrangement ( 11 ) can be capable. The third control electronics ( 152 ) controls the corresponding sensor elements of the third sensor arrangement ( 151 ) for preferably directed transmission or reception of the third measurement signals. This alignment is desirable, but not mandatory. The transmission of third measurement signals by the third sensor arrangement ( 151 ) is controlled by a third sensor control ( 153 ). This is connected to the third control electronics ( 152 ) and provides the transmission parameters of the sensor elements of the sensor arrangement ( 11 ), depending on the settings that are stored in an optional third setting memory using the optional user settings ( 154 ), or by a higher-level control unit. Send parameters that are sent by the third transmitter control ( 153 ) can be controlled, in the case of a RADAR or LIDAR system, for example, burst length, frequency, frequency curve, coding, polarity, phase or amplitude.

• Hier wird davon ausgegangen, dass das zweite Messsystem ebenfalls Messsignale aussendet, deren Echos empfangen werden. Dabei kann es sich beispielsweise um eine LIDAR- oder RADAR-System handeln. Dies ist jedoch nur ein Beispiel. Es kann sich auch um ein anderes Messsystem, wie beispielsweise ein PIR-Infrarotmesssystem handeln, das auf ohnehin im Umfeld des Fahrzeugs befindliche Signale zurückgreift. PIR steht hierbei für „passiv infra red“ (Deutsch: passives Infrarot).

A second group of components forms the second measuring system ( 161 ):

• It is assumed here that the second measuring system also sends out measuring signals whose echoes are received. This can be a LIDAR or RADAR system, for example. However, this is just an example. It can also be a different measuring system, such as a PIR infrared measuring system, which makes use of signals that are already in the vicinity of the vehicle. PIR stands for “passive infra red” (German: passive infrared).

If applicable, the second sensor arrangement sends ( 131 ) sends the measurement signals out into the surroundings of the vehicle and receives the response from the surroundings, in particular the signal response of the objects to be detected or characterized in the surroundings of the vehicle to these emitted measurement signals. The sensor arrangement ( 131 ) converts these received signal responses and other received interference and useful signal signals into measured signal values and presents them to a second signal processor ( 135 ) is available to process them further. The second signal processor ( 135 ) can again be a digital programmable processor in von Neumann or Havard architecture or in another suitable architecture. However, it can also be a suitable analog circuit for preprocessing the signals. In the case of analog neural networks, the processor can optionally also be omitted, but this has so far not proven itself in practice. In the case of a programmable processor, the second signal processor ( 135 ) for example with the first signal processor ( 15th ) and / or the said computer form a unit.

The signals from the second signal processor ( 135 ) processed received signals of the sensor arrangement ( 131 ) are the input signals of the second neural network model ( 136 ), which is typically on a digital memory of a second computer that is connected to the first computer and / or the signal processor ( 15th ) and / or the second signal processor ( 135 ) can be identical, is stored. The second neural network model ( 136 ) can also be used, for example, as special microelectronic Circuit or other functionally equivalent device can be realized. The second neural network model ( 136 ) It is preferred to implement a deep learning process, as described above.

The parameters of the second neural network model ( 136 ) can from a second training data memory ( 137 ) are given preference. The training data are preferably determined in the laboratory by a method as described above.

The second sensor arrangement ( 131 ) is equipped with a second control electronics ( 132 ), which in the case of a LIDAR or radar system can optionally be used for beam shaping, for example by means of phase shifting of the measurement signals transmitted by the individual sensor elements of the sensor arrangement ( 131 ) can be capable. The second control electronics ( 132 ) controls the corresponding sensor elements of the sensor arrangement ( 11 ) for preferably directed transmission or reception of the measurement signals. In the case of a RADAR system, this can be done using phase antennas, for example. This alignment is desirable, but not mandatory. The transmission of measurement signals by the sensor arrangement ( 131 ) is controlled by a second sensor control ( 133 ). This is connected to the second control electronics ( 132 ) and provides the transmission parameters of the sensor elements of the sensor arrangement ( 131 ), depending on the settings that are stored in an optional second settings memory ( 134 ), or by a higher-level control unit. Send parameters that are sent by the second sensor control ( 133 ) can be controlled, in the case of a RADAR system as a second measuring system ( 161 ) For example burst length, frequency, frequency curve, coding, polarity, phase or amplitude.

The output values of the second neural network model ( 136 ) are preferably the positions of the recognized objects and preferably their classification, as well as, for example, a corresponding confidence value that can be assigned to the recognized recognition result as a whole, and / or confidence values that are assigned to the individual objects. These data form the second object recognition result. Further data can be part of the output values. Alternative hypothesis lists are also possible, the hypothesis list including alternative recognition results with different confidence values for these second object recognition results. Every second object recognition result includes one or more objects.

The previously described second measuring system ( 161 ) preferably further comprises a multiplicity of second user controls ( 140a) setting one or more of the second user controls ( 140a) in response to the identification of an arrangement of objects and / or object components or of objects in the second measured values of the second sensor arrangement ( 131 ) and / or takes place in the two- or three-dimensional representations and / or as a function of the second object recognition results.

With the previously described second measuring system ( 161 ) the setting or adjustment of control parameters of one or more of the controls is preferably carried out in response to the identification of the arrangement of objects and / or object components in the second measured values of the second sensor arrangement ( 131 ) and / or in the two- or three-dimensional representations and / or as a function of the second object recognition results. One or more of the second sensor controls ( 133 ) of the second measuring system ( 161 ) or of the vehicle in response to the identification of objects and / or object components and / or the arrangement of objects and / or objects and / or object components in the second measured values of the second sensor arrangement ( 131 ) and / or in the two- or three-dimensional representations and / or as a function of the second object recognition results.

The second sensor arrangement ( 131 ) is equipped with a second control electronics ( 132 ), which are optionally used for beam shaping, for example by means of phase shifting of the transmitted measurement signals of the individual sensor elements of the sensor arrangement 131 ) can be capable. The second control electronics ( 132 ) controls the corresponding sensor elements of the sensor arrangement ( 131 ) for preferably directed transmission or reception of the measurement signals. This alignment is desirable, but not mandatory. The transmission of measurement signals by the sensor arrangement ( 131 ) is controlled by the sensor control ( 133 ). This is connected to the second control electronics ( 132 ) and provides the transmission parameters of the sensor elements of the sensor arrangement ( 131 ), depending on the settings that are stored in an optional second settings memory ( 134 ), or by a higher-level control unit. Send parameters that are used by the sensor control ( 133 ) can be controlled, for example, burst length, frequency, frequency curve, coding, polarity, phase or amplitude.

The proposed overall sensor system also includes a fifth neural network model ( 158 ), which is preferably implemented on another computer system. The other Computer system can form a unit with the aforementioned computer system and / or the aforementioned signal processors. However, they can also be special devices, for example special circuits.

The fourth neural network model ( 156 ) forms a third object recognition result.

The second neural network model ( 136 ) forms a second object recognition result.

The fifth neural network model ( 158 ) now forms a common object recognition hypothesis as a function of the third object recognition result and the second object recognition result. Depending on the common object recognition hypothesis determined and the third object recognition result and the second object recognition result, the fifth neural network model controls ( 158 ) a fourth parameter modification device ( 144 ) for the fourth neural network model ( 156 ) and a second parameter modification device ( 141 ) for the second neural network model ( 136 ).

Both the fourth parameter modification device ( 144 ) and the second parameter modification device ( 141 ) can be implemented on another computer system or one of the aforementioned computer systems and / or signal processors. However, they can also be special devices, for example special circuits.

The fourth parameter modification device ( 144 ) modified depending on the signals of the fifth neural network model ( 158 ) and / or the third object recognition result and / or the second object recognition result, the parameters of the fourth neural network model ( 156 ).

The second parameter modification device ( 141 ) modified depending on the signals of the fifth neural network model ( 158 ) and / or of the third object recognition result and / or of the second object recognition result, the parameters of the second neural network model ( 136 ). This makes the device self-learning.

For security-relevant applications it is necessary to prevent the fifth neural network model ( 158 ) and / or the other neural network models. The proposed overall system therefore includes a second safety device ( 143 ). The role of the second safety device ( 143 ) is to monitor the work of the overall system and to detect obviously incorrect results of the three neural networks. To do this, the second safety device ( 143 ) high-priority direct signals of the third sensor arrangement ( 151 ) in the form of a high-priority signal (H151) from the third sensor arrangement ( 151 ), high-priority signals (H155) of the third signal processor ( 155 ), high-priority direct signals from the second sensor arrangement ( 131 ) in the form of a high-priority signal (H131) from the second sensor arrangement ( 131 ), high-priority signals (H135) of the second signal processor ( 135 ) and user inputs (H142) or other external high-priority signals. All of these high-priority signals are only exemplary. Each of these high-priority signals can be a single signal or a bundle of several signals in time and / or space multiplex. These high-priority signals are preferred but not absolutely necessary. They enable the system to train the decision maker in the form of the fifth neural network model ( 158 ).

Furthermore, the overall measurement system described in this section preferably includes a user control display ( 9 ), which displays settings made in response to the identification of objects and / or object components and / or the arrangement of objects and / or object components in the first ultrasound measurement values and / or in the two- or three-dimensional representations and / or as a function of the first Object recognition result can be made. The user control display ( 9 ) can basically with the display ( 19th ) form a unit.

With the third measuring system described here ( 162 ) preferably activates the user control display ( 9 ) a response to the identification of objects and / or object components and / or the arrangement of objects and / or object components in the third measured values of the third sensor arrangement ( 151 ) and / or in the two- or three-dimensional representations and / or in the third object recognition results control options and displays them.

For the purpose of training the fourth neural network model ( 156 ) also prefers a fourth training image memory ( 157 ), the training images for the fourth neural network model ( 156 ) saves.

With the third measuring system described in this section ( 162 ) the display is ( 19th ) preferably also designed to display the two- or three-dimensional representations, which at least at times also include the information provided by the fourth neural network model ( 156 ) certain Identification and / or a part of the third object recognition result are provided.

The overall measurement system described above preferably also includes a plurality of third user controls ( 145 ), whereby setting one or more of the third user controls ( 145 ) takes place in response to the identification of an arrangement of objects and / or object components or of objects in the third object recognition results and / or in the second object recognition results and / or in the two- or three-dimensional representations and / or as a function of the common object recognition hypothesis.

The preferred image processor used ( 18th ) converts this information into a suitable two- or three-dimensional representation, which is provided by the display ( 19th ) is issued to the driver. The information about object positions and classifications is also given preference to the user controls ( 140a , 145 , 160 ) to hand over. The selection of a saved user control or the adjustment of control parameters of a control in response to the classification is carried out by the user control ( 140a , 145 , 160 ) and transferred to the third setting memory ( 154 ) or from the user control display ( 19th ) is displayed. The third setting memory ( 154 ) also transmits parameters dependent on the classification to the image processor ( 18th ) so that the display ( 19th ) the displayed representation of the objects and their identification can be automatically adapted accordingly.

Finally, it should be expressly pointed out here that the transmission control of the second measuring system ( 161 ) is not absolutely necessary if the second measuring system ( 161 ) no transmission of measuring signals is required for the applied measuring principle. In this case it may even be that the second control electronics ( 134 ), the second sensor control ( 133 ) and the second setting memory ( 134 ) even omitted. As a rule, however, these are still receiving parameters of the second sensor arrangement ( 131 ) determine.

Finally, it should also be expressly pointed out here that the transmission control of the third measuring system ( 162 ) is not absolutely necessary if the third measuring system ( 162 ) no transmission of measuring signals is required for the applied measuring principle. In this case it may even be that the third control electronics ( 154 ), the third sensor control ( 153 ) and the third setting memory ( 154 ) even omitted. As a rule, however, these are still receiving parameters of the third sensor arrangement ( 151 ) determine.

Figure 20

shows a model for training and using an overall system 19th . (Can refer to an overall system 18th easily transferred.)

The first step of the procedure is to carry out ( 171 ) a procedure 7th for the second measuring system ( 161 ) with the determination of the second sensor measured values and determination of a second object recognition result.

As a second step, the procedure comprises the implementation ( 172 ) a procedure 7th for the third measuring system ( 162 ).

As a third step, as a result step of the second step, the method comprises the determination of the third sensor measured values and the determination of a third object recognition result.

The fourth step of the process is installing ( 173 ) of a fifth neural network model ( 158 ) on a computer-based system. This step can also include loading parameters for the fifth neural network model ( 158 ) include.

The method comprises as a fifth step training ( 174 ) of the fifth neural network model ( 158 ) in order to identify the objects and / or object components and / or arrangements of objects and / or object components in the second object recognition result and / or in the third object recognition result

The method comprises, as a sixth step, which is a result step of the fifth step, generating a common object recognition hypothesis.

• • zur Identifizierung von Objekten und/oder Objektkomponenten und/oder von Anordnungen von Objekten und/oder Objektkomponenten auf Basis des zweiten Objekterkennungsergebnisses und des dritten Objekterkennungsergebnisses durch das fünfte neuronale Netzwerkmodell (158) und/oder
• • zur Erzeugung einer gemeinsamen Objekterkennungshypothese auf Basis des zweiten Objekterkennungsergebnisses und des dritten Objekterkennungsergebnisses durch das fünfte neuronale Netzwerkmodell (158).

The method comprises as a seventh step using ( 175 ) of the trained fifth neural network model ( 158 )

• • to identify objects and / or object components and / or arrangements of objects and / or object components based on the second object recognition result and the third object recognition result by the fifth neural network model ( 158 ) and or
• • for generating a common object recognition hypothesis based on the second object recognition result and the third object recognition result by the fifth neural network model ( 158 ).

Figure 21

shows a model for training and storing the parameters of the neural network models of an overall system 19th . (Can refer to an overall system 18th easily transferred.)

As a first step, the procedure includes ( 171 ) a procedure 7th for the second measuring system ( 161 ).

As a second step, as a result step of the first step, the method comprises the determination of the second sensor measurement values and the determination of a second object recognition result.

As a third step, the method includes the step of performing ( 172 ) a procedure 7th for the third measuring system ( 162 ).

As a fourth step, which is a result step of the third step, the method comprises the determination of the third sensor measured values and the determination of a third object recognition result.

The method comprises as a fifth step, the step of installing ( 173 ) of a fifth neural network model ( 158 ) on a computer-based system. This step can also include loading parameters for the fifth neural network model ( 158 ) include.

The method includes, as a sixth step, training ( 174 ) of the fifth neural network model ( 158 ). This is done in order to identify the objects and / or object components and / or arrangements of objects and / or object components in the second object recognition result and / or in the third object recognition result and to generate a common object recognition hypothesis.

As a seventh step, the procedure includes securing ( 185 ) the parameter data of the trained fifth neural network ( 158 ) in a fifth training data memory ( 159 ).

Figure 22

Fig. 12 shows a model for loading the parameters of the neural network models of an overall system 19th and using the entire system prepared in this way. (Can refer to an overall system 18th easily transferred.)

The process includes a first step of loading ( 71 ) the previously saved parameters of the second neural network model ( 136 ) for the second measuring system ( 161 ) from the second training data memory ( 137 ) as current parameters of the second neural network model ( 136 ) for the second measuring system ( 161 ).

As a second step, the method includes loading ( 192 ) the previously saved parameters of the fourth neural network model ( 156 ) from the fourth training data memory ( 157 ) into the fourth neural network model ( 156 ).

As a third step, the method includes loading ( 193 ) the previously saved parameters of the fifth neural network model ( 158 ) from the fifth training data memory ( 159 ) into the fifth neural network model ( 158 ).

• • zur Identifizierung von Objekten und/oder Objektkomponenten und/oder von Anordnungen von Objekten und/oder Objektkomponenten auf Basis des dritten Objekterkennungsergebnisses und des zweiten Objekterkennungsergebnisses durch das fünfte neuronale Netzwerkmodell (158) und/oder
• • zur Erzeugung einer gemeinsamen Objekterkennungshypothese auf Basis des zweiten Objekterkennungsergebnisses und des dritten Objekterkennungsergebnisses durch das fünfte neuronale Netzwerkmodell (158).

The method comprises as a fourth step using ( 175 ) of the trained fifth neural network model ( 158 )

• • for the identification of objects and / or object components and / or of arrangements of objects and / or object components on the basis of the third object recognition result and the second object recognition result by the fifth neural network model ( 158 ) and or
• • for generating a common object recognition hypothesis based on the second object recognition result and the third object recognition result by the fifth neural network model ( 158 ).

Figures 23 and 24

The 23 and 24 are described in the text above.

glossary

Ultrasonic transducer

Ultrasonic transmitter and receiver combined in one component

Neural network / neural network model

Wikipedia (as of January 18, 2019) defines a neural network as follows: In neuroscience, a neural network is any number of interconnected neurons that, as part of a nervous system, form a connection that is supposed to serve a specific function. In computational neuroscience, simplified models of a biological network are also understood as abstract. In computer science, information technology and robotics, their structures are modeled as an artificial neural network and technically reproduced, simulated and modified. From System inspired by the functioning of biological neurons, which serves as the basis for machine learning processes

The nervous system of humans and animals consists of nerve cells (neurons) and glial cells and an environment. The neurons are linked to one another via synapses, which can be understood as connecting points or nodes of an interneuronal network. In addition, there is an exchange in chemical and electrical form between neurons and cells of the neuroglia, in particular oligodendroglia and astroglia, which can change the weighting of signals. The "circuit technology" of neurons usually knows several inputs and one output. If the sum of the input signals exceeds a certain threshold value, the neuron “fires” (excitation generation): An action potential (AP) is triggered on the axon hill, formed in the initial segment and transmitted along the axon (excitation conduction). Action potentials in series are the primary output signals from neurons. These signals can be transmitted to other cells via synapses (transmission of excitation). At electrical synapses, the changes in potential are passed on in direct contact. At chemical synapses, these are converted into a transmitter quantum as a secondary signal, i.e. transmitted by messenger substances (transmission). With divergence, one neuron sends signals to several other neurons, and with convergence, one neuron receives signals from several others.

Characteristic of nerve cells are their cell extensions, with which contacts to individual other cells are made. As dendrites, they are primarily used to receive signals from other cells, while signals to other cells are passed on via the neurite, also called axon in the envelope by glial cells. With branches from its axon as axon collaterals, the signal of a neuron can be transmitted efferent to several other neurons (divergence). A neuron can also receive afferent signals from various other neurons (convergence), primarily via its dendrites as inputs. While the action potential transmitted across the membrane is promptly transferred as an electrical signal in electrical synapses, in chemical synapses it is first transformed into a secondary chemical signal at the presynaptic membrane region of a neuron. This occurs as a potential-dependent neurocrine secretion through the release (exocytosis) of the molecules of a signal substance that are held in stock in synaptic vesicles. After bridging the narrow synaptic gap by diffusion, this messenger substance acts as a neurotransmitter - or as a neuromodulatory cotransmitter - on the membrane region of the postsynaptically assigned cell, if it is equipped with suitable receptor molecules and is receptive to it. With the receptor binding, a transmitter is recognized and directly (ionotropically) or indirectly (metabotropically) a temporary regional change in membrane permeability is initiated. Small ions flowing in or out through the membrane again cause changes in potential postsynaptically, as local electrical signals. The incoming signals converge on the membrane of a neuron, where they are spatially and temporally integrated and summarized. Such postsynaptic potentials (PSP) develop differently, depending on the membrane equipment with receptors and ion channels. As graduated potentials, they can not only be signals of different strength, but also fundamentally different in quality: excitatory (EPSP) stimulate the formation of arousal; inhibitory (IPSP) inhibit the formation of an action potential. With this form of synaptic connection as chemical transmission, signals are given a sign. Furthermore, they can be weighted, strengthened or weakened depending on the process at the connection point. Frequently repeated transmission in rapid succession can lead to longer-lasting changes which, as long-term potentiation (LTP), intensify synaptic transmission. At a low frequency, different types of changes can occur, which lead to a permanent weakening as long-term depression (LTD). In this way, the signal transmission process itself can shape or reshape the synaptic mode (neuronal plasticity). The networking of neurons does not show a rigid connection, but a weighting of the signal path that depends on the previous state and changes with repeated use. There are various, now well-standardized theories about learning in neural networks. The first neural learning rule was described by Donald O. Hebb in 1949 (Hebb's learning rule); Significant developments took place, inter alia, through the work of the Finn Teuvo Kohonen in the mid-1980s. This resulted in typical properties of neural networks that apply equally to natural as well as artificial "neural systems". This includes the property that they can learn complex patterns without any abstraction about the rules on which these patterns are based. This means that neural networks do not follow a logic system, but an intuitive pattern processing (see also artificial intelligence). This also means that the rules do not have to be developed before learning. On the other hand, a possible logic that determined its learning success cannot be determined retrospectively from the neural network. Again, this does not mean that logical behavior and precise rules cannot be learned or applied by neural networks. You just have to go through this Training may be laboriously worked out; for example when learning the grammar of a language over the years. Neural networks do not learn explicitly, but implicitly: the grammar of the mother tongue is first learned implicitly by a toddler. As a schoolchild, they generally learn the rules - again - explicitly, through rules. Especially for the simulation of artificial neural networks in science and technology by means of special circuits or devices or by means of processors with suitable programming, the following applies: The “correct” training of a neural network is a prerequisite for learning success or for the correct processing of a pattern in a nervous system. Conversely, a prediction about the “correct” interpretation of a pattern by a neural network is not precisely possible as long as this specific network is not used or calculated with this specific learning experience. Neural networks thus have the problem that after the learning process, patterns that do not resemble the models implemented in the learning set cause stochastic (ie apparently “random”) behavior of the output neurons. They do not work exactly, but approximately. It is known that the function of a neural network can also be perceived by a Petri network or an HMM recognizer and that a transformation between these images is possible. It is therefore expressly claimed that sub-devices with HMM detectors and / or with Petri networks are also included under neural networks.

Deep learning

Machine learning process that is based on a neural network model with many intermediate layers and independently learns which representation of the data derived from the input data is necessary in order to fulfill the learning task.

Wikipedia (as of January 18, 2019) defines deep learning as follows: Deep learning refers to a class of optimization methods in artificial neural networks that have numerous hidden layers between the input layer and the output layer, making them extensive have internal structure. In extensions of the learning algorithms for network structures with very few or no intermediate layers, such as the single-layer perceptron, the deep learning methods enable stable learning success even with numerous intermediate layers. According to Jürgen Schmidhuber, "deep learning" is just a new term for artificial neural networks and appeared for the first time in 2000 in the publication Multi-Valued and Universal Binary Neurons: Theory, Learning and Applications. The problems solved in the early days of artificial intelligence were intellectually difficult for humans, but easy for computers to process. These problems could be described by formal mathematical rules. The real challenge to artificial intelligence, however, was to solve tasks that are easy for humans to perform, but whose solution is difficult to formulate using mathematical rules. These are tasks that humans solve intuitively, such as speech or face recognition. A computer-based solution to this type of task involves the ability of computers to learn from experience and understand the world in terms of a hierarchy of concepts. Here each concept is defined by its relationship to simpler concepts. By gaining knowledge from experience, this approach avoids the need for human operators to formally specify all of the knowledge the computer needs to do its job. The hierarchy of concepts allows the computer to learn complicated concepts by assembling them from simpler ones. If you draw a diagram that shows how these concepts are built on top of each other, then the diagram is deep, with many layers. For this reason, this approach is called “deep learning” in artificial intelligence. It is difficult for a computer to understand the meaning of raw sensory input data, such as in handwriting recognition, where text initially only exists as a collection of pixels. The conversion of a set of pixels into a chain of numbers and letters is very complicated. Complex patterns have to be extracted from raw data. Learning or evaluating this assignment seems insurmountably difficult if it were programmed manually. One of the most common techniques in artificial intelligence is machine learning. Machine learning is a self-adaptive algorithm. Deep learning, a subset of machine learning, uses a series of hierarchical layers or a hierarchy of concepts to carry out the machine learning process. The artificial neural networks used here are built like the human brain, with the neurons connected to one another like a network. The first layer of the neural network, the visible input layer, processes a raw data input, such as the individual pixels of an image. The data input contains variables that are accessible for observation, hence the "visible layer". This first shift forwards its output to the next shift. This second layer processes the information from the previous layer and also forwards the result. The next layer receives the information from the second layer and processes it further. These layers are known as hidden layers. The the features contained in them are becoming increasingly abstract. Their values are not given in the original data. Instead, the model must determine which concepts are useful in explaining the relationships in the observed data. This continues across all levels of the artificial neural network. The result is output in the last visible layer (English output layer). This divides the desired complex data processing into a series of nested simple associations, each of which is described by a different layer of the model. The technology presented here for sensory processing goes far beyond these approaches. In particular, it provides for the first time a self-learning network of several neural networks that is self-learning in every level of the chain. Each neural network of the network of neural networks represents a node of this network. Each node of a neural network network is therefore itself a neural network.

List of reference symbols

9

User control display;

10

User control;

11

Ultrasonic transducer assembly;

12th

Control electronics;

13th

Transmitter control;

14th

Settings memory;

15th

Signal processor;

16

first neural network model in a first memory;

17th

first training data memory;

18th

Image processor;

19th

Display;

21st

Installation of the first neural network model ( 16 ) for the ultrasonic measuring system ( 164 ) on the computer system;

22nd

Transfer of echo signals from different objects in different positions to the first neural network model for the ultrasonic measurement system ( 164 );

23

Identification of the objects for the first neural network model for the ultrasonic measurement system ( 164 );

24

Transfer of live ultrasound signals to the trained first neural network model for the ultrasonic measurement system in the vehicle and implementation of an object identification by the trained first neural network model to generate a first object recognition result;

25

Use of the first object identification carried out by the first neural network model for the ultrasonic measuring system in the form of a first object recognition result;

35

Saving the parameter data of the trained first neural network model ( 16 ) for the ultrasonic measuring system in a first training data memory ( 17th );

41

Load the previously saved parameters of the first neural network model ( 16 ) for the ultrasonic measuring system from the first training data memory ( 17th ) as the current parameters of the first neural network model ( 16 );

51

Installation of the second neural network model ( 136 ) for the second measuring system ( 161 ) on the computer system;

52

Transfer of sensor signals from different objects in different positions to the second neural network model ( 136 ) for the second measuring system ( 161 );

53

Identification of the objects for the second neural network model ( 136 ) for the second measuring system ( 161 );

54

Transfer of live sensor signals from the second sensor arrangement ( 131 ) to the trained second neural network model ( 136 ) for the second measuring system ( 161 ) and implementation of an object identification by the trained second neural network model ( 136 ) for generating a second object recognition result;

55

Use of the through the second neural network model ( 136 ) for the second measuring system ( 161 ) a second object identification carried out in the form of a second object recognition result;

61

Installation of the fourth neural network model ( 156 ) for the third measuring system ( 162 ) on the computer system;

62

Transfer of sensor signals from different objects in different positions to the fourth neural network model ( 156 ) for the third measuring system ( 162 );

63

Identification of the objects for the fourth neural network model ( 156 ) for the third measuring system ( 162 );

64

Transfer of live sensor signals from the third sensor arrangement ( 151 ) to the trained fourth neural network model ( 156 ) for the third measuring system ( 162 ) and carrying out an object identification by the trained fourth neural network model ( 156 ) for generating a third object recognition result;

65

Saving the parameter data of the trained second neural network model ( 136 ) for the second measuring system ( 161 ) in a second training data memory ( 137 );

65b

Use of the through the fourth neural network model ( 1356 ) for the third measuring system ( 162 ) third object identification carried out in the form of a third object recognition result;

71

Load the previously saved parameters of the second neural network model ( 136 ) for the second measuring system ( 161 ) from the second training data memory ( 137 ) as current parameters of the second neural network model ( 136 ) for the second measuring system ( 161 );

72

Load the previously saved parameters of the fourth neural network model ( 156 ) for the third measuring system ( 162 ) from the fourth training data memory ( 157 ) as the current parameters of the fourth neural network model ( 156 ) for the third measuring system ( 162 );

84

Comparison of the first object recognition results from the first neural network model ( 16 ) for the ultrasonic measuring system ( 164 ) carried out first object identification with the alternative second object recognition results of the second neural network model ( 136 ) for the second measuring system ( 161 ) alternative second object identification carried out and determination of a common object recognition hypothesis;

85

Use of the common object recognition hypothesis;

86

first parameter adjustment of the first neural network model ( 16 ) for the ultrasonic measuring system ( 164 ) depending on the comparison result of the first object recognition results of the first neural network model ( 16 ) for the ultrasonic measuring system ( 164 ) carried out first object identification with the alternative second object recognition results of the second neural network model ( 136 ) for the second measuring system ( 161 ) carried out alternative second object identification in step 84 ;

87

second parameter adjustment of the second neural network model ( 136 ) for the second measuring system ( 161 ) depending on the comparison result of the first object recognition results of the first neural network model ( 16 ) for the ultrasonic measuring system ( 164 ) carried out first object identification with the alternative second object recognition results of the second neural network model ( 136 ) for the second measuring system ( 161 ) carried out alternative second object identification in step 84 ;

88

fourth parameter adjustment of the fourth neural network model ( 156 ) for the third measuring system ( 162 ) depending on the comparison result of the second object recognition results of the second neural network model ( 136 ) for the second measuring system ( 161 ) carried out second object identification with the third object recognition results of the fourth neural network model ( 156 ) for the third measuring system ( 162 ) carried out alternative third object identification in step 204 ;

91

Installation of the third neural network model ( 138 ) for the comparison of the first object recognition results of the first neural network model ( 16 ) for the ultrasonic measuring system ( 164 ) carried out first object identification with the alternative second object recognition results of the second neural network model ( 136 ) for the second measuring system ( 161 ) carried out alternative second object identification and for the determination of a common object recognition hypothesis on the computer system;

92

Training of the third neural network model ( 138 ) by transferring correct and incorrect first object recognition results and correct and incorrect alternative second object recognition results to the third neural network model ( 138 ) together with data about their correctness and / or incorrectness and determination of the parameters of the third neural network model ( 138 ) depending on this data and / or change in the parameters of the third neural network model ( 138 ) depending on this data;

93

Transfer of first live object recognition results and alternative second live object recognition results to the trained third neural network model ( 138 );

94

Determination of a common object recognition hypothesis based on the first live object recognition results and the alternative second live object recognition results by the third neural network model ( 138 );

95

Use of the third neural network model ( 138 ) common object recognition hypothesis determined for the overall measurement system;

105

Saving the parameter data of the trained third neural network ( 138 ) in a third training data memory ( 139 );

111

Load the previously saved parameters of the third neural network model ( 138 ) from the third training data memory ( 139 ) into the third neural network model ( 138 );

130

second user control that is linked to the first user control ( 10 ) can be identical if it is operated in time division multiplex;

131

second sensor arrangement;

132

second control electronics;

133

second sensor control;

134

second setting memory;

135

second signal processor that works with the first signal processor ( 15th ) can be identical if the task is solved in time division multiplex;

136

second neural network model in a second memory;

137

second training data memory;

138

third neural network model;

139

third training data memory;

140

first parameter modification device for the first neural network model ( 16 );

140a

second user control that is linked to the first user control ( 10 ) and / or the third user control ( 160 ) can be identical if it is operated in time division multiplex;

141

second parameter modification device for the second neural network model ( 136 );

142

first safety device;

143

second safety device;

144

fourth parameter modification device for the fourth neural network model ( 156 );

145

third user control;

151

third sensor arrangement;

152

third control electronics;

153

third sensor control;

154

third setting memory;

155

third signal processor that works with the first signal processor ( 13th ) and / or second signal processor ( 135 ) can be identical if the task is solved in time division multiplex;

156

fourth neural network model;

157

fourth training data memory;

158

fifth neural network model;

159

fifth training data memory;

160

third user control that is linked to the first user control ( 10 ) and / or the second user control ( 140a) can be identical if it is operated in time division multiplex;

161

second measuring system;

162

third measuring system;

163

fourth parameter modification device for the fourth neural network model ( 156 );

164

Ultrasonic measuring system;

171

Implementation of a procedure according to 7th for the second measuring system ( 161 ) with determination of the second sensor measurement values and determination of a second object recognition result;

172

Implementation of a procedure according to 7th for the third measuring system ( 162 ) with determination of the third sensor measurement values and determination of a third object recognition result;

173

Install a fifth neural network model ( 158 ) on a computer-based system;

174

Training the fifth neural network model ( 158 ) to identify the objects and / or object components and / or arrangements of objects and / or object components in the second object recognition result and / or in the third object recognition result and to generate a common object recognition hypothesis;

175

Using the trained fifth neural network model ( 158 ) to identify objects and / or object components and / or arrangements of objects and / or object components on the basis of the second object recognition result and the third object recognition result by the fifth neural network model ( 158 ) and / or to generate a common object recognition hypothesis on the basis of the second object recognition result and the third object recognition result by the fifth neural network model ( 158 );

185

Saving the parameter data of the trained fifth neural network ( 158 ) in a fifth training data memory ( 159 );

192

Load the previously saved parameters of the fourth neural network model ( 156 ) from the fourth training data memory ( 157 ) into the fourth neural network model ( 156 );

193

Load the previously saved parameters of the fifth neural network model ( 158 ) from the fifth training data memory ( 159 ) into the fifth neural network model ( 158 );

201

Installation of the fifth neural network model ( 158 ) for the comparison of the second object recognition results of the second neural network model ( 136 ) for the second measuring system ( 161 ) carried out first object identification with the third object recognition results of the fourth neural network model ( 1356 ) for the third measuring system ( 162 ) carried out alternative third object identification and for the determination of a common object recognition hypothesis on the computer system;

202

Training the fifth neural network model ( 158 ) by transferring correct and incorrect second object recognition results and correct and incorrect third object recognition results to the fifth neural network model ( 158 ) together with data about their correctness and / or incorrectness and determination of the parameters of the fifth neural network model ( 158 ) depending on this data and / or change in the parameters of the fifth neural network model ( 158 ) depending on this data;

203

Transfer of second live object recognition results and third live object recognition results to the trained fifth neural network model ( 158 );

204

Determination of a common object recognition hypothesis based on the second live object recognition results and the third live object recognition results by the fifth neural network model ( 158 );

205

Use of the fifth neural network model ( 158 ) common object recognition hypothesis determined for the overall measurement system;

211

Load ( 211 ) the previously saved parameters of the fifth neural network model ( 158 ) from the fifth training data memory ( 159 ) or another data storage device that currently holds this data into the fifth neural network model ( 158 ). This fifth neural network model ( 158 ) are in a completely different, functionally similar overall system than the overall system with which the data to be loaded was created;

215

Fuse ( 215 ) the parameter data of the trained fifth neural network ( 158 ) in a fifth training data memory ( 159 ). This allows the parameters of the fifth neural network model ( 158 ) are used in functionally similar systems by programming and / or construction of these systems or parts of them;

1010

Vehicle part;

1012

Ultrasonic transducer assembly;

1014

Microbeamformer;

1016

Send / receive switch;

1018

Transmission control;

1020

Main jet shaper;

1026

Signal processor;

1028

Image memory;

1030

Image processor;

1032

Scan converter;

1034

Training image memory;

1036

Graphics processor;

1038

User control panel or user control panel;

1040

Image display;

1042

Volume renderer;

1044

Multiplanar reformatter;

1046

Touch screen display control;

1048

User control setting control;

1052

Video processor;

1054

Doppler processor;

1060

Touch screen display;

1080

neural network model;

1090

Installation of the neural network model;

1092

Presentation of images and / or measurement data sets of the same vehicle surroundings situation geometry, which were preferably obtained from a large number of test drives for similar situations, compared to the neural network model;

1094

Training of the neural network model in order to train it to develop the geometry by presenting the known type of situation geometry and its view in the images and / or measurement data sets together with the presentation in step ( 1092 ).

1096

Presentation of environment maps recorded by the ultrasound system and / or measurement data sets compared to the neural network model for identifying the vehicle environment situation geometry as object recognition result;

1098

Use of the identification generated by the neural network model.

H11

high-priority signal of the ultrasonic transducer arrangement ( 11 ) to the first safety device ( 142 );

H15

high priority signal of the first signal processor ( 15th ) to the first safety device ( 142 )

H131

high-priority signal of the second sensor arrangement ( 131 ) to the first safety device ( 142 );

H135

high priority signal of the second signal processor ( 135 ) to the first safety device ( 142 );

H142

User input or other external high priority signal

H151

high-priority signal of the third sensor arrangement ( 151 ) to the second safety device ( 143 );

H155

high priority signal of the third signal processor ( 155 ) to the second safety device ( 143 );

List of the cited writings

• DE 10 2008 036 009 B4
• DE 103 36 638 A1
• [Dror 1995] IE Dror, M. Zagaeski, CF Moss, “Three-dimensional target recognition via sonar: a neural network model”, Neural Networks, vol. 8, no. 1, pp. 149-160,1995
• [LeCun 2015] Y. LeCun, Y. Begio, G. Hinton, "Deep Learning," Nature, vol. 521, pp. 436-444, 2015
• [Zhang 2018] WJ Zhang, G. Yang, Y. Lin, C. Ji and MM Gupta, "On Definition of Deep Learning," 2018 World Automation Congress (WAC), Stevenson, WA, 2018, pp. 1-5, 2018

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• DE 102008036009 B4 [0006, 0318]
• DE 10336638 A1 [0006, 0318]

Non-patent literature cited

• I.E. Dror, M. Zagaeski, C. F. Moss, "Three-dimensional target recognition via sonar: a neural network model", Neural Networks, vol. 8, no. 1, pp. 149-160, 1995 [0318]
• Y. LeCun, Y. Begio, G. Hinton, "Deep Learning," Nature, vol. 521, pp.436-444, 2015 [0318]
• W. J. Zhang, G. Yang, Y. Lin, C. Ji and M. M. Gupta, "On Definition of Deep Learning," 2018 World Automation Congress (WAC), Stevenson, WA, 2018, pp. 1-5, 2018 [0318]

Claims (5)

A safety-relevant measuring system for use in vehicles and / or motor vehicles, characterized in that it has a first safety device (142) and one or more neural network models (16, 136, 138) for evaluating the measurement data of at least one sensor arrangement (11, 131) of the measuring system and that the first safety device (142) monitors at least the neural network model (138) which processes these measurement data or data derived therefrom. Measuring system according to the preceding claim, characterized in that the safety device can be influenced by signals (H11, H131) from the at least one sensor arrangement (11, 131) of the measuring system. Measuring system according to one or more of the preceding claims, characterized in that the safety device (142) can influence the parameters of at least one neural network model (16, 136, 138). Measuring system according to one or more of the preceding claims, characterized in that the safety device (142) can be influenced by a user signal (H142) or an external control signal (H142). Measuring system according to one or more of the preceding claims, characterized in that the safety device (142) can safely prevent safety-endangering and / or potentially safety-endangering signaling of the at least one neural network model (16, 136, 138).

Images