DE102018213427A1 - Driver assistance system - Google Patents

Abstract

A method for detecting a change in the position of a load on a trailer of a motor vehicle, comprising the following steps: capturing (St1) the load at a first point in time on the trailer by means of a camera that can be connected to the motor vehicle as first image data; Capturing (St2) the load at a second point in time on the trailer by means of the camera, as second image data; Detecting (St3) a change in the arrangement of the charge between the first time and the second time; Determining (St4) a reaction to the detected change using a computing unit that can be connected to the motor vehicle.

Description

The present invention relates to a method for detecting a change in position of a load on a trailer of a motor vehicle, and a training system for a vehicle control system for detecting a change in position of a load on a trailer of a motor vehicle.

For securing loads on a trailer, for example, nets, tarpaulins or tension belts are known. Faulty securing of loads is a safety risk in road traffic.

Against this background, the object of the invention is to specify a method by means of which a faulty securing of cargo on a trailer can be detected.

According to the invention, this object is achieved by a method for detecting a change in position of a load on a trailer of a motor vehicle with the features of patent claim 1 and / or by a training system for a vehicle control for detecting a change in position of a load on a trailer of a motor vehicle with the features of patent claim 10 solved.

• - Ein Verfahren zum Detektieren einer Lageänderung einer Ladung auf einem Anhänger eines Kraftfahrzeugs mit den folgenden Schritten: Erfassen der Ladung zu einem ersten Zeitpunkt auf dem Anhänger mittels einer Kamera, die mit dem Kraftfahrzeug verbindbar ist, als erste Bilddaten; Erfassen der Ladung zu einem zweiten Zeitpunkt auf dem Anhänger mittels der Kamera, als zweite Bilddaten; Detektieren einer Änderung der Anordnung der Ladung zwischen dem ersten Zeitpunkt und dem zweiten Zeitpunkt; Ermitteln einer Reaktion auf die detektierte Änderung mittels einer Recheneinheit, die mit dem Kraftfahrzeug verbindbar ist; sowie
• - ein Trainingssystem für eine Fahrzeugsteuerung zum Detektieren einer Lageänderung einer Ladung auf einem Anhänger eines Kraftfahrzeugs, mit wenigstens einer Schnittstelle, um Trainingsdaten, welche jeweils Bilddaten und ein Soll-Reaktionssignal aufweisen, zu erhalten, einer Auswerteeinheit, welche ein künstliches neuronales Netzwerk, insbesondere ein LSTM-Netzwerk, bildet und eingerichtet ist, das künstliche neuronale Netzwerk mit Trainingsdaten vorwärtszuspeisen, um Ist-Reaktionssignale zu ermitteln, und eine veränderte Topologie des künstlichen neuronalen Netzwerks, insbesondere Gewichte, durch Rückwärtsspeisen der Soll-Reaktionssignale in dem künstlichen neuronalen Netzwerk zu ermitteln, wobei die Topologie eingerichtet ist, in der Fahrzeugsteuerung zum Detektieren einer Lageänderung einer Ladung auf einem Anhänger eines Kraftfahrzeugs gespeichert zu werden.

Accordingly, it is provided:

• A method for detecting a change in position of a load on a trailer of a motor vehicle, comprising the following steps: capturing the load at a first point in time on the trailer using a camera that can be connected to the motor vehicle as first image data; Capturing the load at a second point in time on the trailer using the camera, as second image data; Detecting a change in the arrangement of the charge between the first time and the second time; Determining a reaction to the detected change using a computing unit that can be connected to the motor vehicle; such as
• - A training system for a vehicle control for detecting a change in position of a load on a trailer of a motor vehicle, with at least one interface in order to receive training data, which each have image data and a target reaction signal, an evaluation unit, which is an artificial neural network, in particular a LSTM network that forms and is set up to feed the artificial neural network with training data to determine actual reaction signals, and to determine a changed topology of the artificial neural network, in particular weights, by feeding back the desired reaction signals in the artificial neural network, wherein the topology is set up to be stored in the vehicle controller for detecting a change in position of a load on a trailer of a motor vehicle.

Motor vehicles in the sense of this patent application are motor-driven land vehicles.

A towing vehicle is a towing vehicle in a team or trailer or semitrailer. In a tractor with a trailer, the tractor is the towing vehicle. Other examples are trucks in a trailer truck, tractor units in a tractor trailer, cars in a car team or a bicycle with a bicycle trailer.

A trailer is a non-motorized part of a team. For example, trailers, caravans, car trailers and the like are trailers. Passenger cars (abbreviated to passenger cars) are multi-lane vehicles with their own drive for the primary purpose of passenger transportation. In everyday life, they are also called cars (short for automobiles) or technically motor vehicles. Most cars are road vehicles and are used in public road traffic for private transport. Buses and trucks (trucks) are not considered to be cars.

In this application, a load is understood to mean all objects on a trailer. In addition to a load to be transported, this also includes fasteners for the load. A cargo drop is the loss of cargo by the trailer. A charge drop represents a loss of value for the vehicle driver, since the charge is often lost or damaged in the process. Furthermore, a cargo departure is a safety risk for other road users who can collide with the cargo.

Fasteners for loading onto a trailer are, for example, tarpaulins, nets or tension belts.

A camera is a photographic apparatus that can record static or moving images on a photographic film or electronically on a magnetic video tape or digital storage medium or transmit them via an interface. A reversing camera is used to support the driver of a motor vehicle when reversing, in particular as a parking aid. A reversing camera is often connected to a monitor in the vehicle interior via a wiring harness.

Warning signals are perceptible to the driver of a vehicle and increase or focus his attention. Warning signals can be optical, acoustic or haptic signals.

A categorization of a safety relevance is the classification of a situation with regard to a hazard potential. The classification can be made in two (safety-relevant / not safety-relevant) or several categories (safety-relevant / possibly safety-relevant / not safety-relevant).

Conditions are relevant to safety that pose risks to traffic safety. For example, a detaching tension belt is safety-relevant, whereas a tension belt that flutters only at one end is not safety-relevant.

An interface is a device between at least two functional units on which an exchange of logical quantities, e.g. Data, or physical quantities, e.g. electrical signals, either unidirectional or bidirectional. The exchange can be analog or digital. The exchange can also be wired or wireless.

A controller is an electronic module for control or regulation. Controls are used in the automotive sector in all conceivable electronic areas, as well as for the control of machines, systems and other technical processes.

A reaction to a change in position is, for example, a warning signal to a driver or a brake signal for braking the team. Means for executing the signals are reaction units.

A computing unit is a device that processes incoming information and outputs a result resulting from this processing. Electronic circuits such as central processor units or graphics processors are computing units.

Computer program products typically include a sequence of instructions that cause the hardware, when the program is loaded, to perform a specific procedure that leads to a specific result.

An artificial neural network (KNN, English artificial neural network - ANN) is in particular a network of networked artificial neurons simulated in a computer program. The artificial neurons are typically arranged on different layers. The artificial neural network usually comprises an input layer and an output layer, the neuron output of which is the only one of the artificial neural network that is visible. Layers lying between the input layer and the output layer are typically referred to as hidden layers. Typically, an architecture or topology of an artificial neural network is initially initiated and then trained in a training phase for a specific task or for several tasks in one training phase.

The training of the artificial neural network typically includes a change in the weight of a connection between two artificial neurons of the artificial neural network. The weight contains information on the strength of the consideration of an input of a neuron. Training of the artificial neural network can also include developing new connections between artificial neurons, deleting existing connections between artificial neurons, adjusting threshold values of the artificial neurons and / or adding or deleting artificial neurons.

An example of an artificial neural network is a flat artificial neural network (shallow neural network), which often contains only a single hidden layer between the input layer and the output layer and is therefore relatively easy to train. Another example is a deep neural network, which contains several interleaved hidden layers of artificial neurons between the input layer and the output layer. The deep artificial neural network enables improved recognition of patterns and complex relationships

For example, the artificial neural network can be a single or multilayer feedforward network or a recurrent network. Feedforward networks have neurons that are fed exclusively forward, i.e. a neuron is fed exclusively by higher layers.

A recurrent network has bidirectionally connected neurons, i.e. a neuron is also fed by deeper layers. In this way, information from an earlier run can be taken into account when the KNN is run through later, thereby creating a memory.

A training system is a computing unit on which a KNN is trained.

In this application, training data are data pairs from input data that are to be processed by the KNN and result setpoint data that are to be determined by the KNN. During the training, the KNN is adjusted on the basis of a comparison of desired result data with the result determined by the KNN, which results in a training effect.

The input data with which the KNN is fed in this application are image data for a load on a trailer.

When training KNN e.g. Error signal descent method or the gradient method used. This can fall short in the case of several deeper layers. An LSTM network solves this problem by using three types of gate for an LSTM cell for better memory: an input gate, an remember gate and a forget gate, and an output gate. In contrast to conventional KNNs, LSTM enables a kind of memory of previous experiences.

The basic idea of the invention is to monitor image data supplied by a camera for a loaded trailer and to evaluate the image data with regard to a change in the position of the load.

Advantageous refinements and developments result from the further subclaims and from the description with reference to the figures of the drawing.

According to a preferred development of the invention, the reaction is a warning signal to a driver of the towing vehicle and / or to other road users. In this way, the driver of the towing vehicle can be informed about a potential danger from an insufficiently secured load. In addition, other road users can be warned, for example by means of a hazard warning lamp, about possible impending cargo outflows due to insufficient securing of the cargo.

According to a preferred development of the invention, the reaction is a brake signal for braking the motor vehicle. This means that the team can be decelerated in a controlled manner. In this case, it is expedient if the combination is not braked abruptly, since this can lead to a charge escaping, particularly if the load securing is inadequate.

According to a preferred development of the invention, the reaction is a reduced maximum speed of the team or the towing vehicle. This allows a driver to be promptly checked for load securing. Furthermore, the forces that act on the incorrectly secured load can thus be reduced. This can reduce the risk of cargo leaving.

According to a preferred development of the invention, the image data are evaluated by means of a reversing camera. This is particularly advantageous since reversing cameras are already installed as standard in many vehicles. This means that no additional optical sensors are required to carry out the method.

According to a preferred development of the invention, the camera is movable, in particular displaceable or pivotable. The camera can be moved by manual means, that is by hand, or by means of an electronic control on the basis of a control command from a driver or on the basis of a control command from the computing unit. Accordingly, it can be provided that the computing unit recognizes that the image data only partially detect the charge, whereupon a reaction is designed as a displacement and / or pivoting of the camera.

It can be provided that several partial images of the charge are generated, the camera being in a different position in each case. Alternatively, it can be provided that the camera is moved in order to take an overall picture of the charge.

According to a preferred development of the invention, the computing unit is set up to assess the safety relevance of a moving load. This prevents annoying false alarms.

It goes without saying that driver assistance systems for a combination of a motor vehicle and a trailer with a camera for capturing cargo on the trailer at different times as image data, a computing unit for evaluating the captured image data with regard to a change in the arrangement of the cargo and for determining a reaction to the change in the arrangement of the charge and with a reaction unit for carrying out a determined reaction on the basis of the detected change are advantageous.

The computer program product according to one embodiment of the invention carries out the steps of a method according to the preceding description if the computer program product runs on a computer, in particular an in-vehicle computer. When the program in question is used on a computer, the computer program product has an effect, namely the detection of insufficient secured cargo on a trailer, which poses a potential danger to road traffic.

According to a preferred development of the invention, the target reaction signal of the training data has a warning signal which is directed to a driver.

It is advantageous if the warning signal is designed as a haptic, optical or audio warning signal. For example, haptic warning signals can be used to apply vibration signals to objects with which a driver is in contact. For example, a steering wheel or an area of a driver's seat can be subjected to a vibration signal. Alternatively or additionally, the warning signal can be designed as an optical signal which is shown on a display, for example on a head-up display. As an alternative or in addition, audio warning signals, i.e. sounds, are also conceivable.

It goes without saying that a method for training an artificial neural network of a driver assistance system is advantageous with the following steps: providing at least one pair of image data and a target reaction signal; Feeding the artificial neural network with the image data; Determining an actual reaction signal based on the forward feeding; Backfeeding the artificial neural network due to a difference between the actual response signal and the target response signal.

The computer program product according to one embodiment of the invention carries out the steps of a method according to the preceding description if the computer program product runs on a computer, in particular an in-vehicle computer. When the program in question is used on a computer, the computer program product produces an effect, namely machine learning or training a KNN to detect a change in the position of a load on a trailer of a motor vehicle.

• 1 ein Blockschaltbild eines KNN gemäß einer Ausführungsform der Erfindung;
• 2 ein Blockschaltbild einer Ausführungsform der Erfindung;
• 3 ein Blockschaltbild einer Ausführungsform der Erfindung;
• 4 beispielhafte Bilddaten gemäß einer Ausführungsform der Erfindung;
• 5 eines schematische Prinzipskizze einer Ausführungsform der Erfindung;

The present invention is explained in more detail below with reference to the exemplary embodiments given in the schematic figures of the drawings. It shows:

• 1 a block diagram of a KNN according to an embodiment of the invention;
• 2 a block diagram of an embodiment of the invention;
• 3 a block diagram of an embodiment of the invention;
• 4 exemplary image data according to an embodiment of the invention;
• 5 a schematic diagram of an embodiment of the invention;

The accompanying drawings are intended to provide further understanding of the embodiments of the invention. They illustrate embodiments and, in conjunction with the description, serve to explain principles and concepts of the invention. Other embodiments and many of the advantages mentioned result from the drawings. The elements of the drawings are not necessarily shown to scale with respect to one another.

In the figures of the drawings, elements, features and components that are the same, have the same function and have the same effect — unless stated otherwise — are each provided with the same reference symbols.

1 shows a block diagram of a training system 10 according to an embodiment of the invention. The training system 10 includes an interface 12 and an evaluation unit 20 with a KNN 22 , The KNN 22 includes several neurons that are in 1 simplified with 108a-f are shown. The neurons form 108a . b an input layer 102 who have favourited Neurons 108c . d . e form a hidden layer 104 and the neuron 108f forms an output layer 106 ,

The neurons 108a . b the input layer 102 are over the interface 12 with image data 16 forward fed. The image data 16 is in the neurons 108a . b the input layer weighted with initial weights. It can be provided that the image data 16 are broken down into several signal components and the signal components are weighted. It can be provided that one or more functions are applied to the weighted input data. The evaluation of the function forms the output value of a neuron 108a . b that to the neurons 108c . d . e the next lower layer, i.e. the hidden layer 104 , are passed as input values. It can be provided that the hidden layer 104 has several layers.

Similar to the input layer 102 are going to the neurons 108c . d . e weighted input values passed to the hidden layer and one or more functions are applied to the weighted input values. The evaluation of the functions, which is applied to the weighted input values, form the output values of the neurons 108c . d . e , These output values are sent to the neurons of the output layer 106 passed as input values. In 1 are the neurons of the output layer 106 exemplary as a neuron 108f shown. The neuron 108f calculated from the input values by the neurons 108c . d . e the hidden layer 104 were passed, weighting the input values and using one or more functions on the weighted input values an output value. An actual reaction signal can be derived from this output value 24 derived. This process is also known as the forward feed of a KNN.

In a next step, the actual reaction signal 24 with the target reaction signal 18 which of the evaluation unit 20 over the interface 12 was passed, compared.

The next step is the topology of the individual layers 102 . 104 . 106 of the KNN 22 adjusted so that the KNN 22 for the transferred image data 16 the target reaction signal 18 calculated. Adapting the topology 26 may include changing weights, adding connections between neurons, removing connections between neurons, and / or changing functions that are applied to weighted input values. This process is also known as the reverse feed of a KNN.

The 2 shows a block diagram of a method for training a KNN according to an embodiment of the invention. The process includes the steps S1 -S4.

In the step S1 becomes a pair of signals from image data 16 and a target response signal 18 provided.

In the step S2 becomes the KNN 22 with the image data 16 forward fed.

In the step S3 becomes an actual reaction signal 24 due to the forward feed S2 determined.

In the step S4 becomes the artificial neural network 22 due to a difference between the actual reaction signal 24 and the target response signal 18 reverse fed. This changes the topology 26 of the KNN, in particular weights, in order to improve the determination of actual reaction signals due to the forward feeding.

The 3 shows a block diagram of a method for detecting a change in position of a load on a trailer of a motor vehicle. In the step St1 image data of the load are recorded on the trailer at a first time by means of a camera that can be connected to the motor vehicle. In the step St 2 image data of the load are recorded at a second point in time on the trailer by means of the camera. In the step St3 a change in the arrangement of the cargo between the first time and the second time is determined. In the step St 4 a reaction to the detected change is determined by means of a computing unit that can be connected to the motor vehicle.

4 shows a schematic example of image data 200 according to an embodiment of the invention. The image data 200 were taken with a reversing camera of a vehicle and show a trailer 201 with the vehicle on a towbar 209 is to be connected. The trailer includes a substructure 202 as well as a structure that with a tarpaulin 203 is covered. The plans 203 is by means of a fastener 205 , which is designed as a rope, on the substructure 202 attached by pulling the rope through eyelets of the tarp 203 and through holes in the substructure 202 is led. The image data 200 are in a vehicle interior on a monitor 207 displayed. In addition, a computing unit (not shown) is set up, the image data 200 evaluate.

5 shows a schematic diagram of a motor vehicle 213 with a trailer hitch 209 as well as with a swiveling camera 211 , The camera 211 is designed as a fisheye camera with adjustable optical zoom. The camera is accordingly 211 trained to capture image data of a suitable image area with a suitable magnification factor. A computing unit (not shown) is set up to determine the suitable image area and the suitable magnification factor.

LIST OF REFERENCE NUMBERS

10

training system

12

interface

14

training data

16

image data

18

Target response signal

20

evaluation

22

artificial neural network

24

Is response signal

26

topology

102

input layer

104

hidden layer

106

output layer

108a-f

neurons

200

image data

201

pendant

202

substructure

203

Plans

205

fastener

207

monitor

209

trailer hitch

211

camera

213

motor vehicle

S1-S4

steps

St1 - St3

steps

Claims (13)

Method for detecting a change in position of a load on a trailer (201) of a motor vehicle (213), in particular a car, with the following steps: - Detecting (St1) the load at a first point in time on the trailer (201) by means of a camera (211) which can be connected to the motor vehicle (213) as first image data (200); - Detecting (St2) the load at a second point in time on the trailer (201) by means of the camera (211) as second image data (200); - detecting (St3) a change in the arrangement of the charge between the first time and the second time; - Determining (St4) a reaction to the detected change using a computing unit that can be connected to the motor vehicle. Procedure according to Claim 1 , the reaction being a warning signal to a driver of the motor vehicle (213) and / or to other road users. Method according to one of the preceding claims, wherein the reaction is a brake signal for braking the motor vehicle (213). The method of any preceding claim, wherein the response is a reduced maximum speed of the motor vehicle (213). Method according to one of the preceding claims, wherein the camera (211) is designed as a reversing camera of the motor vehicle (213). Method according to one of the preceding claims, wherein the camera (211) is displaceable and / or pivotable. Method according to one of the preceding claims, wherein the computing unit is set up to categorize a movement of a fastening means (205) of the load on the trailer (201) with regard to a safety relevance of the movement. Driver assistance system for a combination of a motor vehicle and a trailer (201) for carrying out a method according to one of the preceding claims a camera (211) for capturing cargo on the trailer (201) at different times as image data (200), - A computing unit for evaluating the captured image data (200) with regard to a change in the arrangement of the load and for determining a reaction to the change in the arrangement of the load and with a reaction unit for carrying out a determined reaction on the basis of the detected change. Computer program product with program code means for a method according to one of the Claims 1 - 7 perform. Training system (10) for a vehicle control for detecting a change in position of a load on a trailer (201) of a motor vehicle (213), with - at least one interface (12) in order to receive training data (14), which each have image data (200) (16) and a target reaction signal (18), - an evaluation unit (20), which forms and is set up an artificial neural network (22), in particular an LSTM network, feed the artificial neural network (22) with training data (14) to determine actual reaction signals (24), and to determine a changed topology (26) of the artificial neural network (22), in particular weights, by feeding back the desired reaction signals (18) in the artificial neural network (22), the topology (26) being set up in the vehicle controller for Detecting a change in position of a load to be stored on a trailer (201) of a motor vehicle (213). Training system according to Claim 10 wherein a target reaction signal (18) of the training data (14) has a warning signal which is directed to a driver. Method for training an artificial neural network (22) of a driver assistance system according to Claim 8 with the following steps: - providing (S1) at least one pair of image data (200) and a desired reaction signal (18); - Forward feeding (S2) of the artificial neural network with the image data (200) (16); - Determining (S3) an actual reaction signal (24) based on the forward feed (S2); - Reverse feeding (S4) of the artificial neural network due to a difference between the actual reaction signal (24) and the target reaction signal (18). Computer program product with program code means to carry out the method according to the Claim 12 perform.

Images