DE102019114399A1 - Method for parking a vehicle with a protrusion - Google Patents

Abstract

The present invention relates to a method for parking a vehicle (10) with a protrusion (12) with a driving support system integrated in the vehicle (10) for carrying out the method, wherein during a training mode (T) at least one trajectory (TR) for the vehicle ( 10) determined from a starting position (SP) to a parking position (PP) and during a playback mode (R) the at least one trajectory (TR) for parking the vehicle (10) is provided, with captured environmental information and dimensions of the vehicle (10) and whose overhang (12) are taken into account, the vehicle dimensions being pre-stored in a memory device. The present invention also relates to a driving support system with means for carrying out at least one step of the method. The present invention also relates to a vehicle (10) with the driving assistance system. The present invention further relates to a computer program with instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the method. The present invention also relates to a data carrier signal which the computer program transmits. The present invention also relates to a computer-readable medium having instructions which, when executed by a computer, cause the computer to carry out the steps of the method.

Description

The present invention relates to a method for parking a vehicle with a protrusion with a driving support system integrated in the vehicle for executing the method, with at least one trajectory for the vehicle from a starting position to a parking position being determined during a training mode and the at least one trajectory being determined during a playback mode is provided for parking the vehicle, wherein captured information about the surroundings, vehicle dimensions and protrusion dimensions are taken into account, the vehicle dimensions being pre-stored in a storage device.

The present invention also relates to a driving support system with means for carrying out the steps of the method.

The present invention also relates to a vehicle that has the driving assistance system.

The present invention also relates to a computer program with instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the method.

The present invention also relates to a data carrier signal which the computer program transmits.

The present invention also relates to a computer-readable medium having instructions which, when executed by a computer, cause the computer to carry out the steps of the method.

A method for parking a vehicle with a protrusion is for example from the laid-open specification WO 2016/116246 A1 known. This document relates to a method for operating a driving support system of a motor vehicle, a distance between the at least one distance sensor and an object being determined by means of at least one distance sensor which is arranged on and / or in a body part of the motor vehicle, the object being above a road surface is arranged on which the motor vehicle is located, wherein the distance to the object is determined by means of the at least one distance sensor while the object is in the direction of travel in front of the motor vehicle, and a clearance height is calculated by means of a control device based on the determined distance by means of a control device describes vertical distance between the road surface and a side of the object facing the road surface.

However, known methods have difficulties in carrying out an intuitive and safe parking maneuver when obstacles, in particular ceiling obstacles, have been detected. This can concern different situations, namely an overhang of a vehicle, e.g. when a bicycle is attached to a rear of the vehicle, roof luggage, e.g. when a trolley is placed on top of the vehicle, a person standing upright extending through the sunroof of the vehicle, or a trailer attached to the vehicle.

The present invention is based on the object of specifying a reliable, cost-effective and safe parking maneuver when obstacles, in particular ceiling obstacles, have been detected.

The problem is solved by the independent claims. Advantageous refinements are given in the subclaims.

• Aktivieren des Trainingsmodus, der die folgenden Schritte aufweist, falls das Fahrzeug erstmalig von seiner Startposition zur Parkposition fährt:
  • Erfassen der Umgebungsinformation durch ein Sensorsystem, die Information über Deckenhindernisse beinhaltet, und
  • Speichern der mindestens einen Trajektorie, der Umgebungsinformation und der Fahrzeugabmessungen in einem Speichersystem,
  • Bestimmen und Speichern einer Schwellenwertbox für akzeptierbare Überstandsabmessungen unter Berücksichtigung der erfassten Umgebungsinformation;
• wenn das Fahrzeug zu seiner Startposition fährt, Starten des Wiedergabemodus, der die folgenden Schritte aufweist, falls die Schwellenwertbox gespeichert ist:
  • Prüfen, ob ein Überstand vorhanden ist, und
  • Simulieren eines Parkvorgangs des Fahrzeugs und seines optionalen Überstands entlang der Trajektorie unter Berücksichtigung der Umgebungsinformation, der Fahrzeugabmessungen und der Überstandsabmessungen.

In particular, the present invention specifies a method for parking a vehicle with a driving assistance system integrated in the vehicle for executing the method, with at least one trajectory for the vehicle being determined from a starting position to a parking position during a training mode and the at least one trajectory being determined during a playback mode is provided for parking the vehicle, wherein captured information about the surroundings, vehicle dimensions and protrusion dimensions are taken into account, the vehicle dimensions being pre-stored in a memory device, with the following steps:

• Activating the training mode, which has the following steps, if the vehicle drives from its starting position to the parking position for the first time:
  • Acquisition of the environmental information by a sensor system, which contains information about ceiling obstacles, and
  • Storing the at least one trajectory, the information about the surroundings and the vehicle dimensions in a storage system,
  • Determining and storing a threshold value box for acceptable protrusion dimensions taking into account the detected environmental information;
• when the vehicle drives to its starting position, starting the playback mode, which has the following steps, if the threshold value box is stored:
  • Check whether there is a protrusion and
  • Simulating a parking process of the vehicle and its optional overhang along the trajectory, taking into account the environmental information, the vehicle dimensions and the overhang dimensions.

The present invention also specifies a driving assistance system which has means for performing at least one step of the method.

The present invention also specifies a vehicle which has the driving assistance system.

The present invention also provides a computer program which contains instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the method.

The present invention also specifies a data carrier signal which the computer program transmits.

The present invention also provides a computer readable medium having instructions which, when executed by a computer, cause the computer to carry out the steps of the method.

The basic idea of the invention is to provide an independent method for reliably and safely driving a vehicle with an overhang from a starting position to a parking position. The system repeatedly checks whether the given circumstances are sufficient to complete the parking process without a collision. The parking process is carried out in a straightforward manner, if possible. This prevents damage to the vehicle, the overhang and the ceiling obstacle. The obstacles include, in particular, ceiling obstacles, which can also include obstacles on the side of the vehicle. It is essential that a new, alternative trajectory does not have to be calculated immediately in every case. The driver is actively involved and can actively remove the excess so that the parking process can be carried out with the trajectory provided at the beginning. In particular, carrying out the method requires only minor adjustments to the vehicle, so that the method according to the invention offers a cost-effective solution for safely parking a vehicle with a protrusion. In particular, it is provided that the vehicle does not protrude in the training mode.

To put it another way, this means, for example, that during the training mode the vehicle is driven from the starting position to the destination, its parking position, with all three-dimensional information about the entire scene being recorded using a sensor system, in particular by a combination of sensors such as LIDAR and / or stereo cameras. This information is stored in a database called a virtual card, also known as a VMAP. The dimensions of the vehicle, which were calculated at the time of manufacture, are also stored and used to calculate the three-dimensional position of the vehicle in the real world. After completion of the training mode, the three-dimensional point cloud is used by the virtual map to calculate a maximum inscribed cuboid that can be fitted into the point cloud of the surroundings along a park trajectory. This can be used as a threshold for accepting the size of the protrusion, such as its maximum length, width and height. During the playback mode, the driver preferably drives the vehicle near the starting position and starts an automatic parking function.

According to a modified embodiment of the invention, it is provided that the checking of whether a protrusion is present is carried out by detecting a protrusion by a sensor device, wherein
if no overhang is detected, the vehicle is parked along the at least one stored trajectory,
if an overhang is detected, a driver of the vehicle is informed of the overhang of the vehicle and a confirmation is requested from the driver that the overhang should be maintained during the parking process,
If the driver does not want to keep the protrusion during the parking process, it is waited until the driver has removed the protrusion, and by detecting a protrusion by the sensor device is checked again whether the protrusion is present, if there is no protrusion, the Vehicle is parked along the at least one stored trajectory,
if the driver wants to maintain the overhang during the parking process, the driving support system requests the overhang dimensions,
the protrusion dimensions are provided,
a vehicle box and a protrusion box for the use of augmented reality are rendered in such a way that the driver can visually verify whether the rendered boxes cover the vehicle and the protrusion, the rendering of the vehicle box and the rendering of the protrusion box based on the vehicle dimensions and the protrusion dimensions based,
the rendered vehicle box and the rendered overhang box are unified into one boundary box,
the bounding box is compared to the threshold box,
if the bounding box does not overlap the threshold value box, the vehicle is parked along the at least one stored trajectory,
if the bounding box overlaps the threshold box, it is calculated whether an alternative trajectory can be used for the parking process in which the bounding box does not overlap the threshold box,
if no alternative trajectory can be used for the parking process, the driver is informed about the overlap in order to ask him to remove the overlap and the playback mode is restarted,
if an alternative trajectory can be used for the parking process, the alternative trajectory is calculated and the vehicle is parked along the alternative trajectory.

According to a modified embodiment of the invention, it is provided that the storage of the at least one trajectory, the environmental information, the vehicle dimensions and / or the threshold value box takes place in a memory system of the vehicle and / or in a memory system of an external server. Storage in the vehicle's storage system enables fast data access. Saving on the storage system of the external server means that the data does not take up the limited storage capacity of the memory of the storage system of the vehicle with storage data and that the data can always be kept up-to-date, even if the vehicle has not been parked in the parking position for a long time is.

According to a modified embodiment of the invention, it is provided that the sensor system has at least one LIDAR sensor and / or one or more stereo cameras and / or the sensor device. It has been found that particularly advantageously useful data for generating the threshold value box and the bounding box can be obtained with these sensors.

According to a modified embodiment of the invention, it is provided that the storage system has a database, so that the at least one trajectory and the environmental information are stored as a virtual map in the database of the storage system. It has been found that the use of this data in a virtual map is particularly suitable for determining an overlap between the threshold value box and the bounding box.

According to a modified embodiment of the invention it is provided that the determination of the threshold value box for acceptable protrusion dimensions is carried out by calculating a maximum possible limit box for the vehicle and its protrusion that fits into a point cloud of the environmental information. It has been found that this is a method that requires relatively little computing power for a reliable and collision-proof result.

According to a modified embodiment of the invention, it is provided that the sensor device consists of proximity sensors of the vehicle for detecting the protrusion. It has been found that proximity sensors are already sufficient to achieve the object of the invention, so that their use offers a cost-effective solution.

According to a modified embodiment of the invention, it is provided that the sensor device is arranged on a roof of the vehicle, on a tailgate of the vehicle and / or at least on a lamp of the vehicle. It has been found that arranging them at these points leads to a reliable and collision-proof solution.

According to a modified embodiment of the invention, it is provided that the projection dimensions have the length, the width and / or the height of the projection. It has been found that taking these dimensions into account leads to a reliable and collision-proof solution. However, it can be sufficient if only the dimensions of the main axis from the vehicle to the obstacle are taken into account. While this results in less detail, it enables faster calculations. This means, for example, that only the height of the overhang is taken into account for a ceiling object.

According to a modified embodiment of the invention, it is provided that the provision of the projection dimensions by manually providing the projection dimensions by the driver and / or by visually viewing the rendered projection box by the driver and changing or verifying the size of the rendered projection box and / or by determining the Overhang dimensions is carried out by the sensor device and the driving support system. It has been found that these provision options are best suited for daily and intuitive use and enable a reliable and collision-proof parking process. Each option in itself is sufficient. However, a combination of these options can increase security. For example, can A manual correction by the driver ensures that he has not made a mistake in the manual entry or has accidentally used incorrect values. The manual correction by the driver can also ensure that the sensor device has not made any incorrect measurements. It is also possible that the sensor device is used for the determination and a manual entry is made at the same time in order to avoid the driver typing incorrectly in the manual entry or accidentally using incorrect values.

According to a modified embodiment of the invention it is provided that in the event that the boundary box does not overlap the threshold value box, the at least one stored trajectory is updated and the vehicle is then parked along the at least one stored trajectory. In this way, the available information can be continuously updated so that existing data can be used in later parking situations, instead of performing new calculation processes that place a load on the system.

• Erfassen eines Spiegels, der das Fahrzeug und seinen Überstand reflektiert,
• Segmentieren des Spiegels in verschiedene Bereiche, die einen Bereich von Interesse enthalten, und vorzugsweise Extrahieren des Bereichs für eine Vorverarbeitung,
• Berechnen der Spiegelgeometrie und Berechnen von Li n senverzerru ng skoeffizienten,
• Entzerren des Bereichs von Interesse, insbesondere durch Ausführen einer Fischaugenkorrektur,
• Erfassen des Fahrzeugs in dem Bereich von Interesse, insbesondere Extrahieren der Ober- und/oder der Rückseite des Fahrzeugs,
• Bestimmen eines Überstands an der Ober- und/oder Rückseite des Fahrzeugs unter Verwendung eines künstlichen neuronalen Netzwerks, insbesondere eines konvolutionellen neuronalen Netzwerks.

According to a modified embodiment of the invention, it is provided that to check whether a protrusion is present, by detecting a protrusion by the sensor device, if a mirror and its mirror reflections are available, the detection of the mirror reflections by the sensor device and the processing of the mirror reflections a processing unit has the following steps:

• Detection of a mirror that reflects the vehicle and its overhang,
• Segmenting the mirror into different areas containing an area of interest and preferably extracting the area for preprocessing,
• Calculating the mirror geometry and calculating line distortion coefficients,
• Rectifying the area of interest, especially by performing fisheye correction,
• Detecting the vehicle in the area of interest, in particular extracting the top and / or rear of the vehicle,
• Determining a protrusion on the top and / or rear of the vehicle using an artificial neural network, in particular a convolutional neural network.

The use of a mirror and the mirror reflection enables a reliable detection of an overhang. This can be used as an alternative to sensors on the vehicle or in addition to sensors mounted on the vehicle to reduce the likelihood of errors. Traffic mirrors in particular are used as mirrors. Traffic mirrors are permanently installed convex mirrors and are regularly part of the street equipment. They are mainly used in areas with difficult visibility and are intended to improve visibility. The appearance of traffic mirrors is not uniformly defined. Traffic mirrors usually consist of a convexly curved mirror surface that is mounted on a rectangular or round carrier plate. This means that more area of the confusing place is shown on the traffic mirror, but the mirror reflection is shown distorted. This distorted display is always compensated for in order to ensure reliable protrusion detection with focus on the area of interest. The use of an artificial neural network then helps to reliably evaluate the captured image. Depending on the complexity of the system, it is also possible to determine the dimensions of the overhang.

Artificial neural networks are computing systems that are vaguely inspired by biological neural networks that make up animal brains. The neural network itself is not an algorithm, but rather a framework for many different machine learning algorithms that work together and process complex data inputs. Such systems learn to carry out tasks using examples, generally without being programmed with task-specific rules. An artificial neural network is based on a collection of connected units, or nodes, called artificial neurons, that freely model the neurons in a biological brain. Each connection, like the synapses in a biological brain, can carry a signal from one artificial neuron to another. An artificial neuron that receives a signal can process it and then signal additional artificial neurons connected to it. In common implementations of artificial neural networks, the signal at a connection between artificial neurons is a real number and the output of each artificial neuron is calculated by a nonlinear function of the sum of its inputs. The connections between artificial neurons are called edges. Artificial neurons and edges usually have a weight that adapts as learning progresses. The weight increases or decreases the strength of the signal on a link. Artificial neurons can have a threshold value so that the signal is only transmitted when the total signal crosses this threshold value. Typically, artificial neurons become layers summarized. Different layers can perform different types of transformations on their inputs. Signals travel from a first layer, also known as the input layer, to a last layer, also known as the output layer, possibly after having passed through the layers several times. The original goal of the artificial neural network approach is to solve problems in the same way that a human brain would.

Convolutional neural networks use a variation of multilayer perceptrons that are designed to require minimal preprocessing. Convolutional neural networks use relatively little preprocessing compared to other image classification algorithms. This means that the network learns the filters that were developed by hand in traditional algorithms. This independence from previous knowledge and human effort in feature design is a great advantage.

• Erfassen des Überstands durch das optische Kamerasystem, vorzugsweise ein Videoüberwachungssystem, das in der Nähe der Parkposition installiert ist, und
• Übertragen der erfassten Daten vom optischen Kamerasystem zum Fahrzeug über ein separates V2X-basiertes Steuersystem. Durch eine sorgfältige und regelmäßige Wartung des externen optischen Kamerasystems kann eine zuverlässige Überstandserfassung gewährleistet werden. Defekte Komponenten des externen optischen Kamerasystems können schnell identifiziert werden, so dass Kollisionen mit höherer Wahrscheinlichkeit vermieden werden können als mit Sensoreinrichtungen, die ausschließlich am Fahrzeug angeordnet sind.

According to a modified embodiment of the invention, it is provided that to check whether a protrusion is present by detecting a protrusion by the sensor device, an external optical camera system is used as a sensor device for detecting the protrusion, the following steps being carried out:

• Detecting the overhang by the optical camera system, preferably a video surveillance system installed in the vicinity of the parking position, and
• Transfer of the recorded data from the optical camera system to the vehicle via a separate V2X-based control system. Careful and regular maintenance of the external optical camera system can ensure reliable protrusion detection. Defective components of the external optical camera system can be identified quickly, so that collisions can be avoided with a higher probability than with sensor devices that are exclusively arranged on the vehicle.

A video surveillance system, also known as a CCTV camera system, can produce images or recordings for surveillance or other private purposes. Cameras can be either video cameras or digital cameras.

V2X is a well-known short form of vehicle-to-everything communication. Vehicle-to-everything communication is the transfer of information from a vehicle to an entity that can influence the vehicle, and vice versa. It is a vehicle communication system that supports other more specific communication types such as V2I, a short form of Vehicle-to-Infrastructure, V2N, a short form of Vehicle-to-Network, V2V, a short form of Vehicle-to-Vehicle, V2P, a short form of Vehicle-to -Pedestrian, V2D, short for Vehicle-to-Device, and V2G, short for Vehicle-to-Grid. The main motivations for V2X are road safety, traffic efficiency and energy savings. There are two types of V2X communication technologies, depending on the underlying technology used, namely WLAN-based and cell-based.

According to a modified embodiment of the invention, it is provided that the computer-readable medium is a computer-readable medium integrated in the vehicle or an external computer-readable medium for simulating the parking process of the vehicle along the trajectory. For a computer-readable medium integrated in the vehicle, the computer-readable medium is preferably a component of the driving assistance system. In particular, in the case of an external computer readable medium, it is intended that the computer readable medium be a mobile device. Mobile devices can be smartphones, smartwatches, data glasses, tablets, laptops or similar functional mobile devices.

These and other aspects of the invention will be apparent and illustrated in the embodiments described below. Individual features that are shown in the embodiments can form an aspect of the present invention on their own or in combination. Features of the various embodiments can be transferred from one embodiment to another embodiment.

• 1 ein zweiteiliges Ablaufdiagramm des erfindungsgemäßen Verfahrens;
• 2 eine schematische Darstellung einer Spiegelreflexion für eine bevorzugte Ausführungsform des erfindungsgemäßen Verfahrens;
• 3 eine schematische dreidimensionale Darstellung eines Fahrzeugs mit einem Überstand auf seiner Oberseite in einem Parkhaus, wobei das Fahrzeug eine Fahrzeugbox bildet und der Überstand eine Überstandsbox bildet, die zusammen eine Begrenzungsbox bilden, gemäß einer ersten Ausführungsform der Erfindung;
• 4 die schematische dreidimensionale Darstellung des Fahrzeugs mit seinen Überstand gemäß 3 während eines Trainingsmodus, wobei Lampenkörper Deckenhindernisse bilden; und
• 5 die schematische dreidimensionale Darstellung des Fahrzeugs mit seinem Überstand gemäß 3 während eines Wiedergabemodus, wobei untere Ränder von Lampenkörperkästen einen oberen Rand einer Schwellenwertbox bilden.

Show it:

• 1 a two-part flowchart of the method according to the invention;
• 2 a schematic representation of a mirror reflection for a preferred embodiment of the method according to the invention;
• 3 a schematic three-dimensional representation of a vehicle with a protrusion on its upper side in a parking garage, wherein the vehicle forms a vehicle box and the protrusion forms a protrusion box, which together form a delimitation box, according to a first embodiment of the invention;
• 4th the schematic three-dimensional representation of the vehicle with its overhang according to 3 during a training mode with lamp bodies forming ceiling obstructions; and
• 5 the schematic three-dimensional representation of the vehicle with its overhang according to 3 during a playback mode wherein bottom edges of lamp body boxes form a top edge of a threshold box.

1 Figure 11 shows a method of parking a vehicle 10 with a supernatant 12 by one in the vehicle 10 Integrated driving support system for carrying out the method, wherein during a training mode T at least one trajectory TR for the vehicle 10 from a starting position SP to a parking position PP determined and during a playback mode R. the at least one trajectory TR for parking the vehicle 10 is provided taking into account captured environmental information, vehicle dimensions and projection dimensions, the vehicle dimensions being pre-stored in a memory device. To this end includes 1 two figures, namely 1a and 1b . 1a shows a training mode T . 1b shows a playback mode R. . This means that the inventive method according to 1a according to the method according to the invention 1b precedes at least once to determine the threshold box.

• - Erfassen der Umgebungsinformation durch ein Sensorsystem T2, wobei die Umgebungsinformation Information über Deckenhindernisse 14 enthält,
• - Speichern der mindestens einen Trajektorie, der Umgebungsinformation und der Fahrzeugabmessungen in einem Speichersystem T3, und
• - Bestimmen und Speichern einer Schwellenwertbox für akzeptierbare Überstandsabmessungen unter Berücksichtigung der erfassten Umgebungsinformation T4.

According to 1a becomes the training mode T activated if the vehicle 10 for the first time from its starting position SP to the parking position PP drives, with the training mode T is activated T1 by performing the following steps:

• - Acquisition of the environmental information by a sensor system T2 , the environment information information about ceiling obstacles 14th contains,
• - Storing the at least one trajectory, the environmental information and the vehicle dimensions in a storage system T3 , and
• - Determination and storage of a threshold value box for acceptable protrusion dimensions taking into account the captured environmental information T4 .

• - Prüfen, ob ein Überstand 12 vorhanden ist, durch Erfassen des Überstands durch eine Sensoreinrichtung R1,
• - falls kein Überstand 12 erfasst wird, Parken des Fahrzeugs 10 entlang der mindestens einen gespeicherten Trajektorie R2,
• - falls ein Überstand erfasst wird, Informieren eines Fahrers des Fahrzeugs 10 über den Überstand 12 des Fahrzeugs 10 und Anfordern einer Bestätigung des Fahrers, das der Überstand während des Parkvorgangs beibehalten werden soll R3,
• - falls der Fahrer den Überstand 12 während des Parkvorgangs nicht beibehalten möchte, warten, bis der Fahrer den Überstand 12 entfernt hat, und erneutes Prüfen, ob der Überstand 12 vorhanden ist, durch Erfassen des Überstands 12 durch die Sensoreinrichtung, wobei, wenn kein Überstand 12 vorhanden ist, das Fahrzeug 10 entlang der mindestens einen gespeicherten Trajektorie geparkt wird R4,
• - falls der Fahrer den Überstand 12 während des Parkvorgangs beibehalten möchte, Anfordern der Überstandsabmessungen durch das Fahrunterstützungssystem R5,
• - Bereitstellen der Überstandsabmessungen R6,
• - Rendern einer Fahrzeugbox 18 und Rendern einer Überstandsbox 20 R7 zum Verwenden von Augmented Reality derart, dass der Fahrer visuell überprüfen kann, ob die gerenderten Boxes 18, 20 das Fahrzeug 10 und den Überstand 12 abdecken, wobei das Rendern der Fahrzeugbox 18 und das Rendern der Überstandsbox 20 auf den Fahrzeugabmessungen und den Überstandsabmessungen basieren,
• - Vereinigen der gerenderten Fahrzeugbox 18 und der gerenderten Überstandsbox 20 zu einer Begrenzungsbox 22 R8,
• - Vergleichen der Begrenzungsbox 22 mit der Schwellenwertbox R9,
• - falls die Begrenzungsbox 22 die Schwellenwertbox nicht überlappt, Parken des Fahrzeugs 10 entlang der mindestens einen gespeicherten Trajektorie TR R10,
• - falls die Begrenzungsbox 22 die Schwellenwertbox überlappt, Berechnen, ob eine alternative Trajektorie ATR für den Parkvorgang verwendet werden kann, bei der die Begrenzungsbox 22 die Schwellenwertbox nicht überlappt R11,
• - falls für den Parkvorgang keine alternative Trajektorie ATR verwendet werden kann, Informieren des Fahrers über die Überlappung, Auffordern zum Beseitigen der Überlappung und erneutes Starten des Wiedergabemodus R R12,
• - falls die alternative Trajektorie ATR für den Parkvorgang verwendet werden kann, Berechnen der alternativen Trajektorie ATR und Parken des Fahrzeugs 10 entlang der alternativen Trajektorie ATR R13.

According to 1b becomes the playback mode R. started if the threshold box is already saved when the vehicle 10 to its starting position SP takes the following steps:

• - Check for a protrusion 12 is present by detecting the protrusion by a sensor device R1 ,
• - if there is no overhang 12 is detected, parking the vehicle 10 along the at least one stored trajectory R2 ,
• - if an overhang is detected, informing a driver of the vehicle 10 about the supernatant 12 of the vehicle 10 and requesting confirmation from the driver that the protrusion should be maintained during the parking process R3 ,
• - in case the driver has the excess 12 does not want to maintain during the parking process, wait for the driver to get the protrusion 12 removed, and rechecking that the supernatant 12 is present by detecting the protrusion 12 by the sensor device, if there is no protrusion 12 is present, the vehicle 10 is parked along the at least one stored trajectory R4 ,
• - in case the driver has the excess 12 would like to maintain during the parking process, request the protrusion dimensions by the driving assistance system R5 ,
• - Providing the protrusion dimensions R6 ,
• - rendering a vehicle box 18th and rendering a protrusion box 20th R7 to use augmented reality in such a way that the driver can visually check whether the rendered boxes 18th , 20th the vehicle 10 and the supernatant 12 cover, rendering the vehicle box 18th and rendering the protrusion box 20th are based on the vehicle dimensions and the protrusion dimensions,
• - Merge the rendered vehicle box 18th and the rendered overhang box 20th to a bounding box 22nd R8 ,
• - Compare the bounding box 22nd with the threshold box R9 ,
• - if the bounding box 22nd the threshold box does not overlap, parking the vehicle 10 along the at least one stored trajectory TR R10 ,
• - if the bounding box 22nd overlaps the threshold box, computing whether an alternate trajectory ATR can be used for the parking process where the boundary box 22nd the threshold box does not overlap R11 ,
• - if there is no alternative trajectory for the parking process ATR can be used, informing the driver of the overlap, prompting to remove the overlap and restarting the playback mode R. R12 ,
• - if the alternative trajectory ATR can be used for the parking process, calculate the alternative trajectory ATR and parking the vehicle 10 along the alternative trajectory ATR R13 .

3 shows a schematic three-dimensional representation of the vehicle 10 with its supernatant 12 on its top in a parking garage, with the vehicle at its starting position SP stands, and where the vehicle 10 a vehicle box 18th forms and the supernatant 12 a protrusion box 20th that together form the bounding box 22nd form, according to a first embodiment of the invention.

4th shows the schematic three-dimensional representation of the vehicle 10 with its supernatant 12 according to 3 during a training mode T , with lamp body ceiling obstacles 14th form. According to 1a is the training mode T activated because the vehicle 10 initially with activated training mode T exemplary from its starting position SP to the parking position PP moves T1 . This means that the sensor system records the environmental information T2 , the information about the ceiling obstacles 14th contains. The at least one trajectory, the information about the surroundings and the vehicle dimensions are then stored in a storage system T3 . As a final step of the method, the threshold value box for acceptable protrusion dimensions is determined and stored taking into account the captured environmental information T4 . This is the training mode T completed.

5 shows the schematic three-dimensional representation of the vehicle 10 with its supernatant 12 according to 3 during a playback mode R. , with lower edges of ceiling obstacles 14th , which are designed as lamp body boxes, form an upper edge of the threshold value box. In the scene shown, one step is comparing the bounding box 22nd with the threshold box R9 . It should be noted that the bounding box 22nd in 5 is not shown in full, but only the protrusion box 20th that overlaps the threshold box as part of the bounding box 22nd . Hence an alternative trajectory ATR calculated at which the bounding box 22nd the threshold box does not overlap R11 . After the alternative trajectory ATR has been calculated, a parking operation of the vehicle 10 along the alternative trajectory ATR at the parking position PP executed R13 .

According to a modified embodiment of the invention, it is provided that the storage of the at least one trajectory TR , the environment information, the vehicle dimensions T3 and / or the threshold value box in a storage system of the vehicle 10 and / or is executed in a storage system of an external server.

According to a modified embodiment of the invention, it is provided that the sensor system has at least one LIDAR sensor and / or one or more stereo cameras and / or the sensor device.

According to a modified embodiment of the invention, it is provided that the memory system has a database so that the at least one trajectory TR and the environment information is stored as a virtual map in the database of the storage system.

According to a modified embodiment of the invention it is provided that the determination of the threshold value box for acceptable projection dimensions T4 by calculating a maximum possible bounding box 22nd for the vehicle 10 and its supernatant 12 which fits into a point cloud of the environmental information.

According to a modified embodiment of the invention, it is provided that the sensor device has proximity sensors of the vehicle 10 to measure the overhang 12 having.

According to a modified embodiment of the invention, it is provided that the sensor device on a roof of the vehicle 10 , on a tailgate of the vehicle 10 and / or on at least one lamp of the vehicle 12 is arranged.

According to a modified embodiment of the invention, it is provided that the protrusion dimensions have a length, a width and / or a height.

According to a modified embodiment of the invention it is provided that the provision of the projection dimensions R6 by manually providing the overhang dimensions by the driver and / or by visually viewing the rendered overhang box 20th by the driver and correcting or verifying the size of the rendered protrusion box 20th and / or by determining the protrusion dimensions by the sensor device and the driving support system.

According to a modified embodiment of the invention, it is provided that in the event that the boundary box 22nd the threshold box does not overlap the at least one stored trajectory TR updated and then the vehicle 10 along the at least one stored trajectory TR is parked R10 .

• - Erfassen eines Spiegels 24, der das Fahrzeug 10 und seinen Überstand 12 reflektiert,
• - Segmentieren des Spiegels 24 in verschiedene Bereiche, die einen Bereich von Interesse enthalten, und vorzugsweise Extrahieren des Bereichs zur Vorverarbeitung,
• - Berechnen der Spiegelgeometrie und Berechnen von Linsenverzerrungskoeffizienten,
• - Entzerren des Bereichs von Interesse, insbesondere durch Ausführen einer Fischaugenkorrektur,
• - Erfassen des Fahrzeug 10 im Bereich von Interesse und insbesondere Extrahieren der Ober- und/oder Rückseite des Fahrzeugs 10,
• - Bestimmen eines Überstands 12 an der Ober- und/oder Rückseite des Fahrzeugs unter Verwendung eines künstlichen neuronalen Netzwerks, insbesondere eines konvolutionellen neuronalen Netzwerks.

According to a modified embodiment of the invention, as shown schematically in 2 is shown, it is provided that to check whether a protrusion 12 is present by detecting the protrusion 12 by the sensor device R1 if a mirror 24 and its mirror reflections 26th are available, the mirror reflections 26th detected by the sensor device and the mirror reflections 26th processed by a processing unit, performing the following steps:

• - Detecting a mirror 24 who is the vehicle 10 and its supernatant 12 reflected,
• - segmenting the mirror 24 into different areas containing an area of interest, and preferably extracting the area for preprocessing,
• - Calculating the mirror geometry and calculating lens distortion coefficients,
• - rectifying the area of interest, in particular by performing a fisheye correction,
• - Detecting the vehicle 10 in the area of interest and in particular extracting the top and / or rear of the vehicle 10 ,
• - Determine a supernatant 12 on the top and / or rear of the vehicle using an artificial neural network, in particular a convolutional neural network.

• - Erfassen des Überstands 12 durch das optische Kamerasystem, vorzugsweise ein Videoüberwachungssystem, das in der Nähe der Parkposition PP installiert ist, und
• - Übertragen der erfassten Daten vom optischen Kamerasystem zum Fahrzeug 10 über ein separates V2X-basiertes Steuersystem.

According to a modified embodiment of the invention it is provided, although this is not shown in any of the figures, that to check whether a protrusion 12 is present by detecting the protrusion 12 by the sensor device R1 , an external optical camera system as a sensor device for detecting the overhang 12 using the following steps:

• - Detecting the excess 12 by the optical camera system, preferably a video surveillance system, which is in the vicinity of the parking position PP is installed, and
• - Transfer of the recorded data from the optical camera system to the vehicle 10 via a separate V2X-based control system.

List of reference symbols

10

vehicle

12

Got over

14th

Ceiling obstruction

18th

Vehicle box

20th

Overhang box

22nd

Bounding box

24

mirror

26th

Mirror reflections

TR

Trajectory

SP

Starting position

PP

Parking position

ATR

Alternative trajectory

T

Training mode

T1

Activate training mode

T2

Acquisition of environmental information

T3

Storing the at least one trajectory, the information about the surroundings and the vehicle dimensions

T4

Determine and store a threshold box for acceptable protrusion dimensions

R.

Playback mode

R1

Check for a protrusion by detecting a protrusion

R2

If no overhang is detected, the vehicle is parked along the at least one stored trajectory

R3

If an overhang is detected, informing the driver of the vehicle of the overhang of the vehicle and requesting confirmation from the driver that the overhang should be maintained during the parking process

R4

If the driver does not want to maintain the protrusion during the parking process, wait until the driver has removed the protrusion and again check whether the protrusion is present by the sensor device, if there is no protrusion, the vehicle along the at least one stored trajectory is parked

R5

If the driver wishes to maintain the protrusion during the parking process, request the protrusion dimensions from the driving assistance system

R6

Providing the protrusion dimensions

R7

Rendering a vehicle box and rendering a protrusion box

R8

Unify the rendered vehicle box and the rendered protrusion box into one bounding box

R9

Compare the bounding box with the threshold box

R10

If the bounding box does not overlap the threshold box, parking the vehicle along the at least one stored trajectory

R11

If the bounding box overlaps the threshold box, calculate whether an alternative trajectory can be used for the parking event in which the bounding box does not overlap the threshold box

R12

If no alternative trajectory can be used for the parking process, inform the driver about the overlap, request to remove the overlap and restart the playback mode

R13

If an alternative trajectory can be used for the parking process, calculate the alternative trajectory and park the vehicle along the alternative trajectory

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• WO 2016/116246 A1 [0007]

Claims (19)

A method for parking a vehicle (10) with a driving support system integrated in the vehicle (10) for carrying out the method, wherein during a training mode (T) at least one trajectory (TR) for the vehicle (10) from a starting position (SP) to a parking position (PP) determined and during a playback mode (R) the at least one trajectory (TR) for a parking process of the vehicle (10) is provided, with captured environmental information, vehicle dimensions and protrusion dimensions being taken into account, the vehicle dimensions being pre-stored in a memory device with the following steps: - If the vehicle (10) drives for the first time from its starting position (SP) to the parking position (PP), activate the training mode (T) (T1), which has the following steps: - Detection of the environmental information by a sensor system (T2), which contains information about ceiling obstacles (14); and - Storing the at least one trajectory, the environmental information and the vehicle dimensions in a storage system (T3); - Determination and storage of a threshold value box for acceptable protrusion dimensions (T4) taking into account the detected environmental information; - If the threshold value box is stored when the vehicle (10) drives to its starting position (SP), starting the playback mode (R), which has the following steps: - Check whether there is a protrusion (12); and - Simulating a parking process of the vehicle (10) and its optional overhang (12) along the trajectory (TR) taking into account the environmental information, the vehicle dimensions and the overhang dimensions. Procedure according to Claim 1 , characterized in that - the checking whether a protrusion (12) is present is carried out by detecting the protrusion by a sensor device (R1); - If there is no overhang (12), the vehicle (10) is parked along the at least one stored trajectory (R2); - If there is an overhang, the driver of the vehicle (10) is informed of the overhang (12) of the vehicle (10) and confirmation from the driver is requested that the overhang should be maintained during the parking process (R3); - If the driver does not want to keep the protrusion (12) during the parking process, it is waited until the driver has removed the protrusion (12), and by detecting the protrusion (12) by the sensor device it is checked again whether the protrusion (12 ) is present, wherein, if there is no protrusion (12), the vehicle (10) is parked along the at least one stored trajectory (R4); - If the driver wishes to maintain the overhang (12) during the parking process, the overhang dimensions are requested by the driving support system (R5) for simulating the parking process for the vehicle (10) along the trajectory (TR); - the protrusion dimensions (R6) are provided; - A vehicle box (18) is rendered and a protrusion box (20) is rendered (R7) for using augmented reality such that the driver can visually verify whether the rendered boxes (18, 20) are the vehicle (10) and the Cover overhang (12), the rendering of the vehicle box (18) and rendering of the overhang box (20) based on the vehicle dimensions and the overhang dimensions; - The rendered vehicle box (18) and the rendered overhang box (20) are combined to form a boundary box (22) (R8); - the boundary box (22) is compared with the threshold value box (R9), - if the boundary box (22) does not overlap the threshold value box, the vehicle (10) is parked along the at least one stored trajectory (TR) (R10); - If the boundary box (22) overlaps the threshold value box, it is calculated whether an alternative trajectory (ATR) can be used for the parking process, in which the boundary box (22) does not overlap the threshold value box (R11); - If no alternative trajectory (ATR) can be used for the parking process, the driver is informed of the overlap and requested to remove the overlap, and the playback mode (R) is restarted (R12); and - if the alternative trajectory (ATR) can be used for the parking process, the alternative trajectory (ATR) is calculated and the vehicle (10) is parked along the alternative trajectory (ATR) (R13). Procedure according to Claim 1 or 2 , characterized in that the storage of the at least one trajectory (TR), the environment information, the vehicle dimensions (T3) and / or the threshold value box is carried out in a storage system of the vehicle (10) and / or in a storage system of an external server. Method according to one of the preceding claims, characterized in that the sensor system has at least one LIDAR sensor and / or one or more stereo cameras and / or the sensor device. Method according to one of the preceding claims, characterized in that the storage system has a database, so that the at least one trajectory (TR) and the environmental information are stored as a virtual map in the database of the storage system. Method according to one of the preceding claims, characterized in that the determination of the threshold value box for acceptable protrusion dimensions (T4) is carried out by calculating a maximum possible limit box (22) for the vehicle (10) and its protrusion (12), which is converted into a point cloud Environmental information fits. Method according to one of the preceding claims, characterized in that the sensor device has proximity sensors of the vehicle (10) for detecting the protrusion (12). Method according to one of the preceding claims, characterized in that the sensor device is arranged on a roof of the vehicle (10), on a tailgate of the vehicle (10) and / or on at least one lamp of the vehicle (12). Method according to one of the preceding claims, characterized in that the protrusion dimensions have a length, a width and / or a height. Method according to one of the preceding claims, characterized in that the provision of the projection dimensions (R6) by manually providing the projection dimensions by the driver and / or by visually viewing the rendered projection box (20) by the driver and correcting or verifying the size of the rendered projection box (20) and / or by determining the protrusion dimensions by the sensor device and the driving support system. Method according to one of the preceding claims, characterized in that in the event that the boundary box (22) does not overlap the threshold value box, the at least one stored trajectory (TR) is updated and then the vehicle (10) is updated along the at least one stored trajectory ( TR) is parked (R10). Method according to one of the preceding claims, characterized in that to check whether a protrusion (12) is present, by detecting the protrusion (12) by the sensor device (R1), if a mirror (24) and its mirror reflections (26) are available are, the mirror reflections (26) are detected by the sensor device and the mirror reflections (26) are processed by a processing unit, the following steps being carried out: - Detecting a mirror (24), which the vehicle (10) and its overhang (12) reflected; - segmenting the mirror (24) into different areas containing an area of interest, and preferably extracting the area for preprocessing; - calculating the mirror geometry and calculating lens distortion coefficients; Rectifying the area of interest in particular by performing a fisheye correction; - Detecting the vehicle (10) in the area of interest and in particular extracting the top and / or rear of the vehicle (10); and - determining an overhang (12) on the top and / or rear of the vehicle (10) using an artificial neural network, in particular a convolutional neural network. Method according to one of the preceding claims, characterized in that to check whether there is a protrusion (12) by detecting the protrusion (12) by the sensor device (R1), an external optical camera system as a sensor device for detecting the protrusion (12) is used, with the following steps: - Detecting the overhang (12) by the optical camera system, preferably a video surveillance system, which is installed in the vicinity of the parking position (PP); and - transmission of the captured data from the optical camera system to the vehicle (10) via a separate V2X-based control system. Driving support system with means for performing at least one step of the method according to at least one of the preceding claims. Vehicle (10) with a driving assistance system according to the preceding claim. Computer program with instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the method according to at least one of the Claims 1 to 13 execute. Data carrier signal that the computer program after Claim 13 transmits. Computer readable medium containing instructions which, when executed by a computer, cause the computer to perform the steps of the method according to at least one of Claims 1 to 13 execute. Computer readable medium according to Claim 18 , characterized in that the computer-readable medium is a computer-readable medium integrated in a vehicle or an external computer-readable medium for simulating the parking process of the vehicle (10) along the trajectory (TR).

Images