WO2021148178A1

WO2021148178A1 - Method and device for carrying out automated lane guidance of a motor vehicle

Info

Publication number: WO2021148178A1
Application number: PCT/EP2020/084204
Authority: WO
Inventors: Alfred Kuttenberger
Original assignee: Robert Bosch Gmbh
Priority date: 2020-01-21
Filing date: 2020-12-02
Publication date: 2021-07-29
Also published as: DE102020200650A1

Abstract

The invention relates to a method for carrying out automated lane guidance of a motor vehicle (1) for guiding the motor vehicle in a traffic lane (11), wherein the automated lane guidance takes image data into consideration, which is determined by means of at least one first sensor system (3). The method is characterised in that an evaluation of an execution of the automated lane guidance takes place, wherein sensor data is determined for the evaluation by means of a second sensor system (4, 5), wherein, based on the determined sensor data, it is evaluated whether the lane guidance is correctly executed. The invention also relates to a corresponding device.

Description

title

Method and device for operating automated tracking of a

Motor vehicle

The present invention relates to a method for operating automated lane guidance of a motor vehicle for guiding the motor vehicle in a lane, the automated lane guidance taking into account image data which are determined by means of at least one first sensor system. The method is characterized in that an implementation of the automated lane guidance is evaluated, with sensor data being determined for the evaluation by means of a second sensor system, the determined sensor data being used to assess whether the lane guidance is being carried out correctly. The invention also relates to a corresponding device for this.

State of the art

Applications are known from the prior art in which functions for lane recognition and lane keeping are represented in a motor vehicle by means of optical sensors such as video cameras or lidar sensors. Here, for example, images are recorded with a video camera and these are then analyzed in real time using algorithms. The images are also searched for lines that belong to a lane and then the vehicle is kept in the lane by means of automated steering interventions.

For example, patent application DE 10 2013 106 769 A1 is known from the prior art. This document relates to a tachograph in which a lane departure warning assistant and a collision warning assistant are integrated and which combines the functions of the tachograph together with the functions of the lane departure warning assistant and the collision warning assistant, the Tachograph can help with a liability distribution in the event of a traffic situation. The images in front of the motor vehicle and the noises in the vicinity of the motor vehicle are recorded by the tachograph according to the invention and forwarded to the control unit, the aforementioned image and sound data, which can be presented as evidence in the event of a traffic accident, after being compressed into the memory unit can be saved.

In systems of this type, however, incorrect lane recognition by means of the optical sensor system can occur. Due to algorithmic weaknesses, it can happen, for example, that the algorithm detects no line or a faulty line. This can then lead to the vehicle being steered off the road if the driver does not take corrective action.

Disclosure of the invention

In contrast, the method according to the invention advantageously enables a reliable detection of faulty vehicle guidance. Automated driving assistance can therefore be carried out longer and with greater security. A lateral deviation of the vehicle from the road during automated driving is avoided. This is made possible by the features of the invention specified in the independent patent claims. Further refinements of the invention are the subject matter of subclaims.

The method according to the invention for operating automated lane guidance of a motor vehicle for guiding the motor vehicle in a lane, with the automated lane guidance taking into account image data determined by means of at least one first sensor system, is characterized in that an implementation of the automated lane guidance is evaluated , with sensor data being determined for the evaluation by means of a second sensor system, with the determined sensor data of the second sensor system being used to evaluate whether the lane guidance is being carried out correctly.

This can be understood to mean a method for executing and correcting automated lane guidance. The automated lane guidance is supported by a second automated function. So there is a Optimization of a driver assistance function for automated lane recognition and lane guidance.

The automated lane guidance takes place taking into account and evaluating the data of a first sensor system. This first sensor system comprises in particular an optical sensor, for example a video camera or a lidar sensor. The first sensor system can also comprise a combination of several sensors, for example a video camera and lidar sensors. A safety function is now being implemented to support and optimize this automated lane guidance. Another sensor system is used to determine data which can be used to evaluate the actual execution of the automated lane guidance. The aim here is to monitor and check whether the motor vehicle is deviating from the lane, for example whether a road edge marking is being driven over and / or whether the hard shoulder is being driven on. To increase the security during the evaluation, a different type of sensor is advantageously used than in the first sensor system. For example, an acoustic sensor and / or a structure-borne sound sensor and / or a movement sensor and / or an inertial sensor can be used as the sensor type for the second sensor. Consequently, the sensor data determined by means of the second sensor are data of a different data type - compared to the image data which are determined by means of the first (optical) sensor system. This also increases the security of the evaluation.

As a further method step, the method can also include the determination of the image data by means of the first (optical) sensor system. The method can also include the method step of evaluating the image data and / or the method step of executing the automated lane guidance on the basis of the evaluated image data.

In an advantageous embodiment, the method is characterized in that a deviation of the motor vehicle from the lane is determined in order to evaluate the execution of the automated lane guidance.

This is understood to mean that the automated lane guidance defines an intended lane on which the motor vehicle is to be moved. The motor vehicle should deviate from this lane by evaluating the sensor data of the second sensor system are recognized. The lane on which the motor vehicle is moving can also be understood, for example, as the lane. For example, there is a deviation from the lane when the motor vehicle changes from the current lane to an adjacent lane or the hard shoulder. Deviating from this lane can therefore correspond to the motor vehicle departing from the lane.

When the vehicle leaves its current lane and comes onto the hard shoulder, it usually has a different texture than the road, which leads to noise and vehicle movements being generated. For the implementation of this evaluation it is therefore advantageous that a sensor system is installed in the vehicle, which can recognize the noises and / or the movement of the vehicle and it can be concluded from this that the vehicle unintentionally deviated from the lane / lane / road is. Such a sensor system can, for. B. be an acceleration, displacement sensor, which is installed in some chassis to enable active chassis control. Alternatively, the inertial sensors and / or structure-borne noise sensors installed in the vehicle can also be used to determine such signals, which are then evaluated by suitable algorithms.

In one possible embodiment, the method is characterized in that, in order to evaluate the execution of the automated lane guidance, driving over a lane boundary is determined.

This is understood to mean that an agreement from the planned lane / lane / lane can already be recognized when driving over a lateral lane boundary. An incorrect execution of the automatic lane guidance can therefore advantageously already be determined before actually driving on the hard shoulder. Advantageously, this makes use of the fact that on some roads the asphalt is provided with periodic depressions at the lateral boundary, which lead to the normal driver being made aware of the noise and vibrations that occur that he is about to be released To get off the road. As a rule, the driver then takes corrective action. Normal lane demarcation lines can also be designed in such a way that they generate an audible noise when they are crossed. Such vibrations and noises can also be determined by sensors and interpreted by a corresponding evaluation as leaving the lane. If leaving the lane is not in accordance with the planned lane guidance, it is assumed that the automatic lane guidance has not been carried out properly.

In a preferred embodiment, the method is characterized in that, as sensor data of the second sensor system, data are determined which enable an indirect assessment of whether the automated lane guidance is being carried out correctly, with the following in particular being determined:

- noises and / or

- vehicle movements.

This is understood to mean that the data ascertained by means of the second sensor system correspond to a data type which only enables a conclusion to be drawn about the correctness of the execution of the automatic track guidance through an appropriately designed evaluation. Such sensor data are advantageously already determined by existing sensor systems in the motor vehicle. As a result, the costs for implementing this monitoring function can be kept low. For example, an inertial sensor already installed in the ESP system determines the movements of the vehicle. This inertial sensor picks up the vibrations that occur when driving over a lane marking. From these sensor data it can be recognized by a suitably set up evaluation that a road edge marking is currently being driven over.

In an alternative development, the method is characterized in that at least one of the following sensor types is used as the second sensor system:

- a sound sensor;

- a structure-borne sound sensor;

- an acceleration sensor;

- an inertial sensor.

This is understood to mean that the second sensor system detects, for example, as a sound sensor in order to record and evaluate audio data in order to detect a crossing of a lane boundary. Alternatively or additionally, the second sensor system can comprise the inertial sensor of the ESP system in order to use to determine the vibrations of the motor vehicle driving over grooves on the edge of the road.

In an advantageous embodiment, the method is characterized in that it is determined in which direction the motor vehicle deviates from the lane, in particular a deviation from the lane to the right or to the left is determined when the sensor data representing the deviation second sensor system on a right side of the vehicle or a left side of the vehicle.

This is understood to mean that during the evaluation of the automatic lane guidance it is determined in which direction the motor vehicle is deviating from the planned lane. The direction of the deviation is advantageously determined taking into account the same sensor data from the second sensor system. For example, it can be determined on which side of the vehicle the sensor data were determined, or were determined first, in order to determine in which direction the deviation occurs. If such a signal is only recognized on one side, it can be assumed that the vehicle is no longer within the lane and there is a deviation in the direction in which the signal is recognized. For example, when the motor vehicle leaves the lane to the right onto the edge strip, the noises and vibrations characteristic of the edge strip are first determined on the right side of the vehicle. Only when the motor vehicle drives with all its wheels on the strip of beach would it be possible to determine the noises and vibrations on both sides of the vehicle.

In one possible embodiment, the method is characterized in that a measure for correcting the lane guidance is carried out if the lane guidance is recognized as incorrect.

This is understood to mean that both a determination and a correction of faulty automated lane guidance should take place by means of the method. Various technical measures can be initiated to correct this. If, for example, in such a case, the structure-borne sound sensor system detects a signal that indicates that the vehicle is traveling e.g. B. indicates to the right of the road, the function can be activated by gently counter-steering to the left and, if necessary, braking Prevent departure from the lane and a correction of the lane guidance can be achieved. In addition, the driver can be informed of the current situation and, if necessary, requested to intervene.

This can also be understood to mean that the automated lane guidance is continued as long as no defined signals are recognized that indicate incorrect lane guidance.

In a preferred development, the method is characterized in that at least one of the following method steps is carried out in the case of a lane guidance recognized as incorrect:

- Degradation of the data from the first sensor system for the automated lane guidance;

- Execution of a steering intervention taking into account the data determined by means of the second sensor system;

- Execution of counter-steering by means of automated steering intervention in order to return to the lane;

- Intervention in the longitudinal guidance of the motor vehicle, in particular automatic braking of the motor vehicle;

- Cancellation of the automated lane guidance;

- Request to a vehicle occupant to take over the lane guidance;

Information about a fault to a vehicle occupant, in particular to a driver;

- Creating a fault entry in a fault memory of the motor vehicle.

In an alternative embodiment, the method is characterized in that in the case of a lane guidance recognized as incorrect, a measure to improve future automated lane guidance is carried out.

This is understood to mean that the method provides for an optimization of the automated lane guidance taking into account image data. For this purpose, a learning process is started if it has been identified during the evaluation that incorrect lane guidance has taken place. An adaptation of the software algorithm is to be understood as a learning process in particular. The learning process can also include a change in the labels in the image data. This can be an internal and / or external learning process. An internal learning process takes place directly in the Motor vehicle. An external learning process takes place outside the motor vehicle, advantageously at the manufacturer of the motor vehicle, or the assistance system or the software. For this purpose, the error sequence (especially image data and evaluation data) is stored and / or transmitted for external revision. A transmission can take place by physically reading out the error memory or, of course, by radio transmission

The method for operating the automated tracking and evaluating the execution can be implemented, for example, in software or hardware or in a mixed form of software and hardware, for example in a control device and / or regulating device. The approach presented here also creates a device which is designed to carry out, control or implement the steps of a variant of a method presented here in corresponding devices. The object on which the invention is based can also be achieved quickly and efficiently through this embodiment variant of the invention in the form of a device.

In the present case, a device can be understood to mean an electrical device that processes sensor signals and outputs control and / or data signals as a function thereof. The device can have an interface which can be designed in terms of hardware and / or software. In the case of a hardware design, the interfaces can, for example, be part of a so-called system ASIC which contains a wide variety of functions of the device. However, it is also possible that the interfaces are separate, integrated circuits or at least partially consist of discrete components. In the case of a software-based design, the interfaces can be software modules that are present, for example, on a microcontroller alongside other software modules. A driver assistance system, for example a lane keeping system, can also be understood as a device.

Furthermore, this can also include a device for performing the automated driving maneuvers, for example a steering actuator. A sensor system for determining environmental data of the vehicle during operation, for example an optical sensor system, can also be understood as a device. The device can also be understood to be a sensor for determining noises or structure-borne noise and / or a sensor for determining vibrations, for example a movement sensor or an inertial sensor. A computer program product or computer program with program code, which can be stored on a machine-readable carrier or storage medium such as a semiconductor memory, a hard disk memory or an optical memory, and for carrying out, implementing and / or controlling the steps of the method according to one of the embodiments described above is also advantageous is used, especially when the program product or program is executed on a computer or device.

Embodiments

It should be pointed out that the features listed individually in the description can be combined with one another in any technically meaningful way and show further embodiments of the invention. Further features and usefulness of the invention emerge from the description of exemplary embodiments on the basis of the accompanying figures.

From the figures shows:

1 shows a schematic representation of a motor vehicle with an assistance system according to an embodiment of the invention; and

2 shows a representation of the method steps of an embodiment of the invention.

1 shows a schematic representation of a motor vehicle 1 with an assistance system 2 configured as a lane keeping system according to an embodiment of the invention. The motor vehicle 1 moves at speed V in the direction of the arrow shown. There is a right-hand bend in the section of route 10 in front of the motor vehicle. The lane 11 in which the motor vehicle 1 is moving is delimited by a right lateral delimitation 12 (lane edge marking) and is thus separated from a hard shoulder 13.

In the motor vehicle 1 there is an optical first sensor system 3 designed as a camera, which creates image data of the route section ahead. The image data can be evaluated in a control device 6. From this image data Existing road markings can be determined in order to determine the route to be followed. The driver assistance system 2 is activated and independently carries out automated steering interventions in order to guide the motor vehicle 1 along the desired route. The automated steering interventions take place, for example, via a device 7 for automated driving maneuvers, such as a steering actuator. The device 7 can be controlled, for example, via a second control device 8. The second control device 8 can accordingly also be designed as a regulating device. This can be the same control device as the one used for evaluating the image data - or, of course, a separate control device. The motor vehicle 1 has a second sensor system 4 designed as a motion sensor, which, for example, can be an inertial sensor already present in the motor vehicle 1. The motion sensor 4 detects vibrations when the motor vehicle 1 drives over a profiled lateral boundary 12 (lane edge marking), for example. Furthermore, the motor vehicle 1 additionally has a second sensor system 5 configured as a sound sensor, which, for example, perceives a change in noise development due to the rolling behavior of the wheels when the motor vehicle, for example, is at least partially driving on the hard shoulder.

FIG. 2 shows an illustration of the method steps of an embodiment of the invention. The method is started in a first step S1. This is done, for example, by manual activation of the assistance system by the driver. In the next step, the respective assistance function is carried out. A so-called lane keeping function, for example, can be implemented as a function. However, the automatic lane guidance can also be part of another automated driving function. For this purpose, image data are first determined in a step S2 by means of the optical sensor system (for example by means of a video camera). In a step S3, an evaluation of the image data takes place, for example in order to determine a lane in the route section lying ahead on the basis of determined lane markings. On the basis of the determined lane, the actual execution of the lane guidance takes place in step S4, for example by means of automated intervention in the steering. While the automated lane guidance is being carried out, it is checked in B1 whether there is a condition for ending the method. Such a condition can be, for example, the manual abort of the lane guidance or also technical abort criteria, for example that no lane markings or no lane sufficiently recognized by the software. If such a condition (Y branch from B1) is present, the method is terminated in step S10, if necessary after a previous takeover request. If there is no condition for a termination (N branch from B1), the method is carried out further.

In a parallel method step S5, further sensor data are determined by means of the second sensor. This sensor is of a different sensor type than the optical sensor system already mentioned. For example, it is a motion sensor for determining vehicle movements. As an alternative or in addition, it is a sound sensor for determining noises. The determined sensor data are evaluated in step S6 to evaluate the lane guidance that has taken place. For example, if the lane guidance is incorrect, the road edge marking may be driven over. These markings are provided with grooves which stimulate a specific vibration in the motor vehicle. The motion sensor picks up these vibrations. By means of a specific evaluation, characteristic patterns can be recognized which allow the conclusion that the motor vehicle is driving on the lane edge marking. Noises can also be determined by means of the sound sensor or structure-borne sound sensor. On the basis of the detected noises or also noise differences, a specific evaluation can be used to determine whether, for example, the vehicle is driving (at least partially) on the hard shoulder, since this usually has a different nature than the lane. If it is determined that the vehicle is (partially) driving on the hard shoulder, the lane guidance has not been carried out correctly because the motor vehicle has deviated from the regular lane.

In a condition B2 it is checked whether there is currently a situation that suggests incorrect execution of the lane guidance. If this is the case (Y branch from B2), then in step S7, for example, the lane keeping was corrected by a slight automated counter-steering. The automated lane guidance can then be continued, for example, in S2 and S5. In addition, an optimization process is started in S8, for example, in order to reduce future incorrect execution of the lane guidance. Furthermore, information and / or a warning can be output to the driver in S9. If no incorrect execution of the lane guidance is recognized in condition B2 (N branch from B2), the automated lane guidance is regularly continued in steps S2 and S5.

Claims

Expectations

1. A method for operating automated lane guidance of a motor vehicle (1) for guiding the motor vehicle (1) in a lane (11), the automated lane guidance taking into account image data which are determined by means of at least one first sensor system (3), thereby characterized in that an evaluation of an implementation of the automated lane guidance is carried out, with sensor data being determined for the evaluation by means of a second sensor system (4, 5), the determined sensor data of the second sensor system (4, 5) being used to evaluate whether the lane guidance is carried out correctly will.

2. The method according to claim 1, characterized in that a deviation of the motor vehicle (1) from the lane (11) is determined to evaluate the execution of the automated lane guidance.

3. The method according to any one of the preceding claims, characterized in that in order to evaluate the execution of the automated lane guidance, driving over a lane boundary (12) is determined.

4. The method according to any one of the preceding claims, characterized in that data are determined as sensor data of the second sensor system (4, 5) which enable an indirect assessment of whether the automated lane guidance is being carried out correctly, with the following in particular being determined:

- noises and / or

- vehicle movements.

5. The method according to any one of the preceding claims, characterized in that at least one of the following sensor types is used as the second sensor system (4, 5):

- a sound sensor;

- a structure-borne sound sensor; - an acceleration sensor;

- an inertial sensor.

6. The method according to any one of the preceding claims, characterized in that it is determined in which direction the vehicle (1) deviates from the lane (11), in particular deviating from the lane (11) to the right or to the left , is determined when the sensor data representing the deviation of the second sensor system (4, 5) are determined on a right side of the motor vehicle or a left side of the motor vehicle.

7. The method according to any one of the preceding claims, characterized in that in the case of a lane guidance recognized as incorrect, a measure for correcting the lane guidance is carried out.

8. The method according to any one of the preceding claims, characterized in that at least one of the following method steps is carried out in the case of a lane guidance recognized as incorrect:

- Degradation of the data from the first sensor system (3) for the automated tracking;

- Execution of a steering intervention taking into account the data determined by means of the second sensor system (4, 5);

- Cancellation of the automated lane guidance;

- Request to a vehicle occupant to take over the lane guidance;

Information about a fault to a vehicle occupant, in particular to a driver;

- Creating a fault entry in a fault memory of the motor vehicle (1).

9. The method according to any one of the preceding claims, characterized in that in the case of a lane guidance recognized as incorrect, a measure to improve future automated lane guidance is carried out.

10. Device (2, 3, 4, 5, 6, 7) which is set up to carry out the method according to one of claims 1 to 11.

11. Computer program that when the program is executed by a

The device according to claim 10 is set up to carry out the method according to one of claims 1 to 9.

12. Machine-readable storage medium on which the computer program according to claim 11 is stored.