DE102014223259B4 - Method for estimating the course of a lane in a lane - Google Patents

Abstract

Method for estimating the course of a lane (F) from the environment data in front of a vehicle recorded by at least one environment sensor, with the following process steps: Lane width (B) of the lane (F) is determined from the intensity profile of the reflected radar beams, b) creating at least one hypothesis about the number of lanes (F1, F2, F3) from the lane width (B) determined in step a), and / or - the detection of other road users on the lane (F) with regard to their movement behavior, and / or - from the information on a digital map, and / or - from traffic sign recognition, c) determining a probability model for the position of the center of the lane (FM1, FM2, FM3) the lanes (F1, F2, F3) based on the hypothesis of the number of lanes (F1, F2, F3), andd) using the probability model, determining a value with the greatest probability value as the position of the center of the lane (FM1, FM2, FM3) of the lanes (F1, F2, F3).

Description

The invention relates to a method for estimating the course of the lane of a roadway.

Driver assistance systems of vehicles that work on the basis of lane information support the driver, for example, when changing lanes, in lane guidance or when overtaking. In such driver assistance systems, image sensors are used to detect the lanes, which detect the traffic area in front of the vehicle. From this image data, lane markings are detected that indicate the boundaries of the lane and thus their course. However, there are situations in which such lane markings do not exist or cannot be detected by image sensors, for example because they are covered by snow or the view of these lane markings is impaired by the weather conditions due to heavy snowfall, rain or fog. In such a case, it is impossible not only for image sensors but also for the human driver to perceive these lane markings.

From the publication by KASPRZAK, Wlodzimierz; NIEMANN, Heinrich: Adaptive road recognition and ego-state tracking in the presence of obstacles; International Journal of Computer Vision, Vol. 28, 1998, a recursive method for determining lane parameters, such as lane width, number of lanes and curve shape and for locating a host vehicle from image sequences of a mono camera is known. The lane is recognized in the steps of vanishing point recognition, classification of an object contour, determination of the course of the lane and determination of the lane. Several vanishing points and several roadway hypotheses are pursued in parallel and the best hypothesis is selected for further use. When determining the vanishing points, all of the line-like objects that determine the roadway are detected and processed from the image sequences recorded by the mono camera. In particular, these are also the median and edge strips of the roadway.

the DE 10 2012 224 498 A1 describes a method for lane recognition and for recognizing a lane of a host vehicle. In this known method, stationary obstacles, such as guard rails and median strips, are detected by means of a radar sensor and the width of the road is calculated from the distance to these stationary obstacles. In addition, the width of the lane of the host vehicle is determined from the image data from an image sensor. The number of lanes can be determined from the calculated lane width and the determined lane width of the host vehicle. In the DE 10 2010 013 403 A1 a method is described for determining a lane geometry in the area of a vehicle on the basis of image data from a camera or an image acquisition device. On the basis of a Gaussian distribution, estimated values of a road and lane geometry are generated from the image data, measurements from the past being used. This known estimation method is based on assumptions of averaging and normal distribution. Instead of using data from the past or normalized distributions, the use of GPS data and data from the vehicle kinematics together with image data from an image acquisition device is proposed.

Also the DE 10 2011 109 569 A1 is based on the task of specifying a method for recognizing a lane that can reliably detect the lane despite incomplete or missing lane markings. In this method, the edge of the lane is detected by a camera regardless of the existence of lane markings and the lane is estimated from the position of the edge of the lane or the position of the edge of the lane is at least taken into account when the lane is recognized.

Also the DE 10 2012 008 780 A1 is based on the problem that, for example, lane departure warning systems cannot function or only function to a limited extent if there are no lane markings. This problem is solved by a method for detecting at least one edge of the road, in particular an unmarked edge of the road, with which a lane model can be created for the vehicle. This method is based on the image data of the traffic area recorded in front of the vehicle as well as a certain number of previously recorded images of a reversed optical flow. Areas to be assumed to be drivable are selected from these image data, search areas are defined and dominant orientations are determined within the search areas, by means of which at least one vanishing point used to create the track model is generated.

Furthermore describes the DE 10 2013 018 561 A1 likewise a method in which, due to the use of general environmental features, reliable lane recognition is realized without the need for lane markings. This method consists in an analysis and evaluation of environmental features in the area of a lane of a vehicle lying ahead, each environmental feature being assigned a position and orientation in a recorded image and corresponding environmental features occurring multiple times in the image being determined. Based on the distribution Road course hypotheses are generated from correspondences, whereby self-similarities of the environmental features are taken into account, irrelevant correspondences with a length with the value zero are filtered and discarded when generating the road course hypotheses.

A method for recognizing the condition of a road surface using a 3D camera as a passive system is described by the DE 10 2012 101 085 A1 the applicant, according to which the 3D camera recorded at least one image of the surroundings in front of the vehicle, from the image data of the 3D camera along a plurality of lines, height profiles of the road surface transverse to the direction of travel of the vehicle are determined in order to use the determined height profiles To determine the condition of the road surface. From the determined height profile, road boundaries, for example curbs, can also be recognized.

The object of the invention is to specify a further method for recognizing the lanes of a roadway of a vehicle, which method can reliably detect the lanes despite missing or unrecognizable roadway markings.

This object is achieved by a method having the features of claim 1 and the features of claim 2.

1. a) Bestimmen der Fahrbahnbreite der Fahrbahn vor dem Fahrzeug aus den mittels eines Radarsensors aufgenommenen Umfelddaten, wobei die Fahrbahnbreite der Fahrbahn aus dem Intensitätsverlauf der reflektierten Radarstrahlen ermittelt wird,
2. b) Erstellen wenigstens einer Hypothese über die Anzahl der Fahrstreifen aus
   • - der mit Verfahrensschritt a) ermittelten Fahrbahnbreite, und/oder
   • - der Detektion der anderen Verkehrsteilnehmer auf der Fahrbahn hinsichtlich ihres Bewegungsverhaltens, und/oder
   • - aus der Information einer digitalen Karte, und/oder
   • - aus einer Verkehrszeichenerkennung,
3. c) Ermitteln eines Wahrscheinlichkeitsmodells für die Lage der Fahrstreifenmitte der Fahrstreifen auf der Basis der Hypothese über die Anzahl der Fahrstreifen, und
4. d) Bestimmen eines Wertes mittels des Wahrscheinlichkeitsmodells mit dem größten Wahrscheinlichkeitswert als Lage der Fahrstreifenmitte der Fahrstreifen.

According to the first-mentioned solution, this method for estimating the course of the lane of a roadway from environment data in front of a vehicle recorded by at least one environment sensor comprises the following method steps:

1. a) Determining the lane width of the lane in front of the vehicle from the environment data recorded by means of a radar sensor, the lane width of the lane being determined from the intensity profile of the reflected radar beams,
2. b) Create at least one hypothesis about the number of lanes
   • - the lane width determined with method step a), and / or
   • - the detection of the other road users on the road with regard to their movement behavior, and / or
   • - from the information on a digital map, and / or
   • - from a traffic sign recognition,
3. c) determining a probability model for the position of the center of the lane on the basis of the hypothesis about the number of lanes, and
4. d) Determining a value by means of the probability model with the greatest probability value as the position of the center of the lane.

1. a) Bestimmen der Fahrbahnbreite der Fahrbahn vor dem Fahrzeug aus mittels eines Bildsensors aufgenommen Umfelddaten, wobei ein Höhenprofil des Bereichs vor dem Fahrzeug quer zu dessen Fahrtrichtung aus den Bilddaten des Bildsensors ermittelt und die Fahrbahnbreite der Fahrbahn aus dem Verlauf des Höhenprofils detektiert wird,
2. b) Erstellen wenigstens einer Hypothese über die Anzahl der Fahrstreifen aus
   • - der mit Verfahrensschritt a) ermittelten Fahrbahnbreite, und/oder
   • - der Detektion der anderen Verkehrsteilnehmer auf der Fahrbahn hinsichtlich ihres Bewegungsverhaltens, und/oder
   • - aus der Information einer digitalen Karte, und/oder
   • - aus einer Verkehrszeichenerkennung,
3. c) Ermitteln eines Wahrscheinlichkeitsmodells für die Lage der Fahrstreifenmitte der Fahrstreifen auf der Basis der Hypothese über die Anzahl der Fahrstreifen, und
4. d) Bestimmen eines Wertes mittels des Wahrscheinlichkeitsmodells mit dem größten Wahrscheinlichkeitswert als Lage der Fahrstreifenmitte der Fahrstreifen.

According to the second-mentioned solution, this method for estimating the course of the lane of a roadway from environment data recorded by at least one environment sensor in front of the vehicle comprises the following method steps:

1. a) determining the lane width of the lane in front of the vehicle from environment data recorded by means of an image sensor, a height profile of the area in front of the vehicle being determined transversely to its direction of travel from the image data of the image sensor and the lane width of the lane being detected from the course of the height profile,
2. b) Create at least one hypothesis about the number of lanes
   • - the lane width determined with method step a), and / or
   • - the detection of the other road users on the road with regard to their movement behavior, and / or
   • - from the information on a digital map, and / or
   • - from a traffic sign recognition,
3. c) determining a probability model for the position of the center of the lane on the basis of the hypothesis about the number of lanes, and
4. d) Determining a value by means of the probability model with the greatest probability value as the position of the center of the lane.

With this method according to the invention, information about the course of the lane can now be generated on the basis of other features when lane markings are not available. For this purpose, the area in which the road is located is first determined, whereby the entire asphalted area is understood by the road, which is divided into several lanes by means of markings. Since the roadway represents a comparatively flat surface that is usually delimited by raised objects such as crash barriers, soundproof walls or waves or at least by a flat area such as a meadow, the roadway can be distinguished from its delimitations with the environment sensor. In this way, with known lane boundaries, the lane width of the lane can be determined from the environment data and a hypothesis can be created about the number of lanes. For example, if the vehicle is on a Motorway and if a width of, for example, 11.25 m is detected, a known width of a lane of a motorway of 3.75 m with associated hard shoulder of two lanes is assumed as a hypothesis about the number of lanes. In the subsequent method step c), the probability for the lane is modeled, ie a probability model for the position of the center of the lane of the lane is determined on the basis of the hypothesis about the number of lanes. Finally, in a last method step d), a value is determined by means of the probability model with the greatest probability value as the position of the center of the traffic lane.

According to the first-mentioned solution, it is provided that the surroundings data are recorded by means of a radar sensor and the lane width of the lane is determined from the intensity profile of the reflected radar beams. Such a radar sensor system ensures that, even in poor visibility conditions, the development of the edge of roadways, such as crash barriers, but also a grassy area can be detected, since these lead to a stronger reflection than the actual, e.g. asphalted, roadway.

In particular, information about possible lane courses can be generated permanently with a radar sensor that cannot detect any lane markings. This can serve to check the plausibility of the lane courses recognized by an image sensor, such as a camera, or to implement an emergency function in the event of a camera failure. Due to the geometry of the lanes and the wide foresight, errors in the azimuth calibration of the sensors involved can be easily identified. Furthermore, due to the long range of a radar sensor compared to a camera, an estimate of the lane course outside the camera's field of view can be obtained.

According to the second solution mentioned, it is provided that the surroundings data are recorded by means of an image sensor, a height profile of the area in front of the vehicle is determined transversely to its direction of travel from the image data of the image sensor and the lane width of the lane is detected from the course of the height profile. With a camera as an image sensor, the smooth roadway can be separated from raised or unstructured areas on the basis of a determined height profile. Further features, such as the texture, can also be used to identify the roadway. Finally, it is also possible to use objects recognized as moving to identify the lane.

• e) mittels den Umfelddaten des Radarsensors und mittels der Umfelddaten des Bildsensors jeweils die Verfahrensschritte b) bis d) durchgeführt werden, und
• f) aus den gemäß Verfahrensschritt d) als Lage der Fahrstreifenmitte der Fahrstreifen bestimmten zwei Wahrscheinlichkeitswerten ein Wert als Lage der Fahrstreifenmitte kombiniert wird.

According to a preferred embodiment of the invention, it is provided that

• e) method steps b) to d) are carried out in each case by means of the environment data of the radar sensor and by means of the environment data of the image sensor, and
• f) a value is combined as the position of the center of the lane from the two probability values determined in accordance with method step d) as the position of the center of the traffic lane.

• f) zusammengeführt, indem die Fahrstreifenhypothesen der beiden Auswertekanäle überlagert werden und hieraus ein einziger Wert als Lage der Fahrstreifenmitte erzeugt wird. Vorzugsweise wird aus einer derart ermittelten Lage der Fahrstreifenmitte der Fahrstreifenverlauf bestimmt.

With this method, two evaluation channels are available, namely a first evaluation channel with which the environment data of a radar sensor are evaluated, and a second evaluation channel with which the environment data of an image sensor is evaluated. The results with regard to the position of the middle of the lane, which result from the two evaluation channels, are in the method step

• f) merged in that the lane hypotheses of the two evaluation channels are superimposed and a single value is generated from this as the position of the center of the lane. The course of the lane is preferably determined from such a position of the center of the lane.

An advantageous further development results from the fact that, in addition to the hypothesis about the number of lanes, a hypothesis is created about the lane widths of the lanes and, on the basis of the hypothesis about the number of lanes and the lane widths of the lanes, a probability model for the location of the center of the lane is created is determined.

Finally, it is provided according to a further development that a Gaussian distribution with a mean value and a variance is used as the probability model.

• 1 ein Ablaufdiagramm eines Ausführungsbeispieles gemäß der Erfindung, und
• 2 Diagramme zur Erläuterung des Ausführungsbeispiels gemäß 1.

The invention is explained in more detail below with reference to the accompanying figures. Show it:

• 1 a flow chart of an embodiment according to the invention, and
• 2 Diagrams to explain the embodiment according to 1 .

To carry out the method according to the invention according to 1 a vehicle with two environment sensors, namely a radar sensor and a camera, is provided. The environment data of these environment sensors each form an evaluation channel, namely a first evaluation channel for evaluating the radar data of the radar sensor, its function with the method steps S1 until S4 described and a second evaluation channel for evaluating the image data of the camera, its function with the process steps S5 until S8 to be discribed.

The radar sensor generates radar data from the surroundings in front of the vehicle, which in a first process step S1 to be provided.

In one process step S2 the width of the lane is derived from this radar data B. a roadway F. determined in the direction of travel of the vehicle by a left and right shoulder R1 and R1 limited, being at the left marginal strip R1 a central guardrail MLP connects. Such a traffic environment in front of the vehicle is shown schematically in 2a) shown.

The width of the lane B. is determined on the basis of differing intensities of the reflected radar beams. For this purpose, the area in which the road is located is first determined F. is located, which represents a comparatively flat surface and raised objects such as guard rails, soundproof walls or waves or at least an uneven area such as a meadow as a border R. is limited. This allows the roadway from the radar sensor F. be distinguished from their limitations. The edge development, such as guardrails, or grass-like, especially uneven edge strips R1 and R2 lead to stronger reflections of the radar beams than from the flat and homogeneous surface of the roadway F. .

The procedural steps carried out so far S1 and S2 for the first evaluation channel processing the environment data of the radar sensor are also carried out in parallel for the second evaluation channel in accordance with the method steps S5 and S6 performed, which processes the environmental data recorded by the camera.

The camera generates from the in 2a) with a lane F. presented environment in front of the vehicle image data, which in a first process step S5 of the second evaluation channel.

In a subsequent process step S6 the lane width is derived from this image data B. the roadway F. certainly. For this purpose, the height profile of the roadway F. Detected and evaluated transversely to the direction of travel of the vehicle, so that the area of the even and smooth roadway F. or the unstructured area of the roadway F. of raised or structured areas, such as the boundaries of the roadway F. can be separated. With the camera, other features, such as the texture, can be used to identify the roadway.

The following are the procedural steps S3 and S4 of the first evaluation channel together with the procedural steps S7 and S8 of the second evaluation channel, since identical functions are implemented there that are in one process step S9 be merged.

In the procedural steps S3 and S7 a hypothesis about the number of existing lanes and the associated lane width is derived on the basis of the radar data or the image data. For this purpose, the process step S2 respectively. S6 certain lane width B. used. Is it known that the vehicle is, for example, on a motorway and for the width of the lane B. If a value of 11.25 m is detected, the hypothesis is that there are two lanes and one hard shoulder, since it is known that a lane on a motorway is 3.75 m wide. It is also possible to create a hypothesis on the number of lanes in terms of their movement behavior by detecting the other road users on the roadway. Finally, it is also possible to use the information from a digital map, e.g. a navigation system, alone or in addition to generate the hypothesis about the number of lanes in order to obtain information about the number of existing lanes and the hard shoulder, since there are also motorway sections where there is no continuous hard shoulder.

However, prior knowledge of the lane width can also be incorporated, for example, that the left lane is narrower than the right lane in a construction site.

Information from traffic sign recognition can also be used to create a hypothesis about the number of lanes. In the case of a lane with more than two lanes, for example, a traffic sign indicates the number of lanes and how long they are available. Construction site signs indicating construction sites can also be detected and their information used to create a hypothesis about the number of lanes. If construction site signs are recognized, it is very likely that the width of the lanes is narrower and therefore needs tight control. In addition, such traffic signs can contain the information about the direction in which the lanes run when the lane is narrowed, how many lanes are available and what width, for example, the left lane has. For example, there is a traffic sign stating that the width of the lane is less than 2 m, while the usual width of the left lane is 2.50 m to 2.70 m. This information can be compared with the detected roadsides and so on very reliable information about the number and width of lanes can be estimated.

In the embodiment according to 1 and 2 a hypothesis is created that three lanes F1 , F2 and F3 available. The hypothesis about the lane width is derived from this, whereby in the simplest case it is assumed that the lanes are evenly distributed over the lane width.

In one to the procedural step S 3 respectively. S7 subsequent process step S4 respectively. S8 becomes the probability W. for the location of the lanes on the basis of the process step S3 respectively. S7 created hypothesis is modeled.

This can be done, for example, on the basis of the course of the center of the lane. In order to model inaccuracies in perception, the width of the lane is used B. the probability W. indicated that there is a center of the lane at the corresponding point FM1 , FM2 and FM3 is located, as shown schematically in 2 B) is shown. One possibility for the realization is a parametric modeling in which the probability W. for example as a Gaussian distribution with a mean value µ and a variance σ is pictured. In the example according to 2 three distributions are determined, namely distributed over the width of the lane for each lane from the hypothesis. Corresponding 2c ) become a Gaussian distribution with a mean µ1 and a variance σ1 for the first lane, a mean value µ2 and a variance σ2 for the second lane and an average µ3 and a variance σ3 created for the third lane. The variances of these three Gaussian distributions are derived from the hypothesis about the width of the lane in accordance with the process step S3 respectively. S7 derived.

A discretized representation of the probability distribution is also available 2d ), which shows a 1-dimensional representation. For a practical implementation, however, a 2-dimensional representation as a grid is advantageous. One axis runs transversely to the road surface and a second axis runs in the direction of the road surface. This means that a changing number of lanes over the length of the road surface can also be modeled. In addition to a square grid, it can be advantageous to choose a grid with rectangular cells. The cell size across the lane is selected to be smaller than in the direction of the lane in order to enable a more precise modeling of the lateral position of the center of the lane.

After both the first evaluation channel for the radar sensor as the first data source and the second evaluation channel for the camera as the second data source, a probability modeling of the center of the lane for the lanes according to the hypothesis about the number of lanes according to the method steps S4 and S8 is carried out in the next process step S9 the lane hypotheses of the two evaluation channels are superimposed. This can be done in the parametric representation by association and combination of the mean values µ and the variances σ of the Gaussian distributions or, in the grid-based, i.e. discrete representation, by combining the cell values of the various grids. For example, a binary Bayesian filter or the Dempster-Shafer theory can be used for this.

As a result of this process step S9 are in one process step S10 the three lane courses F1 , F2 and F3 extracted.

These lane courses determined in this way can be used for trajectory planning or a lane-keeping driver assistance function in which the vehicle is guided along the next maximum of the represented probabilities. This maximum represents the center of the lane. For a visual representation in which the width of a possible lane course is also to be displayed, starting from the maximum, the intersection points of the probability values with a minimum value, e.g. the first time falling below the 50% probability, are displayed as Lane boundaries selected.

Further advantages arise in the case of lane markings that are covered by snow or poor visibility conditions, for example due to heavy snowfall, rain or fog, since the driver can then be offered support in lateral guidance of the vehicle based on the estimated lane courses. This can be implemented, for example, using a display in the cockpit that shows the driver how far he is deviating from the center of the estimated lane to the left or right. In particular, however, the estimated course of the lane can also be displayed over the real traffic scene by means of a large-area head-up display. Since the estimate may be imprecise, in particular the estimated center of the lane can be emphasized more strongly than the edge areas, since this is best within the real lane.

In addition to displaying the course of the lane, the driver can be given a visual, acoustic or haptic warning when leaving the lane, similar to a lane-keeping system, or lateral guidance support is implemented via a superimposed steering torque. Compared to the detection of the lane markings, the intervention thresholds have to be relocated further in the direction of the center of the lane due to the greater uncertainty.

Furthermore, additional assistance functions can be made available with the estimated information about the course of the lane.

Claims (6)

Method for estimating the course of the lane of a roadway (F) from environment data recorded by at least one environment sensor in front of a vehicle with the following method steps: a) Determination of the lane width (B) of the lane (F) in front of the vehicle from the environmental data recorded by means of a radar sensor, the lane width (B) of the lane (F) being determined from the intensity profile of the reflected radar beams, b) Creating at least one hypothesis about the number of lanes (F1, F2, F3) - the lane width (B) determined with method step a), and / or - the detection of the other road users on the roadway (F) with regard to their movement behavior, and / or - from the information on a digital map, and / or - from a traffic sign recognition, c) Determining a probability model for the position of the center of the lane (FM1, FM2, FM3) of the lanes (F1, F2, F3) on the basis of the hypothesis about the number of lanes (F1, F2, F3), and d) Determining a value by means of the probability model with the greatest probability value as the position of the center of the lane (FM1, FM2, FM3) of the lanes (F1, F2, F3). Method for estimating the course of the lane of a roadway (F) from environment data recorded by at least one environment sensor in front of a vehicle with the following method steps: a) Determination of the lane width (B) of the lane (F) in front of the vehicle from the environment data recorded by means of an image sensor, a height profile of the area in front of the vehicle transversely to its direction of travel being determined from the image data of the image sensor and the lane width (B) of the lane (F) is detected from the course of the height profile, b) Creating at least one hypothesis about the number of lanes (F1, F2, F3) - the lane width (B) determined with method step a), and / or - the detection of the other road users on the roadway (F) with regard to their movement behavior, and / or - from the information on a digital map, and / or - from a traffic sign recognition, c) Determining a probability model for the position of the center of the lane (FM1, FM2, FM3) of the lanes (F1, F2, F3) on the basis of the hypothesis about the number of lanes (F1, F2, F3), and d) Determining a value by means of the probability model with the greatest probability value as the position of the center of the lane (FM1, FM2, FM3) of the lanes (F1, F2, F3). Procedure according to Claim 1 and 2 , characterized in that e) method steps b) to d) are carried out in each case by means of the environment data of the radar sensor and by means of the environment data of the image sensor, and f) from the method step d) as the position of the center of the lane (FM1, FM2, FM3) of the Lanes (F1, F2, F3) determine two probability values and a value is combined as the position of the center of the lane (FM1, FM2, FM3). Method according to one of the preceding claims, characterized in that b1) in addition to the hypothesis about the number of lanes, a hypothesis about the lane widths of the lanes (F1, F2, F3) is created, and c1) on the basis of the hypothesis about the number of Lanes (F1, F2, F3) and the lane widths of the lanes (F1, F2, F3) a probability model for the position of the middle of the lane (FM1, FM2, FM3) of the lanes (F1, F2, F3) is determined. Procedure according to Claim 3 and 4th , characterized in that the course of the lane is determined from the position of the center of the lane (FM1, FM2, FM3) determined in accordance with method step f). Method according to one of the preceding claims, characterized in that a Gaussian distribution with a mean value and a variance is used as the probability model.

Images