DE102018112177A1 - Lane detection based on lane models - Google Patents

Abstract

An improved tracking of a traffic lane for a vehicle should be provided. Thus, there is provided a method comprising the steps of providing individual lanes of the lane, extracting lane features (8) from the frames, and providing a first lane model (10). Then, the first lane model (10) is updated with the lane features (12). A second lane model (13) based on an outlier detection algorithm is provided. A tracked confidence value for the updated first lane model and a static confidence value for the static second lane model (13) are calculated. Based on both confidence values, it is decided whether to use the updated first lane model or the static second lane model to calculate current lane information.

Description

The present invention relates to a method of following a lane with the steps of providing lane images, extracting lane features from the frames, and providing a lane model. Further, the present invention relates to a driver assistance system and apparatus each adapted to perform such a method. In addition, the present invention is directed to a computer program product capable of executing the lane following method.

Various driver assistance systems are known in the art. Such driver assistance systems can have different levels of automation, starting with the level 1 such as the ACC (automatic distance control), to the level 5 completely autonomous driving. To implement these driving functions that assist the driver, or even the fully autonomous driving functions, it is necessary that such systems can detect objects, obstacles or other road users ahead. In addition, it is also important in most of the systems to assess whether such objects or obstacles are in the lane or driving range that is likely to be traversed by the own vehicle, for example to adjust the speed and / or a certain safety margin to a preceding vehicle observed. In such a case, if, for example, a preceding vehicle is detected but is not in the lane of the own vehicle, it is either not necessary to warn the driver or automatically decelerate the vehicle. Therefore, it is necessary not only to know each time whether front objects such as preceding vehicles are basically present, but also to know as reliably as possible whether or not a vehicle driving ahead of the own vehicle is located in the lane area or traversed by the own vehicle , Other specific applications for using a lane following method are possible.

Lane detection using camera systems was one of the first developed computer vision ADAS (advanced driver assistance system) functions. Such lane detection systems may be based on lane models for multi-camera and fisheye camera systems.

The document WO 2013/022153 A1 describes an apparatus and method for detecting a traffic lane. The device includes a camera module that captures an image. Further, the apparatus includes a control unit that extracts a plurality of feature points from the image, performs a lane fitting to connect the plurality of feature points with a single line, and tracks the adjusted lane. A display unit displays the tracked lane, wherein the lane tracking adjustment includes performing a short-distance adjustment based on feature points existing in a short-range range among the plurality of feature points, determining an offset representing a lateral inclination of the lane on the basis a result of the short-distance adaptation and performing a curve fitting based on the offset.

Further, the document discloses US 2013/0141520 A1 a lane tracking system with a camera. The camera is designed to receive images of a road from a wide-angle field of view. One or more lane boundaries are detected, each lane boundary comprising a plurality of lane boundary points. A reliability-weighted model driving lane line is adapted to the plurality of points.

The object of the present invention is to provide a method for tracking a lane in an improved quality. Furthermore, a corresponding device and a corresponding computer program product are to be provided.

According to the present invention, this object is achieved by a method, an apparatus and a computer program product as defined in the independent claims.

• - Bereitstellen von Einzelbildern der Fahrspur,
• - Extrahieren von Fahrspurmerkmalen aus den Einzelbildern und
• - Bereitstellen eines ersten Fahrspurmodells, sowie
• - Aktualisieren des ersten Fahrspurmodells mit den Fahrspurmerkmalen,
• - Bereitstellen eines statischen zweiten Fahrspurmodells auf der Basis eines Ausreißerdetektionsalgorithmus,
• - Berechnen eines verfolgten Vertrauenswerts für das aktualisierte erste Fahrspurmodell und eines statischen Vertrauenswerts für das statische zweite Fahrspurmodell und
• - Entscheiden auf der Basis beider Vertrauenswerte, ob das aktualisierte erste Fahrspurmodell oder das statische zweite Fahrspurmodell zum Berechnen von gegenwärtigen Informationen über die Fahrspur verwendet werden soll.

Thus, a method of tracking a lane is provided with the steps

• Providing lanes of the traffic lane,
• - Extract lane features from the frames and
• - Provide a first lane model, as well
• - Updating the first lane model with the lane features,
• Providing a static second lane model based on an outlier detection algorithm,
• Calculating a tracked confidence score for the updated first lane model and a static confidence score for the static second lane model and
• Decide whether to use the updated first lane model or the static second lane model to calculate current lane information based on both confidence values.

For example, a vehicle traveling on or in a lane may need to track the lane to provide assistance information to the driver as current lane information. The vehicle may include one or more cameras (eg, a multi-camera system or a fisheye camera system) for providing images (also called frames) of the lane. Lane features such as heading angle, lateral position (with respect to the vehicle), lane width and curvature are extracted from the frames. A first lane model, preferably for predicting a lane position, etc., is provided. The first lane model is updated with current lane features (such as heading angle, lateral position, lane width or curvature).

In addition, a static second lane model is provided. The term "static" means that the model is not updated with current lane features. Rather, the second lane model is based on a fixed outlier detection algorithm. This algorithm distinguishes between outliers and nests according to given criteria. With respect to the extracted lane features, a confidence score is calculated for each model based on, for example, a statistical hypothesis. Specifically, a tracked confidence value for the updated first lane model and a static confidence value for the static second lane model are calculated. Using these confidence values, it is decided to use the updated first lane model or the static second lane model for further calculations with respect to the tracked lane. Preferably, the model that provides the higher confidence value is used. This guarantees that the calculated current lane information (for example, lane coordinates) is more accurate.

In a preferred embodiment, the updating of the first lane model is performed by predicting a new version of the first lane model using Kalman filtering and adapting the new version of the first lane model to the extracted lane features. Such a procedure makes it possible to easily update the model with a plurality of feature elements. These feature elements describe the lane and form a state vector for the Kalman filter.

Preferably, the first lane model and the second lane model track a lane geometry over time. Thus, the lane models preferably provide current information about lane dimensions, lane position, and / or lane orientation. Such information may be classified in relation to the type of lane.

In a further advantageous development, the lane features relate to lane markings, and prior to updating the first lane model with lane markers, the lane markers are divided into two separate left boundary and one right boundary sets by calculating a division line using the first lane model. Preferably, the dividing line is a second order polynomial. The sign of the lateral distance from the dividing line may be used to classify the lane markings into left lane markings and right lane markings. Such a separation into a left and right lane boundary ensures high quality when adapting the corresponding lane model.

In another embodiment, the static lane model is evaluated based on a best fit lane hypothesis for the extracted lane features using a sample match and a least square fit. In particular, a specific lane model hypothesis can be calculated for each frame. This can be done in two steps: Sample Matching (RANSAC) and Least Square Fit. This results in current model parameters used for the static lane model.

According to another embodiment, a predetermined number of features are used for each of a left and a right border of the lane to pass through the static second lane model Perform polynomial fitting to calculate. For example, six features are randomly selected for each boundary to compute a best fit candidate model.

According to another embodiment, one of the first lane model and the second lane model is used in a first time step, and this model is still used in a second time step immediately following the first time step, if the direct confidence value and the static confidence value are below a predetermined threshold lie. This means that the lane model is fixed between two time steps. Consequently, a change of the lane model is only performed if the respective confidence value is high enough, i. H. above the predetermined threshold.

Moreover, the direct confidence value for a hypothesis regarding the first lane model may be calculated, and the static confidence value is calculated for a hypothesis regarding the second lane model based on the lane characteristics from the frames. Consequently, lane markings can be reliably classified with respect to the two different models.

In another embodiment, the confidence score for each of the first and second lane models is calculated by first calculating independent confidence values for the left and right limits of the lane and then calculating an average of these independent confidence values for the confidence value for the respective first or second lane model. Consequently, the different lane models can be weighted and compared. In particular, a confidence score can be calculated for each model hypothesis to weight and compare the models. The mean value as a common trust is more reliable than any single trustworthiness.

In a preferred embodiment, detected lane markers are separated into enrollees and outliers with respect to the first lane model and only the mates are used to update the first lane model. In this case, model matching is done only with locator marks. As a result, the model matching has improved quality.

The above-mentioned object can also be achieved by a driver assistance system configured to perform a method according to any lane following method as described above. Any other lane tracking device may be configured to perform the described methods. Modifications and advantages of the method according to the invention can also apply to the driver assistance system according to the invention or the device according to the invention.

In addition, the above object is also achieved by a computer program product having program code means stored in particular in a computer readable medium to perform the lane keeping method according to any one of the preceding claims when the computer program product is executed on a computer device of an electronic control unit.

Further features of the invention will become apparent from the claims, the figures and the description of the figures. The features and combinations of features mentioned above in the description as well as the features and feature combinations mentioned below in the description of the figures and / or shown alone in the figures can be used not only in the respectively specified combination but also in other combinations without departing from the scope of the invention , Thus, embodiments of the invention are to be regarded as encompassed and disclosed, which are not explicitly shown and explained in the figures, however, emerge and can be produced by separated combinations of features from the embodiments explained. Embodiments and combinations of features are also to be regarded as disclosed, which thus do not have all the features of an originally formulated independent claim. Moreover, embodiments and combinations of features, in particular by the embodiments set out above, are to be regarded as disclosed which go beyond the feature combinations set out in the back references of the claims or deviate therefrom.

• 1 ein Prinzipblockdiagramm eines Fahrspurerfassungssystems;
• 2 ein Blockdiagramm des Verarbeitungsschemas eines Fahrspurmodellmoduls;
• 3 ein Diagramm eines Fahrspurmodells;
• 4 ein Diagramm für die Trennung von Fahrspurmarkierungen;
• 5 ein Diagramm für die Eigenbewegungskompensation und
• 6 ein Aktivitätsdiagramm für das Fahrspurmodellmodul von 2.

The present invention will now be described in more detail in connection with the attached figures, which show in:

• 1 a principle block diagram of a lane detection system;
• 2 a block diagram of the processing scheme of a lane model module;
• 3 a diagram of a lane model;
• 4 a diagram for the separation of lane markings;
• 5 a diagram for the self-motion compensation and
• 6 an activity diagram for the lane model module of 2 ,

The following embodiments are preferred examples of the present invention.

1 shows a lane detection system. A system input module 1 provides, for example, a front image, a left image, a right image and a back image. These images are sent to a lane marker detection module that detects lane markers from the images (also referred to herein as frames). A transformation module 3 transforms the image information of the lane marker detection module 2 for example, in world coordinates. Thereafter follows a feature tracking module 4 the features like lane markings. For this analysis, the feature tracking module 4 Vehicle geometry data from the system input module 1 receive.

The vehicle geometry information from the system input module 1 along with feature tracking information from the feature tracking module 4 be in a lane position logic module 5 which may also be called lane tracking module. Consequently, the lane position logic module receives 5 Information about detected lane markings (ie, for example, bounding boxes and inner feature positions) preferably in vehicle coordinates. From this information the module tries 5 to adapt a lane consisting of preferably two parallel polynomials of second order as left and right boundaries and the position and orientation of the own vehicle within that lane. Furthermore, the module 5 Track the road geometry over time. The output of the module 5 becomes a lane boundary classification module 6 which classifies, for example, the type and color of the lane boundary. In a system output module 7 may be a lane type of the lane boundary classification module 6 and providing an estimated lane geometry for the current timestamp as results from the whole lane detection system.

2 shows a processing scheme of the lane position logic module or lane tracking module 5 from 1 , data segments 8th from the feature tracking module 4 be in a comparison unit 9 inputting a predicted lane from a lane prediction unit 10 with the data segments 8th from the feature tracking module 4 compares. The lane prediction unit 10 based on a first lane model. It can odometry data from an odometry unit 11 receive. The comparison unit 9 provides information for an update unit 12 that updates lane tracking, ie, updates the first lane model.

A calculation unit 13 calculates in parallel a best model from the measurements. In particular, it calculates for each timestamp a most appropriate lane hypothesis from the incoming lane marking features, ie the data segments from the feature tracking module 4 , The calculation module uses this 13 preferably a combination of RANSAC and Least Square Fit. The result may be called "static best model" (also called static second lane model herein).

A decision logic 14 uses confidence values for a given hypothesis to decide whether the updated or tracked first model from the update unit 12 or the static second lane model from the calculation unit 13 should be used to correct the lane tracking. For this correction receives a correction unit 15 the respective information from the decision unit 14 , Consequently, the lane model for the lane prediction unit becomes 10 updated or the lane model remains unchanged with respect to the previous time step. It is preferable to hold to the result of the previous time step when both the static second lane model and the tracked first lane model provide too low confidence values. The current lane model eventually provides output information 16 which may include classification information such as lane type or lane geometry.

A lane model class may combine two line classes for the left and right boundaries to a lane model. It includes methods for adapting the lane model through the 2D feature positions. This class also calculates the overall confidence for the lane.

der die Fahrspur beschreibt, besteht aus den folgenden Merkmalselementen: $$\widehat{x_k}=\left(\begin{array}{c}Fahrtrichtungswinkel\\ seitlichePosition\\ Fahrspurbreite\\ Krümmung\end{array}\right)$$

wobei der Fahrtrichtungswinkel h (vergleiche 3) die Drehung des Fahrzeugs 17 innerhalb der Fahrspur 18 beschreibt Die Fahrspur 18 weist eine linke Grenze 19 und eine rechte Grenze 20 auf. Diese Grenzen 19 und 20 definieren die Fahrspurbreite w der Fahrspur 18. Die seitliche Position It des Fahrzeugs 17 ist der Euklidische Abstand des Fahrzeugursprungs von der Mittellinie 21 der Fahrspur 18. Die Krümmung der Fahrspur ist der quadratische Betrag a des Polynoms zweiter Ordnung der Straßengrenzen (siehe folgende Beschreibung der Least-Square-Fit des Modells).In the following, an algorithm for the method of tracking a traffic lane according to the invention will be described in more detail. Preferably, the tracking of the first lane model is performed by means of a Kalman filter. The Kalman filter state vector $\widehat{x_k}.$

which describes the lane consists of the following feature elements: $$\widehat{x_k}=\left(\begin{array}{c}FaHrtricHtunGswinkel\\ seitlicHePOsitiOn\\ FaHrspurbreite\\ KrümmunG\end{array}\right)$$

the direction of travel angle h (cf. 3 ) the rotation of the vehicle 17 within the lane 18 describes the lane 18 has a left boundary 19 and a right border 20 on. These limits 19 and 20 define the lane width w of the lane 18 , The lateral position It of the vehicle 17 is the Euclidean distance of the vehicle origin from the centerline 21 the lane 18 , The curvature of the lane is the square amount a the second-order polynomial of the road boundaries (see the following description of the least square fit of the model).

The state parameters are estimated based on 2D features of the lane markers in vehicle coordinates. In contrast to what can be found in the literature, in the present sensor model, the 2D features are not used directly to modify the state. Instead, the 2D features are used to adjust a lane for the current timestamp as an intermediate step. This adjusted lane is the current measurement for the filter update step. This leads to a sensor model in which the measurement vector z _k consists of the same elements as the state vector, which is the observation model H _k simplified to a unit metric. For simplicity, all elements of the state vector are independently estimated. This means that there are actually four independent Kalman filters - one for each element of the state vector.

reduziert wird. Andererseits ist die Konfiguration des Filters viel einfacher, da jedes Element des Filters unabhängig abgestimmt werden kann. Das Risiko von Überanpassungsproblemen wird auch verringert.The disadvantage of this model is that it does not make use of any possible covariants between the elements of the state vector. This means that the state covariant matrix P _k actually to a variance vector p _k with the same dimensions as $\widehat{x_k}$

is reduced. On the other hand, the configuration of the filter is much simpler since each element of the filter can be tuned independently. The risk of over-adjustment problems is also reduced.

• - das erste Fahrspurmodell, d. h. das Aktualisierungsmodell, das unter Verwendung von Fahrspurmarkierungen des aktuellen Zeitstempels berechnet wird, die mit Hilfe des vorhergesagten Fahrspurmodells gefiltert werden und
• - das zweite Fahrspurmodell, d. h. das statische (beste) Modell, das unter Verwendung von Fahrspurmarkierungen des aktuellen Zeitstempels berechnet wird, die mit Hilfe einer RANSAC gefiltert werden.

In the tracking system (cf. 2 ) always two measurements (ie two lane models) are calculated in parallel for each timestamp:

• the first lane model, ie the update model calculated using lane markings of the current timestamp, which are filtered using the predicted lane model, and
• the second lane model, ie the static (best) model calculated using lane markings of the current timestamp, which are filtered using a RANSAC.

Later - in the lane position logic block 5 (see 1 ) - it is decided by trusting which of the two measurements will be used to update the Kalman filter state.

Separation of lane markings for left and right lane boundaries is performed prior to adapting the lane model. The lane markings 22 are split into two separate sets for the left and the right border. This is done by calculating the dividing line 21 performed from the predicted lane model. The dividing line 21 is (like the model boundaries: predicted left boundary 19 and predicted right border 20 ) a second order polynomial having the same parameters as the two lane boundaries 19 . 20 except that the constant amount (c) of the polynomial is based on the average of the left and right limits 19 . 20 is set so that the line passes through the center of the predicted lane. The sign of the lateral distance of the center of the lane marking 22 to the dividing line determines whether a lane marking 22 within a left boundary assignment area 23 or a right-hand border assignment area 24 and is therefore assigned to the left or right border list.

In a preferred embodiment, a calculation of static lane model hypotheses for the current frame is performed. For example, the likelihood of the hypotheses is computed that a lane marker of the current frame belongs to the left boundary of the lane. For each frame can be calculated as a lane model of a best hypothesis. This is done in two steps. First, a RANSAC (sample match) is used to select left and right feature set entrant features. With these embedders, a least square fit is performed to find the current model parameters.

First, pre-filtering is performed using model-driven RANSAC. From the list of features from the left and the right border, n features ( 6 default) for each border. These features compute a best fit candidate model using a polynomial fit (see calculations below). If a valid candidate is found, a confidence score for the fit is calculated and it is checked which of the left and right lane markers are near the matched polynomial. These steps are repeated until either all of the lane markers have been classified as a rounder or the number of iterations reaches a certain value that is configurable and can be 100 by default.

wobei $$A=X\text{'}X$$

und $$B=X\text{'}Y$$

Furthermore, a model least square fit is performed. This means that a model fit for each hypothesis model is performed by a least squares fit method. The goal is to find a solution to the following equation: $$Ax=B$$

in which $$A=X\text{'}X$$

and $$B=X\text{'}Y$$

wobei x_l,n und x_r,n die Merkmals-x-Koordinaten der linken und der rechten Merkmalspositionen sind.The solution vector x contains the polynomial coefficients of the fitted parabolic curve. The matrix X is a (n _l + n _r ) × 4 matrix constructed from the feature samples, where n _l the number of features for the left lane boundary is and _nr is the number of features for the right lane boundary. The matrix has the following form: $$X=\left(\begin{array}{cccc}{x}_{l\mathrm{,1}}^2& x_{l\mathrm{,1}}& 1& 0\\ ⋮& ⋮& ⋮& ⋮\\ x_{l.n_l}^2& x_{l.n_l}& 1& 0\\ x_{r\mathrm{,1}}^2& x_{r\mathrm{,1}}& 0& 1\\ ⋮& ⋮& ⋮& ⋮\\ x_{r.n_r}^2& x_{r.n_r}& 0& 1\end{array}\right)$$

in which x _{l, n} and x _{r, n} are the feature x coordinates of the left and right feature positions.

The matrix Y is an (n _l + n _r ) × 1 matrix containing the y values of the sample points: $$Y=\left(\begin{array}{c}{y}_{l\mathrm{,1}}\\ ⋮\\ y_{l.n_l}\\ y_{r\mathrm{,1}}\\ ⋮\\ y_{r.n_r}\end{array}\right)$$

Therefore we can write for A = X'X: $$A=\left(\begin{array}{cccc}{\displaystyle \Sigma x^4}& {\displaystyle \Sigma x^3}& {\displaystyle <figure-callout\; id="2"\; label="\Sigma \; x\; l"\; filenames="DE102018112177A1\_0024.png"\; state="\{\{state\}\}">\Sigma \; </figure-callout>x_l^{}{}_2^{}}& {\displaystyle <figure-callout\; id="2"\; label="\Sigma \; x\; r"\; filenames="DE102018112177A1\_0024.png"\; state="\{\{state\}\}">\Sigma \; </figure-callout>x_r^{}{}_2^{}}\\ {\displaystyle \Sigma x^3}& {\displaystyle \Sigma x^2}& {\displaystyle \Sigma x_l}& {\displaystyle \Sigma x_r}\\ {\displaystyle <figure-callout\; id="2"\; label="\Sigma \; x\; l"\; filenames="DE102018112177A1\_0024.png"\; state="\{\{state\}\}">\Sigma \; </figure-callout>x_l^{}{}_2^{}}& {\displaystyle \Sigma x_l}& n_l& 0\\ {\displaystyle <figure-callout\; id="2"\; label="\Sigma \; x\; r"\; filenames="DE102018112177A1\_0024.png"\; state="\{\{state\}\}">\Sigma \; </figure-callout>x_r^{}{}_2^{}}& {\displaystyle \Sigma x_r}& 0& n_r\end{array}\right)$$

And for B = X'Y: $$B=\left(\begin{array}{c}{\displaystyle \Sigma \left(x^2*y\right)}\\ {\displaystyle \Sigma x*y}\\ {\displaystyle \Sigma y_l}\\ {\displaystyle \Sigma y_r}\end{array}\right)$$

Whereby x _l . x _r and x refer to the x positions for the left, the right and both borders and y _l . y _r and y relate to the y-positions for the left, the right and both boundaries.

To solve equation (0.1), one can use Cramer's rule or any other valid mechanism.

A confidence score is calculated for each model hypothesis to weight and compare the models. The confidence value for the model is performed by first computing independent confidence values for the left and right bounds, and then computing the mean value as a common trust. The calculation of a confidence score is a common statistical technique for estimating a hypothesis regarding the membership of a parameter at a given interval.

Another essential step is to update the tracked first model. It should be noted that the following notation x _{n | m represents} the estimate of the filter state for every n given observations up to and including m≤n, respectively.

Once the adaptation of a lane model has been successful, this model is retained and updated for subsequent time steps. In general, the tracking is performed with a Kalman filter in two steps: in the first step - the prediction step - the model state is transferred from the previous timestamp t-1 to the current time t using a system model. In the present case, the system model assumes that the lane model is fixed between two time steps. This is approximately true if we assume sufficiently small time increments between two consecutive processing steps. The error introduced by this assumption becomes indirect in the state variance _{pk | k-1} dragged. Thus, in the present case, the prediction step consists only of compensating the vehicle's own motion (see next paragraph) and updating the model state variance values (ie, increasing by the variance of the process noise q _k .) $$p_{k|k-1}=p_{k-1|k-1}+q_k$$

After the model has been updated to the current time, the next step is to assign lane markers detected in the current time step to the existing model. This step separates the lane markings in the stoker and the outlier with respect to the tracked model. With the locator tags, you can do a model fit similar to the static best fit, except that you do not have to do RANSAC before the least square fit.

The self-motion compensation will now be used in conjunction with 5 described. The left part of 5 shows the vehicle 17 in the time step t-1. The right part of 5 shows the vehicle 17 in the time step t , It follows a lane 18 that is between the left border 19 and the right border 20 lies.

1. 1. Von beiden Fahrspurgrenzen 19, 20 des t-1-Fahrspurmodells Auswählen von drei Punkten des Polynoms bei x = -1m, x = 0m und x = +1m.
2. 2. Mit Hilfe von gegebenen Eigenbewegungsinformationen (Translationsvektor $\overrightarrow{s}$
   
   und δω), Transformieren dieser drei Punkte in das aktuelle Fahrzeugkoordinatensystem. Dies wird durch zuerst Drehen der Punkte um das neue Fahrzeugzentrum mit - δω, und dann Verschieben um $-\overrightarrow{s}$
   
   durchgeführt.
3. 3. Durchführen einer Parabelanpassung durch die drei Punkte für die linke und die rechte Fahrspurgrenze 19, 20.

Before assigning lane markers of the current time step to the existing model, the movement of the own vehicle must 17 be compensated. This self-motion compensation is performed in three steps:

1. 1. From both lane boundaries 19 . 20 of the t-1 lane model Select three points of the polynomial at x = -1m, x = 0m and x = + 1m.
2. 2. Using given eigenmotion information (translation vector $\overrightarrow{s}$
   
   and δω), transforming these three points into the current vehicle coordinate system. This is done by first turning the points around the new vehicle center with - δω, and then moving around $-\overrightarrow{s}$
   
   carried out.
3. 3. Perform a parabola fit through the three points for the left and right lane boundaries 19 . 20 ,

In a next step, an assignment of lane markings to the existing model is analyzed. That is, after the self-motion compensation has been performed, the next step is to assign lane markers to the predicted model.

• - Wenn der Zuordnungsschritt vorher Fahrspurmarkierungen für nur eine der zwei Grenzen gefunden hat, wird die Fahrspur durch Anpassen des Polynoms für diese Seite und Spiegeln derselben auf die andere Seite mit Hilfe der bekannten Fahrspurbreite rekonstruiert
• - In Gleichung (0.2) des Abschätzers kleinster Quadrate wird ein zusätzliches Gewicht für jedes Merkmal eingeführt. Dieses Gewicht wird beispielsweise von der Positionsvarianz des Merkmals oder dem Detektionsvertrauen des Merkmals abgeleitet. In beiden Fällen führt eine höhere Positionsvarianz/ein höheres Vertrauen zu einem niedrigeren Gewicht am Ergebnis.

$$A=X\text{'}WX$$

$$A=\left(\begin{array}{cccc}{\displaystyle \sum w{x}^4}& {\displaystyle \sum w{x}^3}& {\displaystyle \sum w_{\mathrm{<figure-callout\; id="2"\; label="l\; x\; l"\; filenames="DE102018112177A1\_0024.png"\; state="\{\{state\}\}">l</figure-callout>}}{x}_l^{}{}_2^{}}& {\displaystyle \sum w_{\mathrm{<figure-callout\; id="2"\; label="r\; x\; r"\; filenames="DE102018112177A1\_0024.png"\; state="\{\{state\}\}">r</figure-callout>}}{x}_r^{}{}_2^{}}\\ {\displaystyle \sum w{x}^3}& {\displaystyle \sum w{x}^2}& {\displaystyle \sum w_l{x}_l}& {\displaystyle \sum w_r{x}_r}\\ {\displaystyle \sum w_{\mathrm{<figure-callout\; id="2"\; label="l\; x\; l"\; filenames="DE102018112177A1\_0024.png"\; state="\{\{state\}\}">l</figure-callout>}}{x}_l^{}{}_2^{}}& {\displaystyle \sum w_l{x}_l}& {\displaystyle \sum w_l}& 0\\ {\displaystyle \sum w_{\mathrm{<figure-callout\; id="2"\; label="r\; x\; r"\; filenames="DE102018112177A1\_0024.png"\; state="\{\{state\}\}">r</figure-callout>}}{x}_r^{}{}_2^{}}& {\displaystyle \sum w_r{x}_r}& 0& {\displaystyle \sum w_r}\end{array}\right)$$

wobei w alle Gewichte sind, w_l die Gewichte für die linken Punkte sind und w_r die Gewichte der rechten Gewichte sind.Thereafter, an update model z. B. calculated on the basis of the following calculation steps. With the pre-filtered list of lane markings from the previous processing step, a lane model is adjusted using the least squares method. This least squares operation is performed in a similar way to the static lane model, but with some differences:

• If the assignment step has previously found lane markings for only one of the two boundaries, the lane is reconstructed by fitting the polynomial for that page and mirroring it to the other side using the known lane width
• In equation (0.2) of the least squares estimator, an additional weight is introduced for each feature. This weight is derived, for example, from the positional variance of the feature or the detection confidence of the feature. In both cases, a higher positional variance / confidence leads to a lower weight on the result.

$$A=X\text{'}WX$$

$$A=\left(\begin{array}{cccc}{\displaystyle \Sigma w{x}^4}& {\displaystyle \Sigma w{x}^3}& {\displaystyle \Sigma w_{\mathrm{<figure-callout\; id="2"\; label="l\; x\; l"\; filenames="DE102018112177A1\_0024.png"\; state="\{\{state\}\}">l\; </figure-callout>}}{x}_l^{}{}_2^{}}& {\displaystyle \Sigma w_{\mathrm{<figure-callout\; id="2"\; label="r\; x\; r"\; filenames="DE102018112177A1\_0024.png"\; state="\{\{state\}\}">r\; </figure-callout>}}{x}_r^{}{}_2^{}}\\ {\displaystyle \Sigma w{x}^3}& {\displaystyle \Sigma w{x}^2}& {\displaystyle \Sigma w_l{x}_l}& {\displaystyle \Sigma w_r{x}_r}\\ {\displaystyle \Sigma w_{\mathrm{<figure-callout\; id="2"\; label="l\; x\; l"\; filenames="DE102018112177A1\_0024.png"\; state="\{\{state\}\}">l\; </figure-callout>}}{x}_l^{}{}_2^{}}& {\displaystyle \Sigma w_l{x}_l}& {\displaystyle \Sigma w_l}& 0\\ {\displaystyle \Sigma w_{\mathrm{<figure-callout\; id="2"\; label="r\; x\; r"\; filenames="DE102018112177A1\_0024.png"\; state="\{\{state\}\}">r\; </figure-callout>}}{x}_r^{}{}_2^{}}& {\displaystyle \Sigma w_r{x}_r}& 0& {\displaystyle \Sigma w_r}\end{array}\right)$$

where w are all weights, w _l the weights for the left points are and w _r the weights of the right weights are.

In the least squares estimator, an additional model curvature adjustment factor is introduced which prevents the curvature value from becoming too high, ie overmodulation of the model to lane data is avoided. The regulatory parameter is very straightforward to add. The cost function takes the following additional parameters: $$\begin{array}{l}S={\displaystyle {\Sigma }_{i=1}^n{w}_{r.j}}{\left(y_{r.i}-a_3{x}_{r.i}^2-a_2{x}_{r.i}-a_1\right)}^2+{\displaystyle {\Sigma }_{j=1}^m{w}_{l.j}}{\left(y_{l.i}-a_3{x}_{l.j}^2-a_2{x}_{l.j}-a_0\right)}^2+\\ \lambda {\displaystyle {\Sigma }_{k=1}^3{a}_{\mathrm{<figure-callout\; id="2"\; label="k"\; filenames="DE102018112177A1\_0024.png"\; state="\{\{state\}\}">k</figure-callout>}}^2}\end{array}$$

Equivalent matrix form $$\left(X\text{'}X+\lambda I\right)A=X^{T}Y$$

And with the weights described in (0.5): $$\left(X\text{'}WX+\lambda I\right)A=X^{T}WY$$

See https://www.cs.ubc.ca/~schmidtm/Documents/2005_Notes_Lasso.pdf. Solving (0.8) is equivalent to minimizing by (0.6) if one of the diagonals in the above I is set to zero (equivalent to a ₀ ). $$\left(X^{T}WX+\lambda K\right)A=X^{T}WY$$

Where the K-diagonal is set to zero in all elements except for the one we want to suppress. For example, if we wanted to suppress the curvature parameter, we would buy the upper left element 1 and set all other entries to 0.

The Lane Position Logic (LPL) Block 5 (see 1 ) decides whether to use the tracked first lane model or the static second (best fit) model to adapt the system output model based on the availability and confidence of the calculated models. 6 shows the activity diagram for this LPL function block. The function starts at the starting point 25 , First, an initial parameter regarding a valid update model in the block 26 set to false. In a first decision block 27 it is then decided whether the input data relates to a first frame. If so, a decision block determines 28 whether a static (best) lane model is valid. If not, the activity ends at the exit 29 ,

If the decision in the decision block 27 is no, determines another decision block 30 whether the tracked first lane model is valid. If so, a decision block decides 31 whether the static second (best) lane model is valid. If so, a decision block checks 32 Whether the static confidence value is higher than twice the tracked confidence value. If so, and if so, the decision from the decision block 28 yes, is in the block 33 set a variable with respect to the update model to the static second (best) lane model.

If the decision in the decision block 30 "No" is, decides a decision block 34 whether the static second (best) lane model is valid. If so, another decision block decides 35 whether the static confidence value is higher than the tracked confidence value. If so, the requirement for the block 33 is met in order to set the variable regarding the update model to the "static second lane model of the (best) value".

If the decision in the decision block 34 "No" is and also the decision of the decision block 35 Is "no", the algorithm ends at the endpoint 36 , The algorithm also ends at an endpoint 37 if the decision of the decision block 31 "no is.

If the decision in the decision block 32 Is "no", furthermore, the variable regarding the update model in the block 38 set to the "tracked first lane model". As a consequence of the blocks 33 and 38 the parameter is related to the presence of a valid update model in the block 39 set to "true". After the block 39 the algorithm also ends at the endpoint 36 ,

∀ feat ∈ {Fahrtrichtung, seitlicher Versatz, Breite, Krümmung} wobei $r_k^{feat}$

die Varianz des Beobachtungsrauschens für jedes Merkmal ist. Die Kalman-Verstärkung wird dann verwendet, um den neuen Zustand und die Zustandsvarianz zu aktualisieren: $${\widehat{x}}_{k|k}={\widehat{x}}_{k|k}+k_k\left(z_k-{\widehat{x}}_{k|k-1}\right)$$

$$p_{k|k}=\left(I-k_k\right)p_{k|k-1}$$

wobei der Messvektor z_k , der in Gl. (0.11) verwendet wird, aus den abgeschätzten Modellparametern für das angepasste Modell besteht, die als beste Messung im LPL-Modul gewählt wurden (siehe vorheriger Abschnitt).If necessary, a model update takes place. Ie. the last step of the tracking may be the update or correction step. Here, the state vector (direction of travel, lane width, lateral offset and curvature) is updated from the LPL block using the selected model. For each of the four independent filters one can calculate four independent Kalman amplification factors, which are in the vector
k _k can be bundled. $$k_k^{feat}=\frac{p_{k|k-1}^{feat}}{p_{k|k-1}^{feat}+r_k^{feat}}$$

∀ feat ∈ {direction of travel, lateral offset, width, curvature} where $r_k^{feat}$

is the variance of the observation noise for each feature. The Kalman gain is then used to update the new state and state variance: $${\widehat{x}}_{k|k}={\widehat{x}}_{k|k}+k_k\left(z_k-{\widehat{x}}_{k|k-1}\right)$$

$$p_{k|k}=\left(I-k_k\right)p_{k|k-1}$$

where the measurement vector z _k which in Eq. (0.11), which consists of the estimated model parameters for the fitted model that were chosen as the best measurement in the LPL module (see previous section).

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• WO 2013/022153 A1 [0004]
• US 2013/0141520 A1 [0005]

Claims (13)

Method for following a lane (18) with the steps - providing individual images of the lane (18), - extracting lane features (2) from the individual images and - providing a first lane model (10), characterized by - updating (12) the first one Lane model (10) with the lane features, - providing a static second lane model (13) based on an outlier detection algorithm, - calculating a tracked confidence score for the updated first lane model and a static confidence score for the static second lane model (13) and decision (14 ) based on both confidence values, whether to use the updated first lane model or the static second lane model to calculate current lane information (18). Method according to Claim 1 characterized in that the updating (12) of the first lane model (10) is performed by predicting a new version of the first lane model using Kalman filtering and adapting the new version of the first lane model to the extracted lane features. Method according to Claim 1 or 2 characterized in that the first lane model (10) and the second lane model (13) track a lane geometry over time. Method according to one of the preceding claims, characterized in that the lane features relate to lane markings (22) and before updating the first lane model (10) with lane markings, the lane markings (22) into two separate sets for a left boundary (19) and a right boundary (20) by calculating a dividing line (21) using the first lane model (10) are divided. A method according to any one of the preceding claims, characterized in that the static second lane model (13) is evaluated on the basis of a best fit lane hypothesis for the extracted lane features using a combination of sample matching and least square fit. Method according to Claim 5 characterized in that, for each of a left and a right border (19, 20) of the lane (18), a predetermined number of features are used to calculate the static second lane model (13) by performing polynomial fitting. Method according to one of the preceding claims, characterized in that one of the first lane model (10) and the second lane model (13) is used in a first time step and this still uses a model in a second time step immediately following the first time step if the tracked confidence value and the static confidence value are below a predetermined threshold. Method according to one of the preceding claims, characterized in that the tracked confidence value for a hypothesis with respect to the first lane model (10) is calculated and the static confidence value for a hypothesis with respect to the second lane model (13) on the basis of the lane characteristics of the individual images is calculated. Method according to Claim 8 characterized in that the confidence value for each of the first and second lane models (10, 13) is calculated by first calculating independent confidence values for the left and right boundaries (19, 20) of the lane (18) and then calculating an average of these independent ones Confidence values is calculated as said confidence value for the respective first or second lane model (10, 13). Method according to one of the preceding claims, characterized in that detected lane markings (22) are separated into stalls and outliers with respect to the first lane model (10) and only the stalls are used to update the first lane model (10). Driver assistance system, which is designed to carry out a method according to any one of the preceding claims. Apparatus designed to perform a method according to any one of Claims 1 to 10 perform. Computer program product with program code means, in particular stored in a computer-readable medium, for the method of following a lane (18) according to any one of Claims 1 to 10 perform when the computer program product is executed on a computer device of an electronic control unit.

Images