CN117036422A

CN117036422A - Method, device, equipment and storage medium for tracking lane lines

Info

Publication number: CN117036422A
Application number: CN202311088049.2A
Authority: CN
Inventors: 陈洪林; 韩锐; 苗乾坤
Original assignee: Neolix Technologies Co Ltd
Current assignee: Neolix Technologies Co Ltd
Priority date: 2023-08-25
Filing date: 2023-08-25
Publication date: 2023-11-10

Abstract

The application discloses a lane line tracking method, a lane line tracking device, lane line tracking equipment and a storage medium, and belongs to the technical field of computers. The method comprises the following steps: acquiring a lane line point cloud and the current position of a vehicle; the lane line point cloud comprises a first lane line point cloud and a second lane line point cloud, and the first lane line point cloud is subjected to fitting processing by utilizing a spline curve so as to obtain a first control point of the first lane line point cloud and a first fitting curve corresponding to the first control point; selecting at least one first sampling point from a first fitting curve by utilizing a preset first sampling strategy based on the current position of the vehicle and the first control point; predicting and updating at least one first sampling point based on the current position of the vehicle and the second road line point cloud by using a Kalman filtering algorithm to obtain a predicted value of a lane line and an estimated value of the lane line; and obtaining a tracking result of the lane line based on the predicted value of the lane line and the estimated value of the lane line.

Description

Method, device, equipment and storage medium for tracking lane lines

Technical Field

The application relates to the technical field of computers, in particular to the technical fields of intelligent traffic, automatic driving and the like, and particularly relates to a lane line tracking method, a lane line tracking device, lane line tracking equipment and a storage medium.

Background

Lane tracking generally refers to using the lane marker position of the previous frame to estimate and predict the position of the lane marker of the next frame. The lane line tracking technology is mainly used for carrying out fitting prediction on lane lines based on an algorithm trunk and a parameter model. Among them, kalman filtering is generally used as an algorithm backbone. Common parametric models include a third order polynomial based parametric model, a spiral based parametric model, and the like.

However, the lane tracking technique in the related art still has some drawbacks as follows: depending on the local coordinate system, a road side local coordinate system needs to be continuously established; the relative relation between the road side coordinate system and the vehicle body coordinate system is difficult to convert, and the calculated amount is large; the calculation result is not intuitive.

Disclosure of Invention

The application provides a method, a device, equipment and a storage medium for tracking a lane line, which can simplify the reasoning process and reduce the calculated amount so as to reduce the burden of data processing while ensuring the reliability of the lane line tracking, and the technical scheme is as follows:

in a first aspect, a method of lane line tracking is provided, the method comprising:

acquiring a lane line point cloud and the current position of a vehicle; the lane line point cloud comprises a first lane line point cloud and a second lane line point cloud, wherein the first lane line point cloud is the lane line point cloud of the initial frame, and the second lane line point cloud is the lane line point cloud of each frame after the initial frame;

Fitting the first lane line point cloud by using a spline curve to obtain a first control point of the first lane line point cloud and a first fitting curve corresponding to the first control point;

selecting at least one first sampling point from the first fitting curve by utilizing a preset first sampling strategy based on the current position of the vehicle and the first control point;

predicting and updating the at least one first sampling point based on the current position of the vehicle and the second lane line point cloud by using a Kalman filtering algorithm to obtain a predicted value of a lane line and an estimated value of the lane line;

and obtaining a tracking result of the lane line based on the predicted value of the lane line and the estimated value of the lane line.

In one possible implementation manner, the first lane line point cloud includes a plurality of first lane line point cloud points, the fitting processing is performed on the first lane line point cloud by using a spline curve to obtain a first control point of the first lane line point cloud and a first fitting curve corresponding to the first control point, including:

determining the path length between a first lane line point cloud point and each i-th first lane line point cloud point and the total path length of the first lane line point cloud, wherein i is a natural number greater than 1;

Obtaining a path length ratio sequence based on the path length and the total path length;

and calculating a first control point of the first lane line point cloud and a first fitting curve corresponding to the first control point by using a spline curve based on the path length ratio sequence and the first lane line point cloud.

In one possible implementation manner, the selecting, based on the current position of the vehicle and the first control point, at least one first sampling point from the first fitting curve by using a preset first sampling strategy includes:

determining a starting sampling point based on the current position of the vehicle and the first control point;

and uniformly sampling the first fitting curve based on the initial sampling point to select at least one first sampling point.

In one possible implementation manner, the predicting and updating the at least one first sampling point based on the current position of the vehicle and the second lane line point cloud by using a kalman filtering algorithm to obtain a predicted value of a lane line and an estimated value of the lane line includes:

using a Kalman filtering prediction model to predict the at least one first sampling point based on the current position of the vehicle so as to obtain a predicted value of the lane line;

And updating the predicted value of the lane line based on the second lane line point cloud by using a Kalman filtering measurement model to obtain an estimated value of the lane line.

In one possible implementation manner, the measuring model using kalman filtering updates the predicted value of the lane line based on the second lane line point cloud to obtain the estimated value of the lane line, and includes:

obtaining a second fitting curve corresponding to the predicted value of the lane line based on the predicted value of the lane line;

uniformly dividing the second fitting curve to obtain a plurality of line segments;

respectively calculating the distance between each line segment and the corresponding second lane line point cloud to obtain a plurality of distances;

and obtaining an estimated value of the lane line by using a Kalman filtering measurement model based on the plurality of distances and the predicted value of the lane line.

In one possible implementation manner, the predicting and updating the at least one first sampling point based on the current position of the vehicle and the second lane line point cloud by using a kalman filtering algorithm to obtain a predicted value of a lane line and an estimated value of the lane line, further includes:

Based on the estimated value of the lane line, obtaining a third fitting curve corresponding to the estimated value of the lane line;

acquiring a current position corresponding to the estimated value of the lane line based on the current position of the vehicle;

determining the running direction of the vehicle based on the current position corresponding to the estimated value of the lane line and the first sampling point;

selecting at least one second sampling point from the third fitting curve by utilizing a preset second sampling strategy based on the running direction of the vehicle;

and carrying out iterative prediction and updating processing on the at least one second sampling point and the second lane line point cloud by using a Kalman filtering algorithm so as to obtain a predicted value of the lane line and an estimated value of the lane line after the iterative prediction and updating processing.

In one possible implementation of the present invention,

the predicting module for predicting the at least one first sampling point based on the current position of the vehicle to obtain a predicted value of the lane line, wherein the predicting module comprises:

determining a jacobian matrix of the kalman filtered predictive model based on the current location of the vehicle and the at least one first sampling point;

And obtaining a predicted value of the lane line based on the jacobian matrix and the at least one first sampling point.

In a second aspect, there is provided an apparatus for lane line tracking, the apparatus comprising:

the acquisition unit is used for acquiring the lane line point cloud and the current position of the vehicle; the lane line point cloud comprises a first lane line point cloud and a second lane line point cloud, wherein the first lane line point cloud is the lane line point cloud of the initial frame, and the second lane line point cloud is the lane line point cloud of each frame after the initial frame;

the obtaining unit is used for carrying out fitting processing on the first lane line point cloud by utilizing a spline curve so as to obtain a first control point of the first lane line point cloud and a first fitting curve corresponding to the first control point;

the sampling unit is used for selecting at least one first sampling point from the first fitting curve by utilizing a preset first sampling strategy based on the current position of the vehicle and the first control point;

the prediction unit is used for predicting and updating the at least one first sampling point based on the current position of the vehicle and the second lane line point cloud by using a Kalman filtering algorithm so as to obtain a predicted value of a lane line and an estimated value of the lane line;

And the tracking unit is used for obtaining a tracking result of the lane line based on the predicted value of the lane line and the estimated value of the lane line.

In a third aspect, there is provided a computer readable storage medium having stored therein at least one instruction that is loaded and executed by a processor to implement the method of the aspects and any one possible implementation as described above.

In a fourth aspect, there is provided an electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the aspects and methods of any one of the possible implementations described above.

In a fifth aspect, there is provided an autonomous vehicle comprising an electronic device as described above.

The technical scheme provided by the application has the beneficial effects that at least:

as can be seen from the above technical solution, in the embodiment of the present application, the current position of the vehicle and the current position of the lane line point cloud may be obtained, where the lane line point cloud includes a first lane line point cloud and a second lane line point cloud, the first lane line point cloud is a lane line point cloud of an initial frame, the second lane line point cloud is a lane line point cloud of each frame after the initial frame, and further, a spline curve may be used to perform a fitting process on the first lane line point cloud, so as to obtain a first control point of the first lane line point cloud and a first fitting curve corresponding to the first control point, based on the current position of the vehicle and the first control point, at least one first sampling point is selected from the first fitting curve by using a preset first sampling strategy, a kalman filtering algorithm is used to predict and update the at least one first sampling point based on the current position of the vehicle and the second lane line point cloud, so as to obtain a predicted value of the lane line and an estimated value of the lane line, and the first sampling point cloud can be used to perform a fitting process on the first lane line, so as to obtain a new curve, and further, the curve can be more accurately calculated by using the first sampling point and the first sampling point cloud is more, and the curve can be more optimized, and the curve can be more better fitted by performing an iterative calculation, so as to obtain a curve fitting process.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the application or to delineate the scope of the application. Other features of the present application will become apparent from the description that follows.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a method for lane tracking according to an embodiment of the present application;

FIG. 2 is a schematic diagram of obtaining sampling points based on control points and a fitting curve in a lane tracking method according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a second lane line point cloud and a second fitted curve distance calculated in a lane line tracking method according to an embodiment of the present application;

FIG. 4 is a schematic diagram of a change in a fitted curve of a lane line in a forward driving situation in a lane line tracking method according to an embodiment of the present application;

FIG. 5 is a schematic diagram of a change in a fitted curve of a lane line in a backward driving situation in a lane line tracking method according to an embodiment of the present application;

FIG. 6 is a flow chart of a method for lane tracking according to another embodiment of the present application;

fig. 7 is a schematic diagram of an application scenario of a lane tracking method according to another embodiment of the present application;

FIG. 8 is a schematic illustration of a Catmull-Rom spline parameterized fitted curve for a different shape lane line of a lane line tracking method provided by another embodiment of the present application;

FIG. 9 is a schematic diagram of a fitted curve of predicted values of lane lines of a prediction model of a lane line tracking method according to another embodiment of the present application;

FIG. 10 is a schematic diagram of a real-time lane tracking result of a prediction model of a lane tracking method according to another embodiment of the present application;

FIG. 11 is a block diagram of an apparatus for lane tracking according to still another embodiment of the present application;

FIG. 12 shows a schematic block diagram of an example electronic device 1200 that may be used to implement an embodiment of the application.

Detailed Description

Exemplary embodiments of the present application will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present application are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

It will be apparent that the described embodiments are some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

It should be noted that, the terminal device in the embodiment of the present application may include, but is not limited to, smart devices such as a mobile phone, a personal digital assistant (Personal Digital Assistant, PDA), a wireless handheld device, and a Tablet Computer (Tablet Computer); the display device may include, but is not limited to, a personal computer, a television, or the like having a display function.

In addition, the term "and/or" herein is merely an association relationship describing an association object, and means that three relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship.

The lane line tracking technology mainly comprises an algorithm trunk and a parameter model, wherein the algorithm trunk adopts Kalman filtering, and the commonly used parameter model comprises a parameter model such as a cubic polynomial, spiral line fitting and the like.

However, in the process of tracking the lane line, a parameter model based on polynomial fitting and a parameter model based on spiral line fitting are required to establish a coordinate conversion relation between a vehicle coordinate system and a lane coordinate system in real time, so that the local linearization direction of the lane curve is continuously calculated, the reasoning is complex, and the calculated amount is large.

Therefore, it is desirable to provide a lane line tracking method that can simplify the reasoning process and reduce the calculation amount to reduce the burden of data processing while ensuring the reliability of lane line tracking.

Referring to fig. 1, a flow chart of a lane tracking method according to an embodiment of the application is shown. The lane line tracking method specifically comprises the following steps:

step 101, acquiring a lane line point cloud and the current position of a vehicle; the lane line point cloud comprises a first lane line point cloud and a second lane line point cloud, wherein the first lane line point cloud is the lane line point cloud of the initial frame, and the second lane line point cloud is the lane line point cloud of each frame after the initial frame.

And 102, fitting the first lane line point cloud by using a spline curve to obtain a first control point of the first lane line point cloud and a first fitting curve corresponding to the first control point.

And 103, selecting at least one first sampling point from the first fitting curve by utilizing a preset first sampling strategy based on the current position of the vehicle and the first control point.

And 104, predicting and updating the at least one first sampling point based on the current position of the vehicle and the second lane line point cloud by using a Kalman filtering algorithm to obtain a predicted value of a lane line and an estimated value of the lane line.

And 105, obtaining a tracking result of the lane line based on the predicted value of the lane line and the estimated value of the lane line.

It should be noted that, the lane line point cloud and the current position of the vehicle may be obtained in real time. The lane line point cloud may be a point cloud unified with a coordinate system. Specifically, a lane line point cloud image related to a vehicle, which is acquired by an on-board sensor, can be converted into a transverse mercator projection coordinate system (Universal Transverse Mercator, UTM) to obtain the lane line point cloud. Or the lane line point cloud image, which is acquired by the vehicle-mounted sensor and related to the vehicle, can be converted into an odometer coordinate system Odom coordinate system to obtain the lane line point cloud.

The execution subject of steps 101 to 105 may be an application located in the local terminal, or may be a functional unit such as a plug-in unit or a software development kit (Software Development Kit, SDK) provided in the application located in the local terminal, or may be a processing engine located in a server on the network side, or may be a distributed system located on the network side, for example, a processing engine or a distributed system in an autopilot platform on the network side, which is not particularly limited in this embodiment.

It will be appreciated that the application may be a native program (native app) installed on the native terminal, or may also be a web page program (webApp) of a browser on the native terminal, which is not limited in this embodiment.

Therefore, the first control point and the first fitting curve can be obtained by carrying out parameterization fitting on the lane line point cloud of the initial frame by utilizing the spline curve, the first fitting curve can be sampled, and only the sampling point is subjected to iterative prediction updating processing by utilizing the Kalman filtering algorithm, so that a more accurate lane line tracking result is obtained more rapidly and effectively, the reliability of lane line tracking is ensured, the operation reasoning process is simplified, the calculated amount is reduced, the burden of data processing is reduced, and the lane line tracking effect is optimized.

Optionally, in one possible implementation manner of this embodiment, the first lane line point cloud includes a plurality of first lane line point cloud points, in step 102, a path length between a first lane line point cloud point and each i-th first lane line point cloud point and a total path length of the first lane line point cloud may be specifically determined, i is a natural number greater than 1, and further, a path length ratio sequence may be obtained based on the path length and the total path length, so that a first control point of the first lane line point cloud and a first fitting curve corresponding to the first control point may be calculated by using spline curves based on the path length ratio sequence and the first lane line point cloud.

In this implementation, the spline curves may include, but are not limited to, catmull-Rom spline curves, B-spline curves, bezier curves, and the like.

In a specific implementation procedure of this implementation manner, first, a total path length of the first lane line point cloud may be calculated based on the first lane line point cloud. And secondly, the path length between the first lane line point cloud point and each ith lane line point cloud point can be calculated based on each first lane line point cloud point in the first lane line point cloud.

For example, for the acquired original point cloud data of the initial lane line, i.e., the first lane line point cloud, p _m ＝{p ₁ ,p ₂ ,…,p _N }. First, can p _m Is provided for the total path length s of (a). Secondly, calculating a first lane line point cloud point p ₁ To the ith first lane line point cloud point p _i A ratio of the path length to the total path length s, a sequence of path length ratios is obtained: t= { T ₀ ,t ₁ ,…,t _i ,…,t _N }

In another specific implementation process of the implementation manner, the spline curve may be a Catmull-Rom spline curve, and specifically, based on a path length ratio sequence, the Catmull-Rom spline curve is used to fit the first lane line point cloud, and a first control point of the first lane line point cloud and a first fitting curve corresponding to the first control point are obtained through calculation.

In the specific implementation process, the path length ratio sequence can be substituted into a Catmull-Rom spline curve, and the first lane line point cloud is fitted to obtain the state quantity of the first lane line point cloud, namely the parameterized first lane line point cloud. The state quantity comprises a first control point and a first fitting curve corresponding to the first control point. Here, the number of first control points may be 4, i.e. control point P ₀ ,P ₁ ,P ₂ ,P ₃ And obtaining a corresponding first fitting curve based on the first control point.

In this way, the spline curve is utilized to perform fitting processing on the first lane line point cloud, so that a first control point of the first lane line point cloud and a first fitting curve corresponding to the first control point are obtained, namely, the parameterized first lane line point cloud is obtained, so that the first control point and the first fitting curve corresponding to the first control point can be used as initial state quantity in the follow-up process, and a Kalman filtering algorithm is input to realize the prediction and updating of lane lines, so that a more accurate lane line tracking result is obtained.

Moreover, based on the Catmull-Rom spline curve, the control points obtained by carrying out lane line parameterization are the geometric shapes of the lane lines through the lane lines, and the geometric shapes of the lane lines can be intuitively and effectively fitted. And the Catmull-Rom spline curve is not limited by a coordinate system, and can approximate a curve with any shape in the expression space. For the lane lines, the lane lines with any bit values and any shapes can be globally expressed in the same coordinate system, and the coordinate conversion relation between the vehicle coordinate system and the lane coordinate system is not required to be established in real time, so that the operation amount is saved.

Optionally, in one possible implementation manner of this embodiment, in step 103, a starting sampling point may be specifically determined based on the current position of the vehicle and the first control point, and further the first fitting curve may be uniformly sampled based on the starting sampling point, so as to select at least one first sampling point.

In this implementation, the first control point may include P ₀ ,P ₁ ,P ₂ ,P ₃ Based on the current position of the vehicle, P may be set ₁ Aligned with any part of the vehicle, e.g. P may be provided ₁ Laterally aligned with the vehicle head position.

Fig. 2 is a schematic diagram of obtaining sampling points based on control points and a fitting curve in a lane tracking method according to another embodiment of the present application, as shown in fig. 2 below. In FIG. 2, the blue rectangle represents the vehicle, the yellow solid line represents the fitted curve, P ₀ ,P ₁ ,P ₂ ,P ₃ Represents control points, P ₁ Transversely aligned with the position of the head according to P ₁ ,P ₂ Uniformly sampling the first fitting curve, and generating t E [0,1 ]]Evenly divided into T= { T ₀ ,t ₁ ,…t _(N-1) ,t _N And t is }, where ₀ Is 0, t _N 1.0, respectively corresponding to two ends of the first fitting curve. Substituting T into the curve equation results in a plurality of first sampling points, which can be noted as: g= { G ₀ ,g ₁ ,g ₂ ,…g _i ,…g _N }. First control point { P } of vehicle at current time K ₀ ,P ₁ ,P ₂ ,P ₃ } _K The geometry of the lane lines perceived by the vehicle can be described.

Preferably, the number of first sampling points may be 10.

Therefore, a plurality of first sampling points can be selected by uniformly sampling the first fitting curve, and the lane line can be predicted and updated by using a Kalman filtering algorithm based on the first sampling points, so that a lane line tracking result is obtained, the calculated amount in the data processing process is reduced, and the reliability and effect of the lane line tracking are effectively improved.

It should be noted that, the specific implementation procedure provided in the present implementation manner may be combined with the various specific implementation procedures provided in the foregoing implementation manner to implement the lane line tracking method of the present embodiment. The detailed description may refer to the relevant content in the foregoing implementation, and will not be repeated here.

Optionally, in one possible implementation manner of this embodiment, in step 104, specifically, a prediction model of kalman filtering may be used, based on the current position of the vehicle, to perform prediction processing on the at least one first sampling point to obtain a predicted value of the lane line, and further, based on the second lane line point cloud, a measurement model of kalman filtering may be used to update the predicted value of the lane line to obtain an estimated value of the lane line.

In this implementation, the Kalman filtering algorithm may include a predictive model and a measurement model.

In a specific implementation process of the implementation manner, a jacobian matrix of the kalman filtering prediction model may be determined specifically based on the current position of the vehicle and the at least one first sampling point, and then a predicted value of the lane line may be obtained based on the jacobian matrix and the at least one first sampling point.

In the specific implementation process, at least one first sampling point may correspond to the current time K of the vehicle, and further, the predicted value of the lane line may be a state value of the lane line at the time of predicting k+1.

In another specific implementation process of the implementation manner, a second fitting curve corresponding to the predicted value of the lane line can be obtained specifically based on the predicted value of the lane line, further the second fitting curve can be subjected to uniform division processing to obtain a plurality of line segments, distances between each line segment and the corresponding second lane line point cloud are calculated respectively to obtain a plurality of distances, and based on the plurality of distances and the predicted value of the lane line, a Kalman filtering measurement model is utilized to obtain the estimated value of the lane line.

Fig. 3 is a schematic diagram of a second lane line point cloud and a second fitted curve distance calculated in the lane line tracking method according to an embodiment of the present application, as shown in fig. 3 below.

One case of the specific implementation is that firstly, the spline curve can be usedAnd uniformly sampling the second fitting curve by t in the line to obtain n+1 sampling points. The second fitting curve can be divided into N line segments, namely short broken line segments, by connecting N+1 sampling pointsThen, the second lane line point cloud p= { p _i } and polyline segment->And (3) corresponding to each other one by one, and calculating the distance from the second lane line point cloud to the line segment.

In another case of the specific implementation process, one line segment may correspond to a plurality of second lane line point clouds, and further, first, one second lane line point cloud may be selected as a corresponding point cloud of one line segment. And secondly, calculating N distances based on each line segment and a second lane line point cloud corresponding to each line segment. Again, a jacobian matrix of the measurement model may be calculated based on the N distances and the predicted values of the lane lines. Finally, the estimated value of the lane line can be calculated based on the jacobian matrix of the measurement model and the predicted value of the lane line.

Therefore, the calculation amount can be further effectively reduced by selecting one second lane line point cloud as the corresponding point cloud of one line segment.

In still another specific implementation procedure of this implementation manner, further, iterative prediction and update processing may be performed on the predicted value of the lane line and the estimated value of the lane line. Specifically, a third fitting curve corresponding to the estimated value of the lane line may be obtained based on the estimated value of the lane line, and further, based on the current position of the vehicle, the current position corresponding to the estimated value of the lane line may be obtained, the driving direction of the vehicle may be determined based on the current position corresponding to the estimated value of the lane line and the first sampling point, based on the driving direction of the vehicle, at least one second sampling point may be selected from the third fitting curve by using a preset second sampling strategy, and iterative prediction processing may be performed on the at least one second sampling point and the second lane line point cloud by using a kalman filtering algorithm, so as to obtain the predicted value of the lane line and the estimated value of the lane line after iterative prediction and update processing.

In this particular implementation, the direction of travel of the vehicle may include forward travel and rearward travel.

In this specific implementation process, the current position corresponding to the estimated value of the lane line may be the position of the vehicle corresponding to the predicted time. The first sampling point may be a sampling point corresponding to a time immediately before the predicted time of the estimated value of the lane line, or the first sampling point may be a sampling point corresponding to a time of the previous iterative process. For example, the current position corresponding to the estimated value of the lane line may be the position of the vehicle at time K, and the first sampling point may be the sampling point corresponding to time K-1.

In one case of this specific implementation, it may be determined whether the traveling direction of the vehicle is forward traveling or backward traveling based on a relationship between the first sampling point and a current position corresponding to the estimated value of the lane line.

In another case of the specific implementation process, when the driving direction of the vehicle is forward driving, at least one second sampling point is selected from the third fitting curve by using a preset second sampling strategy based on forward driving, and a prediction model of kalman filtering can be used to predict the at least one second sampling point based on the current position corresponding to the estimated value of the lane line, so as to obtain a predicted value of a new lane line. Fig. 4 is a schematic diagram of a change of a fitted curve of a lane line in a forward driving situation in a lane line tracking method according to an embodiment of the present application, as shown in fig. 4 below. For the moment K+1, the vehicle runs for a distance, and at the moment, the second sampling point closest to the vehicle head is g _i The vehicle can be approximated to a state value { P over time K ₀ ,P ₁ ,P ₂ ,P ₃ } _K The determined movement of the lane line is g ₀ →g _i . The discrete arc length of forward motion can also be calculatedFIG. 4Wherein, the blue rectangle represents the vehicle, the yellow solid curve can represent the third fitting curve, and the green solid curve can represent the fitting curve corresponding to the new predicted value.

In still another case of the specific implementation process, when the driving direction of the vehicle is backward driving, at least one second sampling point is selected from the third fitting curve by using a preset second sampling strategy based on the backward driving, and a prediction model of kalman filtering can be used to perform prediction processing on the at least one second sampling point based on the current position corresponding to the estimated value of the lane line, so as to obtain a predicted value of a new lane line. Fig. 5 is a schematic diagram of a change of a fitted curve of a lane line in a backward driving situation in the lane line tracking method according to an embodiment of the present application, as shown in fig. 5 below. Based on the third fitting curve, calculate P ₁ The tangential direction of the point, and extracting sampling point on the backward extension line, namely the second sampling point, the sampling interval can be g ₀ g ₁ . At this time, the second sampling point closest to the vehicle head is g _i ' calculating the discrete arc length of the backward running movementCan be approximated as motion g ₀ →g′ _i And g is equal to _N →g _j Equal. In fig. 5, a blue rectangle indicates a vehicle, a yellow solid curve may represent a third fitted curve, and a green solid curve may represent a fitted curve corresponding to the new predicted value.

It will be appreciated that the preset first sampling strategy and the preset second sampling strategy may be the same, the number of the first sampling points and the second sampling points may be the same, the sampling intervals of the first sampling points and the second sampling points may be the same, i.e. P may be selected each time ₁ To P ₂ Sampling points at fixed interval positions of the fitting curve are fitted, so that matrix coefficients of spline curves corresponding to the sampling points are fixed.

Therefore, the fitting curves can be uniformly sampled based on t in the spline curves, so that t sequences sampled each time can be identical, the data operand can be reduced, and the lane line tracking efficiency is optimized.

In this way, iterative prediction processing can be performed on at least one second sampling point and the second lane line point cloud by using a Kalman filtering algorithm, so that the predicted value of the lane line and the estimated value of the lane line after iterative prediction and updating processing are obtained, and the lane line tracking result of the vehicle can be continuously obtained in real time according to the predicted value of the lane line and the estimated value of the lane line, thereby providing the efficiency and the reliability of the lane line tracking.

Optionally, in one possible implementation manner of the present embodiment, in step 105, specifically, the output frequency of the lane line tracking result may be determined based on the actual driving scenario of the vehicle, and in response to determining that the output frequency of the lane line tracking result is high, the predicted value of the lane line is taken as the lane line tracking result; and in response to determining that the output frequency of the lane line tracking result is low, taking the estimated value of the lane line as the lane line tracking result.

In this implementation, the output frequency is greater than a preset first frequency threshold, which may be determined to be a high frequency, and exemplary, the preset first frequency threshold may be 50 times/second. The output frequency is less than a preset second frequency threshold, which may be 5 times/second, for example, the low frequency may be determined.

It can be understood that, in the actual running process of the vehicle, the lane tracking result can be calculated and output in real time, and usually, the predicted value of the lane is output as the lane tracking result.

For a better understanding of the method according to the embodiment of the present application, the following describes the method according to the embodiment of the present application with reference to the accompanying drawings and specific application scenarios. In the application scenario, the spline curve may be a Catmull-Rom spline curve.

Fig. 6 is a flowchart of a method for lane tracking according to another embodiment of the present application, as shown in fig. 6.

Step 601, acquiring a lane line point cloud of an initial frame and a lane line point cloud of each frame after the initial frame.

Step 602, acquiring a current position of a vehicle in real time.

And 603, carrying out parameterization processing on the lane line point cloud of the initial frame by utilizing a Catmull-Rom spline curve so as to obtain an initial state quantity corresponding to the lane line point cloud of the initial frame.

In this embodiment, the initial state quantity includes the control point and the first fitting curve corresponding to the control point.

Specifically, the Catmull-Rom spline curve herein may be a third-order Catmull-Rom spline curve, where the definition of the Catmull-Rom spline curve is as shown in formula (1):

through 4 control points P ₀ ,P ₁ ,P ₂ ,P ₃ Two intermediate points P representing the lane line of the current road section and four control points ₁ ,P ₂ The shape between the two is C (t), t is E [0,1 ]]T introduces an argument, t is used to describe any point on the curve, and tensor τ describes the degree of curve bending at the control point, preferably taking τ=0.5.

The formula (1) can also be expressed as formula (2):

further, the formula (1) can be abbreviated as the formula (3):

C(t)＝B(t)X (3)

wherein, the derivative of any point C (t) with respect to t can be solved to obtain the formula (4):

may also be related to P ₀ ,P ₁ ,P ₂ ,P ₃ And (5) to obtain the derivative of formula (5):

it will be appreciated that for any shape lane line, the parameterization process is to fit the lane line using Catmull-Rom to obtain the control points.

The initial control point is obtained by constructing an overdetermined equation set. For initial lane line original point cloud data p _m ＝{p ₀ ,p ₁ ,p ₂ ,…,p _N The total path length of the original point cloud can be calculated, and the ratio of the path length from the first point cloud point to the i-th point cloud point to the total length can be calculated to obtain a ratio sequence T= { T ₀ ,t ₁ ,…,t _i ,…,t _N Bringing into formula 3, respectively, to build an overdetermined system of equations, as shown in formula (6):

wherein B (t) _i ) Is corresponding to t _i Coefficient of time, x= { P ₀ ,P ₁ ,P ₂ ,P ₃ Equation (6) is denoted as equation (7):

BX＝P (7)

pseudo-inverse B of B matrix being multiplied left on both sides respectively ^L Then equation (8) is obtained:

X＝B ^L P (8)

based on the above formulas (1) to (8), the initial state corresponding to the lane line point cloud of the initial frame can be calculatedQuantity X ₀ I.e. x= { P ₀ ,P ₁ ,P ₂ ,P ₃ Initial value X of } ₀ 。

It will be appreciated that the state quantity X is obtained here ₀ After the control point of (2), the state quantity X is calculated ₀ Is input to the kalman filter algorithm.

Step 604, selecting a plurality of sampling points from the initial state quantity by using a preset sampling strategy based on the current position of the vehicle and the initial state quantity.

In this embodiment, the preset sampling strategy may be uniform sampling based on t of the Catmull-Rom spline curve.

For example, for time K, the lane lines identified by the vehicle perception are distributed in front of the vehicle. After the parameterization of step 603, get the Catmull-Rom control point { P ] of the road section where the current vehicle is located ₀ ,P ₁ ,P ₂ ,P ₃ } _K ，P ₁ Laterally aligned with the vehicle head position as shown in fig. 2. Sampling the fitting curve corresponding to the initial state quantity according to the control point of the initial state quantity, and setting t' E [0,1 ]]Evenly divided into T= { T ₀ ,t ₁ ,…t _N-1 ,…t _N And t is }, where ₀ ＝0,t _N =1.0, corresponding to the two ends of the fitted curve, respectively. Substituting T into equation (1) can yield n+1 sampling points, which can be noted as: g= { G ₀ ,g ₁ ,g ₂ ,…g _i ,…g _N }. When the vehicle is stationary, { P ₀ ,P ₁ ,P ₂ ,P ₃ } _K The geometry of the road section ahead of the vehicle can be described.

When the vehicle is traveling forward or backward, the control point describing the shape of the lane line needs to slide { P } along the direction of the K moment ₀ ,P ₁ ,P ₂ ,P ₃ } _K →{P ₀ ,P ₁ ,P ₂ ,P ₃ } _K+1 。{P ₀ ,P ₁ ,P ₂ ,P ₃ } _K →{P ₀ ,P ₁ ,P ₂ ,P ₃ } _K+1 State transition X for Ying Kaer Manfiltered _k →X _k+1 . Thus, in step 605, the process may,in the iterative process, in the process of predicting and updating the plurality of sampling points, the running direction of the vehicle can be judged first, and then the corresponding sampling processing can be performed based on the forward running condition or the backward running condition of the vehicle.

Step 605, performing iterative prediction and updating processing on a plurality of sampling points by using a kalman filtering algorithm based on the current position of the vehicle and the lane line point cloud of each frame after the initial frame so as to obtain a predicted value of the lane line and an estimated value of the lane line.

In this embodiment, the kalman filter algorithm can be expressed as the following formula (9) and formula (10):

wherein equation (9) may represent a prediction model of the kalman filter, i.e. a prediction equation, and equation (10) may represent a measurement model of the kalman filter, i.e. a measurement equation. F (F) _k-1 May be a jacobian matrix of a predictive model,is the inverse matrix, accordingly->And->May be an estimated error covariance, Q' _k Is the covariance matrix of the noise of the prediction model, < +.>Can represent the predicted value, ">The estimated value may be represented. />The estimated value after update correction at the previous time (time k-1) can be represented. y is _k Is the lane line point cloud (point cloud value) of each frame after the initial frame, +. >Is to predict state quantity +.>Nonlinear metrology model mapped to measurement space, R' _k Is the noise matrix of the measurement model, i.e. normally distributed measurement noise, K _k For Kalman gain, G _k Is a jacobian matrix of the measurement model, +.>Is the inverse matrix accordingly. Here, the measurement model predicts the value of the model +.>And linearizing the above.

Specifically, in the process of predicting and updating the plurality of sampling points of the lane line point cloud by using a kalman filtering algorithm, firstly, the plurality of sampling points of the lane line point cloud of the initial frame may be predicted by using a prediction model of kalman filtering based on the current position of the vehicle, so as to obtain the predicted value of the lane line. And secondly, updating the predicted value of the lane line based on the second lane line point cloud by using a Kalman filtering measurement model so as to obtain the estimated value of the lane line. And after the running direction of the vehicle is determined, sampling a fitting curve of the state quantity based on the running direction of the vehicle to obtain a plurality of new sampling points, further, predicting the plurality of new sampling points based on the current position of the vehicle by using a Kalman filtering prediction model again to obtain a predicted value of the lane line at the next time of the current position, namely, a predicted value of the new lane line, and updating the predicted value of the new lane line based on a second lane line point cloud of a frame at the next time by using a Kalman filtering measurement model to obtain the estimated value of the lane line at the next time, namely, the estimated value of the new lane line. Based on the processing, the iterative prediction and updating processing of a plurality of sampling points of the lane line point cloud are completed, and the predicted value of the lane line and the estimated value of the lane line are obtained.

In the present embodiment, the vehicle traveling direction may include forward traveling or backward traveling.

On the one hand, in the process of obtaining the predicted value of the lane line, after determining that the vehicle traveling direction is forward traveling, the state quantity of the prediction model in the forward traveling case can be obtained specifically by the following calculation processes of the formulas (11) to (15)I.e. the predicted value of the lane line.

For example, as shown in fig. 4, time k+1 is the time next to time K, and for time k+1, the vehicle has traveled a distance, and the sampling point closest to the vehicle head may be g _i . Here, the vehicle can be approximately considered to be relative to the K-time control point { P } ₀ ,P ₁ ,P ₂ ,P ₃ } _K The determined movement of the lane line is g ₀ →g _i And can calculate the discrete arc length of forward running movement

In forward driving movement, as shown in FIG. 4, discrete sampling points { g } _i ,g _i+1 ,…g _N-1 ,g _N The second sampling point appears at the moment K and the moment k+1 simultaneously, and the formula (11) can be obtained by simultaneous equations based on the definition formula of substituting the common discrete sampling points into the Catmull-Rom spline curve:

wherein { B _k (t _i ),…,B _k (t _N ) The } is the Catmull-Rom coefficient of the sampling point at time k, { B _k+1 (t ₀ ),…,B _k+1 (t _N-i ) The "k+1" is the Catmull-Rom coefficient of the sample point at time k (t) _i ) And B (t) ₀ ) Correspondingly, B (t) _N ) And B is connected with _k+1 (t _N-i ) Corresponding to the above.

Further, equation 11 may be arranged as a matrix form of equation (12) as:

can be respectively M ₁ And M ₂ Representing the corresponding Catmull-Rom coefficient matrix, resulting in equation (13):

M ₁ X _k ＝M ₂ X _k+1 (13)

equality two sides simultaneously multiply M ₂ The inverse of (2), which here may be a pseudo-inverse based on SVD solution, may result in equation (14):

and (3) recording:the transition of the state quantity at the time k and the time k+1 can be expressed as formula (15):

X _k+1 ＝MX _k (15)

here, X obtained by the calculation process of the formulas (11) to (15) _k+1 Is the state quantity of the prediction model in the forward driving conditionI.e. the predicted value of the lane point cloud in forward driving situations, where M may represent the prediction modelJacobian matrix F _k-1 。

On the other hand, after determining that the vehicle traveling direction is backward traveling, the prediction model in the case of backward traveling can be obtained specifically by the following calculation procedures of the formula (16) to the formula (18)

As shown in fig. 5, for the backward running motion, in order to acquire the motion of the vehicle at the time k+1 with respect to the time K, P may be calculated ₁ The tangential direction of the point, and extracting sampling points on the backward extension line, wherein the sampling interval is the same as g ₀ g ₁ . At this time, the sampling point closest to the head is g _i ' the discrete arc length of backward running can be calculated Can be approximated as motion g ₀ →g′ _i And g is equal to _N →g _j Equal.

The common sampling points before and after the backward running at the moment K and the moment K+1 are recorded as follows: { g ₀ ,…g _j-1 ,g _j }. Equation (16) can also be derived based on the series of common simultaneous equations of sampling points and arranged into a matrix form:

as in the forward driving process, M can be used respectively ₁ And M ₂ Representing a corresponding Catmull-Rom spline coefficient matrix: the transition of the state quantity at the time k and the time k+1 can be expressed as formula (17):

M ₁ X _k ＝M ₂ X _k+1 (17)

and (3) recording:the transition of the state quantity at the time k and the time k+1 can be expressed as the formula (18):

X _k+1 ＝MX _k (18)

here, X obtained by the calculation process of the formulas (16) to (18) _k+1 Is the state quantity of the prediction model in the backward driving conditionI.e. the predicted value of the lane point cloud in the backward driving situation, where M may represent the jacobian matrix F of the prediction model _k-1 。

It will be appreciated that here, in the actual operation, both forward running and backward running may be performed by sampling only a fixed number of points, and the coefficient array is a constant matrix, which may not impose a significant computational burden.

It will be appreciated that in the algorithm of the Kalman filtering, the state quantity of the Catmull-Rom spline curve can be in the form of a vector, the lane geometry of the road section near the vehicle is modeled by a set of control points, 4 control points are used, each control point has two values of X and y, and the state quantity is a vector X= { p with the length of 8 as a whole _0.x ,p _0.y ,p _1.x ,p _1.y ,p _2.x ,p _2.y ,p _3.x ,p _3.y }. M in equation (15) and equation (18) can be expanded to 8 x 8 form as shown in equation (19), and equation (15) and equation (18) can be expanded to equation (20):

in this embodiment, the lane line point cloud of each frame after the initial frame is the second lane line point cloud, which can be used as the observed quantity of the kalman filter measurement model.

And (3) updating the predicted value of the lane line based on the second lane line point cloud by using a Kalman filtering measurement model, wherein in the process of obtaining the estimated value of the lane line, the estimated value of the lane line can be obtained through calculation from the formula (21) to the formula (31).

The state quantity predicted by the prediction model, namely the predicted value of the lane line, can be continuously fused and corrected by using the Kalman filtering measurement model. The second lane line point cloud may be used as an observation point, and the second lane line point cloud is formed by discrete observation points. The consistency between the second lane line point cloud and the Catmull-Rom spline curve determined by the predicted value X of the lane line, i.e. the second fitted curve, can be expressed using the point-to-curve distance.

For the second fitting curve determined by the state quantity X determined by the prediction model, firstly, uniformly sampling N+1 points according to the Catmull-Rom spline t to form N short broken line segments Then, the second road line point cloud, namely the observed quantity p= { p _j } and polyline segment->One-to-one correspondence and calculating the distance from the point to the polyline: d=f (p _j ,g _i ,g _i+1 )。

It will be appreciated that the samplings may be the same as those in the predictive model described above, and may be just a fixed number of samplings.

Specifically, the distance d from the cloud point of the second lane line point to the line segment is just the residual error of the filtering updating stage, and a nonlinear equation is arranged between d and the control point of the state quantity X determined by the prediction model.

Illustratively, in a first segmentAnd a corresponding one thereofSecond lane line point cloud p _m (x _m ,y _m ) As an example. Determination of ∈two-point>The linear equation is shown in the formula (21):

point p _m Line to line segmentDistance d of (2) ₀ As shown in equation (22):

here, the point-to-line distance d is a scalar, and signed distances d may be used, where positive d indicates that the point is to the left of the line segment and negative d indicates that the point is to the right of the line segment.

In particular, in order to calculate the jacobian of the measurement model, the derivative of d with respect to the state quantity X can be calculated. According to the chain derivative rule, equation (23) is obtained:

here, the derivative of d with respect to X can be decomposed into the product of the derivative of d with respect to the sampling point G and the derivative of G with respect to X. The derivative of G to X can be directly calculated by a Catmull-Rom spline curve formula (5).

d derivative of sampling point G can be calculated according to Catmull-Rom spline curve formula (8)Equation (24) can be derived:

/>

equation (25) can be derived from the definition of the chain derivative rule and the full derivative:

sorting into a form of vector multiplication yields equation (26):

sampling point g according to Catmull-Rom spline curve formula (5) _i The jacobian matrix for the control points can be as shown in equation (27):

for a group ofThe jacobian matrix for X is shown in equation (28): />

d derivative of X, yielding equation (29):

recording deviceFurther, the formula (29) can be abbreviated as the formula (30):

it will be appreciated that the estimate is derived here from the distance of a second lane line point cloud point from a polyline segment. In actual calculation, one line segment usually corresponds to a plurality of second lane line point cloud points, and the plurality of point cloud points can sample one point from the plurality of point cloud points to calculate, so that the calculated amount can be reduced.

Further, based on the N line segments and the corresponding second lane line point cloud points, derivatives of all the second lane line point cloud points to the state quantity of the prediction model may form a jacobian matrix:

/>

here, based on the formulas (21) to (31), the update processing of the predicted value of the lane line by the kalman filter measurement model may be implemented, resulting in the estimated value of the lane line. Wherein equation (22) corresponds to the Kalman filtered measurement model Equation (31) is the jacobian matrix G of the measurement model _k 。

And 606, obtaining a tracking result of the lane line based on the predicted value of the lane line and the estimated value of the lane line.

In this embodiment, fig. 7 is a schematic diagram of an application scenario of a lane tracking method according to another embodiment of the present application, as shown in fig. 7. In this application scenario, the spline curve is a Catmull-Rom spline curve. Firstly, the lane line point cloud of the initial frame can be parameterized by Catmull-Rom, the parameterized result, namely a plurality of sampling points, can be input into a Kalman filtering prediction model, and meanwhile, the vehicle position information under the UTM/Odom coordinate system can be input into the Kalman filtering prediction model. Secondly, inputting the lane line point cloud of each frame after the initial frame into a Kalman filtering measurement model, and carrying out iterative prediction and updating on a plurality of sampling points based on the Kalman filtering prediction model and the measurement model and the lane line point cloud of each frame after the initial frame so as to obtain and output a lane line tracking result in real time.

In this embodiment, fig. 8 is a schematic diagram of a fitted curve parameterized by a Catmull-Rom spline curve of a lane line of different shape according to another embodiment of the present application, as shown in fig. 8. The figure comprises a fitting curve of a transverse S shape, a fitting curve of a longitudinal S shape and a C-shaped fitting curve. It can be seen that the control points of the lane line parameterization based on the Catmull-Rom spline can pass through the lane line itself, thus intuitively and effectively fitting the geometry of the lane line, and being independent of a local coordinate system, thus optimizing the tracking effect of the lane line.

In this embodiment, fig. 9 is a schematic diagram of a fitted curve of predicted values of lane lines of a prediction model of a lane line tracking method according to another embodiment of the present application, as shown in fig. 9. The prediction model in this embodiment may be an equal ratio driving model, which can effectively predict the control point position at the next moment in the forward direction or backward direction, and predict the geometry of a section of lane line. As shown in fig. 9, the red line segment is the shape of a section of lane line at time K, the red point is the control point at time K, two of the control points are two end points of the fitted curve of the lane line, the blue line segment predicts the section of lane line at time k+1 according to the prediction model, and the blue point is the control point at time k+1, where Δt represents the prediction period, such as Δt=0.2, Δt=0.3, Δt=0.4.

In this embodiment, based on iterative progression of prediction update of the kalman filtering algorithm, noise errors caused by single prediction or measurement can be weakened, so that the state estimation value is closer to the theoretical true value in probability, and a smooth estimation result is obtained. The lane line tracking result is obtained based on the method of the embodiment, the whole modeling of the lane line is carried out, and a lane line vectorization diagram of real-time tracking can be obtained. Fig. 10 is a schematic diagram of a real-time lane tracking result of a prediction model of a lane tracking method according to another embodiment of the present application, as shown in fig. 10. The left graph shows the lane line tracking result obtained without the method of the embodiment, and the right graph shows the lane line tracking result obtained with the method of the embodiment, and it can be seen that the blue area in front of the vehicle (white vehicle) and the lane lines around the vehicle in the right graph are clearer and cleaner, the display effect is better, the vehicle is closer to the real lane line, and the reliability of the lane line tracking result is better.

It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present application is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present application.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.

Fig. 11 is a block diagram showing the structure of an apparatus for lane line tracking according to an embodiment of the present application, as shown in fig. 11. The apparatus 1100 for lane line tracking of the present embodiment may include an acquisition unit 1101, an acquisition unit 1102, a sampling unit 1103, a prediction unit 1104, and a tracking unit 1105. Wherein, the obtaining unit 1101 is configured to obtain a lane line point cloud and a current position of a vehicle; the lane line point cloud comprises a first lane line point cloud and a second lane line point cloud, wherein the first lane line point cloud is the lane line point cloud of the initial frame, and the second lane line point cloud is the lane line point cloud of each frame after the initial frame; an obtaining unit 1102, configured to perform fitting processing on the first lane line point cloud by using a spline curve, so as to obtain a first control point of the first lane line point cloud and a first fitting curve corresponding to the first control point; a sampling unit 1103, configured to select at least one first sampling point from the first fitting curve by using a preset first sampling strategy based on the current position of the vehicle and the first control point; a prediction unit 1104, configured to perform prediction and update processing on the at least one first sampling point based on the current position of the vehicle and the second lane line point cloud by using a kalman filtering algorithm, so as to obtain a predicted value of a lane line and an estimated value of the lane line; a tracking unit 1105, configured to obtain a tracking result of the lane line based on the predicted value of the lane line and the estimated value of the lane line.

It should be noted that, part or all of the lane tracking apparatus in this embodiment may be an application located at a local terminal, or may be a functional unit such as a plug-in unit or a software development kit (Software Development Kit, SDK) disposed in the application located at the local terminal, or may be a processing engine located in a server on a network side, or may be a distributed system located on the network side, for example, a processing engine or a distributed system in an autopilot platform on the network side, which is not limited in this embodiment.

Optionally, in one possible implementation manner of this embodiment, the obtaining unit 1102 may specifically be configured to determine a path length between a first lane line point cloud point and each i-th first lane line point cloud point, and a total path length of the first lane line point cloud, i is a natural number greater than 1, obtain a path length ratio sequence based on the path length and the total path length, and calculate, using a spline curve, a first control point of the first lane line point cloud and a first fitting curve corresponding to the first control point based on the path length ratio sequence and the first lane line point cloud.

Optionally, in one possible implementation manner of this embodiment, the sampling unit 1103 may specifically be configured to determine a starting sampling point based on the current position of the vehicle and the first control point, and uniformly sample the first fitting curve based on the starting sampling point to select at least one first sampling point.

Optionally, in one possible implementation manner of this embodiment, the prediction unit 1104 may specifically be configured to perform, based on the current position of the vehicle, a prediction process on the at least one first sampling point to obtain a predicted value of the lane line, and update, based on the second lane point cloud, a predicted value of the lane line using a kalman filter measurement model to obtain an estimated value of the lane line.

Optionally, in one possible implementation manner of this embodiment, the prediction unit 1104 may be specifically configured to obtain, based on the predicted value of the lane line, a second fitted curve corresponding to the predicted value of the lane line, perform uniform division processing on the second fitted curve to obtain a plurality of line segments, respectively calculate distances between each line segment and the corresponding second lane line point cloud, obtain a plurality of distances, and obtain, based on the plurality of distances and the predicted value of the lane line, an estimated value of the lane line by using a kalman filter measurement model.

Optionally, in one possible implementation manner of this embodiment, the prediction unit 1104 may be specifically configured to obtain a third fitted curve corresponding to the estimated value of the lane line based on the estimated value of the lane line, obtain the current position corresponding to the estimated value of the lane line based on the current position of the vehicle, determine the driving direction of the vehicle based on the current position corresponding to the estimated value of the lane line and the first sampling point, select at least one second sampling point from the third fitted curve based on the driving direction of the vehicle by using a preset second sampling strategy, and perform iterative prediction and update processing on the at least one second sampling point and the second lane line point cloud by using a kalman filtering algorithm to obtain the predicted value of the lane line and the estimated value of the lane line after the iterative prediction and update processing.

Optionally, in one possible implementation manner of this embodiment, the prediction unit 1104 may specifically be configured to determine a jacobian matrix of the kalman filtering prediction model based on the current location of the vehicle and the at least one first sampling point, and obtain the predicted value of the lane line based on the jacobian matrix and the at least one first sampling point.

In the embodiment, the lane line point cloud and the current position of the vehicle are acquired through an acquisition unit; the lane line point cloud comprises a first lane line point cloud and a second lane line point cloud, the first lane line point cloud is the lane line point cloud of an initial frame, the second lane line point cloud is the lane line point cloud of each frame after the initial frame, and further the first lane line point cloud can be subjected to fitting processing by an obtaining unit through a spline curve so as to obtain a first control point of the first lane line point cloud and a first fitting curve corresponding to the first control point, at least one first sampling point is selected from the first fitting curve by a sampling unit based on the current position of a vehicle and the first control point by a preset first sampling strategy, the predicting unit predicts and updates the at least one first sampling point by a Kalman filtering algorithm based on the current position of the vehicle and the second lane line point cloud, so that the tracking unit can obtain a predicted value of the lane line and an estimated value of the lane line based on the predicted value of the lane line and the first fitting curve corresponding to the first control point, the first sampling point is more accurately calculated by the sampling unit through the Kalman filtering algorithm based on the current position of the vehicle and the second lane line, the curve can be more easily fitted by the first fitting curve, and the curve can be more accurately calculated, and the load of the lane line is more accurately calculated by the curve is reduced.

One embodiment of the present application provides a computer-readable storage medium having stored therein at least one instruction that is loaded and executed by a processor to implement a method of lane line tracking as described above.

One embodiment of the present application provides an electronic device comprising a processor and a memory having at least one instruction stored therein, the instruction being loaded and executed by the processor to implement a method of lane line tracking as described above.

One embodiment of the present application provides an autonomous vehicle including an electronic device as described above. Specifically, the autonomous vehicle may be a vehicle of the L2 class and above.

In the technical scheme of the application, the related processes of collecting, storing, using, processing, transmitting, providing, disclosing and the like of the personal information of the user accord with the regulations of related laws and regulations, and the public order is not violated.

FIG. 12 shows a schematic block diagram of an example electronic device 1200 that may be used to implement an embodiment of the application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the applications described and/or claimed herein.

As shown in fig. 12, the electronic device 1200 includes a computing unit 1201 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 1202 or a computer program loaded from a storage unit 1208 into a Random Access Memory (RAM) 1203. In the RAM 1203, various programs and data required for the operation of the electronic device 1200 may also be stored. The computing unit 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input/output (I/O) interface 1205 is also connected to the bus 1204.

Various components in the electronic device 1200 are connected to the I/O interface 1205, including: an input unit 1206 such as a keyboard, mouse, etc.; an output unit 1207 such as various types of displays, speakers, and the like; a storage unit 1208 such as a magnetic disk, an optical disk, or the like; and a communication unit 1209, such as a network card, modem, wireless communication transceiver, etc. The communication unit 1209 allows the electronic device 1200 to exchange information/data with other devices through a computer network, such as the internet, and/or various telecommunications networks.

The computing unit 1201 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 1201 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 1201 performs the respective methods and processes described above, for example, a lane line tracking method. For example, in some embodiments, the method of lane line tracking may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 1208. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 1200 via the ROM 1202 and/or the communication unit 1209. When the computer program is loaded into the RAM 1203 and executed by the computing unit 1201, one or more steps of the method of lane line tracking described above may be performed. Alternatively, in other embodiments, the computing unit 1201 may be configured to perform the lane line tracking method in any other suitable manner (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present application may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present application, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present disclosure may be performed in parallel, sequentially, or in a different order, so long as the desired result of the technical solution of the present disclosure is achieved, and the present disclosure is not limited herein.

The above embodiments do not limit the scope of the present application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of the present application.

Claims

1. A method of lane line tracking, the method comprising:

2. The method of claim 1, wherein the first lane-line point cloud comprises a plurality of first lane-line point cloud points, wherein the fitting the first lane-line point cloud using spline curves to obtain a first control point of the first lane-line point cloud and a first fitted curve corresponding to the first control point comprises:

3. The method of claim 1, wherein the selecting at least one first sampling point from the first fitted curve using a preset first sampling strategy based on the current location of the vehicle and the first control point comprises:

4. The method of claim 1, wherein the predicting and updating the at least one first sampling point based on the current location of the vehicle and the second lane line point cloud using a kalman filtering algorithm to obtain a predicted value of a lane line and an estimated value of a lane line comprises:

5. The method of claim 4, wherein the updating the predicted value of the lane line based on the second lane line point cloud by using the kalman filter measurement model to obtain the estimated value of the lane line comprises:

6. The method according to claim 4 or 5, wherein the predicting and updating the at least one first sampling point based on the current position of the vehicle and the second lane line point cloud by using a kalman filtering algorithm to obtain a predicted value of a lane line and an estimated value of a lane line, further comprises:

7. The method of claim 4, wherein the predicting the at least one first sampling point based on the current location of the vehicle using the kalman filter prediction model to obtain the predicted value of the lane line comprises:

8. An apparatus for lane line tracking, the apparatus comprising:

9. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-7.

11. An autonomous vehicle comprising the electronic device of claim 9.