CN110503009A

CN110503009A - Lane line tracking and Related product

Info

Publication number: CN110503009A
Application number: CN201910719667.XA
Authority: CN
Inventors: 杨臻
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2019-07-31
Filing date: 2019-07-31
Publication date: 2019-11-26
Anticipated expiration: 2039-07-31
Also published as: CN110503009B

Abstract

The embodiment of the present application provides a kind of lane line tracking and Related product, and a kind of lane line tracking includes: to be predicted according to N number of first lane model each of of the driving parameters of vehicle to the t-1 moment in t moment, obtain N number of second lane model；N number of second lane model is updated according to the lane line feature in target image, obtains N number of third lane model, the target image is the image of vehicle front predeterminable area described in the t moment；The adaptation probability of each third lane model in N number of third lane model is calculated according to probability parameter；Determine that the third lane model that maximum probability is adapted in N number of third lane model, the third lane model of the adaptation maximum probability are used to track the lane line of the vehicle running surface.The embodiment of the present application is conducive to improve the accuracy of lane line tracking.

Description

Lane line tracking method and related product

Technical Field

The application relates to the technical field of automatic driving, in particular to a lane line tracking method and a related product.

Background

With the development of computer vision technology, the input image is subjected to feature extraction through a neural network algorithm to obtain lane line features of lane lines in the input image, the lane line features are processed by adopting a lane model, and the lane lines in front of a vehicle are output to realize unmanned driving. At present, there are two commonly used lane models, the first is a lane model based on a parallel hypothesis, that is, all lane lines are assumed to be parallel to each other, and the second is a lane model based on a non-parallel hypothesis, that is, all lane lines are assumed to be non-parallel, due to the constraint of self conditions of the lane model based on the parallel hypothesis or the non-parallel hypothesis, when a vehicle runs at a complex intersection, for example, when there are parallel lane lines and non-parallel lane lines at the same time, the lane lines may be missed or the lane lines output do not conform to the rules, so that each lane line at the complex intersection cannot be accurately tracked by using the lane model.

Disclosure of Invention

The embodiment of the application provides a lane line tracking method and related products, which are beneficial to enabling a vehicle to adapt to various complex driving scenes, so that the accuracy of tracking the lane line is improved, and further the traffic safety is improved.

In a first aspect, an embodiment of the present application provides a lane line tracking method, including:

at the time t, predicting each of N first lane models at the time t-1 according to driving parameters of a vehicle to obtain N second lane models, wherein the first lane models are used for tracking multiple groups of lanes, lanes A and lanes B are not parallel, the lanes A and the lanes B are lanes in any two groups of the multiple groups of lanes respectively, the lanes in each group of lanes in the multiple groups of lanes are parallel to each other, and N is an integer greater than or equal to 1;

updating the N second lane models according to lane line characteristics in a target image to obtain N third lane models, wherein the target image is an image of a preset area in front of the vehicle at the time t;

calculating the adaptation probability of each third lane model in the N third lane models according to the probability parameters, wherein the adaptation probability is used for representing the adaptation degree of the third lane model and the lane line of the driving road surface of the vehicle;

and determining a third lane model with the maximum adaptation probability in the N third lane models, wherein the third lane model with the maximum adaptation probability is used for tracking a lane line of the driving road surface of the vehicle.

The first lane model is used for tracking a plurality of groups of lanes, the lanes of the plurality of groups of lanes are not parallel to each other, and the lanes contained in each group of lanes are parallel to each other, so that the third lane model obtained at the moment t can be used for tracking parallel lane lines and non-parallel lane lines, the accuracy of lane line tracking is improved, and vehicles can adapt to various complex driving environments and the traffic safety is improved; and the third lane model with the maximum adaptation probability is adopted to track the lane line, so that the accuracy of lane line tracking is further improved.

In some possible embodiments, the predicting each of the N first lane models at the time t-1 according to the driving parameters of the vehicle to obtain N second lane models includes:

obtaining a prediction matrix according to the driving parameters of the vehicle;

and predicting each of the N first lane models at the t-1 moment according to the prediction matrix to obtain N second lane models.

Therefore, the lane model at the current moment is predicted based on the lane model at the previous moment and the vehicle driving parameters at the current moment, so that the data at the two moments are correlated, the predicted lane model can contain the existing driving information of the vehicle, and the predicted lane model can better accord with the current driving scene.

In some possible embodiments, the updating the N second lane models according to the lane characteristics in the target image to obtain N third lane models includes:

dividing the target image into T sub-images, wherein the distance between the area corresponding to the ith sub-image and the vehicle is less than the distance between the area corresponding to the (i + 1) th sub-image and the vehicle, i is an integer, i is greater than or equal to 1 and less than or equal to T, and T is an integer greater than or equal to 2;

acquiring lane line characteristics in the ith sub-image;

selecting U target lane models matched with lane line characteristics of the ith sub-image from N first reference lane models, performing ith updating operation on each of the U target lane models according to the lane line characteristics to obtain U first reference lane models, wherein the ith updating result comprises the U updated first reference lane models, when i is 1, the N first reference lane models are the N second lane models, when i is greater than 1, the N first reference lane models are updating results of the i-1 th time, and U is greater than or equal to 0 and less than or equal to N;

and when the i is equal to T, the T-th updating result obtained after the T-th updating operation is executed is the N third lane models.

Therefore, the predicted lane model is updated based on the target image of the preset area in front of the vehicle, and when the model is updated, a segmented matching updating mode is adopted, so that the condition that the mistaken detection lane line is updated mistakenly is avoided, the updating error is eliminated, and the updated third lane model is more adaptive to the current driving environment.

In some possible embodiments, the selecting, from the N first reference lane models, U target lane models that match lane line features of the ith sub-image includes:

acquiring an observation vector of a lane line in the target image according to the lane line feature in an image coordinate system;

acquiring M predicted observation vectors corresponding to a lane line in the target image and a lane model A according to the lane line characteristics under a vehicle coordinate system, wherein M is the number of the lane lines tracked by the lane model A, the lane model A is any one reference lane model of the N first reference lane models, and M is an integer greater than or equal to 1;

determining M Mahalanobis distances corresponding to the observation vectors and the M predicted observation vectors;

and determining the minimum mahalanobis distance in the M mahalanobis distances, and when the minimum mahalanobis distance is smaller than a distance threshold, determining the lane model A as a target lane model matched with the lane line characteristics of the ith sub-image.

In some possible embodiments, the method further comprises:

when i is equal to T, if f minimum mahalanobis distances corresponding to a lane line C in the target image are all greater than or equal to the distance threshold, creating N fourth lane models according to the N third lane models, where the N fourth lane models are consistent with lane line parameters of the N third lane models, the lane line C is any one lane line in the target image, the f minimum mahalanobis distances are minimum mahalanobis distances corresponding to lane line features of the lane line C in f sub-images, the f sub-images are sub-images including the lane line features of the lane line C in the T sub-images, the f is an integer greater than or equal to 1, and f is equal to or less than T;

processing each of the N fourth lane models to obtain N new lane models;

and taking the N fourth lane models and the N new lane models as the first lane model at the time t.

Therefore, when the untracked lane line is detected to exist, a new lane model is created, so that the untracked lane line is tracked, the problem of missing detection of the lane line is avoided, and the accuracy of tracking the lane line is improved.

In some possible embodiments, the processing each of the N fourth lane models to obtain N new lane models includes:

acquiring the relative distance between the lane line C and the vehicle;

initially assigning a lane line parameter corresponding to the lane line C in each fourth lane model according to the relative distance to obtain N fifth lane models;

fitting the lane line characteristics of each sub-image in the T sub-images to obtain at least one fitting equation;

if a target fitting equation is firstly obtained in the kth sub-image of the T sub-images, adopting the lane line characteristics of the lane line C in the kth sub-image to update each of the N fourth lane models to obtain the current latest N fourth lane models, wherein the target fitting equation is a fitting equation of which the difference value between the intercept and the relative distance in the at least one fitting equation is smaller than a distance threshold value;

and starting from the (k + 1) th sub-image in the T sub-images, sequentially adopting the lane line characteristics in each sub-image to update the N fourth lane models to obtain the current latest N fourth lane models, and obtaining the N new lane models after the current latest N fourth lane models are updated by adopting the lane line characteristics in the T sub-images.

Therefore, the newly created lane model is updated by the target image so as to be in a tracking state, and the lane line which is not tracked at the time t is tracked by the lane model at the time t +1, so that the missing detection of the lane line is avoided.

In some possible embodiments, the probability parameters include a target matching probability, a prior probability, and an adaptation probability of the first lane model B at the time t-1; the adaptation probability of the third lane model B' is obtained by the target matching probability, the prior probability and the adaptation probability of the first lane model B at the t-1 moment; the first lane model B is any one of the N first lane models;

the target matching probability is used for representing the matching degree of all lane lines in the target image and the third lane model B';

the prior probabilities are used to characterize sources of the third lane model B ', including the third lane model B' by performing an update operation on the first lane model B.

In some possible embodiments, the method further comprises:

obtaining a target third lane model according to the adaptation probability of each third lane model, wherein the target third lane model is a third lane model of which the adaptation probability is smaller than a probability threshold value in the N third lane models;

deleting the target third lane model with the tracking duration being greater than the duration threshold, wherein the tracking duration is the total duration of the target third lane model from the creation time to the t time.

It can be seen that, in the embodiment, the lane model with the probability lower than the probability threshold is deleted in time, so that the calculation amount of the lane line when the lane line is tracked is reduced, and the calculation burden of the vehicle-mounted device is reduced, so that the lane model matched with the current lane line can be found more quickly, and the tracking efficiency is improved.

In a second aspect, an embodiment of the present application provides a lane line tracking apparatus, including:

the vehicle driving prediction system comprises a prediction unit, a prediction unit and a control unit, wherein the prediction unit is used for predicting each of N first lane models at the time t-1 according to driving parameters of a vehicle at the time t to obtain N second lane models, the first lane models are used for tracking multiple groups of lanes, lanes A and lanes B are not parallel, the lanes A and the lanes B are lanes in any two groups of the multiple groups of lanes respectively, the lanes in each group of the multiple groups of lanes are parallel to each other, and N is an integer greater than or equal to 1;

the updating unit is used for updating the N second lane models according to lane line characteristics in a target image to obtain N third lane models, and the target image is an image of a preset area in front of the vehicle at the time t;

the calculation unit is used for calculating the adaptation probability of each third lane model in the N third lane models according to the probability parameters, and the adaptation probability is used for representing the adaptation degree of the third lane model and the lane line of the driving road surface of the vehicle;

and the tracking unit is used for determining a third lane model with the maximum adaptation probability in the N third lane models, and the third lane model with the maximum adaptation probability is used for tracking a lane line of the driving road surface of the vehicle.

In some possible embodiments, in terms of predicting each of the N first lane models at time t-1 according to the driving parameters of the vehicle to obtain N second lane models, the prediction unit is specifically configured to: obtaining a prediction matrix according to the driving parameters of the vehicle; and predicting each of the N first lane models at the t-1 moment according to the prediction matrix to obtain N second lane models.

In some possible embodiments, in terms of updating the N second lane models according to the lane line features in the target image to obtain N third lane models, the updating unit is specifically configured to:

acquiring lane line characteristics in the ith sub-image;

In some possible embodiments, in selecting U target lane models matching the lane line feature of the ith sub-image from the N first reference lane models, the updating unit is specifically configured to:

In some possible embodiments, the apparatus further comprises a creating unit, configured to:

processing each of the N fourth lane models to obtain N new lane models;

In some possible embodiments, in terms of processing each of the N first lane models according to the lane line characteristics of the lane line C to obtain N new lane models, the creating unit is specifically configured to:

in respect of processing each of the N fourth lane models to obtain N new lane models, the creating unit is specifically configured to:

acquiring the relative distance between the lane line C and the vehicle;

In some possible embodiments, the apparatus further comprises a deletion unit, configured to:

In a third aspect, an embodiment of the present application provides another lane line tracking apparatus, including:

the device comprises a processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are connected through electric signals;

the processor is used for predicting each of N first lane models at the time t-1 according to the driving parameters of the vehicle at the time t to obtain N second lane models, the first lane models are used for tracking multiple groups of lanes, a lane A and a lane B are not parallel, the lane A and the lane B are lanes in any two groups of the multiple groups of lanes respectively, lanes contained in each group of the multiple groups of lanes are parallel to each other, and N is an integer greater than or equal to 1;

the processor is further configured to update the N second lane models according to lane line features in a target image to obtain N third lane models, where the target image is an image of a preset area in front of the vehicle at the time t;

the processor is further configured to calculate an adaptation probability of each of the N third lane models according to a probability parameter, where the adaptation probability is used to represent an adaptation degree of the third lane model to a lane line of the driving road surface of the vehicle;

the processor is further configured to determine a third lane model with the largest adaptation probability in the N third lane models, where the third lane model with the largest adaptation probability is used to track a lane line of a driving road of the vehicle.

In a fourth aspect, the present application further provides a computer-readable storage medium, where a computer program is stored, where the computer program is executed by hardware (for example, a processor, and the like) to perform part or all of the steps of any one of the methods performed by the lane line tracking device in the present application.

In a fifth aspect, embodiments of the present application provide a computer program product comprising instructions which, when run on a lane line tracking apparatus, cause the lane line tracking apparatus to perform some or all of the steps of the lane line tracking method of the above aspects.

Drawings

Some drawings to which embodiments of the present application relate will be described below.

Fig. 1A is a schematic view of a lane line in a vehicle coordinate system according to an embodiment of the present disclosure;

fig. 1B is a schematic view of a driving scene according to an embodiment of the present disclosure;

fig. 1C is a schematic structural diagram of an on-board device according to an embodiment of the present application;

fig. 1D is a schematic flowchart of a lane line tracking method according to an embodiment of the present disclosure;

fig. 2A is a schematic flowchart illustrating a method for updating a second lane model according to an embodiment of the present disclosure;

fig. 2B is a schematic diagram of dividing a target image according to an embodiment of the present disclosure;

fig. 2C is a schematic flowchart of another method for updating a lane model according to an embodiment of the present disclosure;

fig. 3A is a schematic flowchart of a method for initializing a lane model according to an embodiment of the present disclosure;

fig. 3B is a schematic view of a high-speed ramp junction scenario provided in the embodiment of the present application;

FIG. 4 is a schematic flow chart illustrating a method for creating a new lane model according to an embodiment of the present application;

FIG. 5 is a flowchart illustrating a method for managing a lane model according to an embodiment of the present disclosure;

FIG. 6 is a schematic diagram of a deleted lane model according to an embodiment of the present disclosure;

fig. 7 is a schematic view of a lane line tracking apparatus according to an embodiment of the present disclosure;

fig. 8 is a schematic view of another lane line tracking apparatus according to an embodiment of the present disclosure.

Detailed Description

The terms "first," "second," "third," and "fourth," etc. in the description and claims of this application and in the accompanying drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

To understand the solution of the present application, a lane model is first introduced.

And modeling the lane line under a vehicle coordinate system to obtain a lane line equation, and expressing equation parameters (lane line parameters) in the lane line equation by vectors to obtain a lane model. At present, a lane line is modeled by a cubic spiral line shown in formula (1) commonly used, and a lane line equation f (l) is obtained, wherein a lane model corresponding to the lane line equation is x ═ C₁C₀b Y₀]。

As shown in fig. 1A, in the vehicle coordinate system, C₁Rate of change of curvature of lane line, C₀Is the curvature of the lane line, b is the slope of the tangent between the origin of coordinates and the lane line, Y₀Is the lateral offset of the lane line from the origin of coordinates, l is the distance along the lane line of the intersection of the lane line and the y-axis, and f (l) is the lateral offset from the intersection of the lane line and the y-axis relative to the x-axis.

At present, when modeling a plurality of lane lines, two ways are generally adopted, the first way is based on the parallel hypothesisI.e. assuming that all lane lines are parallel, C of a plurality of lane lines₁、C₀B is the same; the second method is based on the non-parallel assumption that each lane line has an independent C if all lane lines are not parallel₁、C₀B, obtaining a lane line equation based on a parallel hypothesis and a non-parallel hypothesis based on the modeling method in the formula (1), namely as shown in the formula (2) and the formula (3):

based on the parallel hypothesis, a plurality of lane lines are tracked as a whole, and a lane model is constructed to track the plurality of parallel lane lines, wherein the lane model is x ═ C₁C₀b Y₀…Y_n](ii) a Based on the non-parallel assumption, a lane model needs to be constructed for each lane line, and then the lane model corresponding to the ith lane line is as follows:

based on the parallel hypothesis, when a lane model is adopted to track lane lines, due to the existence of parallel constraint, non-parallel lane lines can be missed to be detected when the lane lines are tracked, due to the existence of non-parallel constraint, based on the non-parallel hypothesis, parallel lane lines can be missed to be detected, and due to the jolt of a vehicle when the vehicle goes up a slope or goes down a slope, the upper and lower slopes of a lane surface are inconsistent, when the lane lines are projected under an image coordinate system, the parallel lane lines are projected to be the non-parallel lane lines, so that the lane lines with the shape of an inner Chinese character eight or an outer Chinese character eight are output. In order to solve the defect of tracking the lane line at present, the technical scheme of the application is specially provided.

The lane model referred to in this application will be described.

The lane model related in the embodiment of the application is used for tracking N groups of lanes and M lane lines corresponding to the N groups of lanes, wherein any two groups of lanes in the N groups of lanes are not parallel, the lanes contained in each group of lanes are parallel to each other, and specific incoming traffic can belong to a group of lanes with a plurality of lanes which are parallel to each other when tracking the parallel lanes, so that each group of lanes contains 0 or more parallel lane lines. It will be appreciated that if each set of lanes contains only one lane, there are no parallel lanes in the set of parallel lanes. N is an integer greater than or equal to 2, M is an integer greater than or equal to 2, and N and M can be set by a user independently according to different driving scenes or automatically set by a system according to different driving scenes.

For example, when the vehicle is traveling in the scene shown in fig. 3B, the first lane and the second lane belong to lanes that are parallel to each other, so the first lane and the second lane are attributed to a set of lanes lane0 for tracking, and the third lane is not parallel to the first lane and the second lane, so the third lane is tracked as a set of lanes lane1 alone.

Therefore, the lane models in the present application are used to track parallel lanes (parallel lane lines) and non-parallel lanes (non-parallel lane lines), so a state vector of each lane model is created based on the lane group tracked in each lane model.

For example, when the vehicle is driving on a highway as shown in fig. 1B, a lane model for tracking three lanes lane0, lane1 and lane2 is created, and since lane line y0 and lane line y1 are parallel lane lines, a group is sharedThe lane lines y2 and y3 share a groupy4 and y5 share a groupTherefore, the lane model created for the road shown in FIG. 1B is

Therefore, when the lane model in the application is popularized to be used for tracking N groups of lanes and M lane lines, the corresponding lane model is

Referring to fig. 1C, fig. 1C is a schematic structural diagram of an in-vehicle device 100 according to an embodiment of the present application, where the in-vehicle device 100 includes: the system comprises an image acquisition module 101, a lane line feature extraction module 102, a lane line tracking module 103, a lane model management module 104, a lane line output module 105 and a driving parameter input module 106, wherein:

the image acquisition module 101 is used for acquiring a target image of a preset area in front of the vehicle at the time t;

the lane line feature extraction module 102 is configured to perform feature extraction on the target image to obtain lane line features of each lane line;

the driving parameter input module 106 is used for inputting the driving parameter input of the vehicle to the lane line tracking module 103;

the lane line tracking module 103 is configured to predict each of N first lane models at the time t-1 according to vehicle driving parameters to obtain N second lane models, update the N second lane models according to lane line characteristics in a target image to obtain N third lane models, where the first lane models are used to track multiple groups of lanes, a lane a and a lane B are not parallel, the lane a and the lane B are lanes in any two groups of the multiple groups of lanes, the lanes in each group of lanes in the multiple groups of lanes are parallel to each other, and N is an integer greater than or equal to 1;

the lane model management module 104 is configured to calculate an adaptation probability of each of the N third lane models according to the probability parameter, where the adaptation probability is used to represent an adaptation degree of the third lane model to a lane line in the target image, and determine a third lane model with a maximum adaptation probability among the N third lane models, and the third lane model with the maximum adaptation probability is used to track the lane line of the driving road of the vehicle;

the lane line output module 105 is configured to output the tracked lane line.

In the embodiment of the application, the lane model is used for tracking a plurality of groups of lanes, the lanes in each group of lanes are parallel to each other, and any two groups of lanes are not parallel to each other, so that the lane lines in each group of lanes are parallel to each other, the lane lines in any two groups of lanes are not parallel to each other, when the vehicle runs in parallel lanes, the vehicle is tracked through one group of lanes, and when the vehicle runs in a scene containing non-parallel lanes, the vehicle is tracked through two or more groups of lanes, so that the vehicle can adapt to various complex driving scenes, and the output lane lines are more accurate; and moreover, the lane line is tracked by adopting the lane model with the maximum adaptation probability, so that the accuracy of the lane line tracking is further improved.

Referring to fig. 1D, fig. 1D is a schematic flowchart of a lane line tracking method according to an embodiment of the present application, where the method includes, but is not limited to, the following steps:

101: at the time t, the vehicle-mounted equipment predicts each of N first lane models at the time t-1 according to driving parameters of a vehicle to obtain N second lane models, wherein the first lane models are used for tracking multiple groups of lanes, lanes A and lanes B are not parallel, the lanes A and the lanes B are lanes in any two groups of the multiple groups of lanes respectively, the lanes in each group of the multiple groups of lanes are parallel to each other, and N is an integer greater than or equal to 1.

Wherein, because the lane line is located on the lane, the tracking lane and the tracking lane line are consistent.

The states of the first lane model and the second lane model are described consistently, and only the corresponding lane line parameters are inconsistent at each moment. For example, if the first lane model is an n-dimensional vector, the second lane model is also an n-dimensional vector.

The N first lane models comprise a lane model obtained by updating at the time of t-1 and a newly created lane model at the time of t-1.

Optionally, a prediction matrix is obtained based on the correlation between the t-1 time and the t-time lane model and the driving parameters of the vehicle, and each of the N first lane models at the t-1 time is predicted based on the prediction matrix to obtain N second lane models.

For example, assuming that the change in curvature of each lane line is continuous, the rate of change C in curvature of the lane line at time t_1,tRate of change of curvature C of lane line at time t-1_1,t-1Same, the curvature c of the lane line at time t_0,t＝v*Δt*c_1,t-1+c_0,t-1Where v is the vehicle speed, c_0,t-1Is the lane line curvature at time t-1. Therefore, the driving parameters of the vehicle can be combined into a prediction matrix, and the lane line parameters in the lane model at the time t-1 are converted into the lane line parameters corresponding to the time t through the prediction matrix, so that N second lane models at the time t can be obtained.

102: and the vehicle-mounted equipment updates the N second lane models according to the lane line characteristics in the target image to obtain N third lane models, wherein the target image is an image of a preset area in front of the vehicle at the time t.

The preset area in front of the vehicle is a front area shot by shooting equipment (such as a camera, a laser radar and the like) in the vehicle-mounted equipment.

103: and the vehicle-mounted equipment calculates the adaptation probability of each of the N third lane models according to the probability parameters, wherein the adaptation probability is used for representing the adaptation degree of the third lane models and the lane lines in the target image.

And the matching condition of the lane line tracked by the third lane model and the lane line of the driving road surface of the vehicle at the time t and the lane attribution of the tracked lane line in the third lane model are jointly determined.

For example, if the vehicle travels on the road surface at time t, there are 4 lane lines y0, y1, y2 and y3, y0 and y1 belong to lane0, y2 and y3 belong to lane1, if the number of lane lines tracked by the third lane model is 4, if the third lane model divides the attribution of the tracked first lane line and the tracked second lane line into one lane, and the tracked third lane line and the tracked fourth lane line belong to another lane, the third lane model has the greatest degree of adaptation to the lane lines of the road surface traveled by the vehicle at time t.

In addition, the probability calculation process is specifically referred to in step 501, and will not be described in detail here.

104: and the vehicle-mounted equipment determines a third lane model with the maximum adaptation probability in the N third lane models, wherein the third lane model with the maximum adaptation probability is used for tracking a lane line of the driving road surface of the vehicle.

Optionally, the lane line is tracked by using a third lane model with the largest adaptation probability, and the tracked lane line is output to a visualization interface, which can be visualized as a visualization interface of the vehicle-mounted device or a device associated with the vehicle-mounted device.

It can be seen that, in the lane model in this embodiment, multiple sets of lanes are tracked, lanes included in each set of lanes are parallel to each other, and lanes included in different sets are not parallel, so that when a vehicle runs in a scene including only parallel lanes, one set of lanes in the lane model are tracked, and when the vehicle runs in a scene including non-parallel lanes, two or more sets of non-parallel lanes in the lane model are used for tracking, so that lane lines under various running scenes can be tracked, and the lane line tracking accuracy is improved; and moreover, the lane line is tracked by adopting the lane model with the maximum adaptation probability, so that the accuracy of the lane line tracking is further improved.

In some possible embodiments, each of the N first lane models is predicted based on equation (4), resulting in N second lane models.

Wherein,for the jth first lane model of the N first lane models,is a pair ofThe j second lane model obtained after prediction, F_t-1Is a first prediction matrix, g_t-1Is a second prediction matrix;

r＝[0 0 … w*Δt]，l＝v*Δt。

the driving parameters of the vehicle comprise vehicle yaw velocity and vehicle speed, wherein w is the vehicle yaw velocity at the moment l is t-1, v is the vehicle speed, and delta t is the time interval between the moment t and the moment t-1.

It can be understood that after the target image is acquired, lane line prediction is performed on the target image, and the predicted lane lines are associated with lane line features, if Q lane lines exist in the target image, each lane line needs to be processed to update the lane model, in this application, the lane line C, that is, the qth lane line of the Q lane lines, is taken as an example to specifically explain that an update operation process corresponding to the lane line is executed, and Q is greater than or equal to 1 and less than or equal to Q.

Referring to fig. 2A, fig. 2A is a schematic flowchart illustrating a method for updating a second lane model according to an embodiment of the present application, where the method includes, but is not limited to, the following steps:

201: the in-vehicle apparatus divides the target image into T sub-images.

As shown in fig. 2B, the target image is divided into T sub-images from near to far, where a distance between a region corresponding to the i-th sub-image and the vehicle is smaller than a distance between a region corresponding to the i + 1-th sub-image and the vehicle, the region corresponding to each sub-image is a real region in a preset region in front of the vehicle, but not an image region, i is an integer, i is greater than or equal to 1 and less than or equal to T, and T is an integer greater than or equal to 2.

And the area of the region corresponding to the (i-1) th sub-image is smaller than that of the region corresponding to the ith sub-image.

202: and the vehicle-mounted equipment acquires the lane line characteristics in the ith sub-image.

The lane line feature is a set of pixel points of the lane line in the ith sub-image.

203: the vehicle-mounted equipment selects U target lane models matched with the lane line characteristics of the ith sub-image from the N first reference lane models, and performs the ith updating operation on each of the U target lane models according to the lane line characteristics to obtain U first reference lane models.

Wherein, the ith updating result comprises the U updated first reference lane models, when i is 1, the N first reference lane models are the N second lane models, when i is more than 1, the N first reference lane models are the updating results of the i-1 th time, U is more than or equal to 0 and less than or equal to N,

optionally, if the ith sub-image includes multiple lane lines, the lane line features of each lane line are sequentially obtained, then the lane line features of each lane line are matched with the lane models, then the lane line features are used for performing update operation on the matched lane models, then other lane line features in the sub-image are used for performing model matching on the update result obtained in the last update operation, and the matched models are updated until after the lane line features of all the lane lines in the ith sub-image are matched and updated, the ith update operation corresponding to the ith sub-image is executed, so that the ith update result is the update result obtained after the match-update operation is executed on all the lane lines in the ith sub-image.

When the lane line characteristics are completely matched with the N first reference lane models, the N first reference models all participate in updating operation to obtain N updated first reference lane models, the N updated first reference lane models serve as a current updating result, when the lane line characteristics are partially matched with the N first reference models, only the first reference models of the matched parts are updated to obtain U updated first reference lane models, the updated U first reference lane models and the un-updated (N-U) first reference lane models serve as the current updating result, when the lane line characteristics are completely not matched with the N first reference models, updating operation is not performed, and the N first reference models serve as the current updating result.

204: the vehicle-mounted device determines whether Q is less than or equal to Q.

If so, let q be q +1, and execute step 202;

if not, go to step 204.

205: the vehicle-mounted equipment determines whether the i is less than or equal to the T;

if so, let i be i +1, and execute step 202; if not, step 206 is executed, namely, the updating is finished, and the N third lane models are obtained.

206: and the vehicle-mounted equipment finishes updating the lane models to obtain N third lane models.

It can be seen that, in this embodiment, the target image is divided, and the divided sub-images are used to sequentially update the second lane model, so that the problem of false detection caused by matching the whole lane line with the lane model is avoided, and the updated third lane model is more adapted to the current driving scene.

Referring to fig. 2C, fig. 2C is a schematic flowchart illustrating another method for updating a second lane model according to an embodiment of the present application, where the method includes, but is not limited to, the following steps:

2031: and under an image coordinate system, the vehicle-mounted equipment acquires the observation vector of the lane line in the target image according to the lane line characteristics.

Fitting the lane line under an image coordinate system to obtain a linear equation, obtaining an observation vector z of the lane line and lane line characteristics of the lane line according to the linear equation, andthe above-mentionedIs the intersection of the linear equation and the linear equation x ═ l, theAt the point of the linear equationA normal vector of (a)Is the slope of the line equation.

Of course, the characteristics of the lane line can be fitted into a curve, a cubic spiral line and the like, the observation vector of the lane line C is obtained based on the curve or the cubic spiral line, in addition, the coordinates of two points can be taken in a fitting equation to obtain the observation vector, the observation vector is obtained mainly based on extended kalman filtering in the application, and the specific mode for obtaining the observation vector is not uniquely limited.

2032: and under a vehicle coordinate system, the vehicle-mounted equipment acquires M predicted observation vectors corresponding to the lane line and the lane model A in the target image according to the lane line characteristics.

The method comprises the following steps that M is the number of lane lines tracked by a lane model A, the lane model A is any one of N first reference lane models, M is an integer larger than or equal to 1, and the lane line features are pixel point sets of the lane lines in an image coordinate system.

The jth predicted observation vector of the M predicted observation vectors isAnd isIn the vehicle coordinate system, f_j(l) Lane line equation for the jth lane line tracked by lane model A, saidl,f_j(l) Is the intersection of the lane line equation with the line x ═ l, and p (l, f)_j(l) Is a point (l, f) in the vehicle coordinate system_j(l) To a projection point obtained under the image coordinate system,to be at the projection point (l, f)_j(l) A normal vector at a location of the (c),j is more than or equal to 1 and less than or equal to M, and is a partial derivative of the projection point in an image coordinate system.

The manner of obtaining the predicted observation vector is also only illustrated by way of example, and the obtaining manner is not limited uniquely in the present application.

Optionally, the point (l, f) is described above_j(l) Projection onto image coordinate system to obtain projection point p (l, f)_j(l) Obtained in particular by a projection operation p (-) which performs a projective transformation by means of camera parameters comprising: the transformation process of the external reference matrix of the camera, namely the matrix formed by the installation angle and the installation position of the camera on the vehicle, the internal reference matrix of the camera, namely the matrix formed by the optical center and the focal length of the camera, and the radial distortion coefficient and the tangential distortion coefficient of the camera lens is the prior art and is not repeated.

2033: and the vehicle-mounted equipment determines M Mahalanobis distances corresponding to the observation vectors and the M predicted observation vectors.

The mahalanobis distance of the observation vector corresponding to each predicted observation vector is calculated by equation (5).

In the above formulaMahalanobis distance of said observation vector to said jth predicted observation vector, Y being z_qAndresidual error of (1), thenP is an extended Kalman filter covariance matrix, H is an observation model represented by a Jacobian matrix, and is determined by projection operation, and R is an observation noise matrix.

2034: and the vehicle-mounted equipment determines the minimum mahalanobis distance in the M mahalanobis distances, and determines the lane model A as a target lane model matched with the lane line characteristics of the ith sub-image when the minimum mahalanobis distance is smaller than a distance threshold.

Further, when it is determined that the lane model a is the matched lane model, the lane model a is updated through the formula (6), so as to obtain a lane model a'.

Wherein,the model of the lane is a, and the lane model is a,is a lane model A',estimation of the extended Kalman covariance matrix for time t-1 versus time t, z_tIn order to observe the vector, the vector is,the predicted observation vector corresponding to the minimum mahalanobis distance in the M predicted observation vectors is used, and H is an observation model represented by a Jacobian matrix and is determined by projection operation.

While updating the second lane model, the updating is synchronizedTo obtain P_t|t，P_t|tIs time tSo as to adopt P at the time t +1_t|tObtaining an estimate of the extended Kalman covariance matrix at time t to time t +1To update the second lane model at the time t +1, the updating process can be referred to as formula (7).

Wherein,and estimating the extended Kalman covariance matrix at the time t-1, and determining the estimation H as an observation model represented by a Jacobian matrix by projection operation.

Since the vehicle-mounted device is in a non-operating state before the vehicle is started, a lane model does not exist in a lane line management module of the vehicle-mounted device, and the lane model needs to be initialized after the vehicle is started.

The following describes a process of initializing a lane model in a specific embodiment.

Referring to fig. 3A, fig. 3A is a schematic flowchart illustrating a method for initializing a lane model according to an embodiment of the present application, where the method includes, but is not limited to, the following steps:

301: the vehicle-mounted device determines whether a lane line feature exists in the target image.

Correspondingly, after the vehicle is started, the vehicle-mounted equipment always detects whether a lane line exists in front of the vehicle, and if the lane line exists, step 302 is executed; otherwise, step 305 is performed without creating a lane model.

302: the vehicle-mounted device determines whether the confidence of the lane line feature is greater than a confidence threshold.

The confidence is used for representing the probability that the pixel points are the pixel points of the lane line, and when the vehicle-mounted equipment detects the lane line characteristics in the target image, the probability value of the pixel point of each pixel point belonging to the lane line is determined, so that when the confidence is greater than the confidence threshold, the pixel point is determined to belong to the lane line, and when the pixel point set corresponding to the lane line characteristics is greater than the confidence threshold, the step 302 is executed; if the confidence level is less than the confidence threshold, it is determined that the lane marking feature is not substantially a lane marking, and step 306 is performed.

303: the in-vehicle apparatus creates a lane model.

When the lane line is detected, a completely new lane model is created, and all the lane line parameters in the created lane model are in an inactivated state and are all 0.

For example, when driving on a highway scene as shown in fig. 3B, the lane model is used to track the left and right lane lines of the lane where the vehicle is located, and the left and right lane lines outside the left and right lane lines, and the state vector of the initial lane model is The lane line parameter corresponding to the first lane,the lane line parameter corresponding to the second lane,respectively, the lateral offset of the four lane lines.

304: and the vehicle-mounted equipment performs initial assignment on the lane line parameters in the lane model to obtain a middle lane model.

Wherein the initial assignment of the lane-line parameters is determined by the width of the lane itself.

For example, given a standard width of 3.5m for a lane, in the initial assignment of the lane model, assuming all lane lines are straight and belong to the same set of lane0, the lane lines are assignedAssigned a value of 0, falseIf the vehicle runs in the center of the lane, the lateral deviation of the left lane line and the right lane line on the two sides of the vehicle is 1.75 and-1.75 respectively, the lateral deviation of the left lane line and the right lane line on the left side of the vehicle is 5.25 and-5.25 respectively, and the like, the lateral deviation Y of the jth lane line on the left side of the vehicle is obtained_jRespectively 1.75+ (j-1) × 3.5, and the lateral deviation of the jth lane line on the right side of the vehicle is respectively- (1.75+ (j-1) × 3.5), wherein j is a positive integer. So when driving on the freeway ramp shown in fig. 3B, the lane model is initially assigned x₀＝[00005.251.75-1.755.25]。

305: and the vehicle-mounted equipment updates the middle lane model by adopting the lane line characteristics in the target image to obtain an initial lane model.

Firstly, the target image is divided into T sub-images, and step 201 is consistent and is not repeated.

Then, starting from the first sub-image, sequentially obtaining the lane line characteristics of each lane line in each sub-image, fitting the lane line characteristics to obtain a fitting equation, obtaining the intercept of each fitting equation and the y-axis, and obtaining a target fitting equation corresponding to the lane line tracked by the middle lane model according to the intercept, wherein the intercept of the target fitting equation and the y-axis is closest to the distance between the lane line and the vehicle, if the target fitting equation corresponding to the lane line is obtained at first in the kth sub-image, updating the middle lane model by using the lane line characteristics corresponding to the target fitting equation, obtaining the target fitting equation corresponding to each lane line tracked by the middle lane model, and performing an updating process shown in fig. 2A by using the remaining sub-images after the updating operation, so as to obtain the initial lane model.

For example, the fitting equation with the intercept closest to 1.75 may be used as the target fitting equation for the left lane line, the fitting equation with the intercept closest to-1.75 may be used as the target fitting equation for the right lane line, and so on, the target fitting equation corresponding to each lane line may be determined. Because the distance between each sub-image and the vehicle is different, the shooting fields of vision are different, so the first sub-image may only contain the left lane line and the right lane line, and the left lane line and the right lane line may exist in the subsequent sub-images, therefore, the target fitting equation corresponding to each lane line is sequentially obtained from the first sub-image in each sub-image, the updating operation is executed, and after the target fitting equations corresponding to all lane lines are obtained in a certain sub-image, the updating process shown in fig. 2A is executed by using the remaining sub-images from the sub-image, and the initial lane model is obtained.

306: the in-vehicle apparatus ends the creation of the initial lane model.

It can be seen that, in the embodiment, when the vehicle starts, the initial lane model is automatically created to adapt to the current driving scene, so that the traffic safety is improved.

Based on the lane model updating method shown in fig. 2A, when the lane line C has no matched lane model in the lane line characteristics of each sub-image, it is determined that the lane line C is in an untracked state, which indicates that a non-parallel lane appears in a preset area in front of the vehicle, and a new lane model needs to be created to track the lane line C.

A method of creating a new lane model is provided below.

Referring to fig. 4, fig. 4 is a schematic flowchart illustrating a method for creating a new lane model according to an embodiment of the present application, where the method includes, but is not limited to, the following steps:

401: the vehicle-mounted device determines whether an untracked lane line exists in the target image. Based on the updating method shown in fig. 2A, when i is equal to T, if f minimum mahalanobis distances corresponding to the lane line C are all greater than or equal to the distance threshold, N fourth lane models are created according to the N third lane models, where the N fourth lane models are consistent with lane line parameters of the N third lane models, the f minimum mahalanobis distances are minimum mahalanobis distances corresponding to lane line features of the lane line C in f sub-images, the f sub-images are sub-images in the T sub-images that include the lane line features of the lane line C, the f is an integer greater than or equal to 1, f is equal to or less than T, and the lane line C is any one lane line in the target image; when it is determined that the lane line C is in an untracked state, step 402 is executed; otherwise, step 406 is performed without creating a lane model.

402: the vehicle-mounted device determines whether the confidence of the untracked lane line is greater than a confidence threshold.

When the confidence is greater than the confidence threshold, determining that the untracked lane line is the lane line, and executing step 403; if it is less than the confidence threshold, then it is determined that the untracked lane line is not substantially a lane line, then step 406 is performed.

403: and the vehicle-mounted equipment creates N fourth lane models according to the N third lane models.

Wherein the N fourth lane models are copied from the N third lane models.

For example, when the vehicle is traveling on a highway scene as shown in fig. 3B, only one third lane model is in the tracking state before the vehicle reaches the on-ramp, and the third lane model is assumed to beSince the lane line y has not been detected before₃Therefore and y₃The corresponding lane marking parameter is in an inactive state, i.e. the lane marking parameterIs 0, y cannot be tracked₃So as to copy x₀Obtaining a new lane model x₁And is andwherein,indicating the lane line y₃And if the lane model belongs to lane1, the number of the first lane models corresponding to the time t is expanded to 2.

404: and the vehicle-mounted equipment processes each of the N fourth lane models to obtain N new lane models.

Optionally, processing each of the N fourth lane models to obtain N new lane models may include: acquiring the relative distance between the lane line C and the vehicle; initially assigning a lane line parameter corresponding to the lane line C in each fourth lane model according to the relative distance to obtain N fifth lane models; fitting the lane line characteristics in each sub-image in the T sub-images to obtain at least one fitting equation; if a target fitting equation is firstly obtained in the kth sub-image of the T sub-images, adopting the lane line characteristics of the lane line C in the kth sub-image to update each of the N fourth lane models to obtain the current latest N fourth lane models, wherein the target fitting equation is a fitting equation of which the difference value between the intercept and the relative distance in the at least one fitting equation is smaller than a distance threshold value; and starting from the (k + 1) th sub-image in the T sub-images, sequentially adopting the lane line characteristics in each sub-image to update the N fourth lane models to obtain the current latest N fourth lane models, and obtaining the N new lane models after the current latest N fourth lane models are updated by adopting the lane line characteristics in the T sub-images.

Firstly, which lane line the untracked lane line is determined according to the relative distance between the lane line C and the vehicle, and then, initial assignment is performed on the lane line parameters corresponding to the untracked lane line, wherein the assignment process can be seen in the process shown in step 303. For example, as shown in fig. 3B, since the standard width of the lane is 3.5m, the relative distance between the lane line C and the vehicle is 5.25m, and the lane line C is determined as the right and right lane lines of the vehicle, the parameters in the fourth lane model are usedThe initial assignment is 000-5.25, and other lane line parameters are consistent with the lane line parameters in the third lane model;

then, starting from the first sub-image, fitting all lane line features in each sub-image to obtain at least one fitting equation, then, determining that the sub-image k including the lane line C is the first to obtain the target fitting equation, and performing an updating operation by using the lane line features of the lane line C in the sub-image k to obtain the current latest fourth lane model, and finally, starting from the sub-image k +1 and ending to the last sub-image, sequentially updating the current latest N first lane models, wherein the specific updating process is consistent with that shown in fig. 2A, and is not repeated herein.

405: and the vehicle-mounted equipment takes the N fourth lane models and the N new lane models as the first lane model at the time t.

406: the in-vehicle apparatus ends the creation of the new lane model.

It can be seen that, in the embodiment, at time t, when an untracked lane line appears in a preset area in front of a vehicle, it is determined that a non-parallel lane appears on a driving road surface of the vehicle, and all current lane models cannot track the lane line in the non-parallel lane, so that a new lane model needs to be created to track the untracked lane line, and miss detection of the lane line is avoided.

Based on the above method for creating lane models, as the travel time of the vehicle increases, many new lane models are created, and the management of the multi-lane models by the vehicle-mounted device tends to be saturated, which brings a burden when calculating the tracking lane line. Therefore, it is necessary to manage the lane models and delete some of the irrelevant lane models. The following provides a management method of a lane model.

Referring to fig. 5, fig. 5 provides a schematic flow chart of a method for managing a lane model according to an embodiment of the present application, including, but not limited to, the following steps:

501: and the vehicle-mounted equipment calculates the adaptation probability of each third lane model in the N third lane models according to the probability parameters.

The probability parameters comprise target matching probability, prior probability and probability of the first lane model B at the t-1 moment; the probability of the third lane model B' is the product of the target matching probability, the prior probability and the probability of the first lane model B at the time t-1.

The first lane model B is any one of the N first lane models, and the third lane model B' is a third lane model obtained by updating the lane model B.

Wherein, the matching probability can be P (Z)^t|θ_k，Θ^t-1，Z^t-1) The prior probability may be P (θ)_k|Θ^t-1,Z^t-1) The probability of the first lane model B at the time t-1 can be P (theta)^t-1|Z^t-1) (ii) a The probability of the third lane model B' may be P (Θ)^t|Z^t)；

The target matching probability is used for representing the matching degree of all lane lines at the time t and the third lane model B', and specifically is the target matching probability is the product of W first matching probabilities, the W first matching probabilities are first matching probabilities corresponding to W target lane line features, the W target lane line features are lane line features matched with the current latest lane model, the current latest lane model is a lane model obtained after each updating operation is performed on the first lane model B, and each first matching probability is obtained by a prediction vector corresponding to each target lane line feature, a minimum mahalanobis distance corresponding to the target lane line feature, and a prediction observation vector corresponding to the minimum mahalanobis distance;

specifically, the lane model is updated to obtain the target lane line feature, the minimum mahalanobis distance corresponding to the target lane line feature and the predicted observation amount corresponding to the minimum mahalanobis distance are determined, the measurement residual covariance matrix S of the observation vector of the target lane line feature and the predicted observation vector is determined, the covariance matrix S is used as the variance of the gaussian distribution, the mahalanobis distance between the observation vector and the predicted observation vector is used as the value of the gaussian distribution (x-u), so that the probability under the gaussian distribution is obtained, and the probability is used as the first matching probability corresponding to the target lane line feature.

A priori probability for characterizing a source of the third lane model B ', the source including the third lane model B' being obtained by performing an update operation on the first lane model B, and the priori probability being

In addition, when the probability of each newly created lane model is calculated, the corresponding prior probability is set to be tau, and the corresponding probability of the newly created lane model at the time t-1 is set to be a preset value.

502: and the vehicle-mounted equipment obtains the target third lane model according to the adaptation probability of each third lane model.

And the target third lane model is a third lane model of which the adaptation probability is smaller than a probability threshold value in the N third lane models.

Wherein the probability threshold may be 0.7, 0.8, or other values.

503: and the vehicle-mounted equipment deletes the target third lane model with the tracking duration being greater than the duration threshold.

Wherein the time threshold may be 1 minute, 5 minutes, or other values.

Optionally, referring to fig. 6, the numbers in the boxes in fig. 6 are used to indicate that each lane line belongs to a lane, where, numeral 0 indicates that the lane line belongs to lane0, and numeral 1 indicates that the lane line belongs to lane 1. At the time t0, only one lane model is running, the lane model is used for tracking four lane lines, at the time t1, it is detected that the lane model cannot track the fourth lane line, a new lane model is created, at the time t2, it is also detected that at the time t1, two lane models cannot track the third lane line, two new lane models need to be created on the basis of t1, and at the time t2, four models are running, so that the number of lane models is increased. Therefore, the vehicle-mounted equipment calculates the adaptation probability of each lane model at the time t, and deletes the lane models with the adaptation probabilities lower than the probability threshold value so as to improve the calculation speed of the vehicle-mounted equipment.

Optionally, after the new lane model is created, the new lane model does not necessarily adapt to the current driving scene because the new lane model has just run, and a certain protection period is set for the new lane model, that is, when the vehicle-mounted device deletes the lane model, only the lane model outside the protection period is deleted, and only the adaptation probability of the new lane model is calculated, and if the adaptation probability is smaller than the probability threshold, the new lane model is not deleted.

Referring to fig. 7, fig. 7 is a lane line tracking apparatus according to an embodiment of the present disclosure, which may include:

the prediction unit 710 is configured to predict, at a time t, each of N first lane models at the time t-1 according to a driving parameter of a vehicle, to obtain N second lane models, where the first lane models are configured to track multiple groups of lanes, a lane a is not parallel to a lane B, the lane a and the lane B are lanes in any two groups of the multiple groups of lanes, the lanes included in each group of the multiple groups of lanes are parallel to each other, and N is an integer greater than or equal to 1;

an updating unit 720, configured to update the N second lane models according to lane line features in a target image to obtain N third lane models, where the target image is an image of a preset area in front of the vehicle at the time t;

a calculating unit 730, configured to calculate, according to a probability parameter, an adaptation probability of each of the N third lane models, where the adaptation probability is used to represent an adaptation degree of the third lane model to a lane line of the driving road surface of the vehicle;

a tracking unit 740, configured to determine a third lane model with a largest adaptation probability in the N third lane models, where the third lane model with the largest adaptation probability is used to track a lane line of a driving road of the vehicle.

In some possible embodiments, in predicting each of the N first lane models at time t-1 according to the driving parameters of the vehicle to obtain N second lane models, the prediction unit 710 is specifically configured to: obtaining a prediction matrix according to the driving parameters of the vehicle; and predicting each of the N first lane models at the t-1 moment according to the prediction matrix to obtain N second lane models.

In some possible embodiments, in terms of updating the N second lane models according to the lane line features in the target image to obtain N third lane models, the updating unit 720 is specifically configured to:

acquiring lane line characteristics in the ith sub-image;

and when the i is equal to T, the T-th updating result obtained after the T-th updating operation is executed is the N third lane models. .

In some possible embodiments, in selecting U target lane models from the N first reference lane models that match the lane line characteristics of the ith sub-image, the updating unit 720 is specifically configured to:

In some possible embodiments, the apparatus further comprises a creating unit 750, the creating unit 750 configured to;

processing each of the N fourth lane models to obtain N new lane models;

In some possible embodiments, in processing each of the N fourth lane models to obtain N new lane models, the creating unit 750 is specifically configured to:

acquiring the relative distance between the lane line C and the vehicle;

the prior probabilities are used to characterize sources of the third lane model B ', including the third lane model B' by performing an update operation on the first lane model B. .

In some possible embodiments, the apparatus further includes a deleting unit 760, where the deleting unit 760 is configured to:

Referring to fig. 8, an embodiment of the present application provides a lane line tracking apparatus 800, including:

a processor 830, a communication interface 820, and a memory 810 coupled to one another; such as processor 830, communication interface 820 and memory 810, are coupled by bus 840.

The Memory 810 may include, but is not limited to, a Random Access Memory (RAM), an Erasable Programmable Read Only Memory (EPROM), a Read-Only Memory (ROM), or a portable Read-Only Memory (CD-ROM), and the like, and the Memory 810 is used for related instructions and data.

The processor 830 may be one or more Central Processing Units (CPUs), and in the case that the processor 830 is one CPU, the CPU may be a single-core CPU or a multi-core CPU.

The processor 830 is configured to read the program codes stored in the memory 810, and cooperate with the communication interface 840 to perform part or all of the steps of the method performed by the travel platform 800 in the above-described embodiment of the present application.

For example, the communication interface 820 is used for receiving vehicle driving parameters at the time t;

the processor 830 is configured to predict, at a time t, each of N first lane models at the time t-1 according to a driving parameter of a vehicle, to obtain N second lane models, where the first lane models are configured to track multiple groups of lanes, a lane a is not parallel to a lane B, the lane a and the lane B are lanes in any two groups of the multiple groups of lanes, lanes included in each group of lanes in the multiple groups of lanes are parallel to each other, and N is an integer greater than or equal to 1;

the processor 830 is further configured to update the N second lane models according to lane line features in a target image to obtain N third lane models, where the target image is an image of a preset area in front of the vehicle at the time t;

the processor 830 is further configured to calculate an adaptation probability of each of the N third lane models according to a probability parameter, where the adaptation probability is used to represent an adaptation degree of the third lane model to a lane line of the driving road surface of the vehicle;

the processor 830 is further configured to determine a third lane model with the highest adaptation probability from the N third lane models, where the third lane model with the highest adaptation probability is used for tracking a lane line of a driving road of the vehicle.

In some possible embodiments, in predicting each of the N first lane models at the time t-1 according to the driving parameters of the vehicle to obtain N second lane models, the processor 830 is specifically configured to: obtaining a prediction matrix according to the driving parameters of the vehicle; and predicting each of the N first lane models at the t-1 moment according to the prediction matrix to obtain N second lane models.

In some possible embodiments, the processor 830 is specifically configured to update the N second lane models according to lane line features in the target image to obtain N third lane models:

acquiring lane line characteristics in the ith sub-image;

In some possible embodiments, in selecting U target lane models matching the lane line feature of the ith sub-image from the N first reference lane models, the processor 830 is specifically configured to:

In some possible embodiments, the processor 830 is further configured to:

processing each of the N fourth lane models to obtain N new lane models;

In some possible embodiments, in processing each of the N fourth lane models to obtain N new lane models, the processor 830 is specifically configured to:

acquiring the relative distance between the lane line C and the vehicle;

the prior probability is used to characterize a source of the third lane model B ', the source including the third lane model B' being obtained by performing an update operation on the first lane model B;

in some possible embodiments, the processor 830 is further configured to:

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, fiber optic, digital subscriber line) or wirelessly (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., compact disk), or a semiconductor medium (e.g., solid state disk), among others. In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the foregoing embodiments, the descriptions of the embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is merely a logical division, and the actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted or not executed. In addition, the indirect coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or units, and may be electrical or in other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or may also be implemented in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage media may include, for example: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

Claims

1. A lane line tracking method, comprising:

2. The method of claim 1, wherein predicting each of the N first lane models at time t-1 according to the driving parameters of the vehicle to obtain N second lane models comprises:

3. The method according to claim 1 or 2, wherein the updating the N second lane models according to the lane line features in the target image to obtain N third lane models comprises:

acquiring lane line characteristics in the ith sub-image;

4. The method according to claim 3, wherein the selecting U target lane models matching the lane line characteristics of the ith sub-image from the N first reference lane models comprises:

acquiring M predicted observation vectors corresponding to a lane line in the target image and a lane model A according to the lane line characteristics under a vehicle coordinate system, wherein M is the number of lane lines tracked by the lane model A, the lane model A is any one reference lane model of the N first reference lane models, and M is an integer greater than or equal to 1;

5. The method of claim 4, further comprising:

processing each of the N fourth lane models to obtain N new lane models;

6. The method of claim 5, wherein said processing each of said N fourth lane models to obtain N new lane models comprises:

acquiring the relative distance between the lane line C and the vehicle;

fitting the lane line characteristics in each sub-image in the T sub-images to obtain at least one fitting equation;

7. The method according to any one of claims 4-6, wherein the probability parameters include a target matching probability, a prior probability, and an adaptation probability of the first lane model B at the time t-1; the adaptation probability of the third lane model B' is obtained by the target matching probability, the prior probability and the adaptation probability of the first lane model B at the t-1 moment; the first lane model B is any one of the N first lane models;

8. The method according to any one of claims 1-7, further comprising:

9. A lane line tracking apparatus, comprising:

10. The apparatus of claim 9,

in terms of predicting each of N first lane models at time t-1 according to the driving parameters of the vehicle to obtain N second lane models, the prediction unit is specifically configured to: obtaining a prediction matrix according to the driving parameters of the vehicle; and predicting each of the N first lane models at the t-1 moment according to the prediction matrix to obtain N second lane models.

11. The apparatus of claim 9 or 10,

in terms of updating the N second lane models according to the lane line characteristics in the target image to obtain N third lane models, the updating unit is specifically configured to:

acquiring lane line characteristics in the ith sub-image;

12. The apparatus of claim 11,

in the aspect of selecting, from the N first reference lane models, U target lane models that match lane line features of the ith sub-image, the updating unit is specifically configured to:

13. The apparatus of claim 12, further comprising a creating unit;

the creating unit is configured to, when i is equal to T, create N fourth lane models according to the N third lane models if f minimum mahalanobis distances corresponding to a lane line C in the target image are all greater than or equal to the distance threshold, where the N fourth lane models are consistent with lane line parameters of the N third lane models, the lane line C is any one lane line in the target image, the f minimum mahalanobis distances are minimum mahalanobis distances corresponding to lane line features of the lane line C in f sub-images, the f sub-images are sub-images including the lane line feature of the lane line C in the T sub-images, the f is an integer greater than or equal to 1, and f is equal to or less than T;

processing each of the N fourth lane models to obtain N new lane models;

14. The apparatus of claim 13,

acquiring the relative distance between the lane line C and the vehicle;

15. The apparatus according to any one of claims 12-14, wherein the probability parameters comprise a target matching probability, a prior probability, and an adaptation probability of the first lane model B at the time t-1; the adaptation probability of the third lane model B' is obtained by the target matching probability, the prior probability and the adaptation probability of the first lane model B at the t-1 moment; the first lane model B is any one of the N first lane models;

16. The apparatus according to any one of claims 9-15, wherein the apparatus further comprises a deletion unit;

the deleting unit is used for obtaining a target third lane model according to the adaptation probability of each third lane model, wherein the target third lane model is a third lane model of which the adaptation probability is smaller than a probability threshold value in the N third lane models;

17. A lane line tracking apparatus, comprising:

18. A computer-readable storage medium, characterized in that a computer program is stored, which computer program is executed by hardware to implement the method performed by the lane line tracking apparatus of any one of claims 1 to 8.