WO2021056341A1

WO2021056341A1 - Lane line fusion method, lane line fusion apparatus, vehicle, and storage medium

Info

Publication number: WO2021056341A1
Application number: PCT/CN2019/108208
Authority: WO
Inventors: 许睿; 陈竞
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-09-26
Filing date: 2019-09-26
Publication date: 2021-04-01
Also published as: CN112154449A

Abstract

A lane line fusion method and apparatus, and a vehicle and a storage medium. The lane line fusion method comprises: acquiring an image of the environment around a movable platform, and obtaining an initial lane line set of the movable platform according to the environmental image (S110); and performing fitting optimization on initial lane line data in the initial lane line set to obtain a target lane line set, with the fitting optimization comprising: performing fitting optimization according to the initial lane line data and historical lane line data in a historical lane line set to obtain the target lane line set (S120), wherein the target lane line set comprises lane lines that are not parallel to one another.

Description

Lane line fusion method, lane line fusion device, vehicle and storage medium

Technical field

This application relates to the field of vehicle assisted driving and automatic driving, and in particular to a lane line fusion method, a lane line fusion device, a vehicle, and a storage medium.

Background technique

Lane line detection is one of the main functions that assisted driving (ADAS) and autonomous driving technologies need to achieve. Based on the detected lane lines, a movable platform, such as a vehicle's driving plan, can be planned to ensure safe and reliable driving. The existing lane line detection method is mainly based on the environment image around the current vehicle, and cannot integrate the historical lane line detected at the previous time to optimize the current lane line, and cannot better cope with the driving scene with bifurcated intersections. Therefore, the existing lane line detection has poor versatility and insufficient accuracy.

Summary of the invention

Based on this, this manual provides a lane line fusion method, a lane line fusion device, a vehicle and a storage medium, which are designed to solve the problem that the existing lane line detection cannot better deal with driving scenarios with bifurcated intersections, and its versatility is poor , Technical problems such as insufficient accuracy.

In the first aspect, this specification provides a lane line detection method, including:

Acquiring an image of the environment around the movable platform, and obtaining an initial lane line set of the movable platform according to the environment image;

Fitting and optimizing the initial lane line data in the initial lane line set to obtain a target lane line set;

Wherein, the fitting optimization includes: performing fitting optimization according to the initial lane line data and historical lane line data in the historical lane line set to obtain a target lane line set, and the fitting optimization is used to make the target lane The line set includes lane lines that are not parallel to each other.

In the second aspect, this specification provides a lane line detection device, including a sensor and a processor;

The sensor is used to obtain an image of the environment around the movable platform;

The processor is configured to implement the following steps:

In the third aspect, this specification provides a vehicle, including:

The above-mentioned lane line detection device is used to determine the lane line;

Sports components for driving.

In a fourth aspect, this specification provides a computer-readable storage medium that stores a computer program, and when the computer program is executed by a processor, the processor implements the aforementioned lane-line fusion method.

The embodiment of this specification provides a lane line fusion method, a lane line fusion device, a vehicle, and a storage medium. A number of initial lane line data is determined through the environment image around the movable platform, and then a set of initial lane line data and historical lane line set The corresponding historical lane line data in the corresponding historical lane line data is fitted and optimized, and several target lane lines are obtained. Since the fitting optimization incorporates the historical detection data of the lane line, the lane line detection can be independent of the assumption of the geometric parallelism of the lane line and the assumption of the parallel road surface, which can be applied to the scene of the road including the bifurcation, which improves the versatility and detection accuracy. .

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the disclosure of this specification.

Description of the drawings

In order to explain the technical solutions of the embodiments of this specification more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of this specification. Ordinary technicians can obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic flowchart of a method for detecting lane lines according to an embodiment of this specification;

Fig. 2 is a schematic diagram of a lane line detection scene in an embodiment;

Figure 3 is a schematic diagram of determining local lane lines in a local map through connected domain analysis;

Fig. 4 is a schematic diagram of a sub-process of obtaining a target lane line set by fitting optimization in Fig. 1;

Fig. 5 is a schematic diagram of a sub-process of the initial lane line data corresponding to the target lane line obtained by the fitting optimization in Fig. 4;

Fig. 6 is a schematic block diagram of a remote control device according to an embodiment of the present specification;

Fig. 7 is a schematic block diagram of a movable platform provided by an embodiment of the present specification.

detailed description

The technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by a person of ordinary skill in the art without creative work shall fall within the protection scope of this specification.

The flowchart shown in the drawings is only an example, and does not necessarily include all contents and operations/steps, nor does it have to be executed in the described order. For example, some operations/steps can also be decomposed, combined or partially combined, so the actual execution order may be changed according to actual conditions.

Hereinafter, some embodiments of this specification will be described in detail with reference to the drawings. In the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of a lane line detection method according to an embodiment of the present specification.

The lane line detection method can be applied to a lane line detection device, which can be mounted on a movable platform and has a data connection with the movable platform. Among them, the movable platform may be some mobile devices that can drive on public transportation roads, such as autonomous vehicles. The lane line detection device may also be an auxiliary driving device that has a data connection with the movable platform.

The lane line detection device can be built into a movable platform, such as a system in which the movable platform is integrated into the movable platform, or can be externally connected to the movable platform, such as an auxiliary driving device connected to the outside of the movable platform.

In some embodiments, the lane line detection method can be applied to the lane line detection scene as shown in FIG. 2, wherein the movable platform 10 may be a vehicle driving on a public transportation road. The lane line detection device 100 is built in the movable platform 10, and one or more sensors 101 are also arranged on the movable platform 10, and the lane line detection device 100 can obtain the environment image of the movable platform 10 through the sensor 101. For example, the sensor 101 includes a vision sensor for acquiring an image of the environment around the movable platform.

Wherein, the sensor 101 can be arranged in the front, rear and/or roof of the movable platform 10, and one or more sensors 101 arranged in the movable platform 10 can be arranged in the same position or in different positions. It is not limited in the embodiment of the present invention.

As shown in FIG. 1, the lane line detection method of this embodiment includes steps S110 to S120.

S110: Acquire an environment image around the movable platform, and obtain an initial lane line set of the movable platform according to the environment image.

Exemplarily, a sensor mounted on the movable platform collects a preset range of environment images around the movable platform in real time or according to a preset period, and constructs a local map of the environment where the movable platform is located based on the environment images.

Exemplarily, an image in front of the movable platform is acquired through a sensor mounted on the movable platform, or called a front view, and the lane line image points in the front view are identified based on the image model. It can also combine the pose information of the movable platform and the 3D environment information to convert the lane line image points to a local map in the world coordinate system to obtain a local map of the environment in which the movable platform is located, and determine that the lane line image point is in the local area. The location point of the corresponding lane line on the map.

Exemplarily, a convolutional neural network (Convolutional Neural Networks, CNN) may be used to perform preliminary lane line detection on the front view to obtain lane line image points.

Exemplarily, a visual inertial navigation system (such as Visual-Inertial Navigation System, VINS) may be used to obtain the pose information of the movable platform, such as the current vehicle body position and pose information.

Exemplarily, the binocular stereo matching algorithm SGBM may be used to determine the 3D environment perception information around the movable platform, that is, the 3D environment information.

In some embodiments, after determining the lane line position points in the local map, an initial lane line set may be determined in the local map, and the initial lane line set includes multiple initial lane line data.

Exemplarily, the connected domain analysis is performed based on the lane line position points in the local map to obtain the local lane line set corresponding to the local map.

Exemplarily, the initial lane line data included in the initial lane line set may be determined according to the lane line position points recorded in the local map. In some embodiments, the local map may be a grid map in the world coordinate system, the grid corresponding to the lane line position point is a lane line grid, and each lane line grid includes a semantic information, and the semantic information is used for The grid representing the lane line is the grid corresponding to the position point of the lane line. In view of this situation, each lane line grid in the local map can be analyzed and processed to obtain the local lane line set corresponding to the local map. The local lane line set includes at least one local lane line, which is then determined from the local lane line set An initial lane line set is generated, and the initial lane line set includes at least one initial lane line data.

In some embodiments, the connected domain analysis processing of each lane line grid in the local map can be performed based on the semantic information of each lane line grid to obtain the local lane line set corresponding to the local map, and based on the preset optimization algorithm and The weight value of each local lane line in the local lane line set is optimized to determine the initial lane line to obtain the initial lane line set. Wherein, the weight value of the local lane line is determined according to the characteristic information of the lane line.

Exemplarily, when performing connected domain analysis processing on each lane line grid in the local map based on the semantic information of each lane line grid, each lane line may be determined from the local map according to the semantic information of each lane line grid. Grid, and extract the connected domain of each lane line grid, determine the connected domain label of the lane line grid in a small area according to the semantic information of each lane line grid, and perform lane based on the connected domain label corresponding to each lane line grid Line fitting determines one or more local lane lines corresponding to the lane line position points in a small area, and so on, can determine all the local lane lines in the local map, so as to obtain the local lane line set corresponding to the local map.

Wherein, the lane line grids belonging to the same lane line correspond to the same connected domain label, and the connected domain label is associated with the semantic information of the lane line grid. For example, the semantic information of the lane line grid G1 is used to indicate that the image point corresponding to the lane line grid G1 belongs to the lane line A, and the semantic information of the lane line grid G2 is used to indicate that the image point corresponding to the lane line grid G2 is also Belonging to lane line A, then the lane line grid G1 and lane line grid G2 correspond to the same connected domain label.

Figure 3 shows a schematic diagram of determining local lane lines in a local map through connected domain analysis. As shown in the figure, each grid represents an image point, and the number in the grid represents the connected domain label corresponding to each grid. Among them, a grid with a connected component label of 0 may be a grid of local lane lines that does not belong to a lane line, and a grid with a connected component label of not 0 may be a grid of lane lines that may belong to a lane line. The grid with the connected domain label of 1 is the image point belonging to the A lane line in the local map; the image point with the connected domain label 2 is the image point corresponding to the B lane line in the local map, where the A lane line and the B The lane lines are two different lane lines.

Exemplarily, when the connected domain analysis processing is performed on the lane line grid in the local map, a small-scale grid can be selected from the local map according to the image detection window of the preset size, and determined based on the semantic information of each grid The connected component label of each grid in the image detection window is displayed, where the connected component label corresponding to the non-lane line grid is 0, and the connected component label of the lane line grid is determined by the corresponding semantic information. Since the grids in the same connected component correspond to the same connected component label, the connected components of the image detection window can be determined based on the characteristics of whether the connected component labels are the same, that is, the grids included in each connected component are determined. The preset size of the image detection window may be 3×3 or 5×5, for example.

After determining each grid included in each connected domain, the grid corresponding to each connected domain can be fitted according to the optimal solution algorithm to obtain the local lane line corresponding to each connected domain, and so on, the image detection can be determined The local lane lines corresponding to the lane lines included when the window is located at different positions in the local map, so that all the local lane lines in the local map can be determined to obtain a set of local lane lines.

For example, if the connected component label corresponding to the grid in the image detection window selected from the local lane line is shown in Figure 3, all the lane line grids with the connected component label of 1 can be modeled according to the optimal solution algorithm. Combine a local lane line, such as 501 in the figure, fit all the lane line grids with the connected domain label 2 according to the optimal solution algorithm to fit a local lane line, such as 502 in the figure, and so on. All the lane lines in the image detection window are grid-fitted into multiple local lane lines, so that all the local lane lines in the local map can be determined.

Exemplarily, after the local lane line set corresponding to the local map is determined, at least one initial lane line can be optimally determined based on the weighted maximum clique algorithm and the weight value of each local lane line in the local lane line set. One initial lane line is filtered to obtain the initial lane line set.

Exemplarily, the feature information of each local lane line in the local lane line set is analyzed to obtain the weight value of each local lane line, and then based on preset optimization algorithms, such as the weighted maximum clique algorithm and the weight of each local lane line Value, the optimization determines at least one initial lane line, and the initial lane line set is obtained.

Wherein, the weight value of the local lane line is determined according to the feature information of the lane line. In one embodiment, the feature information of the lane line includes: the geometric feature and/or color feature of the lane line, and the geometric feature includes the length feature and the width feature. And any one or more of the parallel features between lane lines.

As a feasible embodiment, the corresponding relationship between the geometric and color features of the lane line and the hypothetical score can be established in advance. The higher the sum of the hypothetical scores corresponding to a lane line, then the possibility that the lane line is the actual lane line The greater the sex. Exemplarily, the corresponding relationship between the geometric feature, the color feature and the hypothetical score of the lane line may be as shown in Table 1. It can be seen that each local lane line can determine its own corresponding hypothetical score sum through the corresponding relationship shown in Table 1, and the hypothetical score sum is the weight value corresponding to the local lane line.

Table 1 Scores of lane line feature information

Among them, the hypothetical score corresponding to each feature dimension in Table 1 is only an example, which is mainly used to illustrate the principle of determining the weight value of a local lane line through the feature information of the lane line, and it cannot be a limitation to the embodiment of the present invention.

In one embodiment, the feature information of each local lane line can be analyzed, and the weight value of each local lane line can be determined according to the pre-established correspondence relationship between the geometric feature, color feature and the hypothetical score of the lane line. Further, based on the weighted maximum clique algorithm, at least one local lane line with the highest weight value among the local lane lines is solved, that is, at least one initial lane line.

Further, filtering processing may be performed on at least one initial lane line to obtain an initial lane line set. In one embodiment, post-processing may be performed based on the preset a priori information for the initial lane line, so as to filter all the initial lane lines of the at least one initial lane line, and the wrong lanes in all the initial lane lines The line is filtered out, and the initial lane line set is obtained.

Wherein, the preset a priori information for the initial lane line is a preset range or a preset value set based on the national standard of the lane line, and the a priori information specifically includes length information and width information, etc., assuming that the country The standard lane line length is 1.5 meters, and the length information included in the a priori information for the initial lane line can be set to be a part of the length less than the standard lane line length, such as the range of 10 cm to 15 cm, etc.; assuming national standards If the width of the lane line is 15 cm, the width information included in the prior information can be set to a range of 13 cm to 17 cm. Correspondingly, the wrong lane line is the lane line that does not meet the preset prior information.

The initial lane line selected from the set of local lane lines can be considered as the lane line most consistent with the actual lane line.

S120. Perform fitting optimization on the initial lane line data in the initial lane line set to obtain a target lane line set, where the fitting optimization includes: according to the initial lane line data and the history in the historical lane line set The lane line data is fitted and optimized to obtain the target lane line set.

Specifically, after the initial lane line set is determined from the local map, the historical lane line data in the historical lane line set and the initial lane line data in the initial lane line set can be fitted and optimized to obtain the target lane line set. The current lane line detection results can be combined with the historical lane line detection results in time sequence and space, which is beneficial to improve the accuracy of the lane line detection results.

Exemplarily, the initial lane line data filtered in the initial lane line set and the historical lane line data in the historical lane line set are fitted and optimized to obtain the target lane line set to complete the detection of the lane line.

In some embodiments, after the initial lane line set is determined from the local map, it is possible to query whether there is a historical lane line set in the storage area.

Exemplarily, if the historical lane line set does not exist, it can be determined that the current lane line detection is the first lane line detection, and the historical lane line set is determined according to the initial lane line set obtained this time, and the determined historical lane line The set is stored in the storage area to facilitate subsequent fitting optimization with the new initial lane line set. If there is a historical lane line set, the historical lane line data in the historical lane line set and the initial lane line data in the initial lane line set can be fitted and optimized to obtain the target lane line set.

In some embodiments, the fitting optimization is used to make the target lane line set include lane lines that are not parallel to each other.

By fitting and optimizing the initial lane line data and the historical lane line data in the historical lane line collection, the fitting optimization can be independent of the assumption of lane line geometric parallelism and road surface parallelism, and can be applied to roads including bifurcations. The scene, as shown in Figure 2, the road includes lane lines that are not parallel to each other, which improves the versatility of the lane line detection method.

In some embodiments, the fitting optimization based on the initial lane line data and the historical lane line data in the historical lane line set to obtain the target lane line set includes: based on over-fitting constraint conditions and parallel constraint conditions, Fitting optimization is performed according to the initial lane line data and the historical lane line data in the historical lane line set to obtain the target lane line set.

Exemplarily, the initial lane line data to be fitted is determined from the initial lane line set, and the historical lane line matching the initial lane line data is determined from the historical lane line set, and the initial lane line data to be fitted Is any one of the initial lane line set.

Exemplarily, by matching the index information of the initial lane line data to be fitted with the index information of the historical lane line data in the historical lane line set, the target historical lane line matching the initial lane line data is determined.

For example, the lane to which each initial lane line belongs can be determined based on the semantic information of the lane line grid constituting the initial lane line, and then index information is added to each initial lane line data, and each index information indicates the lane to which the initial lane line data belongs.

After the historical lane line matching the initial lane line to be fitted is determined from the historical lane line set, the process is performed according to the over-fitting constraint condition and the parallel constraint condition based on the initial lane line data and the historical lane line data. Fitting optimization to obtain the target lane line corresponding to the initial lane line data.

Determine each initial lane line data in the initial lane line set in turn as the initial lane line to be fitted, and determine the target lane line corresponding to each initial lane line data; obtain the target lane line set according to at least one target lane line; Combine the constraints and the parallel constraints so that the target lane line set includes lane lines that are not parallel to each other.

In some embodiments, the initial lane line data and the historical lane line data are fitted and optimized through a lane model including an over-fitting constraint condition and a parallel constraint condition.

Exemplarily, the mathematical function corresponding to the lane model can be composed of three parts. The first part is the least squares model based on solving the lane line position points corresponding to the initial lane line data and the target lane line obtained by fitting optimization. The minimum error between the curve equations; the second part is the smoothing term of the curve (corresponding to the preset conditions of overfitting), you can use the integration of the third derivative square of the curve equation as a smoothing constraint, mainly to prevent overfitting; The third part is the weak parallel constraint item (corresponding to the parallel constraint condition), which is mainly used to constrain the parallel state between two adjacent lane lines.

Exemplarily, f _i (t) is used to represent the curve equation of the target lane line obtained by fitting and optimizing the initial lane line data of the i-th initial lane line.

Exemplarily, the curve equation of the target lane line is described by a fifth-degree polynomial, and the specific ones are:

f _i (t) = T _i C _i , where T _i = (1,t,t ² ,...,t ⁵ ) and C _i =(c ₀ ,c ₁ ,c ₂ ,...,c ₅ ) ^T

In the formula, T _i represents the polynomial basis vector, t represents the first coordinate of the lane line position point on the local map, C _i represents the fitting coefficient data, which is the quantity to be solved, and the superscript T represents the transposition of the matrix; thus, The curve equation corresponding to the target lane line obtained by fitting optimization is expressed as follows:

f _i (t)=T _i C _i =c ₀ +c ₁ ×t+c ₂ ×t ² +c ₃ ×t ³ +c ₄ ×t ⁴ +c ₅ ×t ⁵

The first part of the lane model is used to solve the minimum error between the lane line position point corresponding to the initial lane line data and the curve equation corresponding to the target lane line obtained by fitting optimization.

The second part of the lane model is the smoothing term of the curve (corresponding to the over-fitting preset condition), and the integral of the third derivative square of the curve equation can be used as a smoothing constraint.

The third part of the lane model is the weak parallel constraint item (corresponding to the parallel constraint condition), which is mainly used to constrain the parallel state between two adjacent lane lines. For example, it is used to constrain the parallel state between the i-th initial lane line and the i+1-th initial lane line.

Exemplarily, the first part, second part, and third part of the lane model can be processed according to the fifth-order polynomial description form of the target lane line curve equation, and the part of the processing result that is not related to the fitting coefficient data C _i It is expressed as a constant, so that _{a lane model in the matrix form of the fitting coefficient data C i} can be obtained, such as an objective function in the matrix form.

In some embodiments, the objective function of the fitting optimization is expressed as obtaining the fitting coefficient data of the curve equations of two adjacent target lanes, so that the minimum value of the sum of the first part, the second part and the third part of the lane model .

Exemplarily, by taking the derivative of the objective function in the form of a matrix, and setting the derivative to 0, an equation for the fitting coefficient data of the curve equations of two adjacent target lane lines can be obtained.

According to this equation, the fitting coefficient data of the curve equations of two adjacent target lanes can be obtained. According to the obtained fitting coefficient data, the optimal curve equations of the two adjacent target lane lines can be obtained, for example, the target lane obtained by fitting corresponding to the i-th initial lane line and the i+1-th initial lane line The curve equation of the line.

Exemplarily, the preservation of historical observation information can be achieved by constructing an accumulation sum for the accumulation part _{of the polynomial basis vector T i} and the fitting coefficient data C _i on both sides of the equal sign of the equation.

Exemplarily, the cumulative sum of the polynomial basis vector T _i is the first aggregate data A _i , and the cumulative sum of the polynomial basis vector T _i and the fitting coefficient data C _i is the second aggregate data B _i .

_{The first aggregated data A i} and the second aggregated data B _{i are} obtained by accumulating the time series of historical observations. The first aggregated data _Ai and the second aggregated data contain more historical observations, which can prevent, for example, the loss of effective observations caused by sliding windows, which reduces the robustness of lane line detection.

At the same time, although the first aggregated data A _i and the second aggregated data B _i are obtained by accumulating the time series of observations, the amount of data is only determined by the order of the lane curve equation and the number of segments of the curve.

Exemplarily, the historical lane line data is obtained by accumulating historical initial lane line data according to a preset fusion rule.

It is understandable that the historical lane line data has the same amount of data.

In some embodiments, it is determined according to the current time, such as the polynomial basis vector and fitting coefficient data at the n+1th time, and the first aggregated data _Ai and the second aggregated data B _i at the nth time can be solved at the current Fitting coefficients of the curve equation of the optimized target lane line at all times.

In some embodiments, the increment of the cumulative sum is determined according to the current time, such as the polynomial basis vector and the fitting coefficient data at the n+1th time, and the newly determined increment is correspondingly added to the first aggregate data A _i , The second aggregated data B _i realizes that the initial lane line data is compressed to the historical lane line data to obtain updated historical lane line data.

It is understandable that the initial lane line data and historical lane line data are processed through a preset data structure. For example, the initial lane line data and historical lane line data can be processed by accumulating the increment determined by the polynomial basis vector and the fitting coefficient data. Wherein, the preset data structure has a certain amount of calculation. For example, the increment at each time and the accumulation of the first aggregated data and the second aggregated data have the same amount of calculation, which will not increase with the accumulation of historical observation data, and the solution is based on the first aggregated data and the second aggregated data. The amount of calculation for calculating the coefficients of the curve equation will not increase with the accumulation of historical observation data.

In some embodiments, as shown in FIG. 4, performing fitting optimization based on the initial lane line data and historical lane line data in the historical lane line set to obtain the target lane line set includes step S121 to step S123.

S121: Determine historical lane line data matching the initial lane line data from the set of historical lane lines.

Exemplarily, by matching the index information of the initial lane line data shown with the index information of the historical lane line data in the historical lane line set, the historical lane line data matching the initial lane line data is determined.

Exemplarily, if a certain initial lane line data corresponds to the i-th lane line, the historical lane line data corresponding to the i-th lane line in the historical lane line set matches the initial lane line data.

S122: Perform fitting optimization on the initial lane line data and the historical lane line data based on the over-fitting constraint condition and the parallel constraint condition, to obtain the target lane line corresponding to the initial lane line data.

Exemplarily, the historical lane line data corresponding to the i-th lane line includes first aggregated data A _i and second aggregated data B _i .

Specifically, as shown in FIG. 5, the initial lane line data and historical lane line data are fitted and optimized based on the over-fitting constraint condition and the parallel constraint condition to obtain the target lane corresponding to the initial lane line data Line, including step S1221 to step S1222.

S1221. Determine the first incremental data and the second incremental data according to the initial lane line data.

Exemplarily, the initial lane line data at time n+1 includes several lane line position points, so that the first incremental data used for superimposing on the first aggregated data and the second incremental data used for superimposing on the second aggregated data can be determined. Incremental data.

Exemplarily, the lane line position point on the map may be represented by a first coordinate value and a second coordinate value. For example, the first coordinate axis corresponding to the first coordinate value is a coordinate axis parallel to the left and right direction of the vehicle, and the second coordinate value corresponds to The second coordinate axis of is a coordinate axis parallel to the traveling direction of the vehicle.

In some embodiments, the determining the first incremental data according to the initial lane line data includes: determining the first incremental data according to the first coordinate value of the initial lane line on the local map, the The local map is determined according to the environment where the movable platform is located.

In some embodiments, the determining the second incremental data according to the initial lane line data includes: according to the first coordinate value of the initial lane line on the local map and the initial lane line on the local map. The second coordinate value determines the second incremental data.

S1222, based on the equation determined according to the over-fitting constraint condition and the parallel constraint condition, determine the corresponding initial lane line data according to the first incremental data, the second incremental data, the first aggregated data, and the second aggregated data Target lane line.

In some embodiments, the determining the target lane line corresponding to the initial lane line data according to the first incremental data, the second incremental data, the first aggregated data, and the second aggregated data includes: according to the The first incremental data, the second incremental data, the first aggregated data, and the second aggregated data determine the polynomial coefficients corresponding to the initial lane line data; determine the target lane line corresponding to the initial lane line data according to the polynomial coefficients .

Specifically, the curve equation corresponding to the i-th lane line obtained by fitting is expressed as follows:

S123: A target lane line set is formed by at least one of the target lane lines.

Exemplarily, the target lane line set is formed according to the two nearest target lane lines on both sides of the vehicle, or the target lane line set is formed according to the nearest target lane line on the left side of the vehicle, or the target lane line is formed according to the nearest target lane line on the right side of the vehicle Line set, or all target lane lines fitted in the sensor field of view form a target lane line set. Correspondingly, a corresponding number of lane lines in the optimized and target lane line set can be fitted later.

Because the lane model is fitting the initial lane line and the target historical lane line, the parallel state constraint is added between the two adjacent lane lines. Therefore, for complex lane detection scenes, such as branch road scenes, urban scenes, etc., the accuracy of lane line detection can be better guaranteed, so that it can be better applied to multiple lane line scenes, and the versatility is improved.

Exemplarily, after the target lane line set is obtained, the target lane lines in the target lane line set may be combined to obtain at least one lane, and the lane center line of the lane is generated to facilitate the driving of the movable platform. For example, the target lane lines may be combined according to attribute information, such as geometric characteristics and/or color characteristics of the lanes, and the geometric characteristics include any one or more of length characteristics, width characteristics, and parallel characteristics between lanes.

In some embodiments, the lane line detection method further includes: updating the historical lane line data according to the initial lane line data, and the updated historical lane line data has the same amount of data as the historical lane line data before the update.

Exemplarily, the updating the historical lane line data according to the initial lane line data includes: accumulating the first incremental data to the first aggregated data, and accumulating the second incremental data To the second aggregated data. The initial lane line data is compressed to the historical lane line data, and the updated historical lane line data is obtained. In addition, the updated historical lane line data has the same amount of data as the historical lane line data before the update, which can avoid the increase of the fusion accumulated observations and increase the calculation complexity, so that the calculation amount of the fitting optimization is constrained The purpose of living.

The lane line detection method provided by the embodiment of this specification determines a number of initial lane line data through the environment image around the movable platform, and then performs fitting optimization based on the initial lane line data and the corresponding historical lane line data in the historical lane line set, Get a number of target lines. Since the fitting optimization incorporates the historical detection data of the lane line, the lane line detection can be independent of the assumption of the geometric parallelism of the lane line and the assumption of the parallel road surface, which can be applied to the scene of the road including the bifurcation, which improves the versatility and detection accuracy. .

Please refer to FIG. 6 in conjunction with the foregoing embodiment. FIG. 6 is a schematic block diagram of a lane line detection device 100 provided in an embodiment of this specification. The lane line detection device 100 includes a sensor 101 and a processor 102. The sensor 101 may include, for example, a visual sensor for acquiring an image of the environment around the movable platform.

Specifically, the processor 102 may be a micro-controller unit (MCU), a central processing unit (Central Processing Unit, CPU), a digital signal processor (Digital Signal Processor, DSP), or the like.

Exemplarily, the lane line detection device 100 further includes a memory 103, which may include, for example, a Flash chip, a read-only memory (ROM, Read-Only Memory) disk, an optical disk, a U disk, or a mobile hard disk.

Wherein, the processor 102 is configured to run a computer program stored in the memory 103, and implement the aforementioned lane line detection method when the computer program is executed.

Exemplarily, the processor is configured to run a computer program stored in the memory 103, and implement the following steps when the computer program is executed:

Exemplarily, when the processor implements the fitting optimization based on the initial lane line data and the historical lane line data in the historical lane line set to obtain the target lane line set, it realizes:

Based on the over-fitting constraint condition and the parallel constraint condition, fitting optimization is performed according to the initial lane line data and the historical lane line data in the historical lane line set to obtain the target lane line set.

Exemplarily, the initial lane line data and historical lane line data are processed through a preset data structure, wherein the preset data structure has a certain amount of calculation.

Determining historical lane line data matching the initial lane line data from the historical lane line set;

Fitting and optimizing the initial lane line data and historical lane line data based on the over-fitting constraint condition and the parallel constraint condition to obtain the target lane line corresponding to the initial lane line data;

At least one target lane line forms a target lane line set.

Exemplarily, the historical lane line data includes first aggregated data and second aggregated data;

When the processor realizes that the initial lane line data and historical lane line data are fitted and optimized based on the over-fitting constraint condition and the parallel constraint condition, and the target lane line corresponding to the initial lane line data is obtained, the realization :

Determining the first incremental data and the second incremental data according to the initial lane line data;

Based on the equation determined according to the over-fitting constraint condition and the parallel constraint condition, the target corresponding to the initial lane line data is determined according to the first incremental data, the second incremental data, the first aggregated data, and the second aggregated data Lane line.

Exemplarily, when the processor realizes the determination of the target lane line corresponding to the initial lane line data, it realizes:

Determining the polynomial coefficients corresponding to the initial lane line data;

The target lane line corresponding to the initial lane line data is determined according to the polynomial coefficient.

Exemplarily, the processor further implements:

The historical lane line data is updated according to the initial lane line data, and the updated historical lane line data has the same amount of data as the historical lane line data before the update.

Exemplarily, when the processor implements the updating of the historical lane line data according to the initial lane line data, it implements:

Accumulating the first incremental data to the first aggregated data;

The second incremental data is accumulated to the second aggregated data.

Exemplarily, when the processor realizes that the first incremental data is determined according to the initial lane line data, it realizes:

The first incremental data is determined according to the first coordinate value of the initial lane line on the local map, and the local map is determined according to the environment where the movable platform is located.

Exemplarily, when the processor realizes that the second incremental data is determined according to the initial lane line data, it realizes:

The second incremental data is determined according to the first coordinate value of the initial lane line on the local map and the second coordinate value of the initial lane line on the local map.

The specific principle and implementation of the lane line detection device provided in the embodiment of this specification are similar to the lane line detection method in the foregoing embodiment, and will not be repeated here.

Please refer to FIG. 7 in conjunction with the foregoing embodiment. FIG. 7 is a schematic block diagram of a movable platform 200 according to an embodiment of the present specification.

As shown in FIG. 7, the movable platform 200 includes the aforementioned lane line detection device 100 for determining the lane line. The movable platform 200 also includes a movement component 210 for driving.

In some embodiments, the movable platform 200 may be a vehicle, for example, a passenger vehicle, a cargo vehicle, an unmanned vehicle, etc., such as a manual driving vehicle or an autonomous driving vehicle.

The specific principles and implementation of the movable platform provided in the embodiment of this specification are similar to the lane line detection method of the foregoing embodiment, and will not be repeated here.

The embodiments of this specification also provide a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor realizes the lane markings provided in the above-mentioned embodiments. Steps of the detection method.

Wherein, the computer-readable storage medium may be an internal storage unit of the lane line detection device described in any of the foregoing embodiments, for example, the hard disk or memory of the lane line detection device. The computer-readable storage medium may also be an external storage device of the lane line detection device, such as a plug-in hard disk equipped on the lane line detection device, a smart memory card (Smart Media Card, SMC), a secure digital ( Secure Digital, SD card, Flash Card, etc.

The lane line fusion device, the movable platform, and the storage medium provided in the above-mentioned embodiments of this specification determine a number of initial lane line data based on the environment image around the movable platform, and then according to the corresponding history in the set of the initial lane line data and the historical lane line The lane line data is fitted and optimized, and several target lane lines are obtained. Since the fitting optimization incorporates the historical detection data of the lane line, the lane line detection can be independent of the assumption of the geometric parallelism of the lane line and the assumption of the parallel road surface, which can be applied to the scene of the road including the bifurcation, which improves the versatility and detection accuracy. .

It should be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the specification.

It should also be understood that the term "and/or" used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

The above are only specific implementations of this specification, but the protection scope of this specification is not limited to this. Any person skilled in the art can easily think of various equivalents within the technical scope disclosed in this specification. Modifications or replacements, these modifications or replacements shall be covered within the protection scope of this manual. Therefore, the protection scope of this specification should be subject to the protection scope of the claims.

Claims

A method for detecting lane lines, which is characterized in that it comprises:

Acquiring an image of the environment around the movable platform, and obtaining an initial lane line set of the movable platform according to the environment image;

Fitting and optimizing the initial lane line data in the initial lane line set to obtain a target lane line set;

Wherein, the fitting optimization includes: performing fitting optimization according to the initial lane line data and historical lane line data in the historical lane line set to obtain a target lane line set, and the fitting optimization is used to make the target lane The line set includes lane lines that are not parallel to each other.
The method according to claim 1, wherein the step of performing fitting optimization based on the initial lane line data and historical lane line data in the historical lane line set to obtain the target lane line set comprises:

Based on the over-fitting constraint condition and the parallel constraint condition, fitting optimization is performed according to the initial lane line data and the historical lane line data in the historical lane line set to obtain the target lane line set.
The method according to claim 1, wherein the initial lane line data and historical lane line data are processed through a preset data structure, wherein the preset data structure has a certain amount of calculation.
The method according to claim 3, wherein the historical lane line data is obtained by accumulating historical initial lane line data according to a preset fusion rule.
The method according to claim 1, wherein the step of performing fitting optimization based on the initial lane line data and historical lane line data in the historical lane line set to obtain the target lane line set comprises:

Determining historical lane line data matching the initial lane line data from the historical lane line set;

Fitting and optimizing the initial lane line data and historical lane line data based on the over-fitting constraint condition and the parallel constraint condition to obtain the target lane line corresponding to the initial lane line data;

At least one target lane line forms a target lane line set.
The method according to claim 5, wherein the historical lane line data includes first aggregated data and second aggregated data;

The fitting and optimizing the initial lane line data and the historical lane line data based on the over-fitting constraint condition and the parallel constraint condition to obtain the target lane line corresponding to the initial lane line data includes:

Determining the first incremental data and the second incremental data according to the initial lane line data;

Based on the equation determined according to the over-fitting constraint condition and the parallel constraint condition, the target corresponding to the initial lane line data is determined according to the first incremental data, the second incremental data, the first aggregated data, and the second aggregated data Lane line.
The method according to claim 6, wherein the determining the target lane line corresponding to the initial lane line data comprises:

Determining the polynomial coefficients corresponding to the initial lane line data;

The target lane line corresponding to the initial lane line data is determined according to the polynomial coefficient.
The method according to claim 6, wherein the method further comprises:

The historical lane line data is updated according to the initial lane line data, and the updated historical lane line data has the same amount of data as the historical lane line data before the update.
The method according to claim 8, wherein the updating the historical lane line data according to the initial lane line data comprises:

Accumulating the first incremental data to the first aggregated data;

The second incremental data is accumulated to the second aggregated data.
The method according to any one of claims 6-9, wherein the determining the first incremental data according to the initial lane line data comprises:

The first incremental data is determined according to the first coordinate value of the initial lane line on the local map, and the local map is determined according to the environment where the movable platform is located.
The method according to claim 10, wherein the determining the second incremental data according to the initial lane line data comprises:

The second incremental data is determined according to the first coordinate value of the initial lane line on the local map and the second coordinate value of the initial lane line on the local map.
A lane line detection device, which is characterized in that it comprises a sensor and a processor;

The sensor is used to obtain an image of the environment around the movable platform;

The processor is configured to implement the following steps:

Acquiring an image of the environment around the movable platform, and obtaining an initial lane line set of the movable platform according to the environment image;

Fitting and optimizing the initial lane line data in the initial lane line set to obtain a target lane line set;

Wherein, the fitting optimization includes: performing fitting optimization according to the initial lane line data and historical lane line data in the historical lane line set to obtain a target lane line set, and the fitting optimization is used to make the target lane The line set includes lane lines that are not parallel to each other.
The device according to claim 12, wherein the processor implements the fitting optimization based on the initial lane line data and the historical lane line data in the historical lane line set to obtain the target lane line set, achieve:

Based on the over-fitting constraint condition and the parallel constraint condition, fitting optimization is performed according to the initial lane line data and the historical lane line data in the historical lane line set to obtain the target lane line set.
The device according to claim 12, wherein the initial lane line data and historical lane line data are processed through a preset data structure, wherein the preset data structure has a certain amount of calculation.
The device according to claim 14, wherein the historical lane line data is obtained by accumulating historical initial lane line data according to a preset fusion rule.
The device according to claim 12, wherein the processor implements the fitting optimization based on the initial lane line data and the historical lane line data in the historical lane line set to obtain the target lane line set, achieve:

Determining historical lane line data matching the initial lane line data from the historical lane line set;

Fitting and optimizing the initial lane line data and historical lane line data based on the over-fitting constraint condition and the parallel constraint condition to obtain the target lane line corresponding to the initial lane line data;

At least one target lane line forms a target lane line set.
The device according to claim 16, wherein the historical lane line data comprises first aggregated data and second aggregated data;

When the processor realizes that the initial lane line data and historical lane line data are fitted and optimized based on the over-fitting constraint condition and the parallel constraint condition, and the target lane line corresponding to the initial lane line data is obtained, the realization :

Determining the first incremental data and the second incremental data according to the initial lane line data;

Based on the equation determined according to the over-fitting constraint condition and the parallel constraint condition, the target corresponding to the initial lane line data is determined according to the first incremental data, the second incremental data, the first aggregated data, and the second aggregated data Lane line.
The apparatus according to claim 17, wherein when the processor implements the determination of the target lane line corresponding to the initial lane line data, it implements:

Determining the polynomial coefficients corresponding to the initial lane line data;

The target lane line corresponding to the initial lane line data is determined according to the polynomial coefficient.
The device according to claim 17, wherein the processor further implements:

The historical lane line data is updated according to the initial lane line data, and the updated historical lane line data has the same amount of data as the historical lane line data before the update.
The apparatus according to claim 19, wherein when the processor implements the updating of the historical lane line data according to the initial lane line data, it implements:

Accumulating the first incremental data to the first aggregated data;

The second incremental data is accumulated to the second aggregated data.
The device according to any one of claims 17-20, wherein when the processor realizes that the first incremental data is determined according to the initial lane line data, it realizes:

The first incremental data is determined according to the first coordinate value of the initial lane line on the local map, and the local map is determined according to the environment where the movable platform is located.
The device according to claim 21, wherein when the processor realizes that the second incremental data is determined according to the initial lane line data, it realizes:

The second incremental data is determined according to the first coordinate value of the initial lane line on the local map and the second coordinate value of the initial lane line on the local map.
A vehicle, characterized in that it comprises:

The lane line detection device according to any one of claims 12-22, which is used to determine the lane line;

Sports components for driving.
A computer-readable storage medium, characterized in that, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor realizes as described in any one of claims 1-11. The lane line fusion method described.