CN115375956A

CN115375956A - Lane line detection method and related device

Info

Publication number: CN115375956A
Application number: CN202110552169.8A
Authority: CN
Inventors: 白东峰; 曹彤彤; 刘冰冰
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2021-05-20
Filing date: 2021-05-20
Publication date: 2022-11-22

Abstract

The embodiment of the application provides a method and a related device for detecting a lane line, which are used for effectively inhibiting the noise of characteristic points of the lane line so as to improve the accuracy of detecting the lane line. The method specifically comprises the following steps: the detection device of the lane line obtains the characteristic points of the initial lane line and the prior information of the road structure in the driving process of the vehicle; then when the detection device of the lane line determines that the prior information of the road structure is effective and the correlation between the characteristic point of the initial lane line and the prior information of the road structure reaches a preset threshold value according to a road prior judgment rule, converting the characteristic point of the initial lane line into a first lane line characteristic point under a road edge coordinate system; inputting the first lane line characteristic points into a Gaussian mixture model to obtain second lane line characteristic points, wherein the Gaussian mixture model is used for classifying and denoising the first lane line characteristic points; and finally outputting a final lane line according to the second lane line characteristic point.

Description

Lane line detection method and related device

Technical Field

The application relates to the field of artificial intelligence, in particular to a method and a related device for detecting lane lines.

Background

The intelligent driving (such as automatic driving, auxiliary driving and the like) technology depends on the cooperative cooperation of artificial intelligence, visual calculation, radar, a monitoring device and a global positioning system, so that the automatic driving of the vehicle can be realized without the active operation of human beings. The first step of automatic driving is acquisition and processing of environmental information, so that the environmental perception module is an interactive correlation port of the intelligent platform and a traffic scene and is also a front-end key input of the intelligent platform motion decision and planning control module. In the autonomous driving system shown in fig. 1, the environment sensing module acquires traffic scene information using sensor data, which is "eyes" and "ears" of the autonomous driving system, and directly determines the safety and stability of the autonomous driving vehicle in the driving task. In the environment perception module, the detection of the lane lines is one of the core tasks of the environment perception module, the lane line detection provides local lane perception information for vehicle motion planning, and the environment perception module plays a key role in a lane keeping (cruising) system and a lane departure early warning system of an intelligent vehicle platform.

The current lane line detection technology mainly provides the following: according to one scheme, a target area containing the ground is segmented from point cloud data of a laser radar, ground point cloud data are extracted from the target area, characteristic points of lane lines are screened out from the ground point cloud data, a plurality of lane lines are obtained through fitting based on the characteristic points of the lane lines, and the lane lines with the distance between the lane lines within a preset lane line distance range are screened out from the lane lines and serve as the target lane lines. In the other scheme, a three-dimensional coordinate system is established by taking the center of a laser radar as an origin, an interested area is set, the road edge of a road is detected, road edge candidate points are extracted according to the geometric characteristics of the road edge, and noise is removed according to the characteristics of the consistency of the direction of the road edge; determining point cloud data of a road surface where a lane line is located according to the space position of a road edge, carrying out layered processing on scanning lines according to the characteristics of different media with different reflection intensities, setting a reflection intensity threshold value to extract lane line candidate points, carrying out density clustering and denoising according to the characteristics of global continuity of the lane line, and finally using a binomial curve to fit the lane line for output.

However, the two modes have poor anti-noise energy, and the detection accuracy of the lane lines is poor when the lane lines are of a dotted line type or are irregular or when the lane road is uneven or is interfered by obstacles.

Disclosure of Invention

The embodiment of the application provides a method and a related device for detecting a lane line, which are used for effectively inhibiting the noise of characteristic points of the lane line so as to improve the accuracy of detecting the lane line.

In a first aspect, the present application provides a method for detecting a lane line, which specifically includes: the detection device of the lane line obtains the characteristic points of the initial lane line and the prior information of the road structure in the driving process of the vehicle; then when the detection device of the lane line determines that the prior information of the road structure is effective and the correlation between the characteristic point of the initial lane line and the prior information of the road structure reaches a preset threshold value according to a road prior judgment rule, converting the characteristic point of the initial lane line into a first lane line characteristic point under a road edge coordinate system; inputting the first lane line characteristic points into a Gaussian mixture model to obtain second lane line characteristic points, wherein the Gaussian mixture model is used for classifying and denoising the first lane line characteristic points; and finally outputting a final lane line according to the second lane line characteristic point.

In this embodiment, the detection device for the lane line may obtain the sensing result (i.e., the priori information of the road structure) for the road structure and the characteristic point of the initial lane line in the driving process of the vehicle in real time according to the technologies such as laser sensing, radar sensing, or spatial positioning.

In the technical solution provided in this embodiment, the detection device of the lane line performs a road prior decision on the road structure prior information and the initial lane line feature point, so as to screen the initial lane line feature point and the road structure prior information in advance, establish a road edge coordinate system by using the road structure prior information, and convert the initial lane line feature point under the road edge coordinate system, thereby ensuring the validity of the initial lane line feature point and the road structure prior information. And finally, inputting a Gaussian mixture model for classification and denoising, wherein the Gaussian mixture model can effectively learn the spatial distribution characteristics of the lane lines of the corresponding type of roads, so that the classification and denoising of the characteristic points of the lane lines are effectively completed, and the accuracy of lane line detection is finally improved.

Optionally, the road a priori decision rule includes but is not limited to:

the length of the road structure exceeds a first preset value. Namely, it is necessary to ensure that the length of the road structure is sufficient to fully reflect the road structure and the information of the extending direction of the road.

The end point position of the curve of the road structure accords with a preset condition, the preset condition is that the distance between the starting point of the road perception curve and the longitudinal starting point of the vehicle body coordinate system is smaller than a second preset value, the distance between the ending point of the road perception curve and the longitudinal starting point of the vehicle body coordinate system is larger than a third preset value, and the second preset value is smaller than the third preset value.

The parallelism of the two road edges of the road structure is greater than the parallelism threshold. In other words, in an actual scene, two road edges on the left and right sides of a road should be detected on a general road, and the parallelism of the left and right road edges should be calculated in consideration of the fact that the road edge curves of the road are not all parallel.

The coupling degree of the road structure and the characteristic point of the side lane line is greater than the coupling degree threshold value. Most of the road edge distribution and the lane line distribution in the road structure have consistency, but there are few cases of inconsistency, and therefore the degree of coupling between the most lateral lane line characteristic point and the road edge curve should be considered.

In a possible implementation manner, the lane line detection apparatus inputs the first lane line feature point into the gaussian mixture model to obtain a confidence, where the confidence is used to indicate a probability value that the second lane line feature point is used as a valid feature point. That is, when the gaussian mixture model outputs a valid feature point, the probability value of the valid feature point is also output. For example, the feature point with the probability value greater than 0.6 in the first lane line feature point is used as the second lane line feature point, so the second lane line feature point further includes the feature point with the probability value of 0.7 and the feature point with the probability value of 0.8.

In one possible implementation manner, if the detection device of the lane line employs a laser sensing technology, the detection device of the lane line may employ the following technical scheme when acquiring the initial lane line feature point and the road structure prior information:

the detection device of the lane line can acquire the prior information of the road structure and the original laser point cloud data of the road surface according to the laser sensing; and then preprocessing the original laser point cloud data to extract the characteristic points of the initial lane line.

In a possible implementation manner, the detecting device of the lane line obtains the valid lane line feature points in the second lane line feature points by using a bayesian classifier with the second and prior clues, where the prior clues include, but are not limited to, the confidence level, the similarity between the second lane line feature points and the historical tracking result, and the probability value of the reflection intensity absolute value of the second lane line feature points, where the reflection intensity absolute value is used to indicate the feature of relative change of point cloud reflection intensity; and finally, optimizing the characteristic points of the effective lane line by a nonlinear optimization method by using a cost function and outputting the final lane line.

Based on the scheme, in one possible implementation mode, the cost function is designed according to a preset rule; the preset rules at least comprise: 1. the spatial distribution of the lane lines and the spatial distribution of the road structure have consistency. In this embodiment, the consistency between the spatial distribution of the lane lines and the spatial distribution of the road structure may be expressed as the parallel similarity between the road edges and the lane line curves. 2. The lane line output of consecutive frame data has continuity and consistency, i.e., the degree of conformance of the lane line characteristic points and the parametric equation (used to represent the final lane line). In this embodiment, the degree of conformity between the characteristic point of the lane line and the parameter equation may be expressed as a mean value of point-line distances from the characteristic point of the lane line to the parameter equation.

In this embodiment, the detection apparatus for lane lines may also train to obtain the gaussian mixture model in an offline state. The method specifically comprises the following steps: the detection device of the lane line acquires an off-line data set; and then training according to the off-line data set to obtain the Gaussian mixture model.

It should be understood that, when the gaussian mixture model is trained, it may be trained by other devices, and then the trained gaussian mixture model is transplanted to the detection device of the lane line, and the specific manner is not limited herein.

Based on the foregoing solution, in a possible implementation manner, the acquiring, by the device for detecting a lane line, an offline data set includes:

the detection device of the lane line acquires off-line laser point cloud data and off-line road structure prior information; then the detection device of the lane line preprocesses the off-line laser point cloud data to obtain off-line lane line characteristic points; and when the correlation between the off-line lane characteristic points and the off-line road structure prior information is determined to reach a preset threshold value according to the road prior judgment rule, the detection device of the lane line converts the off-line lane characteristic points and the off-line road structure prior information into the off-line data set under the road edge coordinate system.

In a possible implementation manner, the training of the detection apparatus for lane lines according to the offline data set to obtain the gaussian mixture model includes: the lane line detection device constructs the Gaussian mixture model according to the off-line lane line characteristic points and the spatial distribution of the off-line road structure prior information, and solves the model parameter values of the Gaussian mixture model.

In one possible implementation manner, the lane line detection device may further update the model parameter values in the gaussian mixture model according to the output condition of the valid lane line feature points in the second lane line feature points in the process of detecting the lane line on line. In the online part, the detection device of the lane line can also update the parameter values of the model (including the mean and variance of each submodel of the gaussian mixture model and the weight of the submodel) according to the distribution of the effective lane line characteristic points finally output by the gaussian mixture model in the road edge coordinate system. Therefore, the spatial distribution characteristics of the lane line characteristic points can be learned by the Gaussian mixture model, and the accuracy of the Gaussian mixture model on the classification and denoising functions is improved.

In a second aspect, the present application provides a method for training a gaussian mixture model, which specifically includes: acquiring off-line laser point cloud data and off-line road structure prior information; preprocessing the off-line laser point cloud data to obtain off-line lane line characteristic points; when the fact that the off-line road structure prior information is valid and the correlation between the off-line lane line characteristic points and the off-line road structure prior information reaches a preset threshold value is determined according to the road prior judgment rule, the lane line detection device converts the off-line lane line characteristic points and the off-line road structure prior information into the off-line data set under a road edge coordinate system; and constructing the Gaussian mixture model according to the off-line lane characteristic points and the spatial distribution of the off-line road structure prior information, and solving to obtain model parameter values of the Gaussian mixture model.

In a third aspect, the present application provides a lane line detection apparatus having a function of implementing the behavior of the lane line detection apparatus in the first aspect. The function can be realized by hardware, and can also be realized by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the functions described above.

In one possible implementation, the apparatus includes means or module for performing each step of the above first aspect. For example, the apparatus includes: the acquisition module is used for acquiring the prior information of the road structure and the characteristic points of the initial lane line;

the processing module is used for converting the initial lane line characteristic points into first lane line characteristic points under a road edge coordinate system when the road structure prior information is determined to be effective according to a road prior judgment rule and the correlation between the initial lane line characteristic points and the road structure prior information reaches a preset threshold; inputting the first lane line characteristic points into a Gaussian mixture model to obtain second lane line characteristic points, wherein the Gaussian mixture model is used for classifying and denoising the first lane line characteristic points; and outputting a final lane line according to the second lane line characteristic point.

Optionally, the system further comprises a storage module, configured to store necessary program instructions and data of the lane line detection device.

In one possible implementation, the apparatus includes: a processor and a transceiver, the processor being configured to support the lane line detection apparatus to perform the respective functions of the method provided by the first aspect. The transceiver is used for indicating the communication between the detection device of the lane line and other equipment. Optionally, the apparatus may further comprise a memory for coupling to the processor, which stores program instructions and data necessary for the lane line detection means.

In a possible implementation, when the device is a chip within a lane line detection device, the chip includes: a processing module and a transceiver module. The transceiver module may be, for example, an input/output interface, a pin, or a circuit on the chip, and transmits various types of information acquired by the sensor to another chip or module coupled to the chip, and the processing module may be, for example, a processor, where the processor is configured to convert the initial lane line feature point into a first lane line feature point in a road edge coordinate system when it is determined that the road structure prior information is valid according to a road prior decision rule and the correlation between the initial lane line feature point and the road structure prior information reaches a preset threshold; inputting the first lane line characteristic points into a Gaussian mixture model to obtain second lane line characteristic points, wherein the Gaussian mixture model is used for classifying and denoising the first lane line characteristic points; and outputting a final lane line according to the second lane line characteristic point. The processing module can execute computer-executable instructions stored in the storage unit to support the lane line detection device to execute the method provided by the first aspect. Alternatively, the storage unit may be a storage unit in the chip, such as a register, a cache, and the like, and the storage unit may also be a storage unit located outside the chip, such as a read-only memory (ROM) or another type of static storage device that can store static information and instructions, a Random Access Memory (RAM), and the like.

In one possible implementation, the apparatus includes a communication interface for obtaining road structure prior information and initial lane line characteristic points; the logic circuit is used for converting the initial lane line characteristic points into first lane line characteristic points under a road edge coordinate system when the road structure prior information is determined to be effective according to a road prior judgment rule and the correlation between the initial lane line characteristic points and the road structure prior information reaches a preset threshold value; inputting the first lane line characteristic points into a Gaussian mixture model to obtain second lane line characteristic points, wherein the Gaussian mixture model is used for classifying and denoising the first lane line characteristic points; and outputting a final lane line according to the second lane line characteristic point.

The processor mentioned in any of the above may be a general Processing Unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of programs of the above-mentioned data transmission methods.

In a fourth aspect, the present application provides a computer-readable storage medium storing computer instructions for executing the method according to any possible implementation manner of any one of the above aspects.

In a fifth aspect, embodiments of the present application provide a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of any one of the above aspects.

In a sixth aspect, the present application provides a chip system comprising a processor for supporting a lane line detection apparatus to implement the functions referred to in the above aspects, such as generating or processing data and/or information referred to in the above methods. In one possible design, the system-on-chip further includes a memory for storing program instructions and data necessary for the lane line detection apparatus to implement the functions of any one of the above aspects. The chip system may be formed by a chip, and may also include a chip and other discrete devices.

In a seventh aspect, an embodiment of the present application provides an intelligent automobile, where the system includes the lane line detection apparatus in the above aspect.

Drawings

FIG. 1 is an exemplary framework diagram of an autopilot system;

fig. 2 is a schematic flowchart of a conventional lane line detection method;

fig. 3 is a schematic flow chart of another conventional lane line detection method;

FIG. 4a is a schematic diagram of a lidar sensing framework;

FIG. 4b is a schematic view of the lane marking detection apparatus according to the embodiment of the present application;

fig. 5 is a schematic diagram of an embodiment of a method for detecting a lane line in an embodiment of the present application;

FIG. 6 is a schematic diagram of a laser road along a line segment and a characteristic point of a side lane line in the embodiment of the present application;

FIG. 7 is a schematic view of a road-edge coordinate system in an embodiment of the present application;

FIG. 8 is a schematic diagram illustrating an example of the transformation of an initial lane marking feature point into a first lane marking feature point in a road edge coordinate system according to the present disclosure;

FIG. 9 is a schematic process flow diagram of the Gaussian mixture model in the off-line and on-line states according to the embodiment of the present application;

FIG. 10 is a schematic diagram of an embodiment of the present application for converting an offline data set in a road-edge coordinate system;

FIG. 11 is a schematic diagram of a Gaussian mixture model constructed in an embodiment of the present application;

FIG. 12 is a point cloud feature diagram output under a Gaussian mixture model in the embodiment of the present application;

FIG. 13 is a schematic flow chart illustrating a process for obtaining a final lane line using a Bayesian classifier and non-linear optimization in the present embodiment;

fig. 14 is a schematic view of a lane line detection device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application are described below with reference to the accompanying drawings, and it is obvious that the described embodiments are only some embodiments of the present application, but not all embodiments. As can be known to those skilled in the art, with the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

The terms "first," "second," and the like in the description and in the claims of the present application and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be implemented in other sequences than those illustrated or described herein. Moreover, the terms "comprises," "comprising," and any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not expressly listed or inherent to such process, method, article, or apparatus. The naming or numbering of the steps appearing in the present application does not mean that the steps in the method flow have to be executed in the chronological/logical order indicated by the naming or numbering, and the named or numbered process steps may be executed in a modified order depending on the technical purpose to be achieved, as long as the same or similar technical effects are achieved. The division of the units presented in this application is a logical division, and in practical applications, there may be another division, for example, multiple units may be combined or integrated into another system, or some features may be omitted, or not executed, and in addition, the shown or discussed coupling or direct coupling or communication connection between each other may be through some interfaces, and the indirect coupling or communication connection between the units may be in an electrical or other similar form, which is not limited in this application. Furthermore, the units or sub-units described as the separation component may or may not be physically separated, may or may not be physical units, or may be distributed in a plurality of circuit units, and a part or all of the units may be selected according to actual needs to achieve the purpose of the present application.

The current lane line detection technology mainly provides the following: as shown in the schematic flow chart of fig. 2, one scheme is to segment a target area including the ground from the point cloud data of the laser radar, extract ground point cloud data from the target area, screen out lane line feature points from the ground point cloud data, fit to obtain a plurality of lane lines based on the lane line feature points, and screen out lane lines having a distance between lane lines within a preset lane line distance range from the plurality of lane lines as target lane lines. In the scheme, after the ground area is extracted, the lane line characteristic points are directly detected, classified and fitted in the ground area, the fitted lane lines are simply filtered by using the width between the preset lane lines, and the final detection result is output, so that the noise resistance is poor, and the detection effect on the dotted line type or irregular lane lines is poor.

As shown in the flow diagram of fig. 3, another scheme is to establish a three-dimensional coordinate system with the center of the laser radar as the origin, set the region of interest and detect the road edge of the road, extract candidate points of the road edge according to the geometric features of the road edge, and remove noise according to the features of the consistency of the direction of the road edge; determining point cloud data of a road surface where a lane line is located according to the space position of the road edge, carrying out layered processing on scanning lines according to the characteristics of different reflection intensities of different media, setting a reflection intensity threshold value to extract lane line candidate points, carrying out density clustering and denoising according to the characteristics of global continuity of the lane line, and finally using a binomial curve to fit the lane line for output. In the technical scheme, the road edge information is used as prior information and only used as a reference for clustering the characteristic points of the lane lines, and the point cloud clustering results obtained by a density-based clustering algorithm (DBSCAN) are aggregated again. But due to the sparsity of the laser characteristic points of the lane lines, the detection distance of the method is short, and the noise resistance is poor. When the road surface is uneven or is interfered by obstacles, the lane line reflection points contain a large amount of noise, and the accuracy of the clustered and fitted lane lines is poor.

In order to solve the problem, an embodiment of the present application provides a method for detecting a lane line, which specifically includes: the detection device of the lane line obtains the characteristic points of the initial lane line and the prior information of the road structure in the driving process of the vehicle; then when the detection device of the lane line determines that the prior information of the road structure is effective and the correlation between the characteristic point of the initial lane line and the prior information of the road structure reaches a preset threshold value according to a road prior judgment rule, converting the characteristic point of the initial lane line into a first lane line characteristic point under a road edge coordinate system; inputting the first lane line characteristic points into a Gaussian mixture model to obtain second lane line characteristic points, wherein the Gaussian mixture model is used for classifying and denoising the first lane line characteristic points; and finally outputting a final lane line according to the second lane line characteristic point.

The embodiments of the present application relate to a lot of related knowledge about lane line detection, and in order to better understand the solution of the embodiments of the present application, the following first introduces related concepts and background knowledge that may be related to the embodiments of the present application.

(1) Wheeled mobile device

The system is an integrated system integrating multiple functions such as environment perception, dynamic decision and planning, behavior control and execution and the like, and can also be called a wheeled mobile robot or a wheeled intelligent body, for example, wheeled construction equipment, an automatic driving vehicle, an auxiliary driving vehicle and the like, and the wheeled mobile equipment is called as the wheeled mobile equipment provided by the application as long as the equipment is provided with wheeled mobile equipment. For convenience of understanding, in the following embodiments of the present application, a wheeled mobile device is taken as an example of an autonomous vehicle, which may be a car, a truck, a motorcycle, a bus, a boat, an airplane, a helicopter, a lawn mower, a recreational vehicle, a playground vehicle, construction equipment, a trolley, a golf cart, a train, a trolley, and the like, and the embodiments of the present application are not particularly limited.

(2) Perception

The perception accuracy is the first condition that automatic driving can be safely carried out, the perception can be provided with various modules from the perspective of a sensor, such as a laser perception module, a visual perception module, a millimeter wave perception module and the like, the laser perception module is one of key modules and widely applied to Advanced Driver Assisted Systems (ADAS) and Automatic Driving Systems (ADS), the laser perception module can provide relevant information (such as lane lines on a road) of the surrounding environment for wheel type mobile equipment (such as an automatic driving vehicle) provided with the system, the relevant information can also be called perception information, and the perception information provides a solid basis for a reasonable control decision.

In a possible implementation manner, the embodiment of the present application detects the information of the lane line based on the laser radar data. The method can be applied to a lane line detection module of a laser radar sensing framework shown in fig. 4 a. The sensing information is utilized to enable a laser sensing module (such as a laser sensor or a laser radar installed on wheeled mobile equipment) to emit laser beams in a fixed direction, the emitted laser beams are reflected when encountering obstacles, so that the time difference between the emission and the arrival of the laser beams can be obtained, and the distance from the sensor to the nearest obstacle in the direction can be obtained by multiplying the speed by two, so that laser point cloud data can be obtained. And according to the laser point cloud data, obtaining a laser sensing result through a laser road edge detection module, a laser lane line detection module, a target object detection module and a travelable area detection module in a laser radar sensing frame, and then sending the laser sensing result to other modules.

The technical scheme provided by the embodiment of the application can be applied to the detection device of the lane line shown in fig. 4b, and the detection device of the lane line mainly comprises a sensor and a computing platform. Wherein the sensors include lidar sensors and integrated navigation systems. The laser radar sensor is mainly used for acquiring three-dimensional point cloud data of an environment and representing the environment around the intelligent vehicle platform; the computing platform is mainly used for operating sensor drive and each perception algorithm; the integrated navigation system is mainly used for acquiring pose information of the intelligent vehicle.

A laser radar in the computing platform is driven to acquire an original signal of the radar, the original signal is analyzed into position information and reflection intensity information of the three-dimensional point cloud, and the position information and the reflection intensity information are sent to a laser road edge sensing module and a laser lane line detection module; the laser road edge sensing module mainly outputs road structure information as important prior information of the laser lane line detection module; the laser lane line tracking module receives the single-frame detection information and the vehicle pose information to perform multi-frame tracking processing, and finally outputs a smooth and stable lane line sensing result to the lane line receiving module. The lane line receiving module may be a fusion module, a motion planning module, a prediction module, etc.

Specifically, referring to fig. 5, a method embodiment of the lane line detection method in the embodiment of the present application includes:

501. the detection device of the lane line acquires laser point cloud data.

The detection device of the lane line obtains laser point cloud data in the real-time driving process of the vehicle through the laser radar.

502. The detection device of the lane line obtains the characteristic points of the initial lane line and the prior information of the road structure according to the laser point cloud data.

The detection device of the lane line carries out denoising and coordinate conversion on the laser point cloud data, and then obtains preprocessed laser point cloud data and a point cloud characteristic diagram in a parallel projection mode, a spherical projection mode and the like; and then carrying out ground segmentation on the preprocessed laser point cloud data and the point cloud characteristic map to obtain road surface point cloud data in the laser point cloud data, and finally extracting the initial lane line characteristic points in the road surface point cloud data by utilizing the reflection intensity change characteristics of the laser point cloud. And meanwhile, carrying out image segmentation according to the preprocessed laser point cloud data and the point cloud characteristic map, and obtaining road structure prior information through a laser road edge perception algorithm.

503. And when the detection device of the lane line determines that the correlation between the initial lane line characteristic point and the prior information of the road structure reaches a preset threshold value according to a road prior judgment rule, converting the initial lane line characteristic point into a first lane line characteristic point under a road edge coordinate system.

In order to detect whether the road structure prior information output by the laser road edge sensing algorithm is valid or not, the detection device of the lane line presets a road prior judgment rule, wherein the road prior judgment rule includes but is not limited to:

1. the length of the road structure exceeds a first preset value. Namely, it is necessary to ensure that the length of the road structure is sufficient to fully reflect the road structure and the information of the extending direction of the road.

2. The end point position of the curve of the road structure accords with a preset condition, the preset condition is that the distance between the starting point of the road perception curve and the longitudinal starting point of the vehicle body coordinate system is smaller than a second preset value, the distance between the ending point of the road perception curve and the longitudinal starting point of the vehicle body coordinate system is larger than a third preset value, and the second preset value is smaller than the third preset value.

3. The parallelism of the two road edges of the road structure is greater than the parallelism threshold. In other words, in an actual scene, two road edges on the left and right sides of a road should be detected on a general road, and the parallelism of the left and right road edges should be calculated in consideration of the fact that the road edge curves of the road are not all parallel.

4. And the coupling degree of the road structure and the characteristic points of the side lane lines is greater than a coupling degree threshold value. Most of the road edge distribution and the lane line distribution in the road structure have consistency, but there are few cases of inconsistency, so the degree of coupling between the characteristic point of the most lateral lane line and the road edge curve should be considered.

And under the road judgment rule, determining the correlation between the initial lane line characteristic point and the road structure prior information. Only if the correlation between the initial lane line feature point and the road structure prior information reaches a preset threshold value under the condition of at least meeting the four constraint conditions, it can be determined that the road structure prior information is valid, and the laser road edge line segment and the side lane line feature point can be as shown in fig. 6. At the moment, the detection device of the lane line converts the initial lane line characteristic points into first lane line characteristic points under a road edge coordinate system according to the road structure prior information.

In this embodiment, as shown in fig. 7, the road edge coordinate system is defined as follows: the coordinate origin is a perpendicular point of a road edge closest to the origin of the current vehicle body coordinate system, the s coordinate is a curve length value of a curve along the road edge, and the d coordinate is a radial distance from the point to the curve along the road edge. The process of converting the initial lane line characteristic points into the first lane line characteristic points in the road-edge coordinate system may be as shown in fig. 8. Fig. 8 shows the initial lane line feature points obtained according to the laser sensing result on the left side, and fig. 8 shows the first lane line feature points on the right side in the road edge coordinate system.

504. The detection device of the lane line inputs the first lane line characteristic point into a Gaussian mixture model to obtain a second lane line characteristic point.

The detection device of the lane line inputs the first lane line characteristic point into a Gaussian mixture model for classification and denoising to obtain a second lane line characteristic point.

In this embodiment, the gaussian mixture model can be expressed as shown in the following formula:

0≤π _k ≤1

wherein x is the radial coordinate of the first lane line characteristic point under the road edge coordinate system, pi is the mixing coefficient of the model, mu and sigma are the mean and variance of Gaussian distribution respectively, and K is the number of Gaussian distribution in the Gaussian mixture model.

When the Gaussian mixture model is used for calculating the effectiveness of the feature points of the first lane line under the road edge coordinate system, the probability of the feature points belonging to each lane line can be output, the classification with the highest probability is selected, the variance value of relative Gaussian distribution in the classification is calculated, a variance threshold value is set, if the calculated variance is smaller than or equal to the threshold value, the feature points are considered to be effective in road prior, and otherwise, the feature points are invalid points. And traversing all the initial lane characteristic points to solve the effectiveness of the initial lane characteristic points to obtain the second lane characteristic points and the classification information thereof. It will be appreciated that the effectiveness can also be solved using deviation values from a mean value. The specific method is not limited herein.

In this embodiment, the gaussian mixture model may be trained in an offline state and then applied in an online state. The specific process can be as shown in fig. 9:

in the off-line part, obtaining off-line lane characteristic points and off-line road structure prior information; then, under the condition that the correlation between the off-line lane feature point and the off-line road structure prior information is determined, converting the off-line lane feature point and the off-line road structure prior information into a road edge coordinate system, thereby forming an off-line data set, as shown in fig. 10; and then constructing the Gaussian mixture model according to the off-line lane characteristic points and the spatial distribution of the off-line road structure prior information, and solving to obtain model parameter values of the Gaussian mixture model, as shown in FIG. 11. It will be appreciated that an E-M algorithm may be employed in solving the model parameter values for the Gaussian mixture model.

In an online part, a detection device of the lane line acquires initial lane line characteristic points and road structure prior information in real time, then under the condition that the initial lane characteristic points and the road structure prior information are determined to have correlation, the initial lane characteristic points are converted into first lane line characteristic points in a road edge coordinate system, then the first lane line characteristic points are input into a Gaussian mixture model obtained by off-line part training to classify and de-noise the first lane line characteristic points to obtain second lane line characteristic points, and model parameter values of the Gaussian mixture model are updated according to the output condition of the second lane line characteristic points; and finally, outputting the final lane line according to the second lane line characteristic point.

Optionally, in the online portion, the lane line detection apparatus may further update parameter values of the model (including a mean and a variance of each submodel of the gaussian mixture model and a weight of the submodel) according to a distribution of the valid lane line feature points finally output by the gaussian mixture model in the road edge coordinate system.

The point cloud characteristic diagram obtained by adopting the scheme can be shown in fig. 12, and the influence of noise such as road marks (characters and arrows) on the lane line detection is effectively inhibited.

505. And the detection device of the lane line outputs a final lane line according to the characteristic point of the second lane line.

In this embodiment, the final lane line may be output by using a flow chart shown in fig. 13. The detection device of the lane line obtains effective lane line characteristic points in the second lane line characteristic points by using a Bayesian classifier for the second lane line characteristic points and prior clues, wherein the prior clues include but are not limited to confidence, similarity between the second lane line characteristic points and historical tracking results, and probability values of reflection intensity absolute values of the second lane line characteristic points, and the reflection intensity absolute values are used for indicating point cloud reflectivity intensity relative change characteristics; and finally, optimizing the characteristic points of the effective lane line by a nonlinear optimization method by using a cost function and outputting the final lane line.

In this embodiment, the confidence is a probability value that the detection device of the lane line inputs the first lane line feature point into the gaussian mixture model, and the confidence is used to indicate the second lane line feature point as a valid feature point. That is, when the gaussian mixture model outputs a valid feature point, the probability value of the valid feature point is also output. For example, the feature points with the probability value greater than 0.6 in the first lane line feature points are taken as the second lane line feature points, so that the feature points with the probability value of 0.7 and the feature points with the probability value of 0.8 are also included in the second lane line feature points.

The similarity between the second lane line feature point and the historical tracking result can be used as an effective prior clue in a lane line detection algorithm. In the specific implementation process, the historical tracking result of the lane line is converted into the coordinate space where the current data frame is located by using vehicle pose data, data association is carried out on the feature points of the lane line of the current data frame and the historical tracking result, the distance from the feature points to the associated tracking result is calculated, and the distance is mapped to the probability space to give the similarity. The vehicle pose data refers to a data stream reflecting the position and attitude of a vehicle obtained by combining an inertial navigation unit (IMU) with a Global Positioning System (GPS).

The probability value of the absolute value of the reflection intensity of the second lane line characteristic point can be obtained according to the following technical scheme: because the extraction of the initial lane line characteristic points depends on the relative change characteristics of the point cloud reflectivity intensity, the absolute value of the reflection intensity of the point cloud on the lane line is also in a high-intensity interval, the interval value is related to the performance of the laser radar, and the absolute value of the reflection intensity and the interval are mapped to a probability space to give a probability value of the absolute value of the reflection intensity. For example: the numerical range of the reflection intensity of the laser radar to the lane line is assumed to be [ I ] _min ,I _max ]For a certain lane line feature point, its prior probability in the reflection intensity dimension can be expressed as follows:

wherein the P is _Intensity A probability value representing an absolute value of the reflection intensity of the second lane line characteristic point; the Intensity represents the absolute value of the reflection Intensity of the characteristic point of the second lane line.

And integrating the probability values of the prior clues, calculating the posterior probability of the characteristic point of the second lane line according to Bayesian inference, setting an effectiveness probability threshold, and if the output posterior probability value is higher than the effectiveness probability threshold, taking the characteristic point of the lane line corresponding to the posterior probability value as the characteristic point of the final lane line. The specific formula can be as follows:

wherein P (valid) is a posterior probability value, P (GMM) is the confidence, P (History) is the similarity between the second lane line feature point and the historical tracking result, and P (Intensity) is a probability value of the absolute value of the reflection Intensity of the second lane line feature point.

In this embodiment, the cost function is designed according to a preset rule; the preset rule at least comprises: 1. the spatial distribution of the lane lines and the spatial distribution of the road structure have consistency. In this embodiment, the consistency between the spatial distribution of the lane lines and the spatial distribution of the road structure can be expressed as the parallel similarity between the road edges and the lane line curves. 2. The lane line output of consecutive frame data has continuity and consistency, i.e., the degree of conformance of the lane line characteristic points and the parametric equation (used to represent the final lane line). In this embodiment, the degree of conformity between the characteristic point of the lane line and the parameter equation may be expressed as a mean value of point-line distances from the characteristic point of the lane line to the parameter equation.

In the technical solution provided in this embodiment, the detection apparatus of the lane line performs a road prior decision on the road structure prior information and the initial lane line feature point, so as to screen the initial lane line feature point and the road structure prior information in advance, establish a road edge coordinate system by using the road structure prior information, and convert the initial lane line feature point under the road edge coordinate system, thereby ensuring the validity of the initial lane line feature point and the road structure prior information. And finally, inputting a Gaussian mixture model for classification and denoising, wherein the Gaussian mixture model can effectively learn the spatial distribution characteristics of the lane lines of the corresponding types of roads, so that the classification and denoising of the feature points of the lane lines are effectively completed, and the accuracy of lane line detection is finally improved. Meanwhile, the effectiveness of the feature points of the lane lines is judged by utilizing various prior clues, and the noise in the feature points is basically filtered; secondly, a multi-constraint cost function is designed, an optimal lane line parameter equation is solved, and stability and accuracy of lane line parametric output are improved.

The above describes the lane line detection method in the embodiment of the present application, and it is understood that the lane line detection apparatus includes a hardware structure and/or a software module corresponding to each function in order to implement the above functions. Those of skill in the art will readily appreciate that the various illustrative modules and algorithm steps described in connection with the embodiments disclosed herein may be implemented as hardware or combinations of hardware and computer software. Whether a function is performed in hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiment of the present application, the detection apparatus of the lane line may be divided into the functional modules according to the method example, for example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. It should be noted that, in the embodiment of the present application, the division of the module is schematic, and is only one logic function division, and there may be another division manner in actual implementation.

The following describes the detection device of the lane line in the embodiment of the present application in detail, specifically please refer to fig. 14, where fig. 14 is a schematic diagram of an embodiment of a detection device 1400 of the lane line in the embodiment of the present application. The lane line detection apparatus 1400 includes: an obtaining module 1401, configured to obtain road structure prior information and initial lane line feature points;

the processing module 1402 is configured to convert the initial lane line feature point into a first lane line feature point in a road edge coordinate system when it is determined according to a road prior decision rule that the correlation between the initial lane line feature point and the road structure prior information reaches a preset threshold; inputting the first lane line characteristic points into a Gaussian mixture model to obtain second lane line characteristic points, wherein the Gaussian mixture model is used for classifying and denoising the first lane line characteristic points; and outputting a final lane line according to the second lane line characteristic point.

Optionally, the road a priori decision rule includes at least one of:

the length of the road structure exceeds a first preset value;

the end point position of the curve of the road structure accords with a preset condition, the preset condition is that the distance between the starting point of the road perception curve and the longitudinal starting point of the vehicle body coordinate system is smaller than a second preset value, the distance between the ending point of the road perception curve and the longitudinal starting point of the vehicle body coordinate system is larger than a third preset value, and the second preset value is smaller than the third preset value;

the parallelism of the road edges on the two sides of the road structure is greater than the parallelism threshold value;

and the coupling degree of the road structure and the characteristic points of the side lane lines is greater than a coupling degree threshold value.

Optionally, the processing module 1402 is further configured to input the first lane line feature point into the gaussian mixture model to obtain a confidence, where the confidence is used to indicate a probability value of the second lane line feature point.

Optionally, the obtaining module 1401 is specifically configured to obtain original laser point cloud data and the road structure prior information according to a laser sensing technology;

the processing module 1402 is specifically configured to pre-process the original laser point cloud data to obtain the initial lane line feature points.

Optionally, the processing module 1402 is specifically configured to obtain valid lane line feature points of the second lane line feature points by using a bayesian classifier on the second lane line feature points and prior clues, where the prior clues include but are not limited to the confidence level, similarity between the second lane line feature points and a historical tracking result, and a probability value of an absolute value of a reflection intensity of the second lane line feature points, where the absolute value of the reflection intensity is used to indicate a feature of relative change of a point cloud reflectivity intensity; and optimizing the characteristic points of the effective lane line by a nonlinear optimization method by using a cost function to output the final lane line.

Optionally, the cost function is designed according to a preset rule;

the preset rules at least comprise:

the spatial distribution of the lane lines and the spatial distribution of the roads have consistency;

the lane line output of consecutive frame data has continuity and consistency.

Optionally, the obtaining module 1401 is further configured to obtain an offline data set;

the processing module 1402 is further configured to obtain the gaussian mixture model according to the offline data set.

Optionally, the obtaining module 1401 is specifically configured to obtain offline road structure prior information and offline lane line feature points;

the processing module 1402 is specifically configured to convert the off-line lane line feature points and the off-line road structure prior information into the off-line data set in the road edge coordinate system when it is determined according to the road prior decision rule that the correlation between the off-line lane line feature points and the off-line road structure prior information reaches a preset threshold.

Optionally, the processing module 1402 is specifically configured to construct the gaussian mixture model according to the off-line lane characteristic points and the spatial distribution of the off-line road structure prior information, and solve to obtain model parameter values of the gaussian mixture model.

Optionally, the processing module 1402 is further configured to update the model parameter value according to the valid lane line feature point.

The detection device of the lane line in the above embodiment may be an intelligent vehicle, or may be a chip applied to an intelligent vehicle, or other combined devices, components, and the like that can implement the functions of the intelligent vehicle. When the detection device of the lane line is an intelligent automobile, the transceiver module may be a transceiver, the transceiver may include an antenna, a radio frequency circuit and the like, and the processing module may be a processor, such as a baseband chip and the like. When the detection device of the lane line is a component having the detection function of the lane line, the transceiver module may be a radio frequency unit, and the processing module may be a processor. When the detection device of the lane line is a chip system, the portion for receiving in the transceiver module may be an input port of the chip system, the portion for transmitting in the transceiver module may be an output interface of the chip system, and the processing module may be a processor of the chip system, for example: a Central Processing Unit (CPU).

In the embodiment of the present application, the lane line detection apparatus further includes a storage module or a memory, and the included memory is mainly used for storing software programs and data, for example, storing the programs described in the above embodiments. The detection device of the lane line also has the following functions:

the transceiver acquires the prior information of the road structure and the characteristic points of the initial lane line;

the processor is used for converting the initial lane line characteristic points into first lane line characteristic points under a road edge coordinate system when the correlation between the initial lane line characteristic points and the road structure prior information is determined to reach a preset threshold value according to a road prior judgment rule; inputting the first lane line characteristic points into a Gaussian mixture model to obtain second lane line characteristic points, wherein the Gaussian mixture model is used for classifying and denoising the first lane line characteristic points; and outputting a final lane line according to the second lane line characteristic point.

Optionally, the road a priori decision rule includes at least one of:

the length of the road structure exceeds a first preset value;

the coupling degree of the road structure and the characteristic point of the side lane line is greater than the coupling degree threshold value.

Optionally, the processor is further configured to input the first lane line feature point into the gaussian mixture model to obtain a confidence level, where the confidence level is used to indicate a probability value of the second lane line feature point.

Optionally, the transceiver is specifically configured to obtain original laser point cloud data and the road structure prior information according to a laser sensing technology;

the processor is specifically configured to pre-process the original laser point cloud data to obtain the initial lane line feature points.

Optionally, the processor is specifically configured to obtain valid lane line feature points of the second lane line feature points by using a bayesian classifier on the second lane line feature points and prior clues, where the prior clues include, but are not limited to, the confidence levels, similarities between the second lane line feature points and historical tracking results, and probability values of absolute values of reflection intensities of the second lane line feature points, where the absolute values of the reflection intensities are used to indicate characteristics of relative changes of point cloud reflectivity intensities; and optimizing the characteristic points of the effective lane line by a nonlinear optimization method by using a cost function to output the final lane line.

Optionally, the cost function is designed according to a preset rule;

the preset rule at least comprises:

the lane line outputs of consecutive frame data are continuous and consistent.

Optionally, the transceiver is further configured to acquire an offline data set;

the processor is further configured to train according to the offline data set to obtain the gaussian mixture model.

Optionally, the transceiver is specifically configured to obtain offline road structure prior information and offline lane line feature points;

the processor is specifically configured to convert the off-line lane line feature points and the off-line road structure prior information into the off-line data set in a road edge coordinate system when it is determined according to the road prior decision rule that the correlation between the off-line lane line feature points and the off-line road structure prior information reaches a preset threshold.

Optionally, the processor is specifically configured to construct the gaussian mixture model according to the off-line lane feature points and the spatial distribution of the off-line road structure prior information, and solve to obtain model parameter values of the gaussian mixture model.

Optionally, the processor is further configured to update the model parameter value according to the valid lane line feature point.

The embodiment of the application also provides a processing device. The processing device comprises a processor and an interface; the interface is used for acquiring road structure prior information and initial lane line characteristic points; the processor is used for converting the initial lane line characteristic points into first lane line characteristic points under a road edge coordinate system when the correlation between the initial lane line characteristic points and the road structure prior information is determined to reach a preset threshold value according to a road prior judgment rule; inputting the first lane line characteristic points into a Gaussian mixture model to obtain second lane line characteristic points, wherein the Gaussian mixture model is used for classifying and denoising the first lane line characteristic points; and outputting a final lane line according to the second lane line characteristic point.

I.e. the processor, for performing the data processing method of any of the method embodiments described above.

It should be understood that the processing device may be a chip, the processor may be implemented by hardware or software, and when implemented by hardware, the processor may be a logic circuit, an integrated circuit, or the like; when implemented in software, the processor may be a general-purpose processor implemented by reading software code stored in a memory, which may be integrated with the processor, located external to the processor, or stand-alone.

The term "implemented by hardware" refers to that the functions of the modules or units are implemented by a hardware processing circuit without a program instruction processing function, and the hardware processing circuit may be composed of discrete hardware components or may be an integrated circuit. In order to reduce power consumption and size, the integrated circuit is usually implemented. The hardware processing circuit may include an ASIC (application-specific integrated circuit), or a PLD (programmable logic device); the PLD may further include an FPGA (field programmable gate array), a CPLD (complex programmable logic device), and the like. These hardware processing circuits may be a semiconductor chip packaged separately (e.g., as an ASIC); for example, various hardware circuits and CPUs may be formed on a silicon substrate and packaged separately into a chip, which is also referred to as SoC, or circuits and CPUs for implementing FPGA functions may be formed on a silicon substrate and packaged separately into a chip, which is also referred to as SoPC (system on a programmable chip).

Embodiments of the present application further provide a computer-readable storage medium, which includes instructions that, when executed on a computer, cause the computer to control a data processing apparatus to perform any one of the implementations shown in the foregoing method embodiments.

Embodiments of the present application further provide a computer program product, which includes computer program code, when the computer program code runs on a computer, the computer is caused to execute any implementation manner shown in the foregoing method embodiments.

The embodiment of the present application further provides a chip system, which includes a memory and a processor, where the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the chip executes any implementation manner shown in the foregoing method embodiments.

An embodiment of the present application further provides a chip system, which includes a processor, where the processor is configured to call and run a computer program, so that the chip executes any implementation manner shown in the foregoing method embodiment.

It should be noted that the above-described embodiments of the apparatus are merely schematic, where the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. In addition, in the drawings of the embodiments of the apparatus provided in the present application, the connection relationship between the modules indicates that there is a communication connection therebetween, and may be implemented as one or more communication buses or signal lines.

Through the above description of the embodiments, those skilled in the art will clearly understand that the present application can be implemented by software plus necessary general-purpose hardware, and certainly can also be implemented by special-purpose hardware including special-purpose integrated circuits, special-purpose CPUs, special-purpose memories, special-purpose components and the like. Generally, functions performed by computer programs can be easily implemented by corresponding hardware, and specific hardware structures for implementing the same functions may be various, such as analog circuits, digital circuits, or dedicated circuits. However, for the present application, the implementation of a software program is more preferable. Based on such understanding, the technical solutions of the present application or portions contributing to the prior art may be embodied in the form of a software product, where the computer software product is stored in a readable storage medium, such as a floppy disk, a usb disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk of a computer, and includes several instructions for causing a computer device to execute the method according to the embodiments of the present application.

In the above embodiments, all or part of the implementation may be realized by software, hardware, firmware, or any combination thereof. When implemented in software, it may be implemented in whole or in part in the form of a computer program product.

The computer program product includes one or more computer instructions. The procedures or functions described in accordance with the embodiments of the application are all or partially generated when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer, first or second network device, computing device, or data center to another website, computer, first or second network device, computing device, or data center by wired (e.g., coaxial cable, fiber optics, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer readable storage medium can be any available medium that a computer can store or a data storage device, such as a first network device or a second network device, a data center, etc., that includes one or more available media integrated therewith. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), among others.

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in the various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

Additionally, the terms "system" and "network" are often used interchangeably herein. It should be understood that in the embodiment of the present application, "B corresponding to a" means that B is associated with a, from which B can be determined. It should also be understood that determining B from a does not mean determining B from a alone, but may also be determined from a and/or other information.

Those of ordinary skill in the art will appreciate that the various illustrative components and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the components and steps of the various examples have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the technical solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It can be clearly understood by those skilled in the art that, for convenience and simplicity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or may also be implemented in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, which are essential or part of the technical solutions contributing to the prior art, or all or part of the technical solutions, may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the embodiments of the present application.

Claims

1. A method for detecting a lane line, comprising:

acquiring road structure prior information and initial lane line characteristic points;

when the correlation between the initial lane line characteristic points and the road structure prior information is determined to reach a preset threshold value according to a road prior judgment rule, converting the initial lane line characteristic points into first lane line characteristic points in a road edge coordinate system;

inputting the first lane line characteristic points into a Gaussian mixture model to obtain second lane line characteristic points, wherein the Gaussian mixture model is used for classifying and denoising the first lane line characteristic points;

and outputting a final lane line according to the second lane line characteristic point.

2. The method of claim 1, wherein the road apriori decision rule comprises at least one of:

the length of the road structure exceeds a first preset value;

the parallelism of the road edges at two sides of the road structure is greater than the parallelism threshold value;

3. The method according to claim 1 or 2, characterized in that the method further comprises:

and inputting the first lane line characteristic point into the Gaussian mixture model to obtain a confidence coefficient, wherein the confidence coefficient is used for indicating the probability value of the second lane line characteristic point.

4. The method according to any one of claims 1 to 3, wherein the obtaining road structure prior information and initial lane line characteristic points comprises:

acquiring original laser point cloud data and the prior information of the road structure according to a laser sensing technology;

and preprocessing the original laser point cloud data to obtain the initial lane line characteristic points.

5. The method of claim 4, wherein the outputting a final lane line according to the second lane line feature point comprises:

obtaining effective lane line characteristic points of the second lane line characteristic points by using a Bayesian classifier through the second lane line characteristic points and prior clues, wherein the prior clues comprise but are not limited to the confidence degrees, the similarity between the second lane line characteristic points and historical tracking results and the probability values of the reflection intensity absolute values of the second lane line characteristic points, and the reflection intensity absolute values are used for indicating the relative change characteristics of the point cloud reflectivity intensity;

and optimizing the characteristic points of the effective lane line by a nonlinear optimization method by using a cost function to output the final lane line.

6. The method according to claim 5, wherein the cost function is designed according to a preset rule;

the preset rules at least comprise:

the spatial distribution of the lane lines and the spatial distribution of the road structure have consistency;

the lane line output of consecutive frame data has continuity and consistency.

7. The method according to any one of claims 1 to 6, further comprising:

acquiring an offline data set;

and training according to the offline data set to obtain the Gaussian mixture model.

8. The method of claim 7, wherein the obtaining the offline data set comprises:

acquiring off-line road structure prior information and off-line lane line characteristic points;

and when the correlation between the off-line lane line characteristic points and the off-line road structure prior information is determined to reach a preset threshold value according to the road prior judgment rule, converting the off-line lane line characteristic points and the off-line road structure prior information into the off-line data set under a road edge coordinate system.

9. The method of claim 8, wherein the training the Gaussian mixture model from the offline data set comprises:

and constructing the Gaussian mixture model according to the off-line lane characteristic points and the spatial distribution of the off-line road structure prior information, and solving to obtain model parameter values of the Gaussian mixture model.

10. The method of claim 9, further comprising:

and updating the model parameter values according to the characteristic points of the effective lane lines.

11. A lane marking detection apparatus, comprising:

the acquisition module is used for acquiring the prior information of the road structure and the characteristic points of the initial lane line;

the processing module is used for converting the initial lane line characteristic points into first lane line characteristic points under a road edge coordinate system when the correlation between the initial lane line characteristic points and the road structure prior information reaches a preset threshold value according to a road prior judgment rule; inputting the first lane line characteristic points into a Gaussian mixture model to obtain second lane line characteristic points, wherein the Gaussian mixture model is used for classifying and denoising the first lane line characteristic points; and outputting a final lane line according to the second lane line characteristic point.

12. The apparatus of claim 11, wherein the road apriori decision rule comprises at least one of:

the length of the road structure exceeds a first preset value;

13. The apparatus of claim 11 or 12, wherein the processing module is further configured to input the first lane line feature point into the gaussian mixture model to obtain a confidence level, and the confidence level is used to indicate a probability value of the second lane line feature point.

14. The apparatus according to any one of claims 11 to 13, wherein the obtaining module is specifically configured to obtain raw laser point cloud data and the road structure prior information according to a laser sensing technique;

the processing module is specifically used for preprocessing the original laser point cloud data to obtain the initial lane line characteristic points.

15. The apparatus according to claim 14, wherein the processing module is specifically configured to use a bayesian classifier to obtain the valid lane characteristic point of the second lane characteristic point from the second lane characteristic point and prior clues, where the prior clues include but are not limited to the confidence level, similarity between the second lane characteristic point and a historical tracking result, and a probability value of an absolute value of a reflection intensity of the second lane characteristic point, where the absolute value of the reflection intensity is used to indicate a feature of a relative change of a point cloud reflectivity intensity; and optimizing the characteristic points of the effective lane line by using a cost function through a nonlinear optimization method, and outputting the final lane line.

16. The apparatus according to claim 15, wherein the cost function is designed according to a preset rule;

the preset rules at least comprise:

the lane line output of consecutive frame data has continuity and consistency.

17. The apparatus according to any one of claims 11 to 16, wherein the obtaining module is further configured to obtain an offline data set;

the processing module is further configured to train according to the offline data set to obtain the gaussian mixture model.

18. The apparatus according to claim 17, wherein the obtaining module is specifically configured to obtain off-line road structure prior information and off-line lane line feature points;

the processing module is specifically configured to convert the off-line lane line feature points and the off-line road structure prior information into the off-line data set in a road edge coordinate system when it is determined according to the road prior decision rule that the correlation between the off-line lane line feature points and the off-line road structure prior information reaches a preset threshold.

19. The apparatus according to claim 18, wherein the processing module is specifically configured to construct the gaussian mixture model according to the off-line lane characteristic points and the spatial distribution of the off-line road structure prior information, and solve to obtain model parameter values of the gaussian mixture model.

20. The apparatus of claim 19, wherein the processing module is further configured to update the model parameter values according to the valid lane line feature points.

21. A lane marking detection apparatus, comprising at least one processor and a memory, the processor being configured to be coupled to the memory, the processor invoking instructions stored in the memory to control the data processing apparatus to perform the method of any one of claims 1 to 10.

22. A computer storage medium storing computer instructions for performing the method of any one of claims 1 to 10.

23. A computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of any one of claims 1 to 10.

24. A smart car characterized in that it comprises the apparatus of any one of claims 11 to 20.