CN114863392A - Lane line detection method, device, vehicle and storage medium - Google Patents

Abstract

The present disclosure relates to a lane line detection method, a device, a vehicle and a storage medium. The lane line detection method obtains a target detection image and inputs the target detection image into a preset lane line detection model, so that the preset lane The line detection model outputs the target position of the lane line in the target detection image, wherein the preset lane line detection model is obtained by training a preset initial model including a segmentation network and a classification network, and the segmentation network is used to determine the lane line Detecting the first predicted position of the lane line in the sample image, the classification network is used to determine the second predicted position of the lane line in the lane line detection sample image, and using the preset lane line detection model to detect the lane line can effectively improve the lane accuracy of line detection results.

Description

Lane line detection method, device, vehicle and storage medium

technical field

The present disclosure relates to the technical field of intelligent networked vehicles, and in particular, to a lane line detection method, device, vehicle and storage medium.

Background technique

With the continuous development of science and technology, autonomous driving technology has gradually moved towards a development climax. Safety has always been the focus of autonomous driving technology. The environmental perception capability of the autonomous driving system plays a decisive role in safety performance, and lane line detection is a key link in environmental perception.

In the process of lane line detection, usually due to the dark light, reflection, occlusion, or the lane line being washed by rain for a long time, the lane line in the captured road image is blurred, and the scene where the lane line is located is ever-changing. Therefore, the detection of lane lines is difficult and the accuracy of detection results is low.

SUMMARY OF THE INVENTION

In order to overcome the problems existing in the related art, the present disclosure provides a lane line detection method, device, vehicle and storage medium.

According to a first aspect of the embodiments of the present disclosure, there is provided a lane line detection method, the method comprising:

Get the target detection image;

inputting the target detection image into a preset lane line detection model, so that the preset lane line detection model outputs the target position of the lane line in the target detection image;

Wherein, the preset lane line detection model is obtained by training in the following ways:

acquiring a plurality of lane line detection sample images, where the lane line detection images include lane line marked positions;

inputting each of the lane line detection sample images into a preset initial model, the preset initial model includes a segmentation network and a classification network, and the segmentation network is used to determine the first predicted position of the lane line in the lane line detection sample image, The classification network is used to determine the second predicted position of the lane line in the lane line detection sample image;

The preset initial model is trained according to the first predicted position, the second predicted position and the lane line marking position to obtain the preset lane line detection model.

Optionally, the preset initial model further includes a feature extraction network, the feature extraction network is coupled with the classification network and the segmentation network, and the preset initial model is used for

Obtain a multi-scale sample feature map corresponding to each of the lane line detection sample images through the feature extraction network, and determine a first sample feature map including global image information according to the multi-scale sample feature map;

Determine the first predicted position of the lane line in the lane line detection sample image according to the first sample feature map by the segmentation network;

The second predicted position of the lane line in the lane line detection sample image is determined by the classification network according to the multi-scale sample feature map.

Optionally, the preset lane line detection model is obtained by training in the following manner:

Obtain the multi-scale sample feature map corresponding to the lane line detection sample image through the feature extraction network in the preset initial model, and determine the first feature map including the global image information according to the minimum-scale sample feature map in the multi-scale sample feature map. A sample feature map;

Determine the first predicted position of the lane line in the lane line detection sample image according to the first sample feature map by the segmentation network;

determining a first loss value corresponding to the first loss function according to the first predicted position and the lane marking position;

Determine the second predicted position of the lane line in the lane line detection sample image according to the minimum scale sample feature map by the classification network;

determining a second loss value corresponding to the second loss function according to the second predicted position and the lane marking position;

determining a third loss value corresponding to the third loss function according to the first loss value and the second loss value;

In the case that the third loss value is greater than or equal to the preset loss value threshold, adjust the model parameters of the preset initial model to obtain an updated target model, and execute the process of passing the preset initial model again. The feature extraction network obtains the multi-scale sample feature map corresponding to the lane line detection sample image, and the step of determining the third loss value corresponding to the third loss function according to the first loss value and the second loss value, Until it is determined that the third loss value is smaller than the preset loss value threshold, the segmentation network in the current target model is deleted to obtain the preset lane line detection model.

Optionally, the feature extraction network includes a feature pyramid network and a global feature extraction network, the feature pyramid network includes a bottom-up sub-network and a top-down sub-network, and the input end of the global feature extraction network is the same as the The output end of the bottom-up sub-network is coupled, the output end of the global feature extraction network is coupled with the input end of the top-down sub-network, and the output end of the global feature extraction network is also coupled with the input end of the segmentation network coupling, the output end of the top-down sub-network is also coupled with the input end of the segmentation network, and the preset initial model is used for,

Obtain the multi-scale sample feature map corresponding to the lane line detection sample image through the bottom-up sub-network in the feature pyramid network, and input the minimum-scale sample feature map in the multi-scale sample feature map into the global feature an extraction network, so that the global feature extraction network outputs the first sample feature map to the segmentation network and the top-down sub-network;

A multi-scale second sample feature map is input to the segmentation network through the top-down sub-network, so that the segmentation network determines the lane according to the first sample feature map and the second sample feature map The first predicted position of the lane line in the line detection sample image.

Optionally, the segmentation network determines the first predicted position of the lane line in the lane line detection sample image according to the first sample feature map and the second sample feature map, including:

Perform up-sampling processing on the first sample feature map and the second sample feature map through the segmentation network to obtain a sample feature map to be segmented;

Segment the feature map of the sample to be segmented according to a preset segmentation method to obtain a plurality of local feature maps;

Performing convolution and splicing processing on the multiple local feature maps to obtain target sample feature maps;

Determine the first probability that each pixel in the lane line detection sample image belongs to the lane line according to the target sample feature map;

The first predicted position of the lane line in the lane line detection sample image is determined according to the first probability of each pixel.

Optionally, the dividing according to the preset manner includes:

The feature maps of the samples to be divided are divided into H local feature maps of C*W, the feature maps of the samples to be divided are divided into C local feature maps of H*W, and the feature maps of the samples to be divided are divided into At least one of W local feature maps of H*C, where H is the number of layers of the feature map of the sample to be segmented, C is the length of the feature map of the sample to be segmented, and W is the width of the feature map of the sample to be segmented.

Optionally, performing convolution and splicing processing on the multiple local feature maps to obtain target sample feature maps, including:

Obtain the current local feature map, and perform a convolution operation on the current local feature map to obtain the specified local feature map after convolution;

After splicing the specified local feature map and the next local feature map corresponding to the current local feature map, an updated current local feature map is obtained;

determining whether the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps;

In the case where it is determined that the next local feature map corresponding to the current local feature map is not the last one of the multiple local feature maps, obtain the current local feature map again, and perform convolution on the current local feature map operation, to obtain the specified local feature map after convolution, to the step of determining whether the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps;

In the case where it is determined that the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps, obtain the current local feature map, and perform a convolution operation on the current local feature map to obtain a volume The specified local feature map after product is used, and the specified local feature map corresponding to the current local feature map is used as the target sample feature map.

Optionally, the output end of the bottom-up sub-network is coupled to the input end of the classification network, and the preset initial model is used for:

Input the minimum scale sample feature map to the classification network through the bottom-up sub-network;

The minimum-scale sample feature map is divided into a plurality of anchor boxes by the classification network, a second probability that each anchor box belongs to the lane line is determined, and the lane line detection sample image is determined according to the second probability The second predicted location of the center lane line.

Optionally, inputting the target detection image into a preset lane line detection model, so that the preset lane line detection model outputs the target position of the lane line in the target detection image, including:

Obtain the target feature map corresponding to the target detection image through the feature extraction network;

The target feature map is input into the classification network to obtain the target position of the lane line in the target detection image output by the classification network.

According to a second aspect of the embodiments of the present disclosure, there is provided a lane line detection device, the device comprising:

an acquisition module, configured to acquire a target detection image;

a determination module, configured to input the target detection image into a preset lane line detection model, so that the preset lane line detection model outputs the target position of the lane line in the target detection image;

Wherein, the preset lane line detection model is obtained by training in the following ways:

acquiring a plurality of lane line detection sample images, where the lane line detection images include lane line marked positions;

inputting each of the lane line detection sample images into a preset initial model, the preset initial model includes a segmentation network and a classification network, and the segmentation network is used to determine the first predicted position of the lane line in the lane line detection sample image, The classification network is used to determine the second predicted position of the lane line in the lane line detection sample image;

The preset initial model is trained according to the first predicted position, the second predicted position and the lane line marking position to obtain the preset lane line detection model.

Optionally, the preset initial model further includes a feature extraction network, the feature extraction network is coupled with the classification network and the segmentation network, and the preset initial model is used for

Obtain a multi-scale sample feature map corresponding to each of the lane line detection sample images through the feature extraction network, and determine a first sample feature map including global image information according to the multi-scale sample feature map;

Determine the first predicted position of the lane line in the lane line detection sample image according to the first sample feature map by the segmentation network;

The second predicted position of the lane line in the lane line detection sample image is determined by the classification network according to the multi-scale sample feature map.

Optionally, the preset lane line detection model is obtained by training in the following manner:

Obtain the multi-scale sample feature map corresponding to the lane line detection sample image through the feature extraction network in the preset initial model, and determine the first feature map including the global image information according to the minimum-scale sample feature map in the multi-scale sample feature map. A sample feature map;

Determine the first predicted position of the lane line in the lane line detection sample image according to the first sample feature map by the segmentation network;

determining a first loss value corresponding to the first loss function according to the first predicted position and the lane marking position;

Determine the second predicted position of the lane line in the lane line detection sample image according to the minimum scale sample feature map by the classification network;

determining a second loss value corresponding to the second loss function according to the second predicted position and the lane marking position;

determining a third loss value corresponding to the third loss function according to the first loss value and the second loss value;

In the case that the third loss value is greater than or equal to the preset loss value threshold, adjust the model parameters of the preset initial model to obtain an updated target model, and execute the process of passing the preset initial model again. The feature extraction network obtains the multi-scale sample feature map corresponding to the lane line detection sample image, and the step of determining the third loss value corresponding to the third loss function according to the first loss value and the second loss value, Until it is determined that the third loss value is smaller than the preset loss value threshold, the segmentation network in the current target model is deleted to obtain the preset lane line detection model.

Optionally, the feature extraction network includes a feature pyramid network and a global feature extraction network, the feature pyramid network includes a bottom-up sub-network and a top-down sub-network, and the input end of the global feature extraction network is the same as the The output end of the bottom-up sub-network is coupled, the output end of the global feature extraction network is coupled with the input end of the top-down sub-network, and the output end of the global feature extraction network is also coupled with the input end of the segmentation network coupling, the output end of the top-down sub-network is also coupled with the input end of the segmentation network, and the preset initial model is used for,

Obtain the multi-scale sample feature map corresponding to the lane line detection sample image through the bottom-up sub-network in the feature pyramid network, and input the minimum-scale sample feature map in the multi-scale sample feature map into the global feature an extraction network, so that the global feature extraction network outputs the first sample feature map to the segmentation network and the top-down sub-network;

A multi-scale second sample feature map is input to the segmentation network through the top-down sub-network, so that the segmentation network determines the lane according to the first sample feature map and the second sample feature map The first predicted position of the lane line in the line detection sample image.

Optionally, the determining, by the segmentation network, the first predicted position of the lane line in the lane line detection sample image according to the first sample feature map and the second sample feature map includes:

Perform up-sampling processing on the first sample feature map and the second sample feature map through the segmentation network to obtain a sample feature map to be segmented;

Segment the feature map of the sample to be segmented according to a preset segmentation method to obtain a plurality of local feature maps;

Performing convolution and splicing processing on the multiple local feature maps to obtain target sample feature maps;

Determine the first probability that each pixel in the lane line detection sample image belongs to the lane line according to the target sample feature map;

The first predicted position of the lane line in the lane line detection sample image is determined according to the first probability of each pixel.

Optionally, the dividing according to the preset manner includes:

The feature maps of the samples to be divided are divided into H local feature maps of C*W, the feature maps of the samples to be divided are divided into C local feature maps of H*W, and the feature maps of the samples to be divided are divided into At least one of W local feature maps of H*C, where H is the number of layers of the feature map of the sample to be segmented, C is the length of the feature map of the sample to be segmented, and W is the width of the feature map of the sample to be segmented.

Optionally, performing convolution and splicing processing on the multiple local feature maps to obtain target sample feature maps, including:

Obtain the current local feature map, and perform a convolution operation on the current local feature map to obtain the specified local feature map after convolution;

After splicing the specified local feature map and the next local feature map corresponding to the current local feature map, an updated current local feature map is obtained;

determining whether the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps;

In the case where it is determined that the next local feature map corresponding to the current local feature map is not the last one of the multiple local feature maps, obtain the current local feature map again, and perform convolution on the current local feature map operation, to obtain the specified local feature map after convolution, to the step of determining whether the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps;

In the case where it is determined that the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps, obtain the current local feature map, and perform a convolution operation on the current local feature map to obtain a volume The specified local feature map after product is used, and the specified local feature map corresponding to the current local feature map is used as the target sample feature map.

Optionally, the output end of the bottom-up sub-network is coupled to the input end of the classification network, and the preset initial model is used for:

Input the minimum scale sample feature map to the classification network through the bottom-up sub-network;

The minimum-scale sample feature map is divided into a plurality of anchor boxes by the classification network, a second probability that each anchor box belongs to the lane line is determined, and the lane line detection sample image is determined according to the second probability The second predicted location of the center lane line.

Optionally, the determining module is configured to:

Obtain the target feature map corresponding to the target detection image through the feature extraction network;

The target feature map is input into the classification network to obtain the target position of the lane line in the target detection image output by the classification network.

According to a third aspect of the embodiments of the present disclosure, there is provided a vehicle including the lane line detection device described in the second aspect above.

According to a fourth aspect of the embodiments of the present disclosure, there is provided a lane line detection device, including:

a memory on which a computer program is stored;

A processor for executing the computer program in the memory to implement the steps of the method in the first aspect above.

According to a fifth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium having computer program instructions stored thereon, the program instructions implementing the steps of the method described in the first aspect of the present disclosure when the program instructions are executed by a processor.

The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects: obtaining a target detection image; inputting the target detection image into a preset lane line detection model, so that the preset lane line detection model outputs the target Detecting the target position of the lane line in the image, wherein the preset lane line detection model is obtained by training a preset initial model including a segmentation network and a classification network, and performing lane line detection through the preset lane line detection model can effectively Improve the accuracy of lane line detection results.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure.

FIG. 1 is a flowchart of a lane line detection method according to an exemplary embodiment of the present disclosure;

2 is a block diagram of a model structure of a preset initial model shown in an exemplary implementation of the present disclosure;

FIG. 3 is a schematic diagram of the principle of dividing a network according to an exemplary embodiment of the present disclosure;

FIG. 4 is a training flow chart of a preset lane line detection model according to an exemplary embodiment of the present disclosure;

5 is a flowchart of a method for detecting lane lines according to the embodiment shown in FIG. 4;

FIG. 6 is a block diagram of a lane line detection apparatus according to an exemplary embodiment of the present disclosure;

Fig. 7 is a block diagram of a lane line detection apparatus according to an exemplary embodiment.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as recited in the appended claims.

It should be noted that all the actions of obtaining signals, information or data in this application are carried out under the premise of complying with the corresponding data protection regulations and policies of the local country and with the authorization of the corresponding device owner.

Before introducing the specific embodiments of the present disclosure in detail, first, the application scenarios of the present disclosure are described below. The present disclosure can be applied to the process of lane line detection, and the lane line detection can be in the process of automatic driving, unmanned driving or route navigation. An important part of lane recognition when perceiving the driving environment. At present, lane recognition methods are mainly divided into two categories, image feature-based methods and model-based methods. The principle of image feature-based recognition is to use the difference between the characteristics of the edge of the lane marking line and the surrounding road conditions of the road image. The usual feature difference There are image texture, edge geometry, boundary continuity, gray value and contrast of the region of interest, etc., and threshold segmentation and other methods can be used to extract image features. The algorithm is simple and easy to implement, but the disadvantage is that it is easily blocked by noise and shadows of trees. , illumination changes, discontinuous marking lines and other factors interfere with the low accuracy of the lane line detection results and the poor reliability of the lane recognition results. The principle based on model matching is to establish the corresponding lane line model according to the geometric features of the structured road, identify the parameters of the road model, and then identify the lane markings. Due to the limitations of the image feature method, in practical research, the image feature method and the model matching method are generally combined to establish a suitable lane line detection model, which can improve the accuracy of lane line recognition on the one hand, and strengthen the system on the other hand. However, there is currently no perfect model that can guarantee the accuracy of lane line detection results in various detection scenarios.

The present disclosure provides a lane line detection method, device, vehicle and storage medium. The lane line detection method obtains a target detection image and inputs the target detection image into a preset lane line detection model, so that the preset lane line The detection model outputs the target position of the lane line in the target detection image, wherein the preset lane line detection model is trained by a preset initial model including a segmentation network and a classification network, and the lane line detection is performed through the preset lane line detection model , which can effectively improve the accuracy of lane line detection results in various detection scenarios.

The technical solutions of the present disclosure will be described in detail below with reference to specific embodiments.

FIG. 1 is a flowchart of a method for detecting lane lines according to an exemplary embodiment of the present disclosure; as shown in FIG. 1 , the method for detecting lane lines may include:

Step 101, acquiring a target detection image.

Wherein, the target detection image may be any image to be detected in various lane line detection scenarios (for example, the detection scene of straight lane lines, the detection scene of hyperbolic lane lines, the detection scene of B-spline curve lane lines, etc.), The to-be-detected image may include to-be-detected lane lines.

Step 102: Input the target detection image into a preset lane line detection model, so that the preset lane line detection model outputs the target position of the lane line in the target detection image.

In this step, the preset lane line detection model is obtained by training the following methods shown in S11 to S13:

S11, acquiring multiple lane line detection sample images.

Wherein, the lane line detection image includes the lane line marked position.

S12, input each of the lane line detection sample images into a preset initial model, where the preset initial model includes a segmentation network and a classification network, where the segmentation network is used to determine the first predicted position of the lane line in the lane line detection sample image, the The classification network is used to determine the second predicted location of the lane line in the lane line detection sample image.

The segmentation network may include multiple convolution layers and classification layers, and the number of convolution kernels corresponding to different convolution layers is different. The segmentation network is used to divide the feature map corresponding to the lane line detection sample image into multiple local feature maps , and then perform convolution and splicing processing on the multiple local feature maps to obtain the target sample feature map, and then input the target sample feature map into the classification layer, so that the classification layer outputs the lane line detection sample image of the lane line in the sample image. Second predicted location.

It should be noted that the segmentation network is divided by feature maps, and the convolution operation on the divided local feature maps can effectively obtain more feature map details, and then classify the feature maps containing more details, which can effectively improve the performance. The accuracy of the classification results can further improve the accuracy of the lane line detection results. The segmentation network can divide the feature map into multiple local feature maps from different directions of the feature map. For example, a The feature map of H*C*W is cut into H local feature maps of C*W, C local feature maps of H*W, or W local feature maps of H*C, where H is the layer of the feature map number, C is the length of the feature map, and W is the width of the feature map. In addition, the classification layer in the segmentation network may be a fully connected layer, or may be other network modules for classification. There are many network modules with classification functions in the prior art, which are not limited in the present disclosure.

S13: Train the preset initial model according to the first predicted position, the second predicted position and the lane marking position to obtain the preset lane line detection model.

It should be noted that, in the training process, the preset lane line detection model is obtained by training a preset initial model including the classification network and the segmentation network, but in the application process, the preset lane line detection model may include The segmentation network and the classification network may also include the classification network, excluding the segmentation network, or may include the segmentation network, excluding the classification network.

In the above technical solution, by acquiring a target detection image; inputting the target detection image into a preset lane line detection model, so that the preset lane line detection model outputs the target position of the lane line in the target detection image, wherein the preset lane line detection model The lane line detection model is obtained by training a preset initial model including a segmentation network and a classification network. Using the preset lane line detection model for lane line detection can effectively improve the accuracy of lane line detection results in various detection scenarios.

FIG. 2 is a block diagram of a model structure of a preset initial model shown in an exemplary implementation of the present disclosure. As shown in FIG. 2 , the preset initial model further includes a feature extraction network, the feature extraction network, the classification network and the Split network coupling, the preset initial model for:

Obtain a multi-scale sample feature map corresponding to each lane line detection sample image through the feature extraction network, and determine a first sample feature map including global image information according to the multi-scale sample feature map;

Determine the first predicted position of the lane line in the lane line detection sample image according to the first sample feature map by the segmentation network;

The second predicted position of the lane line in the lane line detection sample image is determined by the classification network according to the multi-scale sample feature map.

Wherein, the feature extraction network may include a feature pyramid network and a global feature extraction network, the feature pyramid network includes a bottom-up sub-network and a top-down sub-network, and the global feature extraction network may be the Encoder (encoder) in the Transformer network. ), the input of the global feature extraction network is coupled with the output of the bottom-up sub-network, the output of the global feature extraction network is coupled with the input of the top-down sub-network, and the output of the global feature extraction network Also coupled with the input of the segmentation network, the output of the top-down sub-network is also coupled with the input of the segmentation network, the output of the bottom-up sub-network is coupled with the input of the classification network, the preset The initial model is used to:

The multi-scale sample feature map corresponding to the lane line detection sample image is obtained through the bottom-up sub-network in the feature pyramid network (feature map x ₁ , feature map x ₂ , feature map x ₃ , feature map x in Figure 2 ) ₄ ), and input the minimum-scale sample feature map in the multi-scale sample feature map (feature map x ₄ in Figure 2) into the global feature extraction network, so that the global feature extraction network can contribute to the segmentation network and the self The top-down sub-network outputs the first sample feature map (as shown in Figure 2, the feature map x ₄ is converted into a feature map X ₁ through the global feature extraction network);

Input multi-scale second sample feature maps (such as feature map X ₁ , feature map X ₂ , feature map X ₃ , feature map X ₄ in FIG. 2 ) to the segmentation network through the top-down sub-network, so that the The segmentation network determines the first predicted position of the lane line in the lane line detection sample image according to the first sample feature map and the second sample feature map.

It should be noted that, in the feature pyramid network, the feature map located in the low layer (shallow layer) has rich detailed information, and the feature map located in the high layer (deep layer) has rich semantic information. The segmentation network is based on the first sample feature map. The implementation process of determining the first predicted position of the lane line in the lane line detection sample image with the second sample feature map may include the steps shown in the following S21 to S25:

S21 , performing up-sampling processing on the first sample feature map and the second sample feature map through the segmentation network to obtain a sample feature map to be segmented.

For example, the multi-scale second sample feature map may include a feature map X ₁ , a feature map X ₂ , a feature map X ₃ and a feature map X ₄ as shown in FIG. 2 . For the feature map X ₁ , the feature map X ₂ and the feature map X ₃ are respectively upsampled to obtain a feature map with the same size as the feature map X ₄ , and the feature map X ₁ , the feature map X ₂ and the feature map X ₃ corresponding to the feature map The feature map with the same size as X ₄ is subjected to concat (splicing) processing with the feature map X ₄ to obtain the feature map of the sample to be segmented.

S22, segment the feature map of the sample to be segmented according to a preset segmentation method to obtain a plurality of local feature maps.

The pre-set segmentation method includes: dividing the feature map of the sample to be divided into H local feature maps of C*W, dividing the feature map of the sample to be divided into C local feature maps of H*W, and dividing the feature map of the sample to be divided into C local feature maps of H*W. The feature map of the sample to be divided is divided into at least one of W local feature maps of H*C, where H is the number of layers of the feature map of the sample to be divided, C is the length of the feature map of the sample to be divided, and W is the sample to be divided. The width of the feature map.

For example, as shown in FIG. 3, FIG. 3 is a schematic diagram of the principle of a segmentation network shown in an exemplary embodiment of the present disclosure. Segmentation is performed from four directions, that is, the H*C*W feature map of the sample to be segmented is regarded as a Cube, along the height of the cube (corresponding to H) from top to bottom, divide the H*C*W feature map into H C*W local feature maps (SCNN_D in Figure 3), and then follow the The height of the cube is from bottom to top, and the feature map of H*C*W is divided into another H local feature maps of C*W (SCNN_U in Figure 3), along the width of the cube (corresponding to W) from left to Right, divide the feature map of H*C*W into W local feature maps of H*C (SCNN_R in Figure 3), along the width of the cube (corresponding to W) from right to left, divide H*C The feature map of *W is divided into another W local feature maps of H*C (SCNN_L in Figure 3).

S23, performing convolution and splicing processing on the plurality of local feature maps to obtain a target sample feature map.

A possible implementation in this step is:

Obtain the current local feature map, and perform a convolution operation on the current local feature map to obtain the specified local feature map after convolution; after splicing the specified local feature map with the next local feature map corresponding to the current local feature map , obtain the updated current local feature map; determine whether the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps; determine the next local feature map corresponding to the current local feature map If the picture is not the last one of the multiple local feature maps, perform the acquisition of the current local feature map again, and perform the convolution operation on the current local feature map to obtain the convolved specified local feature map, until the current local feature map is determined. The step of determining whether the next local feature map corresponding to the local feature map is the last one of the multiple local feature maps; after determining that the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps In this case, obtain the current local feature map, and perform convolution operation on the current local feature map to obtain the specified local feature map after convolution, and use the specified local feature map corresponding to the current local feature map as the target sample feature. picture.

Illustratively, still taking the example shown in FIG. 3 as an example, after obtaining H local feature maps of C*W from top to bottom (for example, C*W-1, C*W-2 to C*W-H), The convolution operation can be performed on the local feature map C*W-1 (the convolution kernel is 1*1), and then the convolution result corresponding to the local feature map C*W-1 (the specified local corresponding to C*W-1) feature map) and the local feature map C*W-2 to obtain the current local feature map after splicing, and then perform a convolution operation on the current local feature map (the convolution kernel is 2 1*1) to obtain The specified local feature map corresponding to the C*W-2 is spliced with the specified local feature map corresponding to the local feature map C*W-2 and the local feature map C*W-3 to obtain the updated current local feature map, This cycle is performed until the specified local feature map corresponding to the C*W-H is obtained, wherein the dimension of the specified local feature map corresponding to the C*W-H is H*C*W, and then the specified local feature map corresponding to C*W-H (also A feature map of H*C*W) along the high direction, slice from bottom to top to obtain H local feature maps of C*W from bottom to top (for example, C*W+1, C*W+ 2 to C*W+H), for the C*W+1, C*W+2 to C*W+H and then perform convolution splicing operation to obtain the specified local feature map corresponding to C*W+H (dimension is H*C*W), and then the specified local feature map corresponding to C*W+H follows the width of the cube (corresponding to W) from left to right, and divides the feature map of H*C*W into W Local feature maps of H*C to obtain W local feature maps of H*C from left to right (for example, H*C-1, H*C-2 to H*C-W), for the H*C- 1. Perform the preset convolution splicing operation from H*C-2 to H*C-W to obtain the specified local feature map corresponding to H*C-W (dimension is H*C*W), and then the specified local corresponding to H*C-W. The feature map is from right to left along the width of the cube (corresponding to W), and the feature map of H*C*W is divided into local feature maps from right to left (for example, H*C+1, H*C+2 to H*C+W), perform a preset convolution splicing operation on the H*C+1, H*C+2 to H*C+W, obtain the specified local feature map corresponding to the H*C+W, and use the The specified local feature map corresponding to H*C+W is used as the feature map of the target sample.

S24: Determine a first probability that each pixel in the lane line detection sample image belongs to a lane line according to the target sample feature map.

In this step, since the target sample feature map includes more detailed information in the lane line detection sample image, each pixel in the lane line detection sample image obtained according to the target sample feature map belongs to the first lane line detection sample image. Probability is more accurate.

S25: Determine the first predicted position of the lane line in the lane line detection sample image according to the first probability of each pixel.

In this step, the pixel position of which the first probability is greater than or equal to the first preset probability threshold may be used as the first predicted position of the lane line in the lane line detection sample image.

Through the above S21 to S25, the target sample feature map with more detailed information can be obtained through the segmentation network, and then the first probability that each pixel in the lane line detection sample image belongs to the lane line is determined through the target sample feature map. Determining the first predicted position with the first probability can effectively improve the accuracy of the first predicted position.

Optionally, the preset initial model is also used for:

Input the minimum scale sample feature map (feature map x ₄ in FIG. 2 ) to the classification network through the bottom-up sub-network;

The minimum-scale sample feature map is divided into multiple anchor boxes through the classification network, the second probability that each anchor box belongs to the lane line is determined, and the first probability of the lane line in the lane line detection sample image is determined according to the second probability. 2. Predict the location.

It should be noted that there are many ways to divide the feature map x ₄ into anchor boxes in the prior art, and it is relatively easy to obtain, and the present disclosure will not repeat them here. The classification network (which can be a fully connected layer) determines each The specific details of the second probability that the anchor box belongs to the lane line belongs to a common calculation process in the art, which is not limited in the present disclosure. In addition, here, the pixel position corresponding to the anchor frame whose second probability is greater than or equal to the second preset probability threshold may be used as the second predicted position.

With the above technical solution, the second predicted position of the lane line in the lane line detection sample image can be obtained through the classification network, because the second predicted position is the minimum scale sample feature output by FPN (Feature Pyramid Networks, Feature Pyramid Network). As shown in the figure, the minimum scale sample feature contains rich contextual feature information, so it can also effectively improve the accuracy of the second predicted position.

FIG. 4 is a training flow chart of a preset lane line detection model shown in an exemplary embodiment of the present disclosure; as shown in FIG. 4 , the preset lane line detection model can be obtained by training through the following steps:

Step 401: Acquire a plurality of lane line detection sample images, where the lane line detection images include lane line marked positions.

Wherein, the lane line detection sample image may be any image in various lane line detection scenarios (such as the detection scene of straight lane lines, the detection scene of hyperbolic lane lines, the detection scene of B-spline curve lane lines, etc.), The lane line position can be marked in the lane line detection sample image.

Step 402: Obtain the multi-scale sample feature map corresponding to the lane line detection sample image through the feature extraction network in the preset initial model, and determine the first sample including the global image information according to the minimum-scale sample feature map in the multi-scale sample feature map. A sample feature map.

Wherein, the feature extraction network may include a feature pyramid network and a global feature extraction network, the feature pyramid network includes a bottom-up sub-network and a top-down sub-network, and the global feature extraction network may be the Encoder (encoder) in the Transformer network. ), obtain the multi-scale sample feature map corresponding to the lane line detection sample image through the bottom-up sub-network in the feature pyramid network (feature map x ₁ , feature map x ₂ , feature map x 3 , feature map in Figure 2 , feature map x ₃ , feature map Figure x ₄ ), and input the minimum scale sample feature map in the multi-scale sample feature map (feature map x ₄ in FIG. 2 ) into the global feature extraction network, so that the global feature extraction network is directed to the segmentation network and The top-down sub-network outputs the first sample feature map.

Step 403 , determining the first predicted position of the lane line in the lane line detection sample image through the segmentation network according to the first sample feature map.

In this step, the first sample feature map is input into the top-down sub-network, so that the top-down sub-network inputs a multi-scale second sample feature map to the segmentation network (feature map X in FIG. 2 ). _1. The feature map X ₂ , the feature map X ₃ , the feature map X ₄ ), so that the segmentation network determines the number of lane lines in the lane line detection sample image according to the first sample feature map and the second sample feature map. a predicted location.

Wherein, the specific implementation of the segmentation network determining the first predicted position of the lane line in the lane line detection sample image according to the first sample feature map and the second sample feature map may refer to the steps shown in the above S21 to S25, The present disclosure will not be repeated here.

Step 404: Determine a first loss value corresponding to the first loss function according to the first predicted position and the marked position of the lane line.

The first loss function may be a cross-entropy loss function, a Focal Loss (focal loss) loss function, or other loss functions in the prior art.

Step 405: Determine the second predicted position of the lane line in the lane line detection sample image through the classification network according to the minimum scale sample feature map.

Step 406: Determine a second loss value corresponding to the second loss function according to the second predicted position and the marked position of the lane line.

The second loss function may also be a cross-entropy loss function or a Focal Loss loss function, and may also be other loss functions in the prior art.

Step 407: Determine a third loss value corresponding to the third loss function according to the first loss value and the second loss value.

For example, the third loss function may be L _total =αL _cls +βL _seg , where L _total represents the third loss value, and α and β are weighting coefficients. L _cls is the second loss value corresponding to the second loss function, and L _seg is the first loss value corresponding to the first loss function.

Step 408: Determine whether the third loss value is greater than or equal to a preset loss value threshold.

In this step, when the third loss value is greater than or equal to the preset loss value threshold, step 409 is performed, and when it is determined that the third loss value is less than the preset loss value threshold, step 410 is performed.

Step 409: Adjust the model parameters of the preset initial model to obtain an updated target model.

After this step, it is possible to jump to step 402 again to perform steps 402 to 408 again.

Step 410, delete the segmentation network in the current target model to obtain the preset lane line detection model.

In the above technical solution, the preset lane line detection model is jointly trained by the segmentation network and the classification network, which can effectively improve the accuracy of the model detection results. In order to improve the processing speed of the model, the segmentation network can be deleted in the application stage, and only the The feature extraction network and the classification network can not only ensure the accuracy of the lane line detection results, but also effectively improve the detection efficiency of the model.

Optionally, when determining the target position of the lane line in the target detection image by the preset lane line detection model obtained by the steps shown in FIG. 4 above, the following steps shown in FIG. 5 may be included, and FIG. A flow chart of a lane line detection method illustrated; as shown in Figure 5, the method includes:

Step 501, acquiring a target detection image.

Wherein, the target detection image may be any image to be detected in various lane line detection scenarios (for example, the detection scene of straight lane lines, the detection scene of hyperbolic lane lines, the detection scene of B-spline curve lane lines, etc.), The to-be-detected image may include to-be-detected lane lines.

Step 502: Obtain a target feature map corresponding to the target detection image through the feature extraction network.

Wherein, the feature extraction network may include a feature pyramid network, and the feature pyramid network includes a bottom-up sub-network and a top-down sub-network, and the target feature map may be the minimum-scale feature map input by the bottom-up sub-network (as shown in Fig. Feature maps in 2 x ₄ ).

Step 503: Input the target feature map into the classification network to obtain the target position of the lane line in the target detection image output by the classification network.

Determining the target position of the lane line in the target detection image by the methods shown in the above steps 501 to 503 can not only ensure the accuracy of the lane line detection result, but also effectively improve the detection efficiency of the model.

FIG. 6 is a block diagram of a lane line detection apparatus according to an exemplary embodiment of the present disclosure. As shown in FIG. 6 , the apparatus may include:

an acquisition module 601, configured to acquire a target detection image;

The determining module 602 is configured to input the target detection image into a preset lane line detection model, so that the preset lane line detection model outputs the target position of the lane line in the target detection image;

Among them, the preset lane line detection model is obtained by training in the following ways:

Acquiring a plurality of lane line detection sample images, the lane line detection images including the lane line marking positions;

Each of the lane line detection sample images is input into a preset initial model, the preset initial model includes a segmentation network and a classification network, the segmentation network is used to determine the first predicted position of the lane line in the lane line detection sample image, the classification network for determining the second predicted position of the lane line in the lane line detection sample image;

According to the first predicted position, the second predicted position and the lane marking position, the preset initial model is trained to obtain the preset lane line detection model.

In the above technical solution, by acquiring a target detection image; inputting the target detection image into a preset lane line detection model, so that the preset lane line detection model outputs the target position of the lane line in the target detection image, wherein the preset lane line detection model The lane line detection model is trained by a preset initial model including a segmentation network and a classification network, which can effectively improve the accuracy of lane line detection results in various detection scenarios.

Optionally, the preset initial model further includes a feature extraction network, the feature extraction network is coupled with the classification network and the segmentation network, and the preset initial model is used for

Obtain a multi-scale sample feature map corresponding to each lane line detection sample image through the feature extraction network, and determine a first sample feature map including global image information according to the multi-scale sample feature map;

Determine the first predicted position of the lane line in the lane line detection sample image according to the first sample feature map by the segmentation network;

The second predicted position of the lane line in the lane line detection sample image is determined by the classification network according to the multi-scale sample feature map.

Optionally, the preset lane line detection model is obtained by training in the following manner:

The multi-scale sample feature map corresponding to the lane line detection sample image is obtained through the feature extraction network in the preset initial model, and the first sample including the image global information is determined according to the minimum-scale sample feature map in the multi-scale sample feature map. feature map;

Determine the first predicted position of the lane line in the lane line detection sample image according to the first sample feature map by the segmentation network;

Determine a first loss value corresponding to the first loss function according to the first predicted position and the lane marking position;

Determine the second predicted position of the lane line in the lane line detection sample image according to the minimum scale sample feature map by the classification network;

Determine a second loss value corresponding to the second loss function according to the second predicted position and the lane marking position;

Determine a third loss value corresponding to the third loss function according to the first loss value and the second loss value;

In the case that the third loss value is greater than or equal to the preset loss value threshold, adjust the model parameters of the preset initial model to obtain an updated target model, and execute the feature extraction network in the preset initial model again. Obtain the multi-scale sample feature map corresponding to the lane line detection sample image, to the step of determining the third loss value corresponding to the third loss function according to the first loss value and the second loss value, until the third loss is determined. When the value is less than the preset loss value threshold, delete the segmentation network in the current target model to obtain the preset lane line detection model.

Optionally, the feature extraction network includes a feature pyramid network and a global feature extraction network, the feature pyramid network includes a bottom-up sub-network and a top-down sub-network, and the input end of the global feature extraction network is connected to the bottom-up sub-network. The output end of the network is coupled, the output end of the global feature extraction network is coupled with the input end of the top-down sub-network, the output end of the global feature extraction network is also coupled with the input end of the segmentation network, and the top-down sub-network is coupled The output of is also coupled to the input of the segmentation network, and the preset initial model is used to,

The multi-scale sample feature map corresponding to the lane line detection sample image is obtained through the bottom-up sub-network in the feature pyramid network, and the minimum-scale sample feature map in the multi-scale sample feature map is input into the global feature extraction network to obtain causing the global feature extraction network to output the first sample feature map to the segmentation network and the top-down sub-network;

The multi-scale second sample feature map is input to the segmentation network through the top-down sub-network, so that the segmentation network determines the lane in the lane line detection sample image according to the first sample feature map and the second sample feature map The first predicted position of the line.

Optionally, the segmentation network determines the first predicted position of the lane line in the lane line detection sample image according to the first sample feature map and the second sample feature map, including:

Perform up-sampling processing on the first sample feature map and the second sample feature map through the segmentation network to obtain the sample feature map to be segmented;

Segment the feature map of the sample to be segmented according to a preset segmentation method to obtain a plurality of local feature maps;

Perform convolution and splicing processing on the multiple local feature maps to obtain the target sample feature map;

Determine the first probability that each pixel in the lane line detection sample image belongs to the lane line according to the target sample feature map;

The first predicted position of the lane line in the lane line detection sample image is determined according to the first probability of each pixel.

Optionally, the preset division method includes:

The feature map of the sample to be divided is divided into H local feature maps of C*W, the feature map of the sample to be divided is divided into C local feature maps of H*W, and the feature map of the sample to be divided is divided into W H*W local feature maps *At least one of the local feature maps of C, where H is the number of layers of the feature map of the sample to be segmented, C is the length of the feature map of the sample to be segmented, and W is the width of the feature map of the sample to be segmented.

Optionally, performing convolution and splicing processing on the multiple local feature maps to obtain target sample feature maps, including:

Obtain the current local feature map, and perform a convolution operation on the current local feature map to obtain the specified local feature map after convolution;

After splicing the specified local feature map with the next local feature map corresponding to the current local feature map, an updated current local feature map is obtained;

determining whether the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps;

When it is determined that the next local feature map corresponding to the current local feature map is not the last one of the multiple local feature maps, obtain the current local feature map again, and perform a convolution operation on the current local feature map to obtain Obtaining the specified local feature map after convolution, to the step of determining whether the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps;

When it is determined that the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps, obtain the current local feature map, and perform a convolution operation on the current local feature map to obtain the convolutional The specified local feature map of , and the specified local feature map corresponding to the current local feature map is used as the target sample feature map.

Optionally, the output end of the bottom-up sub-network is coupled with the input end of the classification network, and the preset initial model is used for:

Input the minimum scale sample feature map to the classification network through the bottom-up sub-network;

The minimum-scale sample feature map is divided into multiple anchor boxes through the classification network, the second probability that each anchor box belongs to the lane line is determined, and the first probability of the lane line in the lane line detection sample image is determined according to the second probability. 2. Predict the location.

Optionally, the determining module is configured to:

Obtain the target feature map corresponding to the target detection image through the feature extraction network;

The target feature map is input into the classification network to obtain the target position of the lane line in the target detection image output by the classification network.

In the above technical solution, the preset lane line detection model is jointly trained by the segmentation network and the classification network, which can effectively improve the accuracy of the model detection results. In order to improve the processing speed of the model, the segmentation network can be deleted in the application stage, and only the The feature extraction network and the classification network can not only ensure the accuracy of the lane line detection results, but also effectively improve the detection efficiency of the model.

Regarding the apparatus in the above-mentioned embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be described in detail here.

The present disclosure also provides a vehicle including the lane line detection device described above in FIG. 6 .

The present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and when the program instructions are executed by a processor, implement the steps of the lane line detection method provided by the present disclosure.

Fig. 7 is a block diagram of a lane line detection apparatus according to an exemplary embodiment. For example, apparatus 800 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, and the like.

7, the apparatus 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812, a sensor component 814, and communication component 816.

The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the above-described lane line detection method. Additionally, processing component 802 may include one or more modules that facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

Memory 804 is configured to store various types of data to support operations at device 800 . Examples of such data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, and the like. Memory 804 may be implemented by any type of volatile or nonvolatile storage device or combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.

Power component 806 provides power to various components of device 800 . Power components 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to device 800 .

Multimedia component 808 includes screens that provide an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensor can sense not only the boundaries of the touch or swipe action, but also the duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 808 includes a front-facing camera and/or a rear-facing camera. When the apparatus 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each of the front and rear cameras can be a fixed optical lens system or have focal length and optical zoom capability.

Audio component 810 is configured to output and/or input audio signals. For example, audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when device 800 is in operating modes, such as call mode, recording mode, and voice recognition mode. The received audio signal may be further stored in memory 804 or transmitted via communication component 816 . In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and a peripheral interface module, which may be a keyboard, a click wheel, a button, or the like. These buttons may include, but are not limited to: home button, volume buttons, start button, and lock button.

Sensor assembly 814 includes one or more sensors for providing status assessment of various aspects of device 800 . For example, the sensor assembly 814 can detect the open/closed state of the device 800, the relative positioning of the components, such as the display and keypad of the device 800, the sensor assembly 814 can also detect the position change of the device 800 or a component of the device 800, Presence or absence of user contact with device 800 , device 800 orientation or acceleration/deceleration and temperature changes of device 800 . Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

Communication component 816 is configured to facilitate wired or wireless communication between apparatus 800 and other devices. Device 800 may access wireless networks based on communication standards, such as WiFi, 2G or 3G, or a combination thereof. In one exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, apparatus 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components are implemented for implementing the above lane line detection method.

In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as a memory 804 including instructions, is also provided, and the instructions are executable by the processor 820 of the apparatus 800 to complete the lane line detection method described above. For example, the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

In another exemplary embodiment, there is also provided a computer program product comprising a computer program executable by a programmable apparatus, the computer program having, when executed by the programmable apparatus, for performing the above The code section of the lane line detection method.

Other embodiments of the present disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the present disclosure. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or techniques in the technical field not disclosed by the present disclosure . The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims (13)

1. a lane line detection method, is characterized in that, described method comprises: Get the target detection image; inputting the target detection image into a preset lane line detection model, so that the preset lane line detection model outputs the target position of the lane line in the target detection image; Wherein, the preset lane line detection model is obtained by training in the following ways: acquiring a plurality of lane line detection sample images, where the lane line detection images include lane line marked positions; inputting each of the lane line detection sample images into a preset initial model, the preset initial model includes a segmentation network and a classification network, and the segmentation network is used to determine the first predicted position of the lane line in the lane line detection sample image, The classification network is used to determine the second predicted position of the lane line in the lane line detection sample image; The preset initial model is trained according to the first predicted position, the second predicted position and the lane line marking position to obtain the preset lane line detection model. 2. The method according to claim 1, wherein the preset initial model further comprises a feature extraction network, the feature extraction network is coupled with the classification network and the segmentation network, and the preset initial model Used for: Obtain a multi-scale sample feature map corresponding to each of the lane line detection sample images through the feature extraction network, and determine a first sample feature map including global image information according to the multi-scale sample feature map; Determine the first predicted position of the lane line in the lane line detection sample image according to the first sample feature map by the segmentation network; The second predicted position of the lane line in the lane line detection sample image is determined by the classification network according to the multi-scale sample feature map. 3. The method according to claim 2, wherein the preset lane line detection model is obtained by training in the following manner: Obtain the multi-scale sample feature map corresponding to the lane line detection sample image through the feature extraction network in the preset initial model, and determine the first feature map including the global image information according to the minimum-scale sample feature map in the multi-scale sample feature map. A sample feature map; Determine the first predicted position of the lane line in the lane line detection sample image according to the first sample feature map by the segmentation network; determining a first loss value corresponding to the first loss function according to the first predicted position and the lane marking position; Determine the second predicted position of the lane line in the lane line detection sample image according to the minimum scale sample feature map by the classification network; determining a second loss value corresponding to the second loss function according to the second predicted position and the lane marking position; determining a third loss value corresponding to the third loss function according to the first loss value and the second loss value; In the case that the third loss value is greater than or equal to the preset loss value threshold, adjust the model parameters of the preset initial model to obtain an updated target model, and execute the process of passing the preset initial model again. The feature extraction network obtains the multi-scale sample feature map corresponding to the lane line detection sample image, and the step of determining the third loss value corresponding to the third loss function according to the first loss value and the second loss value, Until it is determined that the third loss value is smaller than the preset loss value threshold, the segmentation network in the current target model is deleted to obtain the preset lane line detection model. 4. The method according to claim 2, wherein the feature extraction network comprises a feature pyramid network and a global feature extraction network, the feature pyramid network comprises a bottom-up sub-network and a top-down sub-network, and the The input end of the global feature extraction network is coupled with the output end of the bottom-up sub-network, the output end of the global feature extraction network is coupled with the input end of the top-down sub-network, and the output end of the global feature extraction network is coupled. The output terminal is also coupled to the input terminal of the segmentation network, the output terminal of the top-down sub-network is also coupled to the input terminal of the segmentation network, and the preset initial model is used for: Obtain the multi-scale sample feature map corresponding to the lane line detection sample image through the bottom-up sub-network in the feature pyramid network, and input the minimum-scale sample feature map in the multi-scale sample feature map into the global feature an extraction network, so that the global feature extraction network outputs the first sample feature map to the segmentation network and the top-down sub-network; A multi-scale second sample feature map is input to the segmentation network through the top-down sub-network, so that the segmentation network determines the lane according to the first sample feature map and the second sample feature map The first predicted position of the lane line in the line detection sample image. 5 . The method according to claim 4 , wherein the segmentation network determines the first characteristic of the lane line in the lane line detection sample image according to the first sample feature map and the second sample feature map. 6 . Predicted locations, including: Perform up-sampling processing on the first sample feature map and the second sample feature map through the segmentation network to obtain a sample feature map to be segmented; Segment the feature map of the sample to be segmented according to a preset segmentation method to obtain a plurality of local feature maps; Performing convolution and splicing processing on the multiple local feature maps to obtain target sample feature maps; Determine the first probability that each pixel in the lane line detection sample image belongs to the lane line according to the target sample feature map; The first predicted position of the lane line in the lane line detection sample image is determined according to the first probability of each pixel. 6. The method according to claim 5, wherein the dividing according to a preset method comprises: The feature maps of the samples to be divided are divided into H local feature maps of C*W, the feature maps of the samples to be divided are divided into C local feature maps of H*W, and the feature maps of the samples to be divided are divided into At least one of W H*C local feature maps, where H is the number of layers of the sample feature map to be segmented, C is the length of the sample feature map to be segmented, and W is the width of the sample feature map to be segmented. 7. The method according to claim 5, wherein, performing convolution and splicing processing on the plurality of local feature maps to obtain target sample feature maps, comprising: Obtain the current local feature map, and perform a convolution operation on the current local feature map to obtain the specified local feature map after convolution; After splicing the specified local feature map and the next local feature map corresponding to the current local feature map, an updated current local feature map is obtained; determining whether the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps; In the case where it is determined that the next local feature map corresponding to the current local feature map is not the last one of the multiple local feature maps, obtain the current local feature map again, and perform convolution on the current local feature map operation to obtain the specified local feature map after convolution, to the step of determining whether the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps; In the case where it is determined that the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps, obtain the current local feature map, and perform a convolution operation on the current local feature map to obtain a volume The specified local feature map after product is used, and the specified local feature map corresponding to the current local feature map is used as the target sample feature map. 8. The method according to claim 4, wherein the output end of the bottom-up sub-network is coupled with the input end of the classification network, and the preset initial model is used for: Input the minimum scale sample feature map to the classification network through the bottom-up sub-network; The minimum scale sample feature map is divided into multiple anchor boxes by the classification network, a second probability that each anchor box belongs to the lane line is determined, and the lane line detection sample image is determined according to the second probability The second predicted location of the center lane line. 9 . The method according to claim 2 , wherein the target detection image is input into a preset lane line detection model, so that the preset lane line detection model outputs the target. 10 . Detect the target location of lane lines in an image, including: Obtain the target feature map corresponding to the target detection image through the feature extraction network; The target feature map is input into the classification network to obtain the target position of the lane line in the target detection image output by the classification network. 10. A lane line detection device, wherein the device comprises: an acquisition module, configured to acquire a target detection image; a determination module, configured to input the target detection image into a preset lane line detection model, so that the preset lane line detection model outputs the target position of the lane line in the target detection image; Wherein, the preset lane line detection model is obtained by training in the following ways: acquiring a plurality of lane line detection sample images, wherein the lane line detection images include lane line marked positions; inputting each of the lane line detection sample images into a preset initial model, where the preset initial model includes a segmentation network and a classification network, and the segmentation network is used to determine the first predicted position of the lane line in the lane line detection sample image, The classification network is used to determine the second predicted position of the lane line in the lane line detection sample image; The preset initial model is trained according to the first predicted position, the second predicted position and the lane line marking position to obtain the preset lane line detection model. 11. A vehicle, characterized in that the vehicle comprises the lane line detection device of claim 10 above. 12. A lane line detection device, characterized in that, comprising: a memory on which a computer program is stored; A processor for executing the computer program in the memory to implement the steps of the method of any one of claims 1-9. 13. A computer-readable storage medium on which computer program instructions are stored, characterized in that, when the program instructions are executed by a processor, the steps of the method according to any one of claims 1-9 are implemented.

Images