KR20210130222A - Dotted line detection method, apparatus and electronic device - Google Patents

Abstract

Embodiments of the present invention provide a dotted line detection method, apparatus and electronic device. The dotted line detection method includes: performing feature extraction on a road image to be detected to obtain a feature map of the road image; determining a lane area in the road image and a shortcoming pixel in the road image based on the feature map, wherein the shortcoming pixel is a pixel likely to belong to a shortcoming of a dashed lane in the road image; and determining a dotted line in the road image based on the lane area and the shortcoming pixel.

Description

Dotted line detection method, apparatus and electronic device

[Citation of related applications]

This application is filed on the basis of a Chinese patent application with an application number of 201910944245.2 and an application date of September 30, 2019, and claims the priority of the Chinese patent application, all contents of the Chinese patent application are incorporated herein by reference. do.

[Technology]

The present invention relates to machine learning technology, and more particularly to a method, apparatus and electronic device for detecting a dotted line.

Detecting lane information on the road is helpful for positioning and decision-making in autonomous driving. For example, a lane in an image may be extracted using a detection algorithm such as a Hough transform through the feature of the artificial design. In some machine learning-based lane detection methods, there is a risk that a dotted lane is detected as a continuous lane.

The present invention provides a method, apparatus and electronic device for detecting a dotted line.

A first aspect of the present invention provides a method for detecting a dashed line. The method includes: performing feature extraction on a road image to be detected to obtain a feature map of the road image; determining a lane area in the road image and a shortcoming pixel in the road image based on the feature map, wherein the shortcoming pixel is a pixel likely to belong to a shortcoming of a dashed lane in the road image; and determining a dotted line in the road image based on the lane area and the shortcoming pixel.

In some optional embodiments, determining the lane area in the road image based on the feature map comprises determining an area reliability of each pixel in the road image based on the feature map, wherein the area reliability is each pixel in the image represents a confidence level belonging to the lane region; and determining an area including pixels whose area reliability is not lower than an area threshold value as the next best area.

determining a shortcoming pixel in the road image based on the feature map, determining a shortcoming reliability of each pixel in the road image based on the feature map, wherein the shortcoming confidence indicates that each pixel in the road image is a dotted lane represents the reliability belonging to the shortcomings of -; determining whether the shortcoming confidence of each pixel is not less than a shortcoming threshold; and determining at least one pixel for which the shortcoming reliability is not lower than the shortcoming threshold as the shortcoming pixel.

In some optional embodiments, determining at least one pixel for which the disadvantage reliability is not lower than a disadvantage threshold as the disadvantage pixel comprises: for each pixel for which the disadvantage reliability is not lower than a disadvantage threshold, the disadvantage reliability is a disadvantage. and when it is determined that there is at least one pixel whose disadvantage reliability is not lower than the disadvantage threshold value among adjacent pixels of the pixel not lower than the threshold value, determining the pixel as the disadvantage pixel.

In some optional embodiments, determining a disadvantage pixel in the road image based on the feature map comprises: for each pixel for which the disadvantage confidence is not lower than a disadvantage threshold, the disadvantage confidence among adjacent pixels of that pixel is a disadvantage threshold and determining that the pixel is not the demerit pixel if it is determined that there is no adjacent pixel that is not lower than the value.

In some optional embodiments, the shortcoming pixels located within each preset area range constitute a corresponding set of shortcoming pixels, and determining a dotted line in the road image based on the lane area and the shortcoming pixel comprises: determining point coordinates in the road image based on a point pixel located in the lane area within the set of point pixels; and determining a dotted line in the road image based on the point coordinates in the road image.

In some optional embodiments, determining the corner coordinates in the road image based on a demerit pixel located in the lane area within each set of demerits pixels comprises the coordinates of a demerit pixel located in the lane area within the set of demerits pixels. performing a weighted averaging on ? to obtain the coordinates of a single point in the road image.

In some optional embodiments, determining the dashed lane in the road image based on the lane area and the shortcoming pixel is based on the shortcoming confidence of a shortcoming pixel located in the lane area within the set of shortcoming pixels. and determining the reliability of the one disadvantage in the system, and if the reliability of the determined disadvantage is lower than a preset threshold, removing the determined disadvantage.

In some optional embodiments, determining the dashed lane in the road image based on the endpoint coordinates in the road image comprises determining a near and far shortcomings among the shortcomings in the road image based on the endpoint coordinates in the road image. thing; and determining a dotted line in the road image based on the near and far shortcomings of the lane area and the shortcomings in the road image.

In some optional embodiments, performing feature extraction on the road image to be detected to obtain a feature map of the road image is performed by a feature extraction network, and a lane area in the road image is selected based on the feature map. Determining is performed by an area prediction network, and determining a shortcoming pixel in the road image based on the feature map is performed by a shortcoming prediction network.

In some optional embodiments, the feature extraction network, the area prediction network, and the shortcoming prediction network perform feature extraction on the road sample image using the feature extraction network to obtain a feature map of the road sample image. manipulation, wherein the sample road sample image includes a sample dashed line, and further includes first label information labeling a lane area in the road sample image and second label information labeling a disadvantage pixel of the sample dashed lane; predicting a lane area in the road sample image based on a feature map of the road sample image using an area prediction network to obtain lane area prediction information; an operation of predicting a shortcoming pixel in the road sample image based on the feature map of the road sample image using a shortcoming prediction network to obtain weak point pixel prediction information; determining a first network loss based on a difference between the lane area prediction information and the first label information, and adjusting a network parameter of the feature extraction network and a network parameter of the area prediction network according to the first network loss Operation; and determining a second network loss based on a difference between the shortcoming pixel prediction information and the second label information, and adjusting a network parameter of the shortcoming prediction network and a network parameter of the feature extraction network according to the second network loss It is trained through manipulation.

In some optional embodiments, the shortcoming pixel labeled with the second label information in the road sample image includes a pixel of the actual shortcoming of the sample dashed lane and an adjacent pixel of the pixel of the actual shortcoming.

In some optional embodiments, the method further comprises modifying the positioning information of the smart vehicle on the road indicated in the road image based on the determined shortcoming of the dashed line.

In some optional embodiments, the correction of the location identification information of the smart vehicle on the road corresponding to the road image based on the determined point coordinates of the dotted line may include: based on the determined point coordinates of the dotted line, the image distance measurement method determining a first distance representing a distance between the determined target shortcoming of the dashed lane and the smart vehicle; Based on the location identification information of the smart vehicle and the longitude and latitude of the target point in the driving assistance map used in the smart vehicle, determining a second distance indicating the distance between the target point and the smart vehicle determined based on the driving assistance map ; and correcting the location identification information of the smart vehicle based on an error between the first distance and the second distance.

According to a second aspect of the present invention, there is provided an apparatus for detecting a dotted line. The apparatus includes: a feature extraction module for performing feature extraction on a road image to be detected to obtain a feature map of the road image; a feature processing module for determining a lane area in the road image and a disadvantage pixel in the road image based on the feature map, wherein the disadvantage pixel is a pixel likely belonging to a disadvantage of a dashed line in the road image; and a lane determination module for determining a dotted line in the road image based on the lane area and the shortcoming pixel.

In some optional embodiments, the feature processing module determines an area reliability of each pixel in the road image based on the feature map, and determines an area including a pixel whose area reliability is not lower than an area threshold value as the lane area and a region determining sub-module for determining , wherein the region reliability indicates the reliability that each pixel in the road image belongs to a lane region.

In some optional embodiments, the feature processing module is further configured to determine, based on the feature map, a shortcoming confidence of each pixel in the road image, wherein the shortcoming confidence is that each pixel in the road image belongs to a shortcoming of a dashed lane. indicates reliability -; determining whether the shortcoming confidence of each pixel is not less than a shortcoming threshold; and a shortcoming pixel server module used to determine as the shortcoming pixel at least one pixel for which the shortcoming reliability is not lower than the shortcoming threshold.

In some optional embodiments, the shortcoming pixel sub-module is further configured to: for each pixel for which the shortcoming reliability is not lower than the shortcoming threshold, at least one of the adjacent pixels of the pixel whose shortcoming reliability is not lower than the shortcoming threshold If it is determined that there is one, the pixel is determined as the disadvantageous pixel.

In some optional embodiments, the shortcoming pixel submodule is further configured to: for each pixel for which the shortcoming reliability is not lower than the shortcoming threshold value, there is an adjacent pixel of the pixel whose faulty reliability is not lower than the shortcoming threshold value If it is determined not to, it is determined that the pixel is not the demerit pixel.

In some optional embodiments, the lane determining module is further configured to: determine a corner coordinate in the road image based on a corner pixel located in the lane area within each set of corner pixels, and based on the corner coordinate in the road image Determine the dashed lanes within the road image.

In some optional embodiments, the lane determining module obtains the coordinates of one shortcoming in the road image by performing a weighted average on the coordinates of the shortcoming pixels located in the lane area in the corresponding negative pixel set.

In some optional embodiments, the lane determining module is further configured to determine the reliability of one shortcoming in the road image based on the shortcoming reliability of the shortcoming pixel located in the lane area in the set of the shortcoming pixels, the determined shortcoming When the reliability of is lower than a preset threshold, the determined disadvantage is removed.

In some optional embodiments, the lane determining module is further configured to determine a near shortcoming and a far shortcoming among the shortcomings in the road image based on the coordinates of the shortcoming in the road image, wherein the lane area and the nearest one of the shortcomings in the road image Determine the dashed lanes in the road image based on the shortcomings and the far shortcomings.

In some optional embodiments, the feature extraction module performs feature extraction on the road image to be detected using a feature extraction network to obtain a feature map of the road image, and the feature processing module is configured to: to determine a lane area in the road image based on the feature map, and determine a shortcoming pixel in the road image based on the feature map using a shortcoming prediction network.

In some optional embodiments, the apparatus further comprises a network training module, wherein the network training module performs feature extraction on the road sample image using a feature extraction network to generate a feature map of the road sample image. an operation of acquiring, the road sample image includes sample dashed lanes, and also includes first label information labeling a lane area in the road sample image and second label information labeling a shortcoming pixel of the sample dotted lane lane - ; an operation of predicting a shortcoming pixel in the road sample image based on the feature map of the road sample image using a shortcoming prediction network to obtain weak point pixel prediction information; determining a first network loss based on a difference between the lane area prediction information and the first label information, and adjusting a network parameter of the feature extraction network and a network parameter of the area prediction network in response to the first network loss Operation; and determining a second network loss based on a difference between the shortcoming pixel prediction information and the second label information, and in response to the second network loss, the network parameter of the shortcoming prediction network and the network parameter of the feature extraction network Through the adjusting operation, the feature extraction network, the area prediction network, and the shortcoming prediction network are trained.

In some optional embodiments, the shortcoming pixel labeled with the second label information in the road sample image includes a pixel of the actual shortcoming of the sample dashed lane and an adjacent pixel of the pixel of the actual shortcoming.

In some optional embodiments, the apparatus further comprises a positioning correction module for modifying positioning information of the smart vehicle on the road corresponding to the road image based on the determined shortcoming of the dashed line.

In some optional embodiments, the positioning correction module determines a first distance indicating the distance between the smart vehicle and the target shortcoming of the dotted line determined through the image distance measurement method based on the determined point coordinates of the dotted line, Based on the location identification information of the smart vehicle and the longitude and latitude of the target point in the driving assistance map used in the smart vehicle, a second distance indicating the distance between the target point determined based on the driving assistance map and the smart vehicle is determined, , corrects the location identification information of the smart vehicle based on an error between the first distance and the second distance.

A third aspect of the present invention provides an electronic device. The device includes a processor and a memory for storing instructions, wherein the method described in any embodiment of the present invention is implemented when the instructions are executed by the processor.

A fourth aspect of the present invention provides a computer-readable recording medium. A computer program is stored in the computer-readable recording medium, and when the computer program is executed by a processor, the method described in any embodiment of the present invention is implemented.

According to an embodiment of the present invention, it is possible to detect a lane area and a shortcoming pixel based on a road image, and also, since each segment within the dotted line can be determined based on the lane area and a shortcoming pixel, division detection for a dotted line can be realized

1 is a flowchart of a method for detecting a dotted line according to at least one embodiment of the present invention.
2 is a flowchart of another method for detecting a dotted line according to at least one embodiment of the present invention.
3 is a schematic diagram of a set of shortcomings pixels in accordance with at least one embodiment of the present invention.
4 is a block diagram of a dotted lane detection network in accordance with at least one embodiment of the present invention.
5 is a flowchart of a training method of a network for detecting a dotted line according to at least one embodiment of the present invention.
6 is a flowchart of an image processing sequence according to at least one embodiment of the present invention.
7 is a flowchart of a method for detecting a dotted line according to at least one embodiment of the present invention.
8 is a block diagram of an apparatus for detecting a dotted line according to at least one embodiment of the present invention.
9 is a block diagram of another device for detecting a dotted line according to at least one embodiment of the present invention.
10 is a block diagram of another device for detecting a dotted line according to at least one embodiment of the present invention.

In order to enable those skilled in the art to better understand the technical solutions provided by one or more embodiments of the present disclosure, the technical solutions in one or more embodiments of the present disclosure may be implemented in one or more implementations of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS It will be described clearly and completely below with reference to the accompanying drawings in an example. Obviously, the described embodiments are merely some and not all embodiments of the present disclosure. Based on one or more embodiments of the present disclosure, all other embodiments obtained without creative work by those skilled in the art should fall within the protection scope of the present disclosure.

In autonomous driving, it is possible to assist in positioning a vehicle by detecting some characteristic points on the road. For example, by detecting the feature point, the current position of the vehicle may be more accurately positioned. In this respect, each dashed lane on the road may generally include a plurality of dotted lane segments, and each dotted lane segment may have two disadvantages, which are available road feature points. Accordingly, it is expected to provide a method capable of detecting a dotted line shortcoming.

At least one embodiment of the present invention provides a method for detecting a dashed line. Since the method can accurately detect the lane of each segment and also detect the shortcomings of the dotted line, it is possible to increase the feature points usable for autonomous driving.

1 is a flowchart of a method for detecting a dotted line according to at least one embodiment of the present invention. The method may include the following steps.

In step 100, feature extraction is performed on the road image to be detected to obtain a feature map of the road image. The road image to be detected includes dashed lanes.

In step 102, a lane area in the road image and a demerit pixel in the road image are determined based on the feature map. The shortcoming pixel is a pixel that is likely to belong to the shortcoming of the dashed lane in the road image.

For example, the area reliability of each pixel in the road image may be determined based on the feature map, and an area including pixels whose area reliability is not lower than an area threshold may be determined as the lane area. The area reliability is the reliability that each pixel in the road image belongs to a lane area.

For example, determine the shortcoming reliability of each pixel in the road image based on the feature map, determine whether the shortcoming reliability of each pixel is not lower than a shortcoming threshold, wherein the shortcoming reliability is not lower than a shortcoming threshold A pixel that does not exist may be determined as the disadvantage pixel. The shortcoming confidence is the confidence that each pixel in the road image belongs to the shortcoming of the dashed line.

In step 104, a dashed line in the road image is determined based on the lane area and the shortcoming pixel.

For example, since the dotted line has to be located in the lane area, it is possible to remove the demerit pixels that are not located in the lane area. By doing so, it is possible to determine each shortcoming of the dashed line based only on a plurality of shortcomings pixels located in the lane area, and also obtain the dotted line of each segment based on each of the shortcomings.

According to the dotted line detection method of the present embodiment, it is possible to detect the lane area and the shortcoming pixel in the road image, and to determine each segment in the dotted line based on the lane area and the shortcoming pixel, so that segmentation detection for the dotted line can be realized. have.

2 is a flowchart of another method for detecting a dotted line according to at least one embodiment of the present invention. As shown in FIG. 2 , the method may include the following steps.

In step 200, feature extraction is performed on the road image to be detected to obtain a feature map of the road image.

The road image may be, for example, a road image collected by a vehicle-mounted camera, a road reflectance image collected by a laser radar, or a HD road image captured through a satellite, which may be used for high-precision map creation. For example, the road image may be an image collected by the smart driving device on a road while driving, and the road image may include each type of lane, for example, a solid lane, a dotted lane, and the like.

In step 202, a lane area in the road image is determined based on the feature map.

For example, the reliability of each pixel in the road image belonging to the lane area may be determined based on the feature map, and an area including pixels having a reliability not lower than an area threshold may be determined as the lane area.

For example, an area threshold may be set. If the confidence level that a pixel belongs to the lane is not lower than the threshold value of the area, the pixel is considered to belong to the lane area. It is considered not to be in the lane zone.

In step 204, based on the feature map, it is determined that each pixel in the road image belongs to the shortcoming of the dotted line.

In step 206, a pixel is selected whose defect confidence is not lower than the defect threshold.

In one example, a disadvantage threshold may be set. When the shortcoming reliability of one pixel is lower than the shortcoming threshold, it is considered that the pixel does not belong to the shortcomings of the dotted line, and the pixel may be deleted from the prediction result of the shortcomings. If the shortcoming reliability of one pixel is not lower than the corresponding shortcoming threshold, it may be considered that the pixel is likely to belong to the shortcoming of the dotted line.

In step 208, for each of the selected pixels for which the defect reliability is not lower than the defect threshold, it is determined whether there is at least one adjacent pixel of the pixel whose defect reliability is not lower than the defect threshold.

In one example, further screening may be performed on the selected pixel so that the prediction result of the shortcoming is more accurate. If there is at least one neighboring pixel of a selected pixel whose defect reliability is not lower than the defect threshold, the pixel is withheld. When the defect reliability of all adjacent pixels of a selected pixel is lower than the defect threshold, it means that the pixel is an isolated point. One practical disadvantage of one dashed lane is that it contains multiple adjacent pixels. Therefore, it is unlikely that such an isolated point is a disadvantage of the dashed lane, so it can be excluded.

When the determination result of step 208 is “YES”, step 210 is executed. If the determination result in step 208 is NO, that is, since the pixel is an isolated point, step 212 is executed.

In step 210, it is determined that the pixel is a bad pixel. Step 214 continues with execution.

In step 212, it is determined that the pixel is not a demerit pixel.

In step 214, a point coordinate in the road image is determined based on a point pixel located in the lane area within each set of point pixels.

For example, a single point in a dashed lane may contain a plurality of pixels, which pixels may be the predicted downside pixels. By performing a weighted averaging on the coordinates of the shortcoming pixels located in the lane area within one set of shortcomings pixels, the coordinates of one shortcoming in the road image may be obtained.

The shortcoming pixel set is a set constituted by at least one shortcoming pixel within a preset area range. For example, since a plurality of shortcoming pixels within the range of a lane edge dwelling of a segment of one of the dashed lanes and its adjacent area may constitute one edge pixel set, one edge pixel set may correspond to one segment of one dotted lane. It may include a pixel corresponding to the sub-optimal shortcoming and a pixel in its adjacent area.

As shown in FIG. 3 , within the preset area range L, at least one shortcoming pixel, for example, a shortcoming pixel 31 is included, and these shortcoming pixels constitute one set of shortcomings pixels. Based on this point pixel, the coordinates of the corresponding one point 32 can be determined. Disadvantage 32 may be a disadvantage of one of the dashed lane segments in the road image.

For example, when all the shortcoming pixels within the preset area range L are located in the lane area, a weighted average may be performed on the coordinates of all the shortcomings pixels. If the coordinates of each disadvantage pixel are expressed as (x, y), the x0 coordinate of the disadvantage 32 can be obtained by weighted averaging the X coordinates of all the disadvantage pixels, and the y coordinate of the disadvantage 32 is weighted averaged by the y coordinate of the disadvantage 32 can be obtained Thereby, the coordinates (x0, y0) of the point 32 can be obtained.

In one example, after determining the corner coordinates in the road image in step 214, based on the shortcoming confidence of a shortcoming pixel located in the lane area within each set of shortcoming pixels, for example, a weighted average for these shortcoming confidences is calculated Thus, after obtaining the reliability of each disadvantage in the road image, it is possible to remove a disadvantage whose reliability is lower than a preset threshold among the determined disadvantages in the road image. In this way, it is possible to eliminate some relatively distant ambiguity in the road image.

In step 216, a dotted line in the road image is determined based on the coordinates of the point in the road image.

For example, based on the coordinates of the shortcomings in the road image determined in step 214, it is possible to determine a near and far shortcoming among each of the shortcomings in the road image. For example, among the two shortcomings of one dotted lane segment in one of the dashed lanes, the shortcoming that is close to a smart driving device equipped with an image acquisition device can be called a near shortcoming, and a shortcoming that is far from the smart driving device can be called a far shortcoming. have. Then, the dotted line in the road image may be determined based on the near and far points among the lane areas and the respective points in the road image. For example, one segment of the dashed lanes can be obtained by combining the lane areas after linking one near endpoint with the corresponding distant endpoint.

In another implementation form, a starting point and an ending point of each dotted line segment may be determined by arranging a plurality of points located in one lane area according to coordinates. For example, after setting the image height direction of the road image to the y direction, aligning each shortcoming in one lane area according to their respective y coordinates, a shortcoming with a small y coordinate is a shortcoming that is close to a shortcoming with a large y coordinate. It can be determined as a distant disadvantage.

According to the above embodiment, after the lane area and the shortcoming pixel are detected, selection is performed on the shortcoming pixel, and only the shortcoming pixel located in the lane area can be reserved. After determining the shortcomings of the dashed lane based on the selected shortcoming pixels, it is possible to exclude some distant and ambiguous shortcomings in the road image based on the reliability of each shortcoming. Accordingly, the detection accuracy of the shortcomings of the dotted line may be improved, and the detection accuracy of the dotted line may be improved.

In some examples, the dotted line detection method may be realized using a pre-trained dotted line detection network.

4 illustrates a block diagram of a detection network of a dashed line. The detection network 40 may include a feature extraction network 41 , an area prediction network 42 , and a shortcoming prediction network 43 .

The feature extraction network 41 may obtain a feature map of the road image by extracting image features from the input road image.

The area prediction network 42 may predict the lane area, ie, the probability that each pixel in the road image belongs to a pixel in the lane area, based on the feature map of the road image. If the training of the detection network 40 is not completed, it is possible that some degree of prediction error exists. For example, even pixels not located in the lane area may be predicted as pixels in the lane area.

The shortcoming prediction network 43 may predict and output a shortcoming pixel based on the feature map of the road image, that is, predict the probability that each pixel in the predicted road image is a shortcoming pixel.

In some cases, what the detection network 40 predicts and outputs may be the confidence that the pixel belongs to a certain type. For example, the area prediction network 42 may predict and output the confidence that each pixel in the road image belongs to a lane area, and the shortcoming prediction network 43 predicts the confidence that each pixel in the road image belongs to a shortcoming pixel. can be printed out.

In order to train the detection network 40 , a plurality of road sample images may be acquired in advance. Each road sample image may include a dotted line, and may also include lane area label information and shortcoming pixel label information. The lane area label information labels the lane area in the road sample image, that is, labels pixels belonging to the lane area in the road sample image. The disadvantage pixel label information labels the disadvantage pixels of the dotted line in the road sample image, that is, the pixels that have two disadvantages in each segment of the dotted line are marked as the disadvantage pixels. For example, if one segment of the dotted line has two shortcomings, one preset area range may be marked for each of the two shortcomings, and all pixels within the range may be marked as a shortcoming pixel.

5 shows a flowchart of a method for training the detection network of the dashed line shown in FIG. 4 according to at least one embodiment of the present invention. As shown in FIG. 5 , the method may include the following steps.

In step 500, the obtained plurality of road sample images are input to the feature extraction network 41, wherein each road sample image includes a dashed line to be detected, and further includes lane area label information and shortcoming pixel label information.

In step 502, through the feature extraction network 41, image features of each input road sample image are extracted to obtain a corresponding feature map.

6 illustrates an example in which the feature extraction network 41 is a Fully Convolutional Network (FCN), for example, a sequence in which the detection network 40 processes an input road sample image.

For example, first, a high-dimensional feature conv1 of the road sample image may be obtained by performing convolution (down-sampling) a plurality of times on the input road sample image. Then, deconvolution (up-sampling) is performed on the high-dimensional feature conv1 to obtain a feature map us_conv1. Then, the feature map us_conv1 may be input to the area prediction network 42 and the shortcoming prediction network 43, respectively.

In step 504, the feature map (e.g., feature map us_conv1) is input to the area prediction network 42 and the shortcoming prediction network 43, respectively, and the lane area in the road sample image through the area prediction network 42 predicts and outputs, and predicts and outputs a pixel in the road sample image through the shortcoming prediction network 43 .

For example, through the area prediction network 42 , the reliability of each pixel in the road sample image belonging to the lane region is predicted and outputted, and the reliability of each pixel in the road sample image belonging to the shortcoming pixel through the shortcoming prediction network 43 is predicted and output. can be predicted and output.

In step 506 , the network parameters of the feature extraction network 41 , the area prediction network 42 , and the weakness prediction network 43 are adjusted based on the prediction result.

For example, determine a first network loss based on a difference between a predicted lane area in the road sample image and a lane area labeled through lane area label information in the road sample image, and according to the first network loss The network parameters of the feature extraction network 41 and the network parameters of the area prediction network 42 may be adjusted. Also, determine a second network loss based on a difference between the predicted disadvantage pixel in the road sample image and the disadvantage pixel in the road sample image labeled through label information, and determine the disadvantage according to the second network loss. The network parameters of the prediction network 43 and the network parameters of the feature extraction network 41 may be adjusted.

For example, the network parameters in the detection network 40 may be adjusted by backward propagation. When the network repetition termination condition is satisfied, network training is terminated. The termination condition may be that the iteration reaches a certain number of times, or that the loss value is less than a preset threshold value.

In some cases, when the number of visible segments of the dashed line in one road sample image is small, that is, when the positive sample rate in the sample image is low, the detection accuracy of the detection network after training can be improved by improving the positive sample rate. In this aspect, for example, by enlarging the range of the shortcoming pixels of the labeled dotted line through the shortcoming pixel label information in the road sample image, the labeled shortcoming pixel in the road sample image is the actual of the dotted line in the road sample image. In addition to including the pixel of the disadvantage, it is also intended to include the pixels adjacent to the pixel of the actual disadvantage. This can mark more pixels as a disadvantage, increasing the positive sample rate.

7 is a flowchart of a method for detecting a dotted line using a training-completed detection network according to an embodiment of the present invention. As described above, the detection network for which the training is completed may include a feature extraction network, an area prediction network, and a shortcoming prediction network. 7 , the method may include the following steps.

In step 700, a road image to be detected is received. For example, the road image may be a road image collected by a smart driving device and driven by itself.

In step 702, image features of the road image are extracted using a feature extraction network, and a feature map of the road image is obtained. For example, the feature extraction network may acquire the feature map of the road image through a plurality of operations such as convolution and deconvolution.

In step 704, the feature map is inputted to an area prediction network and a shortcoming prediction network, respectively, using the area prediction network to predict and output a lane area in the road image, and using the shortcoming prediction network to predict and output the lane area in the road image It predicts and outputs the weak pixel.

The feature map obtained in step 702 may be input to two parallel branching networks, namely, an area prediction network and a shortcoming prediction network. Using the area prediction network, the first prediction result of the lane area in the road image including the first confidence that each pixel in the road image belongs to the lane area may be predicted and output. Also, by using the shortcoming prediction network, the second prediction result of the shortcoming pixel in the road image including the second confidence that each pixel in the road image belongs to the shortcoming pixel may be predicted and output.

In an example, based on the acquisition of the first prediction result, the lane area may be determined based on a pixel whose first reliability is not lower than an area threshold. For example, an area threshold may be set. If the first reliability of one pixel is not lower than the corresponding area threshold, the pixel is considered to belong to the next lane. may be considered not to be included.

In an example, based on the acquisition of the second prediction result, a threshold threshold may be further set. When the second reliability of one pixel is lower than the disadvantage threshold, the pixel may be regarded as not belonging to the disadvantage pixel, that is, a pixel having a second reliability lower than the disadvantage threshold may be deleted from the prediction result of the disadvantage pixel. . If the second reliability of one pixel is not lower than the disadvantage threshold, the pixel may be regarded as belonging to the disadvantage pixel.

In step 706, at least one shortcoming pixel located in the lane area among the predicted shortcoming pixels is acquired.

If one predicted shortcoming pixel is not located in the lane area at all, it is unlikely to be a dashed lane weakness. Accordingly, in step 706 , only the shortcoming pixels located in the lane area may be reserved in consideration of the two prediction results obtained in step 704 .

In one example, in order to make the prediction result of the weak pixel more accurate, selection may be further performed on the predicted weak pixel. If there is at least one adjacent pixel of one disadvantageous pixel whose second reliability is not lower than the disadvantage threshold, the disadvantageous pixel is withheld. If the second reliability of all adjacent pixels of one disadvantageous pixel is lower than the threshold threshold, it means that the disadvantageous pixel is an isolated point. One practical disadvantage of one dashed lane is that it contains a plurality of adjacent pixels. Therefore, it is unlikely that such an isolated point is a disadvantage of the dashed lane, and as described above, it can be excluded.

In step 708, a point coordinate of the dashed line is determined based on the acquired at least one point pixel.

In step 710, a dotted line is determined based on the coordinates of a point located in the same lane area.

Some of the steps 702 to 710 may be realized with reference to the related methods among the embodiments described in FIG. 1 or FIG. 2 described above, and will not be described herein again.

In some cases, after the shortcomings of the dotted line are detected, the shortcomings can be used to assist the positioning of the smart driving device. The smart driving device includes various smart vehicles such as an autonomous driving vehicle or a vehicle possessing an auxiliary driving system. Additionally, the detected dotted line lane and point coordinates can be applied to the creation of a high-precision map.

For example, after detecting a dotted line by the method for detecting a dotted line according to an embodiment of the present invention, location identification information of a smart vehicle on a road corresponding to the road image may be corrected based on the detected shortcomings of the detected dotted line. .

For example, the first distance is determined through an image distance measuring method based on the detected shortcoming of the dashed line. The first distance indicates a distance between a target point of the detected dotted line and the smart vehicle. In one illustrative example, when the smart vehicle is driving, the target shortcoming may be the shortcoming of the dotted line of the nearest segment in front of the smart vehicle. For example, when the smart vehicle travels 10 meters and arrives at the target point, the first distance becomes 10 meters.

In addition, the second distance is determined based on the location confirmation longitude and latitude of the smart vehicle itself and the longitude and latitude of the target point in the driving assistance map used in the smart vehicle. The second distance represents a distance between the target point and the smart vehicle itself determined based on the driving assistance map.

Thereafter, based on the error between the first distance and the second distance, the smart vehicle's own positioning latitude is corrected. For example, when the second distance is 8 meters, the error between the first distance and the second distance is 2 meters, and based on this, the smart vehicle's own positioning latitude and longitude may be corrected.

8 provides an apparatus for detecting a dotted line. As shown in FIG. 8 , the apparatus may include a feature extraction module 81 , a feature processing module 82 , and a lane determination module 83 .

The feature extraction module 81 performs feature extraction on the road image to be detected to obtain a feature map of the road image.

The feature processing module 82 determines a lane area in the road image and a shortcoming pixel in the road image based on the feature map. The shortcoming pixel is a pixel in the road image that is likely to belong to the dotted lane shortcoming.

The lane determining module 83 determines a dotted line in the road image based on the lane area and the shortcoming pixel.

In one example, as shown in FIG. 9 , the feature processing module 82 includes the following sub-modules.

The area determination submodule 821 determines the area reliability of each pixel in the road image based on the feature map, and determines an area including pixels whose area reliability is not lower than an area threshold value as the lane area. The area reliability indicates the reliability that each pixel in the road image belongs to a lane area.

The shortcoming pixel sub-module 822 is configured to determine, based on the feature map, the shortcoming confidence of each pixel in the road image, the shortcoming confidence indicating the confidence that each pixel in the road image belongs to the shortcoming of the dashed lane. ; determining whether the shortcoming confidence of each pixel is not less than a shortcoming threshold; and determining at least one pixel for which the shortcoming reliability is not lower than the shortcoming threshold to be the shortcoming pixel.

In an example, the shortcoming pixel sub-module 822 is configured to: for each pixel for which the shortcoming reliability is not lower than the shortcoming threshold value, at least one adjacent pixel of the pixel having the shortcoming reliability not lower than the shortcoming threshold is present If it is decided to do so, the pixel is determined as the disadvantage pixel.

In one example, the shortcoming pixel sub-module 822 is configured to: for each pixel whose fault reliability is not lower than a fault threshold, there is no adjacent pixel of the pixel whose fault reliability is not lower than the fault threshold. If it is determined that the pixel is not, it is determined that the pixel is not the demerit pixel.

In one example, the lane determining module 83 is configured to determine the shortcoming coordinates in the road image based on a shortcoming pixel located in the lane area in each of the set of shortcoming pixels, and based on the shortcoming coordinates in the road image, the Determine the dashed lanes within the road image.

In one example, the lane determining module 83 obtains the coordinates of one shortcoming in the road image by performing a weighted average on the coordinates of the shortcoming pixels located in the lane area in the corresponding shortcoming pixel set.

In an example, the lane determining module 83 is further configured to determine the reliability of one shortcoming in the road image based on the shortcoming reliability of the shortcoming pixel located in the lane area in the set of shortcoming pixels, and If the reliability is lower than a preset threshold, the determined disadvantage is removed.

In an example, the lane determining module 83 is further configured to determine, based on the coordinates of the shortcomings in the road image, near and far shortcomings among the shortcomings in the road image, the lane area and the nearest of the shortcomings in the road image Determine the dashed lanes in the road image based on the shortcomings and the far shortcomings.

In an example, the feature extraction module 81 obtains a feature map of the road image by performing feature extraction on the road image to be detected using the feature extraction network.

The feature processing module 82 is configured to use an area prediction network to determine a lane area in the road image based on the feature map, and use a shortcoming prediction network to determine a shortcoming pixel in the road image based on the feature map. decide

In an example, the apparatus further comprises a network training module. The network training module trains the feature extraction network, the area prediction network, and the weakness prediction network through the following steps. This step includes: performing feature extraction on the road sample image using a feature extraction network to obtain a feature map of the road sample image, wherein the road sample image includes a sample dotted line lane, and further, the road sample first label information for labeling the lane area in the image and second label information for labeling the shortcoming pixel of the sample dotted line lane are included; predicting a lane area in the road sample image based on a feature map of the road sample image using an area prediction network to obtain lane area prediction information; predicting a shortcoming pixel in the road sample image based on a feature map of the road sample image using a shortcoming prediction network; determining a first network loss based on a difference between the lane area prediction information and the first label information; adjusting a network parameter of the feature extraction network and a network parameter of the area prediction network in response to the first network loss; and determining a second network loss based on a difference between the shortcoming pixel prediction information and the second label information, and adjusting a network parameter of the shortcoming prediction network and a network parameter of the feature extraction network in response to the second network loss. includes manipulation.

In one example, the shortcoming pixel labeled with the second label information in the road sample image includes a pixel of the actual shortcoming of the sample dashed lane and an adjacent pixel of the pixel of the actual shortcoming.

In one example, as shown in FIG. 10 , the device includes a positioning correction module 84 for correcting positioning information of a smart vehicle on a road corresponding to the road image based on the determined coordinates of the shortcomings of the dotted line. ) is further included.

In one example, the positioning correction module 84 specifically determines the first distance indicating the distance between the smart vehicle and the target point of the dotted line determined through the image distance measurement method based on the determined point coordinates of the dotted line. a second distance indicating the distance between the smart vehicle and the target point determined based on the driving assistance map based on the location identification information of the smart vehicle and the longitude and latitude of the target point in the driving assistance map used in the smart vehicle and corrects the location identification information of the smart vehicle based on an error between the first distance and the second distance.

The present invention further provides a computer-readable recording medium. A computer program is stored in the computer readable recording medium, and when the computer program is executed by a processor, the dotted line detection method described in any embodiment of the present invention is implemented.

The present invention further provides an electronic device. The electronic device may include a processor; and a memory for storing instructions executable by the processor. When the above command is executed, the dotted line lane detection method described in any embodiment of the present invention is implemented.

Those of ordinary skill in the art should understand that one or more embodiments of the present disclosure may be provided as methods, systems, or computer program products. Accordingly, one or more embodiments of the present disclosure may be implemented as fully hardware embodiments, fully software embodiments, or embodiments combining software and hardware. Further, one or more embodiments of the present disclosure provide a computer program executing on a computer usable storage medium comprising computer usable program codes, which may include, but are not limited to, a disk storage component, a CD-ROM, an optical storage component, and the like. It can be implemented in the form of a product.

Embodiments of the present invention further provide a computer-readable recording medium, in which a computer program can be stored, and when the program is executed by a processor, the dotted line described in any embodiment of the present invention Steps of a method for training a neural network used to detect a lane line are implemented, and/or steps of a method for detecting a dotted line lane described in any embodiment of the present invention are implemented. As used herein, “and/or” refers to having at least one of two candidates, e.g., “A and/or B” includes three instances: A alone, B alone, and both A and B. can do.

Various embodiments in the present description are described in a gradual manner, the emphasis description of each embodiment is different from other embodiments, and the same or similar parts between the various embodiments may be referred to each other. In particular, since they are substantially similar to the method embodiments, the electronic device embodiments are briefly described and reference may be made to some of those descriptions of the method embodiments for related parts.

The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, it may still achieve the expected result even if the actions or steps described in the claims are performed in a different order in the embodiments. Further, even if the processes described in the figures do not follow their specific order or their sequential order as shown, they may still achieve the expected results. In some embodiments, multi-task processing or parallel processing may also be feasible or beneficial.

Embodiments of the subject matter and functional operations described herein may comprise: digital electronic circuitry, tangible computer software or firmware, computer hardware that may include the structures disclosed herein and structural equivalents thereof, or a combination of one or more thereof. can be implemented. Embodiments of the subject matter described in this disclosure provide one or more computer programs, ie, one or more computer program instructions encoded on a tangible non-transitory program carrier for executing by or for controlling operations of data processing equipment. It can be implemented as a module. Alternatively or in addition, the program instructions may be generated for encoding and transmitting information to suitable receiving equipment for execution by the data processing equipment, such as machine-generated electrical, optical, or electromagnetic signals. may be encoded into artificial propagation signals. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access storage device, or a combination of one or more of these.

The processing and logic procedures described herein may be executed by one or more programmable computers executing one or more computer programs to operate based on input data and generate output to perform corresponding functions. The processing and logic procedures may also be executed by dedicated logic circuits, such as field programmable gate arrays (FPGAs) or application specific integrated circuits (ASICs), and device 80 may also be implemented as dedicated logic circuits.

A computer suitable for executing computer programs may include, for example, a general and/or special purpose microprocessor, or any other type of central processing unit. Generally, a central processing unit receives instructions and data from a read-only storage component and/or a random access storage component. The basic components of a computer may include a central processing unit for implementing or executing instructions and one or more storage devices for storing instructions and data. In general, a computer may also include one or more mass storage devices for storing data. Mass storage devices may be, for example, magnetic, optical or magneto-optical disks. Alternatively, the computer may be operatively coupled to the mass storage devices to receive data from or transmit data to the mass storage devices. Otherwise, the above two cases may coexist. However, these devices are not essential to a computer. A computer may also be a mobile phone, personal digital assistant (PDA), mobile audio or video player, game console, global positioning system (GPS) receiver, or portable storage device such as a universal serial bus (USB) flash drive. It may be embedded in other devices, these being mentioned as just a few examples.

Computer-readable media suitable for storing computer program instructions and data may include all forms of non-volatile storage components, media, and storage devices. For example, it may include semiconductor storage devices such as EPROMs, EEPROMs and flash devices, magnetic disks such as internal hard disks or removable disks, magneto-optical disks, CD ROM disks or DVD-ROM disks. The processor and memory may be supplemented by or incorporated into dedicated logic circuitry.

Although this invention includes many specific implementation details, these should not be construed as limiting any scope to be disclosed or protected, but primarily used to describe features of the specific embodiments disclosed. Certain features described in multiple embodiments of the present disclosure may also be combined and implemented in a single embodiment. On the other hand, various features described in a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination. Also, although some features work in certain combinations as described above and are even initially claimed as such, one or more features from a claimed combination may in some cases be eliminated therefrom, and the claimed combination may be a sub-combination or It may refer to a variant thereof.

Similarly, while acts are described in a particular order in the drawings, they should not be construed to mean that the acts must be performed one after the other, sequentially, or completely, based on the particular order shown, to achieve the expected results. . In some cases, multi-tasking or parallel processing may be beneficial. In addition, the separation of various system modules and components in the above embodiments should not be construed as meaning that such separation is essential in all embodiments. It should also be understood that the program components and systems described may generally be integrated together into a single software product, or packaged into multiple software products.

Accordingly, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the expected result. Moreover, for the processing described in the figures, it is not necessary to follow the specific order or sequential order as shown to achieve the expected results. In some implementations, multi-tasking or parallel processing may be beneficial.

The foregoing are only preferred examples of one or more embodiments of the present disclosure, and are not used to limit one or more embodiments of the present disclosure. Any modification, equivalent substitution, improvement, etc. within the spirit and principle of one or more embodiments of the present disclosure shall fall within the protection scope of one or more embodiments of the present disclosure.

Claims (30)

A method for detecting a dotted line, comprising:
performing feature extraction on the road image to be detected to obtain a feature map of the road image;
determining a lane area in the road image and a shortcoming pixel in the road image based on the feature map, wherein the shortcoming pixel is a pixel likely to belong to a shortcoming of a dashed lane in the road image; and
determining a dotted line in the road image based on the lane area and the shortcoming pixel
A method for detecting a dotted line. According to claim 1,
Determining the lane area in the road image based on the feature map comprises:
determining an area reliability of each pixel in the road image based on the feature map, the area reliability indicating a degree of confidence that each pixel in the road image belongs to the lane area; and
determining an area including pixels of area reliability not lower than an area threshold as the next best area
A method for detecting a dotted line. 3. The method of claim 1 or 2,
Determining the shortcoming pixel in the road image based on the feature map comprises:
determining a shortcoming confidence level of each pixel in the road image based on the feature map, the shortcoming confidence indicating a confidence that each pixel in the road image belongs to a shortcoming of a dashed lane;
determining whether the shortcoming confidence of each pixel is not less than a shortcoming threshold; and
determining as the shortcoming pixel at least one pixel for which the shortcoming reliability is not lower than the shortcoming threshold
A method for detecting a dotted line. 4. The method of claim 3,
Determining at least one pixel for which the disadvantage reliability is not lower than the disadvantage threshold as the disadvantage pixel comprises:
For each of the pixels whose disadvantage reliability is not lower than the disadvantage threshold value, if it is determined that there is at least one adjacent pixel of the pixel whose disadvantage reliability is not lower than the disadvantage threshold value, the pixel is determined as the disadvantage pixel including doing
A method for detecting a dotted line. 4. The method of claim 3,
Determining the shortcoming pixel in the road image based on the feature map comprises:
For each pixel whose disadvantage reliability is not lower than the disadvantage threshold, if it is determined that there is no adjacent pixel of the pixel whose disadvantage reliability is not lower than the disadvantage threshold, the pixel is not the disadvantage pixel. further including determining
A method for detecting a dotted line. 6. The method according to any one of claims 1 to 5,
wherein said shortcoming pixels located within each preset area range constitute a corresponding set of shortcomings pixels,
Determining the dashed line in the road image based on the lane area and the shortcoming pixel comprises:
determining point coordinates in the road image based on a point pixel located in the lane area within each set of point pixels; and
determining a dotted line in the road image based on the coordinates of a point in the road image
A method for detecting a dotted line. 7. The method of claim 6,
Determining the corner coordinates in the road image based on a demerit pixel located in the lane area within each set of demerit pixels comprises:
performing a weighted average on the coordinates of the shortcoming pixels located in the lane area within the set of shortcomings pixels to obtain the coordinates of one shortcoming in the road image
A method for detecting a dotted line. 8. The method of claim 7,
Determining the dashed line in the road image based on the lane area and the shortcoming pixel comprises:
determining the reliability of one of the shortcomings in the road image based on the shortcoming reliability of the shortcoming pixels located in the lane area within the set of shortcomings pixels; and
When the reliability of the determined disadvantage is lower than a preset threshold, further comprising removing the determined disadvantage
A method for detecting a dotted line. 9. The method according to any one of claims 6 to 8,
Determining the dotted line in the road image based on the coordinates of the point in the road image comprises:
determining near and far shortcomings among the shortcomings in the road image based on the coordinates of the shortcomings in the road image; and
determining a dotted line in the road image based on the near and far shortcomings of the lane area and the shortcomings in the road image
A method for detecting a dotted line. 10. The method according to any one of claims 1 to 9,
performing feature extraction on the road image to be detected to obtain a feature map of the road image is performed by a feature extraction network;
Determining the lane area in the road image based on the feature map is performed by an area prediction network,
Determining the shortcoming pixel in the road image based on the feature map is performed by a shortcoming prediction network.
A method for detecting a dotted line. 11. The method of claim 10,
The feature extraction network, the area prediction network, and the shortcoming prediction network are
An operation of performing feature extraction on a road sample image using a feature extraction network to obtain a feature map of the road sample image, wherein the sample road sample image includes a sample dashed lane, and a lane area in the road sample image first label information for labeling , and second label information for labeling a pixel of the sample dotted line;
predicting a lane area in the road sample image based on a feature map of the road sample image using an area prediction network to obtain lane area prediction information;
an operation of predicting a shortcoming pixel in the road sample image based on the feature map of the road sample image using a shortcoming prediction network to obtain weak point pixel prediction information;
determining a first network loss based on a difference between the lane area prediction information and the first label information, and adjusting a network parameter of the feature extraction network and a network parameter of the area prediction network according to the first network loss Operation; and
determine a second network loss based on a difference between the shortcoming pixel prediction information and the second label information, and adjust the network parameter of the shortcoming prediction network and the network parameter of the feature extraction network according to the second network loss trained through the
A method for detecting a dotted line. 12. The method of claim 11,
The shortcoming pixel labeled through the second label information in the road sample image includes a pixel of the actual shortcoming of the sample dashed lane and an adjacent pixel of the pixel of the actual shortcoming.
A method for detecting a dotted line. 10. The method according to any one of claims 6 to 9,
Further comprising correcting the location identification information of the smart vehicle on the road corresponding to the road image based on the determined point coordinates of the dotted line
A method for detecting a dotted line. 14. The method of claim 13,
Correcting the location identification information of the smart vehicle on the road corresponding to the road image based on the determined coordinates of the shortcomings of the dotted line is,
determining a first distance indicating a distance between the smart vehicle and a target shortcoming of the dotted line determined through an image distance measuring method based on the determined point coordinates of the dotted line;
Based on the location identification information of the smart vehicle and the longitude and latitude of the target point in the driving assistance map used in the smart vehicle, determining a second distance indicating the distance between the target point and the smart vehicle determined based on the driving assistance map ; and
Based on the error between the first distance and the second distance, comprising correcting the location identification information of the smart vehicle
A method for detecting a dotted line. A dotted line detection device comprising:
a feature extraction module for obtaining a feature map of the road image by performing feature extraction on the road image to be detected;
a feature processing module for determining a lane area in the road image and a disadvantage pixel in the road image based on the feature map, wherein the disadvantage pixel is a pixel likely belonging to a disadvantage of a dashed line in the road image; and
and a lane determination module for determining a dotted lane in the road image based on the lane area and the shortcoming pixel.
A dotted line detection device, characterized in that. 16. The method of claim 15,
The feature processing module,
a region determining submodule for determining region reliability of each pixel in the road image based on the feature map, and determining an region including pixels whose region reliability is not lower than a region threshold as the lane region;
The area reliability indicates the reliability of each pixel in the road image belonging to the lane area.
A dotted line detection device, characterized in that. 17. The method of claim 15 or 16,
The feature processing module,
determining a shortcoming confidence level of each pixel in the road image based on the feature map, the shortcoming confidence indicating a confidence that each pixel in the road image belongs to a shortcoming of a dashed lane; determining whether the shortcoming confidence of each pixel is not less than a shortcoming threshold; and a disadvantage pixel server module used to determine as the disadvantage pixel at least one pixel whose disadvantage reliability is not lower than the disadvantage threshold value.
A dotted line detection device, characterized in that. 18. The method of claim 17,
The disadvantage pixel sub-module is further configured to, when it is determined that for each pixel whose disadvantage reliability is not lower than the disadvantage threshold value, there is at least one adjacent pixel among the adjacent pixels of the pixel whose disadvantage reliability is not lower than the disadvantage threshold value, Determining the pixel as the shortcoming pixel
A dotted line detection device, characterized in that. 18. The method of claim 17,
The disadvantage pixel sub-module is further configured to, for each of the pixels whose disadvantage reliability is not lower than the disadvantage threshold value, when it is determined that there is no adjacent pixel of the pixel whose disadvantage reliability is not lower than the disadvantage threshold value, the adjacent pixel does not exist. to determine that the pixel is not the above-mentioned disadvantage pixel
A dotted line detection device, characterized in that. 20. The method according to any one of claims 15 to 19,
The lane determining module is configured to determine the shortcoming coordinates in the road image based on the shortcoming pixels located in the lane area within each of the shortcoming pixel sets, and determine the dotted line in the road image based on the shortcomings coordinates in the road image doing
A dotted line detection device, characterized in that. 21. The method of claim 20,
The lane determining module is configured to perform a weighted average on the coordinates of the shortcoming pixels located in the lane area within the set of shortcomings pixels to obtain the coordinates of one shortcoming in the road image.
A dotted line detection device, characterized in that. 22. The method of claim 21,
The lane determining module is further configured to determine the reliability of one shortcoming in the road image based on the shortcoming reliability of the shortcoming pixel located in the lane area within the set of shortcoming pixels, and the reliability of the determined shortcoming is greater than a preset threshold. If it is low, it eliminates the determined disadvantage
A dotted line detection device, characterized in that. 23. The method according to any one of claims 20 to 22,
The lane determining module is further configured to determine near and far shortcomings among the shortcomings in the road image based on the coordinates of the shortcomings in the road image, and based on the near and far shortcomings among the shortcomings in the lane area and the road image, Determining the dashed lanes within the road image
A dotted line detection device, characterized in that. 24. The method according to any one of claims 15 to 23,
The feature extraction module performs feature extraction on a road image to be detected using a feature extraction network to obtain a feature map of the road image,
The feature processing module is configured to determine a lane area in the road image based on the feature map using an area prediction network, and determine a shortcoming pixel in the road image based on the feature map using a shortcoming prediction network.
A dotted line detection device, characterized in that. 25. The method of claim 24,
The dotted line detection device further includes a network training module,
The network training module,
An operation of performing feature extraction on a road sample image using a feature extraction network to obtain a feature map of the road sample image, wherein the sample road sample image includes a sample dashed lane, and a lane area in the road sample image first label information for labeling , and second label information for labeling a pixel of the sample dotted line;
predicting a lane area in the road sample image based on a feature map of the road sample image using an area prediction network to obtain lane area prediction information;
an operation of predicting a shortcoming pixel in the road sample image based on the feature map of the road sample image using a shortcoming prediction network to obtain weak point pixel prediction information;
determining a first network loss based on a difference between the lane area prediction information and the first label information, and adjusting a network parameter of the feature extraction network and a network parameter of the area prediction network according to the first network loss Operation; and
determine a second network loss based on a difference between the shortcoming pixel prediction information and the second label information, and adjust the network parameter of the shortcoming prediction network and the network parameter of the feature extraction network according to the second network loss through the operation,
to train the feature extraction network, the area prediction network, and the shortcoming prediction network.
A dotted line detection device, characterized in that. 26. The method of claim 25,
The shortcoming pixel labeled through the second label information in the road sample image includes a pixel of the actual shortcoming of the sample dashed lane and an adjacent pixel of the pixel of the actual shortcoming.
A dotted line detection device, characterized in that. 24. The method according to any one of claims 20 to 23,
Further comprising a positioning correction module for correcting positioning information of a smart vehicle on a road corresponding to the road image based on the determined shortcomings of the dotted line
A dotted line detection device, characterized in that. 28. The method of claim 27,
The positioning correction module is
Determine a first distance indicating the distance between the smart vehicle and the target shortcoming of the dotted line determined through the image distance measurement method based on the determined point coordinates of the dotted line,
Based on the location identification information of the smart vehicle and the longitude and latitude of the target point in the driving assistance map used in the smart vehicle, a second distance indicating the distance between the target point and the smart vehicle determined based on the driving assistance map is determined, ,
correcting the positioning information of the smart vehicle based on the error between the first distance and the second distance
A dotted line detection device, characterized in that. As an electronic device,
processor; and
a memory for storing instructions;
15. When the instruction is executed by the processor, the method for detecting a dotted line according to any one of claims 1 to 14 is implemented.
Electronic device, characterized in that. A computer-readable recording medium having a computer program stored therein, comprising:
When the computer program is executed by a processor, the method for detecting a dotted line according to any one of claims 1 to 14 is implemented
A computer-readable recording medium, characterized in that.

Images