CN112396044B - Method for training lane line attribute information detection model and detecting lane line attribute information - Google Patents
Method for training lane line attribute information detection model and detecting lane line attribute information Download PDFInfo
- Publication number
- CN112396044B CN112396044B CN202110078578.9A CN202110078578A CN112396044B CN 112396044 B CN112396044 B CN 112396044B CN 202110078578 A CN202110078578 A CN 202110078578A CN 112396044 B CN112396044 B CN 112396044B
- Authority
- CN
- China
- Prior art keywords
- lane line
- attribute information
- lane
- training
- line
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method for training a lane line attribute information detection model and detecting lane line attribute information, wherein the method for training the lane line attribute information detection model comprises the following steps: acquiring a plurality of lane line training images, wherein the lane line training images are obtained by shooting along the driving direction of a vehicle; inputting a plurality of lane line training images into an initial lane line attribute information detection model to obtain a lane line attribute information probability map, wherein the longitudinal probability value of the lane line attribute information probability map represents the probability value corresponding to each type of lane line attribute; dividing a plurality of detection anchor lines on the lane line attribute information probability map at preset intervals along the longitudinal direction of the lane line attribute information probability map; and adjusting the parameters of the initial lane line attribute information detection model according to the detection result of the nearest detection anchor line to each lane line in the lane line training image until the loss value meets the target condition, thereby obtaining the lane line attribute information detection model.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a method for training a lane line attribute information detection model and detecting lane line attribute information.
Background
The lane line attribute information includes: the existence/nonexistence of the lane line, the type of the lane line, the color of the lane line, the damage degree of the lane line and the like are important guidelines for a driver or an automatic driving perception system to obtain the current vehicle position and plan a short-term path, and key information is provided for drawing a high-precision map.
In the related art, a classification network is usually adopted to detect the attribute information of the lane lines, but the method needs to extract the pixel points of the lane lines firstly, then the extracted pixel points of the lane lines are spliced into a new picture for detection, and the time consumption and the computational resource consumption of the pixel point searching and merging process are large.
Disclosure of Invention
Therefore, the technical problem to be solved by the present invention is to overcome the defects of time consumption and high computational resource consumption of the lane line attribute information detection method in the prior art, so as to provide a lane line attribute information detection model training and a lane line attribute information detection method.
According to a first aspect, the invention discloses a method for training a lane line attribute information detection model, which comprises the following steps: acquiring a plurality of lane line training images, wherein the lane line training images are obtained by shooting along the driving direction of a vehicle; inputting the plurality of lane line training images into an initial lane line attribute information detection model to obtain a lane line attribute information probability map, wherein the longitudinal probability value of the lane line attribute information probability map represents the probability value corresponding to each type of lane line attribute; dividing a plurality of detection anchor lines on the lane line attribute information probability map at preset intervals along the longitudinal direction of the lane line attribute information probability map; and adjusting the parameters of the initial lane line attribute information detection model according to the detection result of the nearest detection anchor line to each lane line in the lane line training image to the lane line until the loss value meets the target condition, thereby obtaining the lane line attribute information detection model.
Optionally, the loss value is determined by the following formula:
wherein LOSS represents the total LOSS value;
representing the loss value of whether the lane line at the ith detection anchor line exists in the training image; d represents the number of detection anchor lines in the training image; m represents the number of lane line attribute categories;
representing the loss value of the jth lane line attribute information at the ith detection anchor line in the training image;
the method comprises the steps that whether a lane line exists at the ith detection anchor line or not is shown, the value of the lane line is 1, and the value of the lane line does not exist and is 0;
indicating the number of lane lines at the detection anchor line;
is a constant, and represents the weight of the jth lane line attribute information.
Optionally, the nearest detection anchor line to each lane line in the lane line training image is obtained by: acquiring a preset associated mark line; and determining the nearest detection anchor line from each lane line in the lane line training image according to the preset associated mark line.
Optionally, the determining, according to the preset associated flag row, a nearest detection anchor line to each lane line in the lane line training image includes:
wherein the content of the first and second substances,
indicating a detection anchor line closest to the lane line;
a first coordinate axis coordinate representing a preset association point of an association mark row and a lane line; itl denotes a preset interval; argmin represents taking the detection anchor line corresponding to the minimum value;
a second coordinate axis coordinate representing a preset associated mark row; y represents a second coordinate axis coordinate set of all pixel points of a preset associated row;
second coordinate axis coordinates representing pixel points of the preset associated mark lines; n represents the number of pixel points of a preset association row; r represents a vector flag; x represents the first coordinate of the preset associated rowAxis coordinates.
Optionally, the inputting the plurality of lane line training images into an initial lane line attribute information detection model includes: and randomly selecting a preset number of lane line training images from the plurality of lane line training images and inputting the lane line training images into the initial lane line attribute information detection model.
According to a second aspect, the invention also discloses a method for detecting attribute information of a lane line, which comprises the following steps: acquiring an image to be detected; inputting the image to be detected into a lane line attribute information detection model for identification to obtain a lane line attribute probability map to be detected, wherein the lane line attribute information detection model is obtained by training through the lane line attribute information detection model training method of the first aspect or any optional embodiment of the first aspect; dividing a plurality of detection anchor lines on the to-be-detected lane line attribute information probability map at preset intervals along the longitudinal direction of the to-be-detected lane line attribute information probability map; and determining the attribute information of the lane lines in the image to be detected according to the detection results of the plurality of detection anchor lines.
According to a third aspect, the present invention also discloses a training apparatus for a lane line attribute information detection model, comprising: the training image acquisition module is used for acquiring a plurality of lane line training images, and the lane line training images are obtained by shooting along the driving direction of the vehicle; the input module is used for inputting the plurality of lane line training images into an initial lane line attribute information detection model to obtain a lane line attribute information probability map, and the longitudinal probability value of the lane line attribute information probability map represents the probability value corresponding to each type of lane line attribute; the first dividing module is used for dividing a plurality of detection anchor lines on the lane line attribute information probability map at preset intervals along the longitudinal direction of the lane line attribute information probability map; and the adjusting module is used for adjusting the parameters of the initial lane line attribute information detection model according to the detection result of the nearest detection anchor line to each lane line in the lane line training image to the lane line until the loss value meets the target condition, so as to obtain the lane line attribute information detection model.
According to a fourth aspect, the present invention also discloses a lane line attribute information detection apparatus, including: the image acquisition module to be detected is used for acquiring an image to be detected; the recognition module is used for inputting the image to be detected into a lane line attribute information detection model for recognition to obtain a lane line attribute probability map to be detected, wherein the lane line attribute information detection model is obtained by training through the lane line attribute information detection model training method in the first aspect or any optional embodiment in the first aspect; the second dividing module is used for dividing a plurality of detection anchor lines on the to-be-detected lane line attribute information probability map at preset intervals along the longitudinal direction of the to-be-detected lane line attribute information probability map; and the lane line attribute information determining module is used for determining the lane line attribute information in the image to be detected according to the detection results of the plurality of detection anchor lines.
According to a fifth aspect, the present invention also discloses a computer device, comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executable by the at least one processor to cause the at least one processor to perform the steps of the lane line attribute information detection model training method according to the first aspect or any one of the optional embodiments of the first aspect or the steps of the lane line attribute information detection method according to the second aspect.
According to a sixth aspect, the present invention further discloses a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the lane line attribute information detection model training method according to the first aspect or any one of the optional embodiments of the first aspect or the steps of the lane line attribute information detection method according to the second aspect.
The technical scheme of the invention has the following advantages:
1. the invention provides a method and a device for training a lane line attribute information detection model, which are characterized in that a lane line attribute information probability map is obtained by acquiring a plurality of lane line training images which are obtained by shooting along the driving direction of a vehicle and inputting the plurality of lane line training images into an initial lane line attribute information detection model, the longitudinal probability value of the lane line attribute information probability map represents the probability value corresponding to each type of lane line attribute along the longitudinal direction of the lane line attribute information probability map, and dividing a plurality of detection anchor lines on the lane line attribute information probability map at preset intervals, and adjusting the parameters of an initial lane line attribute information detection model according to the detection result of the nearest detection anchor line to each lane line in the lane line training image until the loss value of the target loss function meets the target condition to obtain the lane line attribute information detection model. The method comprises the steps of inputting a training image into an initial lane line attribute information detection model for feature extraction to obtain a lane line attribute information probability map, adapting to a scene with severe change and resisting interference of external noise, segmenting the lane line attribute information probability map according to longitudinal anchor lines, adjusting parameters of the initial lane line attribute information detection model by taking a detection result of a nearest detection anchor line to the lane line in the lane line training image as a detection result, and identifying the pixels without extracting the pixels of the lane line and forming a new image for identification, thereby reducing the consumption of time and computational resources.
2. The invention provides a method and a device for detecting lane line attribute information, which are characterized in that an image to be detected is obtained, the image to be detected is input into a lane line attribute information detection model for identification to obtain a lane line attribute probability map to be detected, a plurality of detection anchor lines are marked on the lane line attribute information probability map to be detected at preset intervals along the longitudinal direction of the lane line attribute information probability map to be detected, and lane line attribute information in the image to be detected is determined according to the detection results of the plurality of detection anchor lines. The invention can detect a plurality of lane lines in parallel and dynamically by inputting the image to be detected into the lane line attribute information detection model for identification, thereby improving the identification efficiency, reducing the time consumption, avoiding extracting pixel points and reducing the consumption of computational resources.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart of a specific example of a method for training a lane line attribute information detection model according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating an exemplary probability map of lane line attribute information according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating an exemplary embodiment of detecting anchor line division;
fig. 4 is a flowchart of a specific example of a lane line attribute information detection method according to an embodiment of the present invention;
FIG. 5 is a diagram showing an exemplary effect of the embodiment of the present invention;
fig. 6 is a schematic block diagram of a specific example of a lane line attribute information detection model training apparatus according to an embodiment of the present invention;
fig. 7 is a schematic block diagram of a specific example of the lane line attribute information detection apparatus in the embodiment of the present invention;
FIG. 8 is a diagram showing an exemplary embodiment of a computer device.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the accompanying drawings, and it should be understood that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the description of the present invention, it should be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; the two elements may be directly connected or indirectly connected through an intermediate medium, or may be communicated with each other inside the two elements, or may be wirelessly connected or wired connected. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
In addition, the technical features involved in the different embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
The embodiment of the invention discloses a method for training a lane line attribute information detection model, which comprises the following steps as shown in figure 1:
s11: a plurality of lane line training images are acquired, and the lane line training images are obtained by shooting along the driving direction of the vehicle.
For example, the lane line training image may be a real image captured by a camera (e.g., a vehicle-mounted camera, a roadside camera, etc.), or may be an artificially synthesized image. The embodiment of the invention does not specifically limit the lane line training image, as long as the lane line training image is obtained by shooting along the driving direction of the vehicle.
The lane line training image can be directly obtained from the camera according to a protocol or obtained by searching from a search engine.
In the embodiment of the invention, the acquired lane line training image can be directly an image with a label, or can be marked after the acquired lane line training image without the label is acquired, so that the lane line training image with the label is obtained.
As an optional implementation manner of the embodiment of the present invention, after a plurality of lane line training images are acquired, the lane line training images are preprocessed. The pre-processing may include: unifying image formats, for example, converting a data format into a TFRecord format, which is a binary format; the pre-processing may further include: the uniformity of the image size, in the embodiment of the present invention, the image size may be set to 512 × 256 × 3; the pre-processing may further include: image blurring, denoising, and the like. The embodiment of the present invention does not specifically limit the pretreatment, and those skilled in the art can determine the pretreatment according to actual situations.
S12: inputting a plurality of lane line training images into the initial lane line attribute information detection model to obtain a lane line attribute information probability map, wherein the longitudinal probability value of the lane line attribute information probability map represents the probability value corresponding to each type of lane line attribute.
For example, the lane line attribute information probability map may be a network probability space map (for example, the lane line attribute information probability map shown in fig. 2), or may also be a planar probability map (probability matrix).
The lane line attribute information may include: the presence or absence of a lane line, the number of lane lines, the color of a lane line (e.g., yellow, white, etc.), the type of a lane line (e.g., solid line, dashed line, etc.), whether a lane line is broken, etc., which is not specifically limited by the embodiment of the present invention and can be determined by a person skilled in the art according to actual circumstances.
In the embodiment of the present invention, the probability value corresponding to each type of lane line attribute refers to a probability value corresponding to each specific lane line attribute, for example, the probability value corresponding to yellow color of a lane line, the probability value corresponding to a solid line type of a lane line, and the like. The longitudinal probability value of the lane line attribute information probability map represents the probability value corresponding to each type of lane line attribute, that is, each column includes the probability value corresponding to each type of lane line attribute, for example, as shown in fig. 2, Pc, C0, C1 … … Cn, T0, T1 … … Tn all represent the prediction probability that a lane line belongs to the type of attribute, An0, An1 … … Ann represent detection anchor lines divided at equal intervals, and for An0 column, it includes: probability values of various colors of the lane line, probability values of various types, probability values of whether the lane line is broken, and the like. The lane line attribute information probability map specifically shows that the probability values corresponding to the lane line attributes can be determined according to actual detection requirements.
For example, determining whether a lane line exists at the currently detected anchor line, determining the lane line color and the lane line type may be defined by the following formulas:
wherein the content of the first and second substances,
a predicted probability representing the presence of a lane line; threshold represents the probability threshold of the existence of the lane line, and the threshold is distributed in [0, 1 ]]Within the interval; the lane line COLOR determining mode is that argmax is used for solving a position index with the maximum probability in a lane line COLOR probability vector COL, and the lane line COLOR of network inference is extracted from a COLOR category list COLOR according to the index position; the lane line type attribute prediction process is completely the same as the color prediction process; t represents a probability vector of a lane line type; r represents a vector flag; m1 represents the number of color categories; m2 representsNumber of lane line types.
In the actual use stage, the attribute types which can be detected by the lane line attribute detection network can be freely added according to the same method.
Inputting a plurality of lane line training images to the initial lane line attribute information detection model may input all of the lane line training images to the initial lane line attribute information detection model, or may input a part (for example, half) of the lane line training images to the initial lane line attribute information detection model at a time. The number of lane line training images input to the initial lane line attribute information detection model each time may be the same or different. The embodiment of the invention does not specifically limit the number of the lane line training images input into the initial lane line attribute information detection model and the number of the lane line training images input into the initial lane line attribute information detection model each time, and can be determined by a person skilled in the art according to actual conditions.
In the embodiment of the invention, a plurality of lane line training images are input into an initial lane line attribute information detection model for convolution calculation to obtain a feature map (namely a lane line attribute information probability map) of the image, wherein the feature map can be a matrix of N, H, W, C, N is a batch size, namely the number of the images input in parallel at one time, H is an image height, W is an image width, and C is a channel.
As an optional implementation manner of the embodiment of the present invention, inputting a plurality of lane line training images to an initial lane line attribute information detection model includes:
and randomly selecting a preset number of lane line training images from the plurality of lane line training images and inputting the lane line training images into the initial lane line attribute information detection model. The preset number may be 100, and the preset data is not specifically limited in the embodiment of the present invention, and may be determined by a person skilled in the art according to an actual situation.
According to the embodiment of the invention, the preset number of lane line training images are randomly selected from the plurality of lane line training images and input into the initial lane line attribute information detection model, so that the initial lane line attribute information detection model is prevented from learning the training images in a fixed sequence, and meanwhile, the model is prevented from emphasizing the training images input finally (forgetfulness of a neural network) when the learning rate is consistent, and the generalization capability of the model is improved. Meanwhile, noise may exist in the training images, the sequence of the samples is randomly disturbed, the training images with noise are mixed in the normal training images, and negative effects of noise data can be reduced.
S13: and dividing a plurality of detection anchor lines on the lane line attribute information probability map at preset intervals along the longitudinal direction of the lane line attribute information probability map.
For example, a coordinate system may be first established for the probability map of the attribute information of the lane lines, as shown in fig. 3, where x represents the row direction of the image coordinate system, y represents the column direction of the image coordinate system,
representing a detection anchor line;
representing a preset association row for determining a detection anchor line closest to each lane line; the thick lines in the longitudinal direction distributed in the image coordinate system are lane line exemplary diagrams.
The preset interval may be randomly set, and the preset interval is not specifically limited in the embodiment of the present invention, and may be determined by a person skilled in the art according to an actual situation of a lane line in an image. For example, the training image column cols =512, and the preset interval is itl =16, so that the training image column needs to be arranged in the column direction
The strip detects the anchor line.
Preferably, the number of the detection anchor lines divided according to the preset interval needs to be greater than the number of the lane lines. The number of lane lines on the current road is generally 4 at most, and therefore, the number of detection anchor lines may be set to be more than 4. In the embodiment of the present invention, in order to ensure that all lane lines can be detected, the number of the divided detection anchor lines may be set to 64.
S14: and adjusting the parameters of the initial lane line attribute information detection model according to the detection result of the nearest detection anchor line to each lane line in the lane line training image until the loss value meets the target condition, thereby obtaining the lane line attribute information detection model.
Illustratively, the nearest detection anchor line to each lane line in the lane line training image may be obtained by:
first, a preset associated marker line is obtained. The preset associated mark line may be self-defined, and the embodiment of the present invention does not specifically limit the preset associated mark line, as long as it is ensured that the preset associated mark line has an intersection with each lane line.
Secondly, determining the nearest detection anchor line from each lane line in the lane line training image according to the preset associated mark line.
Exemplarily, determining a nearest detection anchor line from each lane line in the lane line training image according to the preset associated flag row may specifically be:
wherein the content of the first and second substances,
indicating a detection anchor line closest to the lane line;
a first coordinate axis coordinate (column coordinate) representing a preset association point of the association mark row and the lane line; itl denotes a preset interval; argmin represents taking the detection anchor line corresponding to the minimum value;
a second coordinate axis coordinate (abscissa) representing a preset associated index row; y represents a second coordinate axis coordinate set of all pixel points of a preset associated row;
second coordinate axis coordinates representing pixel points of the preset associated mark lines; n represents the number of pixel points of a preset association row; r represents a vector flag; x denotes a first coordinate axis coordinate (column coordinate) of the preset associated row.
In the embodiment of the present invention, the loss value may be a sum of loss values of each type of lane line attribute information, where each type of lane line attribute may set a different loss function, for example, whether a lane line exists using a binary cross entropy function, a lane line color and a lane line type using a cross entropy function, and the like, specifically:
wherein LOSS represents the total LOSS value;
a loss value representing whether the lane line exists at the ith detection anchor line in the training image,
；
representing a real tag;
representing a prediction result; d represents the number of detection anchor lines in the training image; m represents the number of lane line attribute categories;
representing the loss value of the jth type lane line attribute information at the ith detection anchor line in the training image,
；
a representation true value tag;
representing a prediction probability;
the method comprises the steps that whether a lane line exists at the ith detection anchor line or not is shown, the value of the lane line is 1, and the value of the lane line does not exist and is 0;
indicating the number of lane lines at the detection anchor line;
is a constant, and represents the weight of the jth lane line attribute information.
In the embodiment of the present invention, the target condition may be that the loss value is within a preset range (less than 0.01), and the preset range may be determined according to an actual demand.
When the loss value does not meet the target condition, the loss value is input into an optimizer, an SGD (random gradient descent) optimizer is adopted in the embodiment of the invention, and the optimization steps are as follows: (1) selecting a sample from a plurality of training images; (2) calculating partial derivatives of the loss function to all weights and biases; (3) updating network parameters (weight values and bias values) of the lane line attribute information detection model; (4) go back to step S11 to continue training. The above process is a process of one training step of the model, the above training process is repeatedly executed until the model loss value meets the target condition, and the model training is ended. After the trained lane line attribute information detection model and the network parameters are frozen, the model can be used for deployment and is used for actual industrial production.
The invention provides a method for training a lane line attribute information detection model, which comprises the steps of obtaining a plurality of lane line training images, shooting the lane line training images along the driving direction of a vehicle, inputting the plurality of lane line training images into an initial lane line attribute information detection model to obtain a lane line attribute information probability map, representing the probability value corresponding to each type of lane line attribute by the longitudinal probability value of the lane line attribute information probability map, dividing a plurality of detection anchor lines on the lane line attribute information probability map at preset intervals along the longitudinal direction of the lane line attribute information probability map, adjusting the parameters of the initial lane line attribute information detection model by the detection result of the nearest detection anchor line to each lane line in the lane line training images until the loss value meets the target condition, and obtaining the lane line attribute information detection model. The method comprises the steps of inputting a training image into an initial lane line attribute information detection model for feature extraction to obtain a lane line attribute information probability map, adapting to a scene with severe change and resisting interference of external noise, segmenting the lane line attribute information probability map according to longitudinal anchor lines, adjusting parameters of the initial lane line attribute information detection model by taking a detection result of a nearest detection anchor line to the lane line in the lane line training image as a detection result, and identifying the pixels without extracting the pixels of the lane line and forming a new image for identification, thereby reducing the consumption of time and computational resources.
The embodiment of the invention also discloses a method for detecting the attribute information of the lane line, which comprises the following steps as shown in figure 4:
s21: acquiring an image to be detected; the image to be detected is an image acquired by a vehicle-mounted camera of a target vehicle in real time, and the image to be detected is acquired from the camera by a wired network or wireless network method according to a protocol.
S22: the image to be detected is input into the lane line attribute information detection model for identification to obtain a lane line attribute probability map to be detected, and the lane line attribute information detection model is obtained through training by the lane line attribute information detection model training method of the embodiment, so that a target vehicle can plan a short-term path conveniently.
S23: dividing a plurality of detection anchor lines on the attribute information probability map of the lane line to be detected at preset intervals along the longitudinal direction of the attribute information probability map of the lane line to be detected; the dividing method for detecting the anchor line is consistent with the dividing method for training the lane line attribute information detection model, and reference may be specifically made to the description of step S13 in the above embodiment, which is not described herein again.
S24: and determining the attribute information of the lane lines in the image to be detected according to the detection results of the plurality of detection anchor lines.
For example, the determining of the attribute information of the lane line in the image to be detected according to the detection results of the multiple detection anchor lines may specifically be that a detection anchor line closest to each lane line is determined first, and a probability value corresponding to the detection anchor line closest to each lane line is used as the attribute information of the lane line of each lane line in the image to be detected.
The invention provides a lane line attribute information detection method, which comprises the steps of obtaining an image to be detected, inputting the image to be detected into a lane line attribute information detection model for identification to obtain a lane line attribute probability map to be detected, marking off a plurality of detection anchor lines on the lane line attribute information probability map to be detected at preset intervals along the longitudinal direction of the lane line attribute information probability map to be detected, and determining lane line attribute information in the image to be detected according to the detection results of the plurality of detection anchor lines. The invention can detect a plurality of lane lines in parallel and dynamically by inputting the image to be detected into the lane line attribute information detection model for identification, thereby improving the identification efficiency, reducing the time consumption, avoiding extracting pixel points and reducing the consumption of computational resources.
Fig. 5 is a specific example diagram of an effect diagram in an embodiment of the present invention, in which bold lines represent lane lines, and non-bold vertical lines represent detection anchor lines, and when viewed from left to right, a first detection anchor line is responsible for detecting attribute information of a first lane line, and a second detection anchor line is responsible for detecting attribute information … … of a second lane line, where a detection result of the first lane line is: the material is as follows: the paint and the color are as follows: white, type: the probability of existence of the solid line and the lane line is as follows: 0.97377366, the virtual attribute is "no", i.e. the actual lane line.
The embodiment of the present invention further discloses a device for training a lane line attribute information detection model, as shown in fig. 6, including:
a training image obtaining module 31 configured to obtain a plurality of lane line training images, which are obtained by shooting along a vehicle traveling direction; the specific implementation manner is described in the above embodiment in relation to step S11, and is not described herein again.
The input module 32 is configured to input the multiple lane line training images to the initial lane line attribute information detection model to obtain a lane line attribute information probability map, where a longitudinal probability value of the lane line attribute information probability map represents a probability value corresponding to each type of lane line attribute; the specific implementation manner is described in the above embodiment in relation to step S12, and is not described herein again.
A first dividing module 33, configured to divide a plurality of detection anchor lines on the lane line attribute information probability map at preset intervals along a longitudinal direction of the lane line attribute information probability map; the specific implementation manner is described in the above embodiment in relation to step S13, and is not described herein again.
And the adjusting module 34 is configured to adjust parameters of the initial lane line attribute information detection model according to a detection result of the closest detection anchor line to each lane line in the lane line training image to the lane line until the loss value meets the target condition, so as to obtain the lane line attribute information detection model. The specific implementation manner is described in the above embodiment in relation to step S14, and is not described herein again.
The invention provides a lane line attribute information detection model training device, which obtains a lane line attribute information detection model by obtaining a plurality of lane line training images, wherein the lane line training images are obtained by shooting along the driving direction of a vehicle, the plurality of lane line training images are input into an initial lane line attribute information detection model to obtain a lane line attribute information probability map, the longitudinal probability value of the lane line attribute information probability map represents the probability value corresponding to each type of lane line attribute, a plurality of detection anchor lines are divided on the lane line attribute information probability map at preset intervals along the longitudinal direction of the lane line attribute information probability map, and the parameters of the initial lane line attribute information detection model are adjusted according to the detection result of the nearest detection anchor line to the lane line in the lane line training images until the loss value meets the target condition. The method comprises the steps of inputting a training image into an initial lane line attribute information detection model for feature extraction to obtain a lane line attribute information probability map, adapting to a scene with severe change and resisting interference of external noise, segmenting the lane line attribute information probability map according to longitudinal anchor lines, adjusting parameters of the initial lane line attribute information detection model by taking a detection result of a nearest detection anchor line to the lane line in the lane line training image as a detection result, and identifying the pixels without extracting the pixels of the lane line and forming a new image for identification, thereby reducing the consumption of time and computational resources.
As an alternative implementation manner of the embodiment of the present invention, the loss value may be calculated by the following formula:
wherein LOSS represents the total LOSS value;
representing the loss value of whether the lane line at the ith detection anchor line exists in the training image; d represents the number of detection anchor lines in the training image; m represents the number of lane line attribute categories;
representing the loss value of the jth lane line attribute information at the ith detection anchor line in the training image;
the method comprises the steps that whether a lane line exists at the ith detection anchor line or not is shown, the value of the lane line is 1, and the value of the lane line does not exist and is 0;
indicating the number of lane lines at the detection anchor line;
is a constant, and represents the weight of the jth lane line attribute information.
As an optional implementation manner of the embodiment of the present invention, the device for training a lane line attribute information detection model further includes:
the preset associated mark line acquisition module is used for acquiring a preset associated mark line; the specific implementation manner is described in the relevant description of the corresponding steps in the above embodiments, and is not described herein again.
And the detection anchor line determining module is used for determining the nearest detection anchor line from each lane line in the lane line training image according to the preset associated mark line. The specific implementation manner is described in the relevant description of the corresponding steps in the above embodiments, and is not described herein again.
As an optional implementation manner of the embodiment of the present invention, the detection anchor line determining module includes:
wherein the content of the first and second substances,
indicating a detection anchor line closest to the lane line;
a first coordinate axis coordinate representing a preset association point of an association mark row and a lane line; itl denotes a preset interval; argmin represents taking the detection anchor line corresponding to the minimum value;
a second coordinate axis coordinate representing a preset associated mark row; y represents a second coordinate axis coordinate set of all pixel points of a preset associated row;
second coordinate axis coordinates representing pixel points of the preset associated mark lines; n represents the number of pixel points of a preset association row; r represents a vector flag; x represents a first coordinate axis coordinate of a preset associated row.
As an optional implementation manner of the embodiment of the present invention, the input module includes:
and the selection module is used for randomly selecting a preset number of lane line training images from the plurality of lane line training images and inputting the lane line training images into the initial lane line attribute information detection model. The specific implementation manner is described in the relevant description of the corresponding steps in the above embodiments, and is not described herein again.
The embodiment of the present invention further discloses a device for detecting attribute information of a lane line, as shown in fig. 7, including:
an image to be detected acquisition module 41, configured to acquire an image to be detected; the specific implementation manner is described in the above embodiment in relation to step S21, and is not described herein again.
The recognition module 42 is configured to input the image to be detected into the lane line attribute information detection model for recognition, so as to obtain a to-be-detected lane line attribute probability map, where the lane line attribute information detection model is obtained by training through the lane line attribute information detection model training method in the embodiment; the specific implementation manner is described in the above embodiment in relation to step S22, and is not described herein again.
A second dividing module 43, configured to divide a plurality of detection anchor lines on the to-be-detected lane line attribute information probability map at preset intervals along the longitudinal direction of the to-be-detected lane line attribute information probability map; the specific implementation manner is described in the above embodiment in relation to step S23, and is not described herein again.
And the lane line attribute information determining module 44 is configured to determine lane line attribute information in the image to be detected according to the detection results of the multiple detection anchor lines. The specific implementation manner is described in the above embodiment in relation to step S24, and is not described herein again.
The device for detecting the attribute information of the lane line, provided by the invention, is used for obtaining an image to be detected, inputting the image to be detected into a lane line attribute information detection model for identification to obtain a lane line attribute probability map to be detected, marking off a plurality of detection anchor lines on the lane line attribute information probability map to be detected at preset intervals along the longitudinal direction of the lane line attribute information probability map to be detected, and determining the lane line attribute information in the image to be detected according to the detection results of the plurality of detection anchor lines. The invention can detect a plurality of lane lines in parallel and dynamically by inputting the image to be detected into the lane line attribute information detection model for identification, thereby improving the identification efficiency, reducing the time consumption, avoiding extracting pixel points and reducing the consumption of computational resources.
An embodiment of the present invention further provides a computer device, as shown in fig. 8, the computer device may include a processor 51 and a memory 52, where the processor 51 and the memory 52 may be connected by a bus or in another manner, and fig. 8 takes the example of connection by a bus as an example.
The processor 51 may be a Central Processing Unit (CPU). The Processor 51 may also be other general purpose processors, Digital Signal Processors (SPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, or combinations thereof.
The memory 52 is a non-transitory computer-readable storage medium, and can be used to store a non-transitory software program, a non-transitory computer-executable program, and modules, such as program instructions/modules corresponding to the lane line attribute information detection model training or the lane line attribute information detection method in the embodiment of the present invention (for example, the training image acquisition module 31, the input module 32, the first division module 33, and the adjustment module 34 shown in fig. 6, or the to-be-detected image acquisition module 41, the identification module 42, the second division module 43, and the lane line attribute information determination module 44 shown in fig. 7). The processor 51 executes various functional applications and data processing of the processor by running non-transitory software programs, instructions and modules stored in the memory 52, that is, implements the lane line attribute information detection model training or lane line attribute information detection method in the above-described method embodiment.
The memory 52 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by the processor 51, and the like. Further, the memory 52 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 52 may optionally include memory located remotely from the processor 51, and these remote memories may be connected to the processor 51 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more modules are stored in the memory 52, and when executed by the processor 51, perform a lane wire attribute information detection model training method as in the embodiment shown in fig. 1 or a lane wire attribute information detection method as shown in fig. 4.
The details of the computer device can be understood by referring to the corresponding related descriptions and effects in the embodiments shown in fig. 1 or fig. 4, and are not described herein again.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, abbreviated as HDD), a Solid State Drive (SSD), or the like; the storage medium may also comprise a combination of memories of the kind described above.
Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope defined by the appended claims.
Claims (9)
1. A method for training a lane line attribute information detection model is characterized by comprising the following steps:
acquiring a plurality of lane line training images, wherein the lane line training images are obtained by shooting along the driving direction of a vehicle;
inputting the plurality of lane line training images into an initial lane line attribute information detection model to obtain a lane line attribute information probability map, wherein the longitudinal probability value of the lane line attribute information probability map represents the probability value corresponding to each type of lane line attribute;
dividing a plurality of detection anchor lines on the lane line attribute information probability map at preset intervals along the longitudinal direction of the lane line attribute information probability map;
acquiring a preset associated mark line;
determining a nearest detection anchor line from each lane line in the lane line training image according to the preset correlation mark row;
and adjusting the parameters of the initial lane line attribute information detection model according to the detection result of the nearest detection anchor line to each lane line in the lane line training image to the lane line until the loss value meets the target condition, thereby obtaining the lane line attribute information detection model.
2. The method of claim 1, wherein the loss value is determined by the formula:
wherein the content of the first and second substances,
represents the total loss value;
representing the loss value of whether the lane line at the ith detection anchor line exists in the training image; d represents the number of detection anchor lines in the training image; m represents the number of lane line attribute categories;
representing the loss value of the jth lane line attribute information at the ith detection anchor line in the training image;
the method comprises the steps that whether a lane line exists at the ith detection anchor line or not is shown, the value of the lane line is 1, and the value of the lane line does not exist and is 0;
indicating the number of lane lines at the detection anchor line;
is a constant, and represents the weight of the jth lane line attribute information.
3. The method of claim 1, wherein determining a nearest inspection anchor line from each lane line in a lane line training image according to the preset associated marker row comprises:
wherein the content of the first and second substances,
indicating a detection anchor line closest to the lane line;
a first coordinate axis coordinate representing a preset association point of an association mark row and a lane line;
representing a preset interval;
representing the detection anchor line corresponding to the minimum value;
a second coordinate axis coordinate representing a preset associated mark row;
a second coordinate axis coordinate set representing all pixel points of a preset associated row;
second coordinate axis coordinates representing pixel points of the preset associated mark lines; n represents the number of pixel points of a preset association row; r represents a vector flag; x represents a first coordinate axis coordinate of a preset associated row.
4. The method of claim 1, wherein the inputting the plurality of lane line training images to an initial lane line attribute information detection model comprises:
and randomly selecting a preset number of lane line training images from the plurality of lane line training images and inputting the lane line training images into the initial lane line attribute information detection model.
5. A method for detecting attribute information of a lane line is characterized by comprising the following steps:
acquiring an image to be detected;
inputting the image to be detected into a lane line attribute information detection model for recognition to obtain a lane line attribute information probability map to be detected, wherein the lane line attribute information detection model is obtained by training through the lane line attribute information detection model training method of any one of claims 1 to 4;
dividing a plurality of detection anchor lines on the to-be-detected lane line attribute information probability map at preset intervals along the longitudinal direction of the to-be-detected lane line attribute information probability map;
acquiring a preset associated mark line;
determining a nearest detection anchor line from each lane line in the lane line training image according to the preset correlation mark row;
and determining the attribute information of the lane lines in the image to be detected according to the detection result of the nearest detection anchor line of each lane line in the training image of the lane lines.
6. A lane line attribute information detection model training device is characterized by comprising:
the training image acquisition module is used for acquiring a plurality of lane line training images, and the lane line training images are obtained by shooting along the driving direction of the vehicle;
the input module is used for inputting the plurality of lane line training images into an initial lane line attribute information detection model to obtain a lane line attribute information probability map, and the longitudinal probability value of the lane line attribute information probability map represents the probability value corresponding to each type of lane line attribute;
the first dividing module is used for dividing a plurality of detection anchor lines on the lane line attribute information probability map at preset intervals along the longitudinal direction of the lane line attribute information probability map;
the preset associated mark line acquisition module is used for acquiring a preset associated mark line;
the detection anchor line determining module is used for determining the nearest detection anchor line from each lane line in the lane line training image according to the preset associated mark line;
and the adjusting module is used for adjusting the parameters of the initial lane line attribute information detection model according to the detection result of the nearest detection anchor line to each lane line in the lane line training image to the lane line until the loss value meets the target condition, so as to obtain the lane line attribute information detection model.
7. A lane line attribute information detection device, characterized by comprising:
the image acquisition module to be detected is used for acquiring an image to be detected;
the recognition module is used for inputting the image to be detected into a lane line attribute information detection model for recognition to obtain a lane line attribute information probability map to be detected, wherein the lane line attribute information detection model is obtained by training through the lane line attribute information detection model training method of any one of claims 1 to 4;
the second dividing module is used for dividing a plurality of detection anchor lines on the to-be-detected lane line attribute information probability map at preset intervals along the longitudinal direction of the to-be-detected lane line attribute information probability map;
the preset associated mark line acquisition module is used for acquiring a preset associated mark line;
the detection anchor line determining module is used for determining the nearest detection anchor line from each lane line in the lane line training image according to the preset associated mark line;
and the lane line attribute information determining module is used for determining the lane line attribute information in the image to be detected according to the detection result of the nearest detection anchor line away from each lane line in the lane line training image.
8. A computer device, comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the steps of the lane wire attribute information detection model training method of any one of claims 1-4 or the steps of the lane wire attribute information detection method of claim 5.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the lane wire attribute information detection model training method according to any one of claims 1 to 4 or the steps of the lane wire attribute information detection method according to claim 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110078578.9A CN112396044B (en) | 2021-01-21 | 2021-01-21 | Method for training lane line attribute information detection model and detecting lane line attribute information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110078578.9A CN112396044B (en) | 2021-01-21 | 2021-01-21 | Method for training lane line attribute information detection model and detecting lane line attribute information |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112396044A CN112396044A (en) | 2021-02-23 |
| CN112396044B true CN112396044B (en) | 2021-04-27 |
Family
ID=74625546
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110078578.9A Active CN112396044B (en) | 2021-01-21 | 2021-01-21 | Method for training lane line attribute information detection model and detecting lane line attribute information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112396044B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112906810B (en) * | 2021-03-08 | 2024-04-16 | 共达地创新技术(深圳)有限公司 | Target detection method, electronic device, and storage medium |
| CN113361447A (en) * | 2021-06-23 | 2021-09-07 | 中国科学技术大学 | Lane line detection method and system based on sliding window self-attention mechanism |
| CN113442856B (en) * | 2021-08-31 | 2021-12-10 | 国汽智控(北京)科技有限公司 | Control method and device based on adaptive platform and ROS2 and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111104838A (en) * | 2018-10-26 | 2020-05-05 | 斯特拉德视觉公司 | Method and device for detecting ROI (region of interest) based on lower end line of obstacle |
| CN111259796A (en) * | 2020-01-16 | 2020-06-09 | 东华大学 | Lane line detection method based on image geometric features |
| US20200302189A1 (en) * | 2018-03-09 | 2020-09-24 | Tencent Technology (Shenzhen) Company Limited | Lane line data processing method and apparatus, computer device, and storage medium |
| CN112016463A (en) * | 2020-08-28 | 2020-12-01 | 佛山市南海区广工大数控装备协同创新研究院 | Deep learning-based lane line detection method |
| CN112131914A (en) * | 2019-06-25 | 2020-12-25 | 北京市商汤科技开发有限公司 | Lane line attribute detection method and device, electronic equipment and intelligent equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10423840B1 (en) * | 2019-01-31 | 2019-09-24 | StradVision, Inc. | Post-processing method and device for detecting lanes to plan the drive path of autonomous vehicle by using segmentation score map and clustering map |
-
2021
- 2021-01-21 CN CN202110078578.9A patent/CN112396044B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200302189A1 (en) * | 2018-03-09 | 2020-09-24 | Tencent Technology (Shenzhen) Company Limited | Lane line data processing method and apparatus, computer device, and storage medium |
| CN111104838A (en) * | 2018-10-26 | 2020-05-05 | 斯特拉德视觉公司 | Method and device for detecting ROI (region of interest) based on lower end line of obstacle |
| CN112131914A (en) * | 2019-06-25 | 2020-12-25 | 北京市商汤科技开发有限公司 | Lane line attribute detection method and device, electronic equipment and intelligent equipment |
| CN111259796A (en) * | 2020-01-16 | 2020-06-09 | 东华大学 | Lane line detection method based on image geometric features |
| CN112016463A (en) * | 2020-08-28 | 2020-12-01 | 佛山市南海区广工大数控装备协同创新研究院 | Deep learning-based lane line detection method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112396044A (en) | 2021-02-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112396044B (en) | Method for training lane line attribute information detection model and detecting lane line attribute information | |
| CN107590470B (en) | Lane line detection method and device | |
| CN108090459B (en) | Traffic sign detection and identification method suitable for vehicle-mounted vision system | |
| CN109697397B (en) | Target detection method and device | |
| CN111274926B (en) | Image data screening method, device, computer equipment and storage medium | |
| CN111723854B (en) | Expressway traffic jam detection method, equipment and readable storage medium | |
| CN111967396A (en) | Processing method, device and equipment for obstacle detection and storage medium | |
| EP4170601A1 (en) | Traffic marker detection method and training method for traffic marker detection model | |
| CN111898491A (en) | Method and device for identifying reverse driving of vehicle and electronic equipment | |
| CN112215190A (en) | Illegal building detection method based on YOLOV4 model | |
| CN113902740A (en) | Construction method of image blurring degree evaluation model | |
| US20200250443A1 (en) | Information processing device, server, and traffic management system | |
| KR20150146374A (en) | System for lane recognition using environmental information and method thereof | |
| CN116363100A (en) | Image quality evaluation method, device, equipment and storage medium | |
| CN113743163A (en) | Traffic target recognition model training method, traffic target positioning method and device | |
| CN110555344B (en) | Lane line recognition method, lane line recognition device, electronic device, and storage medium | |
| CN112784817B (en) | Method, device and equipment for detecting lane where vehicle is located and storage medium | |
| CN113989774A (en) | Traffic light detection method and device, vehicle and readable storage medium | |
| CN115393792A (en) | Target abnormal state detection method and device and electronic equipment | |
| CN116311015A (en) | Road scene recognition method, device, server, storage medium and program product | |
| CN112149600A (en) | Big data automobile driving area image analysis method and system | |
| CN112215189A (en) | Accurate detecting system for illegal building | |
| CN104700396B (en) | The method and system of the parameter for estimating the volume of traffic is determined from image | |
| CN112654997B (en) | Lane line detection method and device | |
| CN112364795B (en) | Automatic identification method for number lamps and situation awareness method for two ships meeting |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |