CN114821534A - Lane line detection method, electronic device, medium, and vehicle - Google Patents
Lane line detection method, electronic device, medium, and vehicle Download PDFInfo
- Publication number
- CN114821534A CN114821534A CN202210509064.9A CN202210509064A CN114821534A CN 114821534 A CN114821534 A CN 114821534A CN 202210509064 A CN202210509064 A CN 202210509064A CN 114821534 A CN114821534 A CN 114821534A
- Authority
- CN
- China
- Prior art keywords
- lane line
- line detection
- image
- layer
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention relates to the field of artificial intelligence, particularly provides a lane line detection method, electronic equipment, a medium and a vehicle, and aims to solve the technical problem that the existing lane line detection method is poor in detection accuracy. To this end, the lane line detection method of the present invention includes: acquiring an image to be detected containing a lane line; the method comprises the steps of inputting an image to be detected into a lane line detection model trained by a distillation method to identify a lane line, wherein a main network of a teacher model used by the distillation method comprises a conversion module and at least one distinguishing feature generation module, and the distinguishing feature generation module comprises a convolution layer, a global average pooling layer and at least one full-connection layer which are sequentially connected. Thus, the detection precision of the lane line is improved.
Description
Technical Field
The invention relates to the field of artificial intelligence, and particularly provides a lane line detection method, electronic equipment, a medium and a vehicle.
Background
Currently, the detection of lane lines is mostly realized by an automatic driving perception model. The automatic driving perception model is often very light in weight due to the fact that the automatic driving perception model is required to meet vehicle end deployment, and the performance of the corresponding model has a large promotion space. Knowledge distillation is an effective way to improve the performance of lightweight models without any changes to the lightweight models at the inference stage. However, in practice, there is no distillation method for effectively transferring knowledge distillation to a lane line detection scene, so that the conventional distillation method for a lightweight model has a poor distillation effect, and when the distillation method is used for lane line detection, lane line detection accuracy is poor, and a satisfactory detection effect is difficult to achieve.
Accordingly, there is a need in the art for a new lane line detection method to solve the above problems.
Disclosure of Invention
In order to overcome the above-mentioned drawbacks, the present invention is proposed to solve or at least partially solve the technical problem of poor detection accuracy corresponding to the existing lane line detection method. The invention provides a lane line detection method, an electronic device, a medium and a vehicle.
In a first aspect, the present invention provides a lane line detection method, including the steps of: acquiring an image to be detected containing a lane line; inputting the image to be detected into a lane line detection model trained by a distillation method, identifying a lane line, wherein, the teacher model and the lane line detection model used by the distillation method both comprise a main network and a characteristic pyramid network, wherein the teacher model backbone network comprises a conversion module for converting images from an image domain to a feature domain and at least one discriminant feature generation module connected in series, the discriminant feature generation module comprises a convolutional layer, a global average pooling layer and at least one fully-connected layer which are connected in sequence, wherein the convolutional layer receives the first image feature output by the conversion module and outputs a second image feature, the global average pooling layer and the at least one fully connected layer generate dynamic weights from the second image feature input, and dynamically weighting the second image characteristics by using the dynamic weights to obtain third image characteristics.
In one embodiment, the convolutional layers include a first sub-convolutional layer, a second sub-convolutional layer, and a third sub-convolutional layer, wherein the convolutional kernel size of the first sub-convolutional layer is M × N, the convolutional kernel size of the second sub-convolutional layer is N × N, and the convolutional kernel size of the third sub-convolutional layer is N × M, where N and M are both natural numbers and M < N.
In one embodiment, the discriminant feature generation module further comprises: a shortcut connection path, a residual layer, and an activation function layer, wherein the shortcut connection path transfers the first image feature to the residual layer; and the residual error layer carries out summation operation on the first image characteristic and the third image characteristic and then outputs the summation operation through the activation function layer.
In one embodiment, further comprising: acquiring a lane line training image; and respectively inputting the lane line training images into a teacher model and a lane line detection model, and carrying out knowledge distillation on the lane line detection model through the teacher model to obtain the trained lane line detection model.
In one embodiment, the inputting the lane line training image into a teacher model and a lane line detection model, respectively, and performing knowledge distillation on the lane line detection model through the teacher model further includes: the characteristic pyramid network of the teacher model and the lane line detection model respectively outputs a first characteristic diagram and a second characteristic diagram; determining a distillation loss based on the first and second profiles; determining a base loss based on a detection result of the lane line detection model and the real label; training the lane line detection model based on the distillation loss and the base loss.
In one embodiment, said determining distillation loss based on said first and second profiles comprises: constructing a KL loss function:
in the above formula, LOSS (y) T ,y s ) Represents the distillation loss function, y T 、y s Respectively representing a first feature diagram and a second feature diagram, C representing the number of feature channels, tau representing a hyper-parameter, H and W representing the spatial dimension of the feature, respectively, phi (-) representing a function for converting the first feature into a feature distribution.
In one embodiment, before the lane line training images are input into a teacher model and a lane line detection model, respectively, and the lane line detection model is subjected to knowledge distillation by the teacher model, the method further includes: pre-training the lane line detection model to be trained to obtain pre-trained lane line detection model parameters; and initializing a lane line detection model to be trained by using the parameters.
In a second aspect, the invention provides an electronic device comprising a processor and a storage means, said storage means being adapted to store a plurality of program codes, said program codes being adapted to be loaded and run by said processor to perform the lane line detection method of any of the preceding claims.
In a third aspect, a computer readable storage medium is provided, having stored therein a plurality of program codes adapted to be loaded and run by a processor to perform the lane line detection method of any of the preceding claims.
In a fourth aspect, a vehicle is provided that includes the aforementioned electronic device.
One or more technical schemes of the invention at least have one or more of the following beneficial effects:
the invention provides a lane line detection method, firstly obtaining an image to be detected containing a lane line, then inputting the image to be detected into a lane line detection model trained by a distillation method to identify the lane line, wherein a high-performance teacher model is mainly provided, a global average pooling layer and at least one full-connection layer in the model can generate dynamic weight according to the input image characteristics, and the weighted operation of multiplication or summation of the image characteristics and the dynamic weight output by the full-connection layer is carried out to obtain weighted image characteristics, so that the network performance is effectively improved under the condition of introducing less calculation amount or parameters, the effect of no pain expansion point of a light-weight lane line detection model is realized, the efficient distillation of the model is realized, and the detection precision of the trained lane line detection model is higher, when the method is used for detecting the lane line, the detection precision of the lane line detection is improved, and therefore the effect of safe driving is achieved.
Furthermore, image features can be extracted through three convolution kernels which are sequentially superposed, the parameter quantity of a corresponding network is small relative to one convolution kernel, the problems that the network parameters are large and difficult to optimize due to the fact that the large convolution kernels are directly used are solved, the distillation effect is improved, the distillation time is reduced, meanwhile, the large receptive field is achieved, and when the large receptive field is applied to the identification of the lane line, the detection accuracy of the lane line can be further improved.
Drawings
The disclosure of the present invention will become more readily understood with reference to the accompanying drawings. As is readily understood by those skilled in the art: these drawings are for illustrative purposes only and are not intended to constitute a limitation on the scope of the present invention. Moreover, in the drawings, like numerals are used to indicate like parts, and in which:
fig. 1 is a schematic flow chart of main steps of a lane line detection method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of an overall distillation model including a teacher model and a student model, according to one embodiment of the invention;
FIG. 3 is a diagram illustrating the overall architecture of a backbone network of a teacher model according to one embodiment of the invention;
FIG. 4 is a block diagram of a discriminant feature generation module in a backbone network according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the invention.
Detailed Description
Some embodiments of the invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only for explaining the technical principle of the present invention, and are not intended to limit the scope of the present invention.
In the description of the present invention, a "module" or "processor" may include hardware, software, or a combination of both. A module may comprise hardware circuitry, various suitable sensors, communication ports, memory, may comprise software components such as program code, or may be a combination of software and hardware. The processor may be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and/or signal processing functionality. The processor may be implemented in software, hardware, or a combination thereof. Non-transitory computer readable storage media include any suitable medium that can store program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random-access memory, and the like. The term "a and/or B" denotes all possible combinations of a and B, such as a alone, B alone or a and B. The term "at least one A or B" or "at least one of A and B" means similar to "A and/or B" and may include only A, only B, or both A and B. The singular forms "a", "an" and "the" may include the plural forms as well.
At present, no distillation method for effectively transferring knowledge distillation to a lane line detection scene exists in practice, so that the existing distillation method for the light weight model has poor distillation effect, and when the distillation method is used for lane line detection, the lane line detection precision is poor, and a satisfactory detection effect is difficult to achieve.
Therefore, the application provides a lane line detection method, an electronic device, a medium and a vehicle.
Firstly, an image to be detected containing a lane line is obtained, then the image to be detected is input into a lane line detection model trained by a distillation method to identify the lane line, wherein a teacher model used by the distillation method is a high-performance teacher model, the teacher model comprises a main network and a feature pyramid network, wherein a distinguishing feature generation module in the main network comprises a convolution layer, a global average pooling layer and at least one full-connection layer, a dynamic weight can be generated according to the input second image feature through the global average pooling layer and the at least one full-connection layer, and the dynamic weight is used for dynamically weighting the second image feature to obtain a third image feature, so that the performance of the teacher model is effectively improved under the condition of not changing the calculated amount, the distillation effect is further improved, and thus, the detection accuracy of the trained lane line detection model is higher, when the method is used for detecting the lane line, the detection precision of the lane line detection is improved, and therefore the effect of safe driving is achieved.
Referring to fig. 1, fig. 1 is a flow chart illustrating main steps of a lane line detection method according to an embodiment of the present invention. As shown in fig. 1, the method for distilling a lane line detection model in the embodiment of the present invention mainly includes the following steps S11 to S12.
Step S11: and acquiring an image to be detected containing the lane line.
Specifically, the hardware used for acquiring the to-be-detected image including the lane line may include a vehicle-mounted camera, Lidar, Radar, and the like.
Step S12: inputting the image to be detected into a lane line detection model trained by a distillation method, identifying a lane line, wherein, the teacher model and the lane line detection model (i.e. the student model) used by the distillation method both comprise a main network and a characteristic pyramid network, wherein the teacher model backbone network comprises a sequentially connected conversion module for converting images from an image domain to a feature domain and at least one discriminant feature generation module comprising a sequentially connected convolutional layer, a global average pooling layer, and at least one fully connected layer, the convolution layer receives the first image characteristics output by the conversion module and outputs second image characteristics, the global average pooling layer and the at least one full-connection layer generate dynamic weights according to the input second image characteristics, and the dynamic weights are used for dynamically weighting the second image characteristics to obtain third image characteristics.
Specifically, as shown in fig. 2, the teacher model and the student model in the present application each include a backbone network, a feature pyramid network, and a prediction network, where the feature pyramid network is used to extract a feature map, and the prediction network is used to output a prediction result according to an input feature map.
Specifically, the teacher model backbone network comprises a conversion module and at least one discriminant feature generation module which are connected in sequence, wherein the conversion module is mainly used for converting an image from an image domain to a feature domain, namely extracting feature information of an input image. In addition, the number of the discrimination feature generation modules in the application is at least one.
In the example shown in fig. 3, the backbone network includes one conversion module and four discriminant feature generation modules connected in sequence.
Wherein, the conversion module comprises 3 convolutional layers. The number of output channels of the first convolutional layer is 32, the convolutional kernel k is (3,3), the moving step s is 2, and the void convolution rate d is 1. The number of output channels, convolution kernels and hole convolution rate of the second convolutional layer are the same as the first convolutional layer, except that the step size of the second convolutional layer is 1. The convolution kernels, shift steps, and void convolution rates of the third convolutional layer and the first convolutional layer are all the same, except that the number of output channels of the third convolutional layer is 64.
The number of output channels of the first discriminant feature generation module is 128, the number of output channels of the second discriminant feature generation module is 256, the number of output channels of the third discriminant feature generation module is 512, and the number of output channels of the fourth discriminant feature generation module is 1024.
For example, when the input image is 6 × 3 × 1088 × 1920, that is, 6 lane line training images with the channel number of 3 and the length and width of 1088 × 1920 are input, the output image is 6 × 64 × 544 × 960 after passing through the conversion module, then after being convolved by the first discriminant feature generation module, the output image is 6 × 128 × 272 × 480, after being convolved by the second discriminant feature generation module, the output image is 6 × 256 × 136 × 240, after being convolved by the third discriminant feature generation module, the output image is 6 × 512 × 68 × 120, and after being convolved by the fourth discriminant feature generation module, the output image is 6 × 34 × 60.
In summary, the conversion module outputs the first image feature to the discriminant feature generation module.
Each discriminant feature generation module may include at least one block module, each block module including sequentially connected convolutional layers, global average pooling layers, and at least one fully-connected layer.
In this embodiment, the first discriminant feature generation module may include three blocks, only two blocks are shown in fig. 3, which are sequentially stacked, wherein the number of the second blocks is 2 shown next to the second block by 2. The second discriminant feature generation module may include four blocks, only two blocks superimposed one after the other being illustrated in fig. 3, where the number of second blocks is 3 next to the second block in a 3 manner. The third discriminant feature generation module may include six blocks, only two blocks superimposed one after the other being illustrated in fig. 3, wherein the number of the second blocks is represented by 5 next to the second block. Similarly, the fourth discriminant feature generation module may include three blocks, only two blocks which are sequentially stacked are illustrated in fig. 3, where the number of the second block is 2 by 2 next to the second block. Through the backbone network in the application, the relevant features of the input image can be extracted efficiently.
In one embodiment, as shown in fig. 4, a block in the present application includes a convolutional layer, a global averaging pooling layer (global averaging pooling), and two fully connected layers (fc).
As shown in fig. 4, the convolution layer includes a first sub-convolution layer, a second sub-convolution layer, and a third sub-convolution layer, the convolution kernel of the first sub-convolution layer is 1 × 3, the convolution kernel of the second sub-convolution layer is 3 × 3, and the convolution kernel of the third sub-convolution layer is 3 × 1. Through the three sub convolution layers stacked in sequence, the convolution effect is equivalent to the convolution effect of a convolution kernel of 5 x 5, however, three convolution kernels stacked in sequence in the application are small compared with the prior art that one convolution kernel is used, the parameter quantity of a corresponding network is small, the problems that the network parameter quantity is large and the optimization is difficult due to the fact that a large convolution kernel is directly used are solved, the distillation effect is favorably improved, the distillation time is shortened, meanwhile, the large sensing field is realized, the large sensing field is particularly effective for the identification of the long and thin detection object of the lane line, and the detection accuracy of the lane line can be further improved.
Additionally, in this embodiment, the activation function corresponding to the first sub-convolutional layer and the second sub-convolutional layer may be relu.
Based on the above example, one skilled in the art will appreciate that the convolution kernel size of the first sub-convolution layer of the present application may be M × N, the convolution kernel size of the second sub-convolution layer may be N × N, and the convolution kernel size of the third sub-convolution layer may be N × M, where N and M are both natural numbers and M < N. Specifically, the convolution effect of three convolution kernels of size M × N, N × N and N × M in this order is equivalent to the convolution effect of one convolution kernel of (2N-1) × (2N-1). This is better under the requirement that a larger receptive field is required when the identification of such a slender detected object as a lane line is performed (for example, N >3), and the larger N is, the better the effect is.
The convolution layer receives the first image characteristic output by the conversion module and outputs a second image characteristic.
In a specific embodiment, as shown in fig. 4, a block in the present application includes a convolutional layer, a global averaging pooling layer (global averaging pooling), and two fully-connected layers (fc), where the global averaging pooling layer and the two sequentially-connected fully-connected layers can generate a dynamic weight according to a second image feature output by the convolutional layer, and dynamically weight the second image feature by using the dynamic weight to obtain a third image feature.
The method specifically comprises the steps of performing spatial compression on a second image feature output by a convolutional layer by using a global average pooling layer, and then weighting the feature in a channel dimension by using a dynamic activation value of the number of channels output by two full-connection layers, so as to realize the effect of dynamically adjusting a feature response value according to different inputs.
In addition, the activation function corresponding to the two fully-connected layers may be relu or sigmoid. Illustratively, the activation function corresponding to the first full connection layer may be relu, and the activation function corresponding to the second activation function may be sigmoid. It should be noted that, in the solution of the present application, only one full connection layer may be used, and the activation function may use one of relu and sigmoid.
In FIG. 4
This is shown as a multiplicative weighting, however the application is not so limited and the weighting here may be a product operation or a summation operation.
The third image characteristic is obtained by performing product operation or summation operation on the dynamic weight output by the full connection layer and the second image characteristic, so that the network performance is effectively improved under the condition of introducing less calculated amount or parameters, the effect of no pain expansion point of the light-weight lane line detection model is realized, and the efficient distillation of the model is realized.
As shown in fig. 4, in one embodiment, the discriminant feature generation module further includes: the image processing method comprises the steps of a shortcut connection path, a residual layer and an activation function layer, wherein the shortcut connection path transmits a first image feature to the residual layer; and after the residual error layer sums the first image characteristic and the third image characteristic, outputting the sum through an activation function layer, wherein an activation function corresponding to the activation function layer may be relu.
In summary, based on the foregoing embodiments, it can be seen that the convolutional layer included in the block of the present application can achieve a large receptive field, and the global average pooling layer and the at least one fully-connected layer can achieve dynamic weighting of features, both of which can raise the distillation upper limit.
However, due to the network arrangement in fig. 4, the depth of the network is large. For this reason, the application also transmits the first image feature to the residual layer through the set shortcut connection path. The problem of in the degree of depth network gradient disperse, be difficult to the training is solved, still can guarantee the network when the network degree of depth is great and realize the effect of high-efficient distillation, and then improve student's model's distillation effect.
The process of training the lane marking detection model using the distillation method may be through the following steps S101 to S102.
Step S101: and acquiring a lane line training image.
Specifically, the lane line training image set obtained here may include a plurality of lane line training images, and each lane line training image is correspondingly labeled with labels such as lane line positions and categories, and may be used for subsequent distillation training of a teacher model and a student model.
Step S102: and respectively inputting the lane line training images into a teacher model and a lane line detection model, and carrying out knowledge distillation on the lane line detection model through the teacher model to obtain the trained lane line detection model.
In one embodiment, step S102 further comprises:
respectively outputting a first characteristic diagram and a second characteristic diagram by the characteristic pyramid network of the teacher model and the lane line detection model;
determining a distillation loss based on the first and second profiles;
determining the base loss based on the detection result of the lane line detection model and a real label (ground route); the lane marking detection model is trained based on the distillation loss and the foundation loss.
Specifically, the first characteristic diagram and the second characteristic diagram are respectively output through the characteristic pyramid network of the teacher model and the characteristic pyramid network of the lane line detection model to determine the distillation loss, the distillation loss and the basic loss corresponding to the lane line detection model are matched to train the lane line detection model, and the trained lane line detection model can be obtained until the distillation loss and the basic loss are converged, so that the distillation effect of the lane line detection model is effectively improved.
In one embodiment, determining the distillation loss based on the first profile and the second profile comprises: constructing a KL loss function:
in the above formula, LOSS (y) T ,y s ) Denotes the distillation loss function, y T 、y s Respectively representing a first feature diagram and a second feature diagram, C representing the number of feature channels, tau representing a hyper-parameter, H and W representing the spatial dimension of the feature, respectively, phi (-) representing a function for converting the first feature into a feature distribution.
Specifically, the constructed KL loss function is used as distillation loss, after normalization is performed on each pixel point through H x W dimensionality, KL divergence is performed on each pixel point one by one, for lane line detection, the feature distribution of codes on each channel is the same, compared with the condition that the KL divergence is directly performed on each pixel point through the constructed KL loss function in the prior art, the distribution distance of the significance region between feature pyramid network output features of a teacher model and a light-weight lane line detection model is shortened, the learning difficulty between the features on the channels is reduced, the distillation effect is improved, and the identification accuracy of the lane line detection model is improved.
Based on the steps S11-S12, firstly, an image to be detected containing a lane line is obtained, then the image to be detected is input into a lane line detection model trained by a distillation method to identify the lane line, wherein a teacher model used by the distillation method is a high-performance teacher model, dynamic weights can be generated according to input image characteristics through a global averaging pooling layer and at least one full connection layer in the model, and product operation or summation operation is carried out on the dynamic weights output by the full connection layer and the input image characteristics to obtain weighted image characteristics, so that the performance of a network is effectively improved under the condition of introducing less calculation amount or parameters, the effect of no pain expansion points of a light-weight lane line detection model is realized, efficient distillation of the model is realized, and the detection precision of the trained lane line detection model is higher, when the method is used for detecting the lane line, the detection precision of the lane line detection is improved, and therefore the effect of safe driving is achieved.
In one embodiment, before inputting the lane line training images into the teacher model and the lane line detection model, respectively, and performing knowledge distillation on the lane line detection model by the teacher model, the method further includes:
pre-training a lane line detection model to be trained to obtain pre-trained lane line detection model parameters;
and initializing the lane line detection model to be trained by using the parameters.
Specifically, before knowledge distillation, the lane line detection model is pre-trained, the pre-trained parameters of the lane line detection model are used for initializing the lane line detection model, and the lane line detection model with the initialized parameters and the teacher model are used for distillation training, so that the distillation effect is improved.
It should be noted that, although the foregoing embodiments describe each step in a specific sequence, those skilled in the art will understand that, in order to achieve the effect of the present invention, different steps do not necessarily need to be executed in such a sequence, and they may be executed simultaneously (in parallel) or in other sequences, and these changes are all within the protection scope of the present invention.
It will be understood by those skilled in the art that all or part of the flow of the method according to the above-described embodiment may be implemented by a computer program, which may be stored in a computer-readable storage medium and used to implement the steps of the above-described embodiments of the method when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable storage medium may include: any entity or device capable of carrying said computer program code, media, usb disk, removable hard disk, magnetic diskette, optical disk, computer memory, read-only memory, random access memory, electrical carrier wave signals, telecommunication signals, software distribution media, etc. It should be noted that the computer readable storage medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable storage media that does not include electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.
Furthermore, the invention also provides electronic equipment. As shown in fig. 5, in an embodiment of the electronic device according to the present invention, the electronic device includes a processor 80 and a storage device 81, the storage device may be configured to store a program for executing the distillation method of the lane line detection model or the lane line detection method of the above-described method embodiment, and the processor may be configured to execute a program in the storage device, the program including but not limited to a program for executing the lane line detection method of the above-described method embodiment. For convenience of explanation, only the parts related to the embodiments of the present invention are shown, and details of the specific techniques are not disclosed.
Further, the invention also provides a computer readable storage medium. In one computer-readable storage medium embodiment according to the present invention, the computer-readable storage medium may be configured to store a program that executes the lane line detection method or the distillation method of the lane line detection model of the above-described method embodiment, and the program may be loaded and executed by a processor to implement the above-described lane line detection method. For convenience of explanation, only the parts related to the embodiments of the present invention are shown, and details of the specific techniques are not disclosed. The computer readable storage medium may be a storage device formed by including various electronic devices, and optionally, the computer readable storage medium is a non-transitory computer readable storage medium in the embodiment of the present invention.
Further, the invention also provides a vehicle comprising the electronic equipment.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
Claims (10)
1. A lane line detection method is characterized by comprising the following steps:
acquiring an image to be detected containing a lane line;
inputting the image to be detected into a lane line detection model trained by a distillation method, identifying a lane line,
the teacher model and the lane line detection model used by the distillation method both comprise a main network and a feature pyramid network, wherein the main network of the teacher model comprises a conversion module and at least one distinguishing feature generation module which are sequentially connected and used for converting an image from an image domain to a feature domain, the distinguishing feature generation module comprises a convolution layer, a global average pooling layer and at least one full-connection layer which are sequentially connected, the convolution layer receives a first image feature output by the conversion module and outputs a second image feature, the global average pooling layer and the at least one full-connection layer generate a dynamic weight according to the input second image feature, and the dynamic weight is used for dynamically weighting the second image feature to obtain a third image feature.
2. The lane marking detection method of claim 1, wherein the convolutional layers comprise a first sub-convolutional layer, a second sub-convolutional layer, and a third sub-convolutional layer, wherein the convolutional kernel size of the first sub-convolutional layer is M x N, the convolutional kernel size of the second sub-convolutional layer is N x N, the convolutional kernel size of the third sub-convolutional layer is N x M, wherein N and M are both natural numbers and M < N.
3. The lane line detection method according to claim 1 or 2, wherein the discriminating characteristic generating module further includes: short connection path, residual layer and activation function layer, wherein
The shortcut connection path transmits the first image feature to a residual layer;
and the residual error layer carries out summation operation on the first image characteristic and the third image characteristic and then outputs the summation operation through the activation function layer.
4. The lane line detection method according to any one of claims 1 to 3, further comprising:
acquiring a lane line training image;
and respectively inputting the lane line training images into a teacher model and a lane line detection model, and carrying out knowledge distillation on the lane line detection model through the teacher model to obtain the trained lane line detection model.
5. The lane line detection method according to claim 4, wherein the inputting of the lane line training image into a teacher model and a lane line detection model, respectively, the teacher model performing knowledge distillation on the lane line detection model, further comprises:
the characteristic pyramid network of the teacher model and the lane line detection model respectively outputs a first characteristic diagram and a second characteristic diagram;
determining a distillation loss based on the first and second profiles;
determining a base loss based on a detection result of the lane line detection model and the real label;
training the lane line detection model based on the distillation loss and the base loss.
6. The lane line detection method of claim 5, wherein determining the distillation loss based on the first and second profiles comprises: constructing a KL loss function:
in the above formula, LOSS (y) T ,y S ) Denotes the distillation loss function, y T 、y S Respectively representing a first feature diagram and a second feature diagram, C representing the number of feature channels, tau representing a hyper-parameter, H and W representing the spatial dimension of the feature, respectively, phi (-) representing a function for converting the first feature into a feature distribution.
7. The lane line detection method according to claim 4, wherein before the lane line training image is input to a teacher model and a lane line detection model, respectively, and the lane line detection model is subjected to knowledge distillation by the teacher model, the method further comprises:
pre-training the lane line detection model to be trained to obtain pre-trained lane line detection model parameters;
and initializing a lane line detection model to be trained by using the parameters.
8. An electronic device comprising a processor and a storage means adapted to store a plurality of program codes, characterized in that said program codes are adapted to be loaded and run by said processor to perform the lane line detection method of any of claims 1 to 7.
9. A computer-readable storage medium having stored therein a plurality of program codes, characterized in that the program codes are adapted to be loaded and run by a processor to perform the lane line detection method according to any one of claims 1 to 7.
10. A vehicle characterized by comprising the electronic device of claim 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210509064.9A CN114821534A (en) | 2022-05-10 | 2022-05-10 | Lane line detection method, electronic device, medium, and vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210509064.9A CN114821534A (en) | 2022-05-10 | 2022-05-10 | Lane line detection method, electronic device, medium, and vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114821534A true CN114821534A (en) | 2022-07-29 |
Family
ID=82513876
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210509064.9A Pending CN114821534A (en) | 2022-05-10 | 2022-05-10 | Lane line detection method, electronic device, medium, and vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114821534A (en) |
-
2022
- 2022-05-10 CN CN202210509064.9A patent/CN114821534A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109086824B (en) | Seabed substrate sonar image classification method based on convolutional neural network | |
| CN104299006A (en) | Vehicle license plate recognition method based on deep neural network | |
| CN111027576B (en) | Cooperative significance detection method based on cooperative significance generation type countermeasure network | |
| CN111680176A (en) | Remote sensing image retrieval method and system based on attention and bidirectional feature fusion | |
| CN111160171A (en) | Radiation source signal identification method combining two-domain multi-features | |
| CN113743417B (en) | Semantic segmentation method and semantic segmentation device | |
| CN113988147B (en) | Multi-label classification method and device for remote sensing image scene based on graph network, and multi-label retrieval method and device | |
| CN114821058A (en) | Image semantic segmentation method and device, electronic equipment and storage medium | |
| CN110969104B (en) | Method, system and storage medium for detecting drivable area based on binarization network | |
| CN114399686A (en) | Remote sensing image ground feature identification and classification method and device based on weak supervised learning | |
| CN115546196A (en) | Knowledge distillation-based lightweight remote sensing image change detection method | |
| CN112149526A (en) | Lane line detection method and system based on long-distance information fusion | |
| CN114692750A (en) | Fine-grained image classification method and device, electronic equipment and storage medium | |
| CN110942097A (en) | Imaging-free classification method and system based on single-pixel detector | |
| CN114387454A (en) | Self-supervision pre-training method based on region screening module and multi-level comparison | |
| CN116977747B (en) | Small sample hyperspectral classification method based on multipath multi-scale feature twin network | |
| CN116363526B (en) | MROCNet model construction and multisource remote sensing image change detection method and system | |
| CN113591892A (en) | Training data processing method and device | |
| CN112686184A (en) | Remote sensing house change detection method based on neural network | |
| CN114821534A (en) | Lane line detection method, electronic device, medium, and vehicle | |
| CN115546840A (en) | Pedestrian re-recognition model training method and device based on semi-supervised knowledge distillation | |
| CN115329821A (en) | Ship noise identification method based on pairing coding network and comparison learning | |
| CN110826726B (en) | Target processing method, target processing device, target processing apparatus, and medium | |
| CN113468908B (en) | Target identification method and device | |
| CN117475481B (en) | Domain migration-based night infrared image animal identification method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |