CN111062347A - Traffic element segmentation method in automatic driving, electronic device and storage medium - Google Patents
Traffic element segmentation method in automatic driving, electronic device and storage medium Download PDFInfo
- Publication number
- CN111062347A CN111062347A CN201911332028.4A CN201911332028A CN111062347A CN 111062347 A CN111062347 A CN 111062347A CN 201911332028 A CN201911332028 A CN 201911332028A CN 111062347 A CN111062347 A CN 111062347A
- Authority
- CN
- China
- Prior art keywords
- feature matrix
- traffic
- segmentation
- matrix
- scene image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Multimedia (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a traffic element segmentation method in automatic driving, an electronic device and a storage medium, wherein the method comprises the following steps: acquiring an acquired traffic scene image, and segmenting the traffic scene image through a pre-trained segmentation model; performing convolution calculation on the traffic scene image to obtain a feature matrix, performing down-sampling and up-sampling on the feature matrix, then overlapping the feature matrix with a feature matrix processed based on an ASPP (advanced application Programming protocol) module and an attention mechanism to obtain a target feature matrix, and classifying the target feature matrix through a sigmoid function to obtain a traffic element segmentation result; and simultaneously inputting the segmentation result and the preset standard data into a countermeasure network to train the countermeasure network, and generating a traffic element segmentation image with confidence degree through the countermeasure network. By the scheme, the problem that the existing traffic element segmentation model is poor in generalization capability is solved, the generalization capability of a network model can be effectively improved, and the adaptability to different traffic scenes is enhanced.
Description
Technical Field
The present invention relates to the field of computer vision, and in particular, to a method for segmenting traffic elements in automatic driving, an electronic device, and a storage medium.
Background
In the field of automatic driving, a camera mounted on a vehicle is required to identify and analyze traffic elements such as roads and signs in a traffic scene so that a driving system can make a decision further according to the scene. Generally, various different objects exist in a traffic scene, and in order to ensure the accuracy of detection and identification, different traffic elements need to be accurately segmented.
At present, the image segmentation method based on deconvolution can more accurately segment traffic elements and keep details lost, however, due to the fact that traffic scene images acquired in automatic driving are various, image segmentation models obtained by training through methods such as deconvolution and void convolution are simply adopted according to acquired samples, generalization capability is poor, and the method is difficult to adapt to multi-scene traffic element segmentation recognition.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method, an electronic device, and a storage medium for segmenting traffic elements in automatic driving, so as to solve the problem that an existing traffic scene image segmentation recognition model is poor in generalization capability.
In a first aspect of an embodiment of the present invention, a method for segmenting traffic elements in automatic driving is provided, including:
acquiring a traffic scene image acquired by a vehicle-mounted camera, and segmenting the traffic scene image through a pre-trained segmentation model;
specifically, convolution calculation is carried out on the traffic scene image to obtain a feature matrix, the feature matrix is subjected to down sampling and up sampling, then is superposed with the feature matrix processed based on an ASPP (automatic document Programming protocol) module and an attention mechanism to obtain a target feature matrix, and the target feature matrix is classified through a sigmoid function to obtain a traffic element segmentation result;
and simultaneously inputting the segmentation result and the preset standard data into a countermeasure network to train the countermeasure network, and generating a traffic element segmentation image with confidence degree through the countermeasure network.
In a second aspect of the embodiments of the present invention, there is provided an electronic device for traffic element segmentation in automatic driving, including:
the segmentation module is used for acquiring a traffic scene image acquired by a vehicle-mounted camera and segmenting the traffic scene image through a pre-trained segmentation model;
specifically, convolution calculation is carried out on the traffic scene image to obtain a feature matrix, the feature matrix is subjected to down sampling and up sampling, then is superposed with the feature matrix processed based on an ASPP (automatic document Programming protocol) module and an attention mechanism to obtain a target feature matrix, and the target feature matrix is classified through a sigmoid function to obtain a traffic element segmentation result;
and the generation module is used for simultaneously inputting the segmentation result and the preset standard data to the countermeasure network to train the countermeasure network, and generating the traffic element segmentation image with confidence degree through the countermeasure network.
In a third aspect of the embodiments of the present invention, there is provided an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the steps of the method according to the first aspect of the embodiments of the present invention.
In a fourth aspect of the embodiments of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the method provided in the first aspect of the embodiments of the present invention.
In the embodiment of the invention, a traffic scene image is segmented through a pre-trained segmentation model, a segmentation result and preset standard data are simultaneously input into an countermeasure network to train the countermeasure network, and a traffic element segmentation image with confidence coefficient is generated through the countermeasure network, wherein a feature matrix is obtained by performing convolution calculation on the traffic scene image, after down-sampling and up-sampling are performed on the feature matrix, the feature matrix is superposed with a feature matrix processed based on an ASPP (automatic switched programming) module and an attention mechanism to obtain a target feature matrix, and the target feature matrix is classified through a sigmoid function to obtain a traffic element segmentation result. The segmentation model is reversely adjusted based on the countermeasure network, so that the generalization capability of the neural network model can be enhanced, the problem of poor generalization capability of the existing segmentation model is solved, the dependence of the traditional convolutional neural network on a sample set is reduced, and different traffic scenes can be better adapted. Meanwhile, based on up-sampling and down-sampling, the traffic element segmentation precision can be improved through the ASPP module and attention mechanism processing, more information in the acquired image is acquired, and the trained model has stronger identification capability.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings described below are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a flow chart illustrating a method for segmenting traffic elements in automatic driving according to an embodiment of the present invention;
FIG. 2 is another schematic flow chart of a method for segmenting traffic elements in automatic driving according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an electronic device for traffic element segmentation in automatic driving according to an embodiment of the present invention.
Detailed Description
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the embodiments described below are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons skilled in the art without any inventive work shall fall within the protection scope of the present invention, and the principle and features of the present invention shall be described below with reference to the accompanying drawings.
The terms "comprises" and "comprising," when used in this specification and claims, and in the accompanying drawings and figures, are intended to cover non-exclusive inclusions, such that a process, method or system, or apparatus that comprises a list of steps or elements is not limited to the listed steps or elements.
Referring to fig. 1, fig. 1 is a schematic flow chart of a traffic element segmentation method in automatic driving according to an embodiment of the present invention, including:
s101, acquiring a traffic scene image acquired by a vehicle-mounted camera, and segmenting the traffic scene image through a pre-trained segmentation model;
the traffic scene image is an image acquired in each direction in the driving process of the vehicle, generally is influence data of the vehicle in the road driving process, and traffic factors such as roads, road marking lines, signs, signal lamps and the like can be acquired according to the traffic scene image so as to facilitate a driving system to make driving decisions.
The pre-trained segmentation model is used for detecting and segmenting traffic elements of the traffic scene image, and the pre-trained segmentation model is pre-trained through the constructed traffic element training set, so that the segmentation model based on the convolutional neural network can perform segmentation processing on the traffic scene image.
Preferably, the traffic elements are extracted from the acquired traffic scene images, and the traffic elements are used as samples to train the segmentation model, wherein the iterative update algorithm of the segmentation model is a binary cross entropy loss function BCELoss.
Specifically, convolution calculation is carried out on the traffic scene image in the pre-trained segmentation model to obtain a feature matrix, the feature matrix is subjected to down-sampling and up-sampling and then is superposed with the feature matrix processed based on an ASPP (automatic position protocol) module and an attention mechanism to obtain a target feature matrix, and the target feature matrix is classified through a sigmoid function to obtain a traffic element segmentation result;
and setting related training network parameters to train the deep learning network by taking the traffic scene image as a sample. The traffic scene image is cut to a preset size, the traffic scene image can be conveniently used as a sample to train the segmentation model, and traffic element characteristics are obtained so as to detect traffic elements in the segmented traffic image.
The down-sampling is used for reducing the image to enable the image to conform to the size of a display area, the up-sampling can enlarge the image to enable the image to have more information, and the up-sampling and the down-sampling are realized through a pooling technology, so that the characteristic dimensionality can be reduced, and effective information is reserved. In the downsampling process, the size of the image features is reduced, the number of feature maps is increased, and thus features of various sizes in one image can be obtained as far as possible. The up-sampling is to decode the image, enlarge the feature map by various interpolation methods, then fuse the feature map with the down-sampling same size, and position the feature map at the position of the original image. This facilitates the most efficient classification of each pixel of the feature map and the acquisition of the classified position information.
The use of extended Convolution (Atrous Convolution) in the ASPP module can expand the field of view of feature extraction without sacrificing feature spatial resolution, and ASPP exploits multi-scale information to further enhance the segmentation effect. The attention mechanism (attention mechanism) is used between image coding and decoding, the limited feature storage space occupied in the coding process can be reduced, meanwhile, the decoding process can make the input have difference according to the useful information of different stages, namely, the attention mechanism can reduce the commonality and only pay attention to the difference between the features.
Carrying out convolution calculation on a sample traffic scene image to obtain a characteristic matrix, processing the characteristic matrix through an ASPP module and an attention mechanism to obtain a new characteristic matrix, carrying out down-sampling and convolution calculation on an original characteristic matrix, then continuing to carry out processing through the ASPP module and the attention mechanism, carrying out up-sampling on the characteristic matrix obtained through multiple times of convolution, and superposing the characteristic matrix obtained through processing through the ASPP module and the attention mechanism to obtain a target characteristic matrix.
In another embodiment of the present invention, the process of processing the original traffic scene image to obtain the target feature includes:
performing convolution calculation on the cut traffic scene image to obtain a characteristic matrix R1;
processing the feature matrix R1 through an ASPP module and an attention mechanism to obtain a feature matrix Q1;
after downsampling is carried out on the feature matrix R1, convolution calculation is carried out to obtain a feature matrix R2;
processing the feature matrix R2 through an ASPP module and an attention mechanism to obtain a feature matrix Q2;
after downsampling is carried out on the feature matrix R2, convolution calculation is carried out to obtain a feature matrix R3;
processing the feature matrix R3 through an ASPP module and an attention mechanism to obtain a feature matrix Q3;
the feature matrix R3 is convolved for multiple times to generate a feature matrix W1;
after upsampling, the feature matrix W1 is superposed with the feature matrix Q3 to obtain a feature matrix T1;
performing convolution calculation on the feature matrix T1 to obtain a feature matrix U1, upsampling the feature matrix U1, and overlapping the upsampled feature matrix U1 with the feature matrix Q2 to obtain a feature matrix T2;
performing convolution calculation on the feature matrix T2 to obtain a feature matrix U2, upsampling the feature matrix U2, and overlapping the upsampled feature matrix U2 with the feature matrix Q1 to obtain a feature matrix T3;
the obtained feature matrix T3 is the target feature matrix, and the feature matrix T3 is used as the target feature matrix.
And classifying and outputting the target feature matrix through a sigmoid function so as to detect and segment traffic elements in the traffic scene image.
The ASPP algorithm is added on the basis of the original linknet algorithm, so that the edge accuracy of the extracted traffic elements is higher, and the context information is more. The addition of the Attention algorithm can reduce the computational complexity of the algorithm and can accelerate the reasoning speed of the model.
The sigmoid function is an activation function, and the target feature matrix can be identified and classified through the sigmoid function. Based on the classified output of the sigmoid function to the feature matrix, the traffic elements can be segmented. And performing secondary classification on the target feature matrix through the sigmoid function based on feature learning of historical samples, and segmenting traffic elements from the traffic scene image.
S102, inputting the segmentation result and the preset standard data to a countermeasure network simultaneously to train the countermeasure network, and generating a traffic element segmentation image with confidence degree through the countermeasure network.
The segmentation result is a segmented traffic element obtained by segmenting the trained convolutional neural network model, i.e., a segmentation result obtained by segmenting the model, and the predetermined standard data is standard traffic element image data (or referred to as gold standard). The countermeasure network comprises a generator network and a discriminator network, wherein the generator network mainly learns the mapping relation between the random vector and the real standard data according to the input random vector, and the discriminator network is a two-classifier and used for adjusting the generator network according to the difference between the real sample data and the output of the generator network.
The segmentation performance of the segmentation model can be improved by introducing a countermeasure network into the segmentation network, two input high-dimensional images are mapped into a low-latitude output space in the countermeasure network, and the distance between the two input high-dimensional images is calculated by using a specified distance measurement calculation formula in an output result, so that the similarity between the images is judged, and the segmentation image with confidence coefficient is generated.
In an embodiment, as shown in fig. 2, fig. 2 is a schematic flowchart of a segmentation map with confidence generated by the countermeasure network on the basis of step S102 according to this embodiment, and includes:
the traffic element image segmentation result 2010 obtained by the segmentation of the pre-trained segmentation model is input to the countermeasure network module 2030 together with the gold standard (i.e., the traffic element image of the predetermined standard) 2020, and the segmentation map with confidence is generated by the network module 2030.
Wherein, the updating algorithm of the countermeasure network is as follows:
the parameters of the convolutional neural network are adjusted back by setting the gap between the convolutional neural network output and the gold standard measured by the antagonistic network module 2030, thereby improving the segmentation result. The segmentation module and the confrontation module confront each other in training, the network model which starts training may not achieve an ideal segmentation effect, the accuracy is very low, and a great loss value is caused, while the confrontation network module can easily compare the matching degree between pictures, the matching degree between the pictures at the back is gradually improved, and the segmentation model can also gradually achieve the ideal segmentation effect.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
Fig. 3 is a schematic structural diagram of an electronic device for a traffic element segmentation method in automatic driving according to an embodiment of the present invention, where the electronic device includes:
the segmentation module 310 is configured to acquire a traffic scene image acquired by a vehicle-mounted camera, and segment the traffic scene image through a pre-trained segmentation model;
specifically, convolution calculation is carried out on the traffic scene image to obtain a feature matrix, the feature matrix is subjected to down sampling and up sampling, then is superposed with the feature matrix processed based on an ASPP (automatic document Programming protocol) module and an attention mechanism to obtain a target feature matrix, and the target feature matrix is classified through a sigmoid function to obtain a traffic element segmentation result;
optionally, the segmentation module 310 includes:
the training unit is used for extracting traffic elements from the acquired traffic scene images and training the segmentation model by taking the traffic elements as samples, wherein the iterative update algorithm of the segmentation model is a binary cross entropy loss function.
Optionally, the performing convolution calculation on the traffic scene image to obtain a feature matrix, and after down-sampling and up-sampling the feature matrix, superimposing the feature matrix with a feature matrix processed based on an ASPP module and an attention mechanism to obtain a target feature matrix specifically includes:
performing convolution calculation on the cut traffic scene image to obtain a characteristic matrix R1;
processing the feature matrix R1 through an ASPP module and an attention mechanism to obtain a feature matrix Q1;
after downsampling is carried out on the feature matrix R1, convolution calculation is carried out to obtain a feature matrix R2;
processing the feature matrix R2 through an ASPP module and an attention mechanism to obtain a feature matrix Q2;
after downsampling is carried out on the feature matrix R2, convolution calculation is carried out to obtain a feature matrix R3;
processing the feature matrix R3 through an ASPP module and an attention mechanism to obtain a feature matrix Q3;
the feature matrix R3 is convolved for multiple times to generate a feature matrix W1, and the feature matrix W1 is subjected to upsampling and then is superposed with a feature matrix Q1 to obtain a feature matrix T1;
performing convolution calculation on the feature matrix T1 to obtain a feature matrix U1, upsampling the feature matrix U1, and overlapping the upsampled feature matrix U1 with the feature matrix Q2 to obtain a feature matrix T2;
and performing convolution calculation on the feature matrix T2 to obtain a feature matrix U2, performing upsampling on the feature matrix U2, and then overlapping the upsampled feature matrix U2 with the feature matrix Q2 to obtain a feature matrix T3, wherein the feature matrix T3 is used as a target feature matrix.
And a generating module 320, configured to input the segmentation result and the predetermined standard data to a countermeasure network simultaneously to train the countermeasure network, and generate a traffic element segmentation image with confidence through the countermeasure network.
Optionally, the step of inputting the segmentation result and the predetermined standard data to a countermeasure network simultaneously to train the countermeasure network, and the generating the traffic element segmentation image with the confidence degree by the countermeasure network specifically includes:
and acquiring the traffic element images obtained by segmentation of the pre-trained segmentation model, inputting the segmented traffic element images and the traffic element images with a preset standard into the trained confrontation network, and calculating and judging the similarity of the two images through the trained confrontation network.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those skilled in the art will appreciate that all or part of the steps in the method for implementing the above embodiments may be implemented by a program to instruct associated hardware, where the program may be stored in a computer-readable storage medium, and when the program is executed, the program includes steps S101 to S102, where the storage medium includes, for example: ROM/RAM, magnetic disk, optical disk, etc.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for segmenting traffic elements in automatic driving is characterized by comprising the following steps:
acquiring a traffic scene image acquired by a vehicle-mounted camera, and segmenting the traffic scene image through a pre-trained segmentation model;
specifically, convolution calculation is carried out on the traffic scene image to obtain a feature matrix, the feature matrix is subjected to down sampling and up sampling, then is superposed with the feature matrix processed based on an ASPP (automatic document Programming protocol) module and an attention mechanism to obtain a target feature matrix, and the target feature matrix is classified through a sigmoid function to obtain a traffic element segmentation result;
and simultaneously inputting the segmentation result and the preset standard data into a countermeasure network to train the countermeasure network, and generating a traffic element segmentation image with confidence degree through the countermeasure network.
2. The method of claim 1, wherein the obtaining of the traffic scene image captured by the vehicle-mounted camera, and the segmenting of the traffic scene image by the pre-trained segmentation model further comprises:
extracting traffic elements from the acquired traffic scene images, and training a segmentation model by taking the traffic elements as samples, wherein the iterative update algorithm of the segmentation model is a two-class cross entropy loss function.
3. The method according to claim 1, wherein the performing convolution calculation on the traffic scene image to obtain a feature matrix, and after down-sampling and up-sampling the feature matrix, superimposing the feature matrix with a feature matrix processed based on an ASPP module and an attention mechanism to obtain a target feature matrix specifically comprises:
performing convolution calculation on the cut traffic scene image to obtain a characteristic matrix R1;
processing the feature matrix R1 through an ASPP module and an attention mechanism to obtain a feature matrix Q1;
after downsampling is carried out on the feature matrix R1, convolution calculation is carried out to obtain a feature matrix R2;
processing the feature matrix R2 through an ASPP module and an attention mechanism to obtain a feature matrix Q2;
after downsampling is carried out on the feature matrix R2, convolution calculation is carried out to obtain a feature matrix R3;
processing the feature matrix R3 through an ASPP module and an attention mechanism to obtain a feature matrix Q3;
the feature matrix R3 is convolved for multiple times to generate a feature matrix W1, and the feature matrix W1 is subjected to upsampling and then is superposed with a feature matrix Q3 to obtain a feature matrix T1;
performing convolution calculation on the feature matrix T1 to obtain a feature matrix U1, upsampling the feature matrix U1, and overlapping the upsampled feature matrix U1 with the feature matrix Q2 to obtain a feature matrix T2;
and performing convolution calculation on the feature matrix T2 to obtain a feature matrix U2, performing upsampling on the feature matrix U2, and then overlapping the upsampled feature matrix U2 with the feature matrix Q1 to obtain a feature matrix T3, wherein the feature matrix T3 is used as a target feature matrix.
4. The method of claim 1, wherein the segmentation result and the predetermined standard data are simultaneously input to a countermeasure network to train the countermeasure network, and wherein generating a traffic element segmentation image with confidence by the countermeasure network comprises:
and acquiring the traffic element images obtained by segmentation of the pre-trained segmentation model, inputting the segmented traffic element images and the traffic element images with a preset standard into the trained confrontation network, and calculating and judging the similarity of the two images through the trained confrontation network.
5. An electronic device for traffic element segmentation in autonomous driving, comprising:
the segmentation module is used for acquiring a traffic scene image acquired by a vehicle-mounted camera and segmenting the traffic scene image through a pre-trained segmentation model;
specifically, convolution calculation is carried out on the traffic scene image to obtain a feature matrix, the feature matrix is subjected to down sampling and up sampling, then is superposed with the feature matrix processed based on an ASPP (automatic document Programming protocol) module and an attention mechanism to obtain a target feature matrix, and the target feature matrix is classified through a sigmoid function to obtain a traffic element segmentation result;
and the generation module is used for simultaneously inputting the segmentation result and the preset standard data to the countermeasure network to train the countermeasure network, and generating the traffic element segmentation image with confidence degree through the countermeasure network.
6. The electronic device of claim 5, wherein the segmentation module comprises:
the training unit is used for extracting traffic elements from the acquired traffic scene images and training the segmentation model by taking the traffic elements as samples, wherein the iterative update algorithm of the segmentation model is a binary cross entropy loss function.
7. The electronic device according to claim 5, wherein the convolving the traffic scene image to obtain a feature matrix, and after down-sampling and up-sampling the feature matrix, superimposing the feature matrix with a feature matrix processed based on an ASPP module and an attention mechanism to obtain a target feature matrix specifically comprises:
performing convolution calculation on the cut traffic scene image to obtain a characteristic matrix R1;
processing the feature matrix R1 through an ASPP module and an attention mechanism to obtain a feature matrix Q1;
after downsampling is carried out on the feature matrix R1, convolution calculation is carried out to obtain a feature matrix R2;
processing the feature matrix R2 through an ASPP module and an attention mechanism to obtain a feature matrix Q2;
after downsampling is carried out on the feature matrix R2, convolution calculation is carried out to obtain a feature matrix R3;
processing the feature matrix R3 through an ASPP module and an attention mechanism to obtain a feature matrix Q3;
the feature matrix R3 is convolved for multiple times to generate a feature matrix W1, and the feature matrix W1 is subjected to upsampling and then is superposed with a feature matrix Q1 to obtain a feature matrix T1;
performing convolution calculation on the feature matrix T1 to obtain a feature matrix U1, upsampling the feature matrix U1, and overlapping the upsampled feature matrix U1 with the feature matrix Q2 to obtain a feature matrix T2;
and performing convolution calculation on the feature matrix T2 to obtain a feature matrix U2, performing upsampling on the feature matrix U2, and then overlapping the upsampled feature matrix U2 with the feature matrix Q2 to obtain a feature matrix T3, wherein the feature matrix T3 is used as a target feature matrix.
8. The electronic device of claim 5, wherein the segmentation result and the predetermined standard data are simultaneously input to a countermeasure network for training the countermeasure network, and the generation of the traffic element segmentation image with confidence through the countermeasure network is specifically:
and acquiring the traffic element images obtained by segmentation of the pre-trained segmentation model, inputting the segmented traffic element images and the traffic element images with a preset standard into the trained confrontation network, and calculating and judging the similarity of the two images through the trained confrontation network.
9. An electronic device comprising a processor, a memory and a computer program stored in the memory and running on the processor, characterized in that the processor, when executing the computer program, implements the steps of the method for traffic element segmentation in autonomous driving according to any of claims 1 to 4.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method for partitioning traffic elements in autonomous driving according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911332028.4A CN111062347B (en) | 2019-12-21 | 2019-12-21 | Traffic element segmentation method in automatic driving, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911332028.4A CN111062347B (en) | 2019-12-21 | 2019-12-21 | Traffic element segmentation method in automatic driving, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111062347A true CN111062347A (en) | 2020-04-24 |
| CN111062347B CN111062347B (en) | 2023-04-28 |
Family
ID=70301465
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911332028.4A Active CN111062347B (en) | 2019-12-21 | 2019-12-21 | Traffic element segmentation method in automatic driving, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111062347B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111914482A (en) * | 2020-07-27 | 2020-11-10 | 武汉中海庭数据技术有限公司 | Driving condition generation method and system for automatic driving test |
| CN112597825A (en) * | 2020-12-07 | 2021-04-02 | 深延科技(北京)有限公司 | Driving scene segmentation method and device, electronic equipment and storage medium |
| CN113486716A (en) * | 2021-06-04 | 2021-10-08 | 电子科技大学长三角研究院(衢州) | Airport scene target segmentation method and system thereof |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108961253A (en) * | 2018-06-19 | 2018-12-07 | 深动科技（北京）有限公司 | A kind of image partition method and device |
| CN109191472A (en) * | 2018-08-28 | 2019-01-11 | 杭州电子科技大学 | Based on the thymocyte image partition method for improving U-Net network |
| CN109643377A (en) * | 2018-11-19 | 2019-04-16 | 深圳市汇顶科技股份有限公司 | Fingerprint identification device and electronic equipment |
| CN109685813A (en) * | 2018-12-27 | 2019-04-26 | 江西理工大学 | A kind of U-shaped Segmentation Method of Retinal Blood Vessels of adaptive scale information |
| CN109903292A (en) * | 2019-01-24 | 2019-06-18 | 西安交通大学 | A kind of three-dimensional image segmentation method and system based on full convolutional neural networks |
| CN110111335A (en) * | 2019-05-08 | 2019-08-09 | 南昌航空大学 | A kind of the urban transportation Scene Semantics dividing method and system of adaptive confrontation study |
| CN110188765A (en) * | 2019-06-05 | 2019-08-30 | 京东方科技集团股份有限公司 | Image, semantic parted pattern generation method, device, equipment and storage medium |
| CN110490884A (en) * | 2019-08-23 | 2019-11-22 | 北京工业大学 | A kind of lightweight network semantic segmentation method based on confrontation |
| WO2019238560A1 (en) * | 2018-06-12 | 2019-12-19 | Tomtom Global Content B.V. | Generative adversarial networks for image segmentation |
| CN110598714A (en) * | 2019-08-19 | 2019-12-20 | 中国科学院深圳先进技术研究院 | Cartilage image segmentation method and device, readable storage medium and terminal equipment |
-
2019
- 2019-12-21 CN CN201911332028.4A patent/CN111062347B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019238560A1 (en) * | 2018-06-12 | 2019-12-19 | Tomtom Global Content B.V. | Generative adversarial networks for image segmentation |
| CN108961253A (en) * | 2018-06-19 | 2018-12-07 | 深动科技（北京）有限公司 | A kind of image partition method and device |
| CN109191472A (en) * | 2018-08-28 | 2019-01-11 | 杭州电子科技大学 | Based on the thymocyte image partition method for improving U-Net network |
| CN109643377A (en) * | 2018-11-19 | 2019-04-16 | 深圳市汇顶科技股份有限公司 | Fingerprint identification device and electronic equipment |
| CN109685813A (en) * | 2018-12-27 | 2019-04-26 | 江西理工大学 | A kind of U-shaped Segmentation Method of Retinal Blood Vessels of adaptive scale information |
| CN109903292A (en) * | 2019-01-24 | 2019-06-18 | 西安交通大学 | A kind of three-dimensional image segmentation method and system based on full convolutional neural networks |
| CN110111335A (en) * | 2019-05-08 | 2019-08-09 | 南昌航空大学 | A kind of the urban transportation Scene Semantics dividing method and system of adaptive confrontation study |
| CN110188765A (en) * | 2019-06-05 | 2019-08-30 | 京东方科技集团股份有限公司 | Image, semantic parted pattern generation method, device, equipment and storage medium |
| CN110598714A (en) * | 2019-08-19 | 2019-12-20 | 中国科学院深圳先进技术研究院 | Cartilage image segmentation method and device, readable storage medium and terminal equipment |
| CN110490884A (en) * | 2019-08-23 | 2019-11-22 | 北京工业大学 | A kind of lightweight network semantic segmentation method based on confrontation |
Non-Patent Citations (1)
| Title |
|---|
| 廖旋等: "融合分割先验的多图像目标语义分割", 《中国图象图形学报》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111914482A (en) * | 2020-07-27 | 2020-11-10 | 武汉中海庭数据技术有限公司 | Driving condition generation method and system for automatic driving test |
| CN112597825A (en) * | 2020-12-07 | 2021-04-02 | 深延科技(北京)有限公司 | Driving scene segmentation method and device, electronic equipment and storage medium |
| CN113486716A (en) * | 2021-06-04 | 2021-10-08 | 电子科技大学长三角研究院(衢州) | Airport scene target segmentation method and system thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111062347B (en) | 2023-04-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11475681B2 (en) | Image processing method, apparatus, electronic device and computer readable storage medium | |
| CN111461039B (en) | Landmark identification method based on multi-scale feature fusion | |
| CN112560831B (en) | Pedestrian attribute identification method based on multi-scale space correction | |
| CN112990065B (en) | Vehicle classification detection method based on optimized YOLOv5 model | |
| CN115631344B (en) | Target detection method based on feature self-adaptive aggregation | |
| CN108305260B (en) | Method, device and equipment for detecting angular points in image | |
| CN111062347B (en) | Traffic element segmentation method in automatic driving, electronic equipment and storage medium | |
| CN112733885A (en) | Point cloud identification model determining method and point cloud identification method and device | |
| CN110781980B (en) | Training method of target detection model, target detection method and device | |
| CN111008576B (en) | Pedestrian detection and model training method, device and readable storage medium | |
| CN117058646B (en) | Complex road target detection method based on multi-mode fusion aerial view | |
| CN110852327A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN110991414B (en) | Traffic element high-precision segmentation method, electronic equipment and storage medium | |
| CN114037640A (en) | Image generation method and device | |
| CN115147648A (en) | Tea shoot identification method based on improved YOLOv5 target detection | |
| CN110969104A (en) | Method, system and storage medium for detecting travelable area based on binarization network | |
| CN114332814A (en) | Parking frame identification method and device, electronic equipment and storage medium | |
| Al Mamun et al. | Efficient lane marking detection using deep learning technique with differential and cross-entropy loss. | |
| CN114220087A (en) | License plate detection method, license plate detector and related equipment | |
| CN113989814A (en) | Image generation method and device, computer equipment and storage medium | |
| CN117612136A (en) | Automatic driving target detection method based on increment small sample learning | |
| CN111160282B (en) | Traffic light detection method based on binary Yolov3 network | |
| CN116778470A (en) | Object recognition and object recognition model training method, device, equipment and medium | |
| CN114898306A (en) | Method and device for detecting target orientation and electronic equipment | |
| CN115424250A (en) | License plate recognition method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |