CN109993082A - The classification of convolutional neural networks road scene and lane segmentation method - Google Patents
The classification of convolutional neural networks road scene and lane segmentation method Download PDFInfo
- Publication number
- CN109993082A CN109993082A CN201910211802.XA CN201910211802A CN109993082A CN 109993082 A CN109993082 A CN 109993082A CN 201910211802 A CN201910211802 A CN 201910211802A CN 109993082 A CN109993082 A CN 109993082A
- Authority
- CN
- China
- Prior art keywords
- classification
- road
- neural networks
- convolutional neural
- scene
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
Abstract
The present invention relates to a kind of classification of convolutional neural networks road scene and lane segmentation method, establish double task cooperative structural models based on convolutional neural networks, it is made of an encoder, two decoders, pass through end-to-end training, realize that characteristic information is shared, the classification to the various roads scene such as urban road, backroad and highway is completed, and region can travel to the road in scene and be split, reaches the double goal of road scene classification and road area segmentation.Based on convolutional neural networks, by road scene perceive in scene classification in conjunction with the double task of road extraction, exported in real time by training mode end to end, be beneficial to improve intelligent driving auxiliary system function, effective reliable drive assistance function is provided.Wherein in model to the part of travelable extracted region using full convolutional network in such a way that condition random field combines, can generate high-definition picture optimization output as a result, reaching more accurate segmentation effect.
Description
Technical field
The present invention relates to a kind of image recognition technology, in particular to a kind of convolutional neural networks road scene classification and road
Dividing method.
Background technique
The perception of road environment is the key technology in vehicle assistant drive technology, and the analysis and understanding of road scene are vehicles
Carry the important component of intelligence system.In terms of automatic Pilot, the target of road scene identification is can to obtain road automatically
The high-layer semantic information of image judges scene type belonging to image, and mentions to the travelable region in image
It takes.Such as under the complex conditions background such as sleet, haze, to street, country road and expressway, these road scenes are divided
Class identification makes the early warnings such as speed adjustment to the vehicle of traveling, it will help realize unmanned.In automobile assistant driving
System aspects, carrying out profound semantic understanding to road scene facilitates navigation and the intelligent early-warning of automobile, uses tape label
Road scene image can more describe scene locating for driver, ride safety of automobile is played a crucial role.
Realize scene classification research direction mainly include the following types: the new feature extraction of scene image, based on high level
Semantic scene classification and the scene classification based on deep learning.When scene type reaches thousand classes or more, and database volume is prominent
When breaking million images, traditional method based on low-level image feature and high-level semantic is difficult to handle mass data.And use depth
The method of study then has good performance in this big data, and therefore, the present invention proposes the road field based on deep learning
Scape perception and lane segmentation method.
In terms of deep learning road scene sort research, application No. is 201710894156.2 patents to propose one kind
Driving Scene classification method based on convolutional neural networks.Using traditional convolution with feature extraction layer and Feature Mapping layer
Neural network carries out feature extraction and training to image by convolution sum sub-sampling twice, pixel value is rasterized and connected into
Output of one vector as network generates scene classifier, realizes the classification to variety classes Driving Scene.
Application No. is 201711174595.2 patents to propose a kind of visual scene identification side based on convolutional neural networks
Method arbitrarily grabs two from the Same Scene of initial data respectively and finds similar image from different scenes, as scene
The mode that perception variation and perception offset images pair occurs constructs test set, with convolutional neural networks to the feature of each image pair
Vector makes the difference and is standardized, and obtains feature difference figure, when scene appearance acute variation, can obtain higher accuracy rate.
Application No. is 201610261849.3 patents to propose a kind of traffic scene classification based on convolutional neural networks
Method, the original cut tree phase of characteristic information and image that the RGB-D image of traffic scene is extracted by convolutional neural networks
In conjunction with forming optimal purity cost tree, and carry out the covering of optimal purity, obtain clearly objective contour, increase the standard of classification
True property.
The patent of application number 201510080035.5 proposes a kind of deconvolution depth e-learning based on Weight
Scene recognition method obtains the feature on road scene image different scale using the deconvolution depth network model of Weight
Figure samples to characteristic pattern through spatial pyramid model, forms the feature expression of image, finally complete using supporting vector classifier
Constituent class training and test, obtain the recognition result of different scenes.
Summary of the invention
The problem of the present invention be directed to the identifications of multiclass complexity road scene, proposes a kind of convolutional neural networks road scene
Classify and lane segmentation method, establish double task cooperative structural models based on convolutional neural networks in method, is compiled by one
Code device, two decoders composition realize that characteristic information is shared by end-to-end training, complete to urban road, backroad and
The classification of the various roads scene such as highway, and region can travel to the road in scene and be split, reach road scene
The double goal of classification and road area segmentation.
The technical solution of the present invention is as follows: a kind of convolutional neural networks road scene classification and lane segmentation method, specific to wrap
Include following steps:
1) image in database is selected, randomly assigne is carried out to selection image, a part is used as training set image, another portion
It is allocated as test set image；
2) training set image is sent into double task cooperative structure convolutional neural networks road scene classification and lane segmentation mould
Type carries out convolutional neural networks training, convolutional neural networks after being trained；
Double task cooperative structure convolutional neural networks road scene classification are mentioned with lane segmentation model comprising input layer, feature
Layer, information processing layer and output layer are taken, training set image is unified for an equal amount of RGB image and is sent into input layer, and input layer is defeated
Image is sent into the feature coding device that feature extraction layer is generated by convolutional neural networks out, extracts lane segmentation in feature coding device
The characteristics of image needed with scene classification generates characteristic pattern；Information processing layer is by decoder two solutions of classification decoder and segmentation
Code device composition, the characteristic pattern of feature extraction layer output, which respectively enters two decoders, which carries out scene type judgement and road, can travel
Extracted region；The result of two decoder outputs is sent into output layer, and through joint reasoning, joint reasoning refers to sorting code number device and segmentation
Encoder shares identical characteristic information, and model only trains a weight network in training, and model is carrying out backpropagation ginseng
When number adjustment, the network optimization is carried out by the way of superposition sorting code number device and the loss of partition encoding device, obtains training the road Hou Dui
The convolutional neural networks of road scene perception, achievable scene classification determines and the task of lane segmentation；
3) convolutional neural networks identify after test set image being sent into step 2) training, and achievable scene classification determines
With the task of lane segmentation.
Step 2) the feature coding device uses convolutional neural networks VGG-16 as model；Decoder of classifying uses convolution
The full articulamentum of neural network VGG-16 generates one-dimensional characteristic vector, is then classified using Softmax classifier, completes more
The task of class road scene classification；Divide decoder using full convolutional network in such a way that condition random field combines, completion road
Divide task in road；Whole network is using training method end to end, and model only trains a weight network, in the reversed of inclusion layer
In communication process by superposition classification decoder and divide decoder loss carry out the network optimization, be completed at the same time scene classification and
The task of lane segmentation.
The classification performance of the step 2) classification decoder by calculate accuracy rate Precision and recall rate Recall this
Two indexs are measured, and the formula difference of accuracy rate and recall rate is as follows:
Wherein, TP indicates that the sample size being correctly detecting, FP indicate that the correct sample being not detected, FN indicate wrong
The sample that erroneous detection measures.
Step 2) the segmentation decoder is exactly the evaluation to Pixel-level classification, road point to the evaluation of segmentation result in fact
The problem of cutting can be considered as two classification problems, and any pixel belongs to road surface or belongs to background；Divide decoder and carries out picture
It when element classification, predicts that the classification results of output and mark image carry out Pixel-level comparison, evaluates the prediction output knot of each pixel
Fruit, if it is " positive class " that pixel, which is judged to road surface pixel, it is " negative class " that pixel, which is judged to background pixel point, then the pixel classifications
As a result true positives TP is taken1, false positive FP1, true negative TN1, false negative FN1One of four kinds of results；Wherein TP1Indicate that positive class is sentenced
It is set to positive class, FP1Indicate that negative class determines the class that is positive, TN1Indicate that negative class determines the class that is negative, FN1Indicate that positive class determines the class that is negative.Into
And can in the hope of the accurate rate TPR and recall rate FPR of each image on entire test set,
Segmentation effect is evaluated using the MaxF1 to classify based on pixel, F-Measure is that information retrieval field is common
Evaluation criterion, offset parameter β, then the calculation formula of F-Measure is as follows:
Wherein, when taking β=1, F score is the harmonic-mean of accurate rate and recall rate.
The beneficial effects of the present invention are: convolutional neural networks road scene classification of the present invention and lane segmentation method, base
In convolutional neural networks, by road scene perceive in scene classification in conjunction with the double task of road extraction, by end to end
Training mode is exported in real time, is beneficial to improve intelligent driving auxiliary system function, is provided effectively reliable drive and is assisted function
Energy.Full convolutional network (FCN, Fully Convolutional wherein is used to the part of travelable extracted region in model
Networks) with CRF (the Conditional Random Field) mode that condition random field combines, high-resolution can be generated
Rate image optimization exports as a result, reaching more accurate segmentation effect.
Detailed description of the invention
Fig. 1 is that the double task cooperative structure convolutional neural networks road scene classification of the present invention are illustrated with lane segmentation model
Figure；
Fig. 2 is the structure chart for the convolutional neural networks VGG-16 that the present invention uses.
Specific embodiment
1, system forms
The present invention realizes that road scene classification is divided into two parts with road area, and first part is trained part,
The training of network is carried out first.It is completed offline before system use.Second part is part of detecting, using having completed to train
Network, the road image of input is differentiated.The processing step of two parts is identical, and specific steps see below (3. processing
Process).Two-part difference is input data difference.The input data of first part is prior ready instruction as needed
Practice data set.The input data of second part is test data set.Test data set can be preprepared indiscipline
Image.Training data and test data can also be made of the real-time road image of collection in worksite.
2, data set
The data set that the present invention uses is by 4000 width road scene figures in SUN data set, KITTI data set and CamVid
As composition, training and test are participated in using 4 class label images, classification is divided into desert road, avenue, highway and rural area
4 class road scene of path, each classification include 1000 images.
The ratio that training dataset and test data set is arranged is 3:1.A quarter is randomly selected from each classification
Image forms test set image.Training dataset and test data set can also be as needed, by vehicle-mounted camera to road scene
Video acquisition is carried out, then by Video Quality Metric is that image is constituted.In vehicle travel process, by the original graph of vehicle-mounted camera acquisition
It is preprocessed as needing, then input convolutional neural networks model.
Input picture size is unified for 360 × 480 RGB image.Place is normalized to the contrast of image, brightness
Reason.
3, process flow
Double task cooperative structure convolutional neural networks road scene classification of the invention and lane segmentation model include four layers of knot
Structure: input layer, feature extraction layer, information processing layer and output layer.
Input layer is image input part.Described image refers to above-mentioned training set image or test set image.
Feature information extraction layer is the feature coding device generated by a convolutional neural networks.It is mentioned from the image of input layer
It wins the confidence characteristics of image abundant needed for breath process layer.
Information processing layer is made of two decoders, not to be classification decoder and segmentation decoder.Two decoders are total
The image feature information generated by feature extraction layer is enjoyed, road scene classification and lane segmentation task are respectively completed.
Output layer obtains the output result of convolutional neural networks.The result of two decoders is finally obtained through joint reasoning
The result of road scene perception.
4, specific steps
Since target training dataset is smaller, in order to improve precision as far as possible in the case where low volume data, therefore uses and turn
It moves learning method and carries out network training.Its basic thought be first pass through to be trained on another large-scale data set, and
Then the model parameter that training is obtained is instructed again as the initial value of present networks parameter on target training dataset
Practice, tuning further is carried out to original parameter, is also small parameter perturbations.The principle that shift learning is relied on is certain features similar
General character having the same on training dataset.
By the method training network of shift learning, using the model parameter of the training on large-scale data set as this reality
The problem of initial value for testing model, is tested on this basis, makes up data set too small bring accuracy decline.It will be 2012
It is trained in advance on ILSVRC (the ImageNet Large Scale Visual Recognition Challenge) data set in year
Initial weight of the good VGG-16 network model parameter as feature coding device and classification decoder.It is used when training network
Mini-batch gradient descent method optimizes target loss function J, and Study rate parameter (base_lr) is set as 0.01, moves
Measuring parameter (momentum) is 0.9, and weight decaying (weight_decay) is set as 0.0005, is then constantly trained to network
Until training loss function convergence.After training set picture is disturbed, extracted by the sequence that ten pictures are a batch, it is ensured that repeatedly
The access times of every image are 1 during generation, and training VGGNet network is restrained until loss function.VGG mould is used in experiment
Type realizes image characteristics extraction.
Model part parameter setting will be enumerated below, as shown in table 1.
Table 1
| Parameter | Default value | Function |
| gpu-nu | 0 | Which GPU is selected to accelerate convolution algorithm |
| image_resize | 360×480 | Image size |
| weights | Nothing | The Model Weight of network |
| file_image | Nothing | The input file of image |
In table 1, the effect of parameter gpu-nu is the sequence of decision model image processor GPU used when carrying out feature extraction
Number, it, when default value is 0, is exactly GPU that when multiple GPU are acted on simultaneously, model can select corresponding GPU according to the parameter
Number be 1.Parameter image_resize shows size when image input, in road scene perception task, image,
The picture size that video detection arrives is generally unknown, and usual VGG model uses fixed size 224 × 224 as input, this hair
The bright parameter value for defaulting picture size in an experiment is set as 360 × 480.Parameter weights is the power of convolutional neural networks model
Weight, weight file is relevant with the network model used, when the number of network training is not identical, the updated power of program iteration
Weight is also not quite similar.Initial weight of the invention uses ILSVRC data set training weight, is tested on this basis；
Parameter file_image corresponds to the input address of experimental data set, while containing test set and training set.In training process, make
The objective function for intersecting entropy loss (cross-entropy loss) function as training network is used, it is all in batchsize
Image pixel is both participated in unwise calculation.When the pixel quantity between classification each in training set has very large deviation, based on true
Classification need differentially to weigh loss, this process or processing be known as class between balance.
Specific processing step is as follows.
Step 1: input picture.Input size is unified for 360 × 480 RGB image.
Step 2: using convolutional neural networks VGG-16 as the encoder of model, extracts the feature of road image, generates
Characteristic pattern.
The task of feature extraction layer is processing image and extracts lane segmentation and scene classification abundant that required necessity is schemed
As information.Depth convolutional neural networks VGG16 model can fast and effeciently learn scene image feature, so that scene image is special
Sign has more abstractness, and ability to express is more prominent.
The structure chart of the convolutional neural networks VGG-16 used as shown in Figure 2, VGG-16 include five sections of convolution, each section of volume
Product includes 2-3 convolutional layer, while every segment trailer can connect a maximum pond layer (Maxpool) and be used to downscaled images size.
The quantity of convolution kernel in every section is the same, and the convolution nuclear volume of section more rearward is more: for 64-128-256-512-512.
In cataloged procedure, retain the whole convolutional layers and pond layer of VGG-16, gives up and connect entirely for three of classification below
Connect layer.It the use of size is 3*3, the convolution kernel that step-length is 1 can generate corresponding characteristic pattern when executing convolution operation each time,
Extract the minutia of image.Nonlinear Mapping is carried out to its result with ReLU activation primitive, is then 2*2, step-length with size
Pondization operation is carried out for 2 filter, the feature of image is screened.Each down-sampling becomes the length of characteristic pattern with width
For original half.After multiple convolution, pondization operation, the spy of fixed resolution needed for finally obtaining information processing layer
Sign figure.
Step 3: using dual-task decoder (respectively scene classification decoder and lane segmentation decoder) to step 2
The characteristic pattern of middle generation is decoded analysis, achievees the purpose that scene type determines and road can travel extracted region.
(1) classification decoder
Road scene includes desert road, four class of avenue, highway and country road, and when classification uses
Softmax classifier.If m road scene data set sample is expressed as follows:
{(x1,y1),(x2,y2),……,(xm,ym)}
Wherein, training or test image x are given and extracts feature, x by feature extraction layeriIt is mentioned for i-th of sample through feature
The feature vector exported after taking layer to handle, yiIt is the class label of i-th of sample, yi∈ { 1,2 ..., k }, data set include four altogether
Class road scene image, then k=4.
For given test sample image x, indicate that sample x belongs to the general of a certain classification j with function p (y=j | x)
Rate, wherein j ∈ { 1,2 ..., k }.Sample image x is divided into the probability of classification j in Softmax classifier are as follows:
Wherein, ω=(ω1, ω2..., ωk) be in full articulamentum neuron with export mind i-th of Softmax classifier
The weight parameter being connected through member, ωTX is the weighted term of parameter ω and sample attribute.Pω(xi) it is a probability matrix.Probability
With for 1, in particular to
It is that image is judged to the probability of each scene type to be indicated with the number between 0 to 1 that it, which is acted on, and with matrix
Form output.
When classifier classifies to road image, the probability that image belongs to each classification is calculated, takes probability value maximum
It is classified as the classification of the image.
(2) divide decoder
Full convolutional network (FCN, Fully Convolutional is used when travelable extracted region in road scene
Networks) the mode combined with CRF (Conditional Random Field) condition random field.Original road scene figure
As the size reduced through feature coding device Chi Huahou, low resolution characteristic pattern is generated, to obtain the segmentation figure big with original image etc.,
Traditional FCN (Fully Convolutional Networks) adopt by deconvolution using convolutional layer to output result
Sample generates high-definition picture optimization output as a result, reaching segmentation effect in conjunction with jump link structure, but FCN is to increase image
Size can introduce noise to the way that original image is filled, and the result divided is not fine enough, lack Space Consistency.And incite somebody to action
When FCN is combined with CRF, training simultaneously generates a network end to end, promotes segmentation effect.In the segmentation of road area
In task, first using full convolutional network FCN in input picture road surface and background carry out rough sort, obtain each pixel
Belong to road surface or belongs to the probability of non-road surface (background).Then, use condition random field, to road surface in rough sort result with it is non-
The marginal portion on road surface optimizes, and obtains that edge is more careful, more accurate lane segmentation result on the whole.Condition random field energy
By the RGB feature of input picture and its spatial relation all as reference, maximum a posteriori is carried out in the prediction result of model
Distributed operation, to enhance the segmentation effect of FCN.
Assuming that there are random areas X=(X1, X2..., Xn), n is the quantity of pixel, XiIt is the mark class of pixel i
Not, wherein i ∈ { 1,2 ..., m }, all mark classifications constitute mark collection, can use L=(l1, l2..., lk) indicate, k is classification
Number；There are another random areas M=(M1, M2..., Mn), MiIt is the color feature vector of pixel i.For condition random field
(M, X) provides a probability function:
Wherein, E (X | M) is energy function, is simplified shown as E (X), and X belongs to mark collection L, and Z (M) is normalization factor.It is logical
It crosses minimum energy function and obtains optimal pixel classifications result.The energy object function of condition random field are as follows:
E (X)=∑iψu(xi)+∑i,jψp(xi,xj) (3)
Wherein, i, j indicate pixel.The first item ψ of energy equationu(xi)=- logP (xi), referred to as unitary potential function should
Pixel belongs to class label XiProbability, P (xi) indicate that pixel i belongs to the probability of a certain class mark.The second of energy equation
Item ψp(xi,xj) it is pairs of potential function, belong to of a sort probability P (x for measuring two pixelsi,xj).And:
Wherein, μ (xi,xj) it is label comparison function, xi≠xjWhen, μ (xi,xjOtherwise)=1 is equal to 0, for judging difference
Compatibility between label；P indicates location information；M indicates colouring information；θα、θγFor controlling the scale of location information；θβWith
In the scale of control color similarity；ω is linear combination weight.
The prior probability of the unitary potential function item of condition random field is provided by FCN, primary Calculation go out pixel belong to it is each
Then the probability of classification enhances segmentation effect by pairs of potential function.CRF is in image segmentation task by combining original image
The mode of RGB feature improves the segmentation effect of network.When image edge area color difference is larger, the pixel quilt of both sides of edges
It is larger to be divided into different classes of probability.
Step 4: the result that two kinds of decoders generate exported through joint reasoning to (joint reasoning refers to classification decoder
With segmentation decoder share convolutional layer provide characteristic information, model training when only train a weight network, model into
When row backpropagation parameter adjusts, the network optimization is carried out by the way of superposition classification decoder and segmentation decoder losses), it is right
Road scene carries out effectively perceive.
Full convolutional network FCN and convolutional neural networks model VGG-16 convolutional layer having the same in structure, therefore
Realize that road scene classification with when can exercise extracted region, is used for feature extraction using identical convolutional layer, the spy as model
Encoder is levied, achievees the purpose that feature information extraction layer shares convolutional layer.Decoder of classifying is raw using the full articulamentum of VGG-16
At one-dimensional characteristic vector, then classified using Softmax classifier, completes the task of multiclass road scene classification.Segmentation
Decoder in such a way that condition random field combines, completes lane segmentation task using full convolutional network.Whole network uses end
To the training method at end, model only trains a weight network, passes through superposition classification solution in the back-propagation process of inclusion layer
The loss of code device and segmentation decoder carries out the network optimization, is completed at the same time the task of scene classification and lane segmentation.
The classification performance of model is by calculating this two indexs progress of accuracy rate (Precision) and recall rate (Recall)
It measures.In image classification task, it is the figure for really belonging to the category among the picture of certain classification that accuracy rate expression, which is predicted to be,
Piece ratio.And recall rate indicates among the actual test sample of certain classification picture finally by model partition to such sample ratio
The formula difference of example, accuracy rate and recall rate is as follows:
Wherein, TP indicates that the sample size being correctly detecting, FP indicate that the correct sample being not detected, FN indicate wrong
The sample that erroneous detection measures.
Evaluation to segmentation result is exactly the evaluation to Pixel-level classification in fact.Lane segmentation problem can be considered as one two
Classification problem, any pixel belong to road surface or belong to background.Classifier predicts the classification knot of output when carrying out pixel classifications
Fruit and mark image carry out Pixel-level comparison, and the prediction for evaluating each pixel, which exports as a result, setting, is judged to road surface pixel for pixel
For " positive class ", it is " negative class " that pixel, which is judged to background pixel point, then the pixel classifications result takes true positives (TP1), false positive
(FP1), true negative (TN1), false negative (FN1One of) four kinds of results.Wherein TP1Indicate that positive class determines the class that is positive, FP1It indicates
Negative class determines the class that is positive, TN1Indicate that negative class determines the class that is negative, FN1Indicate that positive class determines the class that is negative.And then it can be in the hope of entire
The accurate rate (TPR) and recall rate (FPR) of each image on test set.
For the present invention using segmentation effect is evaluated based on the MaxF1 that pixel is classified, F-Measure is a kind of statistic,
F-Measure is also known as F-Score, is the common evaluation criterion of information retrieval field.Offset parameter is β, then F-Measure
Calculation formula is as follows:
Wherein, when taking β=1, F score is the harmonic-mean of accurate rate and recall rate.
Accuracy rate, recall rate and segmentation effect F value the heterogeneous networks recognition result pair as shown in table 2 that the method for the present invention obtains
Than.
Table 2
Claims (4)
1. a kind of convolutional neural networks road scene classification and lane segmentation method, which is characterized in that specifically comprise the following steps:
1) image in database is selected, randomly assigne is carried out to selection image, a part is used as training set image, and another part is made
For test set image；
2) by training set image be sent into double task cooperative structure convolutional neural networks road scenes classification and lane segmentation model into
The training of row convolutional neural networks, convolutional neural networks after being trained；
Double task cooperative structure convolutional neural networks road scene classification and lane segmentation model include input layer, feature extraction
Layer, information processing layer and output layer, training set image are unified for an equal amount of RGB image and are sent into input layer, input layer output
Image is sent into the feature coding device that is generated by convolutional neural networks of feature extraction layer, extracted in feature coding device lane segmentation with
The characteristics of image that scene classification needs generates characteristic pattern；Information processing layer is by decoder two decodings of classification decoder and segmentation
Device composition, the characteristic pattern of feature extraction layer output respectively enter two decoders progress scene types and determine to can travel area with road
It extracts in domain；The result of two decoder outputs is sent into output layer, and through joint reasoning, joint reasoning refers to that sorting code number device and segmentation are compiled
Code device shares identical characteristic information, and model only trains a weight network in training, and model is carrying out backpropagation parameter
When adjustment, the network optimization is carried out by the way of superposition sorting code number device and the loss of partition encoding device, to road after being trained
The convolutional neural networks of scene perception, achievable scene classification determines and the task of lane segmentation；
3) convolutional neural networks identify after test set image being sent into step 2) training, and achievable scene classification determines and road
The task of road segmentation.
2. the classification of convolutional neural networks road scene and lane segmentation method according to claim 1, which is characterized in that described
Step 2) feature coding device uses convolutional neural networks VGG-16 as model；Decoder of classifying uses convolutional neural networks VGG-
16 full articulamentum generates one-dimensional characteristic vector, is then classified using Softmax classifier, completes multiclass road scene point
The task of class；Divide decoder using full convolutional network in such a way that condition random field combines, completion lane segmentation task；It is whole
A network only trains a weight network using training method end to end, model, leads in the back-propagation process of inclusion layer
The loss progress network optimization for being superimposed classification decoder and dividing decoder is crossed, times of scene classification and lane segmentation is completed at the same time
Business.
3. convolutional neural networks road scene classification according to claim 1 or claim 2 and lane segmentation method, which is characterized in that
The classification performance of step 2) the classification decoder is by calculating this two indexs of accuracy rate Precision and recall rate Recall
It is measured, the formula difference of accuracy rate and recall rate is as follows:
Wherein, TP indicates that the sample size being correctly detecting, FP indicate that the correct sample being not detected, FN indicate to be examined by mistake
The sample measured.
4. convolutional neural networks road scene classification according to claim 1 or claim 2 and lane segmentation method, which is characterized in that
Step 2) the segmentation decoder is exactly the evaluation to Pixel-level classification to the evaluation of segmentation result in fact, and lane segmentation problem can
It is considered as two classification problems, any pixel belongs to road surface or belongs to background；Divide decoder when carrying out pixel classifications,
The classification results and mark image for predicting output carry out Pixel-level comparison, and the prediction for evaluating each pixel exports as a result, setting picture
It is " positive class " that element, which is judged to road surface pixel, and it is " negative class " that pixel, which is judged to background pixel point, then the pixel classifications result takes very
Positive TP1, false positive FP1, true negative TN1, false negative FN1One of four kinds of results；Wherein TP1Indicate that positive class judgement is positive
Class, FP1Indicate that negative class determines the class that is positive, TN1Indicate that negative class determines the class that is negative, FN1Indicate that positive class determines the class that is negative.And then it can be with
The accurate rate TPR and recall rate FPR of each image on entire test set are acquired,
Segmentation effect is evaluated using the MaxF1 to classify based on pixel, F-Measure is that information retrieval field is commonly evaluated
Standard, offset parameter β, then the calculation formula of F-Measure is as follows:
Wherein, when taking β=1, F score is the harmonic-mean of accurate rate and recall rate.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910211802.XA CN109993082B (en) | 2019-03-20 | 2019-03-20 | Convolutional neural network road scene classification and road segmentation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910211802.XA CN109993082B (en) | 2019-03-20 | 2019-03-20 | Convolutional neural network road scene classification and road segmentation method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109993082A true CN109993082A (en) | 2019-07-09 |
| CN109993082B CN109993082B (en) | 2021-11-05 |
Family
ID=67130681
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910211802.XA Expired - Fee Related CN109993082B (en) | 2019-03-20 | 2019-03-20 | Convolutional neural network road scene classification and road segmentation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109993082B (en) |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109902600A (en) * | 2019-02-01 | 2019-06-18 | 清华大学 | A kind of road area detection method |
| CN110458047A (en) * | 2019-07-23 | 2019-11-15 | 北京理工大学 | A kind of country scene recognition method and system based on deep learning |
| CN110503637A (en) * | 2019-08-13 | 2019-11-26 | 中山大学 | A kind of crack on road automatic testing method based on convolutional neural networks |
| CN110598586A (en) * | 2019-08-27 | 2019-12-20 | 电子科技大学 | Target detection method and system |
| CN110599497A (en) * | 2019-07-31 | 2019-12-20 | 中国地质大学（武汉） | Drivable region segmentation method based on deep neural network |
| CN110717383A (en) * | 2019-08-29 | 2020-01-21 | 阿里巴巴集团控股有限公司 | Remote sensing detection method, device and system |
| CN110781773A (en) * | 2019-10-10 | 2020-02-11 | 湖北工业大学 | Road extraction method based on residual error neural network |
| CN111144418A (en) * | 2019-12-31 | 2020-05-12 | 北京交通大学 | Railway track area segmentation and extraction method |
| CN111160109A (en) * | 2019-12-06 | 2020-05-15 | 北京联合大学 | Road segmentation method and system based on deep neural network |
| CN111161282A (en) * | 2019-12-30 | 2020-05-15 | 西南交通大学 | Target scale selection method for image multi-level segmentation based on depth seeds |
| CN111191654A (en) * | 2019-12-30 | 2020-05-22 | 重庆紫光华山智安科技有限公司 | Road data generation method and device, electronic equipment and storage medium |
| CN111368846A (en) * | 2020-03-19 | 2020-07-03 | 中国人民解放军国防科技大学 | Road ponding identification method based on boundary semantic segmentation |
| CN111611919A (en) * | 2020-05-20 | 2020-09-01 | 西安交通大学苏州研究院 | Road scene layout analysis method based on structured learning |
| CN111898702A (en) * | 2020-08-14 | 2020-11-06 | 海之韵(苏州)科技有限公司 | Unmanned ship environment intelligent sensing method based on deep learning |
| CN111985324A (en) * | 2020-07-14 | 2020-11-24 | 广西大学 | Road detection method combining full convolution regression neural network and conditional random field |
| CN111985378A (en) * | 2020-08-13 | 2020-11-24 | 中国第一汽车股份有限公司 | Road target detection method, device and equipment and vehicle |
| CN112092827A (en) * | 2020-09-23 | 2020-12-18 | 北京百度网讯科技有限公司 | Automatic driving function control method, automatic driving function control device, electronic equipment and storage medium |
| CN112528500A (en) * | 2020-12-11 | 2021-03-19 | 深兰科技（上海）有限公司 | Evaluation method and evaluation equipment for scene graph construction model |
| CN112669316A (en) * | 2021-01-29 | 2021-04-16 | 南方电网调峰调频发电有限公司 | Power production abnormity monitoring method and device, computer equipment and storage medium |
| CN112766092A (en) * | 2021-01-05 | 2021-05-07 | 北京航空航天大学 | Method for quickly identifying background category based on brain-like neural network and application thereof |
| CN112949617A (en) * | 2021-05-14 | 2021-06-11 | 江西农业大学 | Rural road type identification method, system, terminal equipment and readable storage medium |
| CN113015887A (en) * | 2019-10-15 | 2021-06-22 | 谷歌有限责任公司 | Navigation directions based on weather and road surface type |
| CN113096001A (en) * | 2021-04-01 | 2021-07-09 | 咪咕文化科技有限公司 | Image processing method, electronic device and readable storage medium |
| CN113362332A (en) * | 2021-06-08 | 2021-09-07 | 南京信息工程大学 | Depth network segmentation method for coronary artery lumen contour under OCT image |
| CN113408633A (en) * | 2021-06-29 | 2021-09-17 | 北京百度网讯科技有限公司 | Method, apparatus, device and storage medium for outputting information |
| CN113449589A (en) * | 2021-05-16 | 2021-09-28 | 桂林电子科技大学 | Method for calculating driving strategy of unmanned automobile in urban traffic scene |
| EP3901822A1 (en) * | 2020-04-24 | 2021-10-27 | Stradvision, Inc. | Method and device for on-vehicle active learning to be used for training perception network of autonomous vehicle |
| CN115223119A (en) * | 2022-06-15 | 2022-10-21 | 广州汽车集团股份有限公司 | Driving region detection method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106599773A (en) * | 2016-10-31 | 2017-04-26 | 清华大学 | Deep learning image identification method and deep learning image identification system used for intelligent driving, and terminal device |
| CN107609602A (en) * | 2017-09-28 | 2018-01-19 | 吉林大学 | A kind of Driving Scene sorting technique based on convolutional neural networks |
| CN108492272A (en) * | 2018-03-26 | 2018-09-04 | 西安交通大学 | Cardiovascular vulnerable plaque recognition methods based on attention model and multitask neural network and system |
| CN108876796A (en) * | 2018-06-08 | 2018-11-23 | 长安大学 | A kind of lane segmentation system and method based on full convolutional neural networks and condition random field |
-
2019
- 2019-03-20 CN CN201910211802.XA patent/CN109993082B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106599773A (en) * | 2016-10-31 | 2017-04-26 | 清华大学 | Deep learning image identification method and deep learning image identification system used for intelligent driving, and terminal device |
| CN107609602A (en) * | 2017-09-28 | 2018-01-19 | 吉林大学 | A kind of Driving Scene sorting technique based on convolutional neural networks |
| CN108492272A (en) * | 2018-03-26 | 2018-09-04 | 西安交通大学 | Cardiovascular vulnerable plaque recognition methods based on attention model and multitask neural network and system |
| CN108876796A (en) * | 2018-06-08 | 2018-11-23 | 长安大学 | A kind of lane segmentation system and method based on full convolutional neural networks and condition random field |
Non-Patent Citations (2)
| Title |
|---|
| HAO ZHOU 等: "Image Semantic Segmentation Based on FCN-CRF Model", 《2016 INTERNATIONAL CONFERENCE ON IMAGE, VISION AND COMPUTING》 * |
| KIM-HAN THUNG 等: "A brief review on multi-task learning", 《MULTIMEDIA TOOLS AND APPLICATIONS》 * |
Cited By (40)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109902600A (en) * | 2019-02-01 | 2019-06-18 | 清华大学 | A kind of road area detection method |
| CN109902600B (en) * | 2019-02-01 | 2020-10-27 | 清华大学 | Road area detection method |
| CN110458047A (en) * | 2019-07-23 | 2019-11-15 | 北京理工大学 | A kind of country scene recognition method and system based on deep learning |
| CN110599497A (en) * | 2019-07-31 | 2019-12-20 | 中国地质大学（武汉） | Drivable region segmentation method based on deep neural network |
| CN110503637A (en) * | 2019-08-13 | 2019-11-26 | 中山大学 | A kind of crack on road automatic testing method based on convolutional neural networks |
| CN110503637B (en) * | 2019-08-13 | 2022-12-06 | 中山大学 | Road crack automatic detection method based on convolutional neural network |
| CN110598586A (en) * | 2019-08-27 | 2019-12-20 | 电子科技大学 | Target detection method and system |
| CN110717383A (en) * | 2019-08-29 | 2020-01-21 | 阿里巴巴集团控股有限公司 | Remote sensing detection method, device and system |
| CN110717383B (en) * | 2019-08-29 | 2023-05-02 | 阿里巴巴集团控股有限公司 | Remote sensing detection method, device and system |
| CN110781773A (en) * | 2019-10-10 | 2020-02-11 | 湖北工业大学 | Road extraction method based on residual error neural network |
| CN113015887A (en) * | 2019-10-15 | 2021-06-22 | 谷歌有限责任公司 | Navigation directions based on weather and road surface type |
| CN111160109A (en) * | 2019-12-06 | 2020-05-15 | 北京联合大学 | Road segmentation method and system based on deep neural network |
| CN111160109B (en) * | 2019-12-06 | 2023-08-18 | 北京联合大学 | Road segmentation method and system based on deep neural network |
| CN111161282B (en) * | 2019-12-30 | 2021-10-29 | 西南交通大学 | Target scale selection method for image multi-level segmentation based on depth seeds |
| CN111191654A (en) * | 2019-12-30 | 2020-05-22 | 重庆紫光华山智安科技有限公司 | Road data generation method and device, electronic equipment and storage medium |
| CN111161282A (en) * | 2019-12-30 | 2020-05-15 | 西南交通大学 | Target scale selection method for image multi-level segmentation based on depth seeds |
| CN111144418B (en) * | 2019-12-31 | 2022-12-02 | 北京交通大学 | Railway track area segmentation and extraction method |
| CN111144418A (en) * | 2019-12-31 | 2020-05-12 | 北京交通大学 | Railway track area segmentation and extraction method |
| CN111368846A (en) * | 2020-03-19 | 2020-07-03 | 中国人民解放军国防科技大学 | Road ponding identification method based on boundary semantic segmentation |
| CN111368846B (en) * | 2020-03-19 | 2022-09-09 | 中国人民解放军国防科技大学 | Road ponding identification method based on boundary semantic segmentation |
| KR102589764B1 (en) | 2020-04-24 | 2023-10-17 | 주식회사 스트라드비젼 | On-vehicle active learning method and device for learning the perception network of an autonomous vehicle |
| KR20210152025A (en) * | 2020-04-24 | 2021-12-14 | 주식회사 스트라드비젼 | On-Vehicle Active Learning Method and Apparatus for Learning Perception Network of Autonomous Vehicle |
| EP3901822A1 (en) * | 2020-04-24 | 2021-10-27 | Stradvision, Inc. | Method and device for on-vehicle active learning to be used for training perception network of autonomous vehicle |
| CN111611919A (en) * | 2020-05-20 | 2020-09-01 | 西安交通大学苏州研究院 | Road scene layout analysis method based on structured learning |
| CN111985324A (en) * | 2020-07-14 | 2020-11-24 | 广西大学 | Road detection method combining full convolution regression neural network and conditional random field |
| CN111985324B (en) * | 2020-07-14 | 2022-10-28 | 广西大学 | Road detection method combining full convolution regression neural network and conditional random field |
| CN111985378A (en) * | 2020-08-13 | 2020-11-24 | 中国第一汽车股份有限公司 | Road target detection method, device and equipment and vehicle |
| CN111898702A (en) * | 2020-08-14 | 2020-11-06 | 海之韵(苏州)科技有限公司 | Unmanned ship environment intelligent sensing method based on deep learning |
| CN112092827B (en) * | 2020-09-23 | 2022-04-22 | 北京百度网讯科技有限公司 | Automatic driving function control method, automatic driving function control device, electronic equipment and storage medium |
| CN112092827A (en) * | 2020-09-23 | 2020-12-18 | 北京百度网讯科技有限公司 | Automatic driving function control method, automatic driving function control device, electronic equipment and storage medium |
| CN112528500A (en) * | 2020-12-11 | 2021-03-19 | 深兰科技（上海）有限公司 | Evaluation method and evaluation equipment for scene graph construction model |
| CN112528500B (en) * | 2020-12-11 | 2023-08-29 | 深兰人工智能应用研究院(山东)有限公司 | Evaluation method and evaluation equipment for scene graph construction model |
| CN112766092A (en) * | 2021-01-05 | 2021-05-07 | 北京航空航天大学 | Method for quickly identifying background category based on brain-like neural network and application thereof |
| CN112669316A (en) * | 2021-01-29 | 2021-04-16 | 南方电网调峰调频发电有限公司 | Power production abnormity monitoring method and device, computer equipment and storage medium |
| CN113096001A (en) * | 2021-04-01 | 2021-07-09 | 咪咕文化科技有限公司 | Image processing method, electronic device and readable storage medium |
| CN112949617A (en) * | 2021-05-14 | 2021-06-11 | 江西农业大学 | Rural road type identification method, system, terminal equipment and readable storage medium |
| CN113449589A (en) * | 2021-05-16 | 2021-09-28 | 桂林电子科技大学 | Method for calculating driving strategy of unmanned automobile in urban traffic scene |
| CN113362332A (en) * | 2021-06-08 | 2021-09-07 | 南京信息工程大学 | Depth network segmentation method for coronary artery lumen contour under OCT image |
| CN113408633A (en) * | 2021-06-29 | 2021-09-17 | 北京百度网讯科技有限公司 | Method, apparatus, device and storage medium for outputting information |
| CN115223119A (en) * | 2022-06-15 | 2022-10-21 | 广州汽车集团股份有限公司 | Driving region detection method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109993082B (en) | 2021-11-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109993082A (en) | The classification of convolutional neural networks road scene and lane segmentation method | |
| CN108985194B (en) | Intelligent vehicle travelable area identification method based on image semantic segmentation | |
| CN111612807B (en) | Small target image segmentation method based on scale and edge information | |
| CN113159051B (en) | Remote sensing image lightweight semantic segmentation method based on edge decoupling | |
| CN110263786B (en) | Road multi-target identification system and method based on feature dimension fusion | |
| CN108509978A (en) | The multi-class targets detection method and model of multi-stage characteristics fusion based on CNN | |
| US11651302B2 (en) | Method and device for generating synthetic training data for an artificial-intelligence machine for assisting with landing an aircraft | |
| CN110826638A (en) | Zero sample image classification model based on repeated attention network and method thereof | |
| CN107133974A (en) | The vehicle type classification method that Gaussian Background modeling is combined with Recognition with Recurrent Neural Network | |
| CN112464911A (en) | Improved YOLOv 3-tiny-based traffic sign detection and identification method | |
| CN112801270B (en) | Automatic U-shaped network slot identification method integrating depth convolution and attention mechanism | |
| CN112183635A (en) | Method for realizing segmentation and identification of plant leaf lesions by multi-scale deconvolution network | |
| CN111753787A (en) | Separated traffic sign detection and identification method | |
| CN109670392A (en) | Based on mixing autocoder road image semantic segmentation method | |
| CN110659601B (en) | Depth full convolution network remote sensing image dense vehicle detection method based on central point | |
| CN110281949B (en) | Unified hierarchical decision-making method for automatic driving | |
| CN115035361A (en) | Target detection method and system based on attention mechanism and feature cross fusion | |
| CN116110022B (en) | Lightweight traffic sign detection method and system based on response knowledge distillation | |
| CN110008899A (en) | A kind of visible remote sensing image candidate target extracts and classification method | |
| CN113298817A (en) | High-accuracy semantic segmentation method for remote sensing image | |
| CN111310820A (en) | Foundation meteorological cloud chart classification method based on cross validation depth CNN feature integration | |
| CN110310298A (en) | A kind of road target real-time three-dimensional point cloud segmentation method based on cycling condition random field | |
| CN113792631A (en) | Aircraft detection and tracking method based on multi-scale self-adaption and side-domain attention | |
| CN112084897A (en) | Rapid traffic large-scene vehicle target detection method of GS-SSD | |
| CN115205568B (en) | Road traffic multi-element detection method based on multi-scale feature fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20211105 |