CN109635642A - A kind of road scene dividing method based on residual error network and expansion convolution - Google Patents

Abstract

The invention discloses a kind of based on residual error network and expands the road scene dividing method of convolution, constructs convolutional neural networks in the training stage, the Residual block that hidden layer is set gradually by 10 is formed；The original road scene image of every in training set is input in convolutional neural networks and is trained, the corresponding 12 width semantic segmentation prognostic chart of every original road scene image is obtained；By calculate set that the corresponding 12 width semantic segmentation prognostic chart of every original road scene image is constituted and corresponding true semantic segmentation image procossing at 12 width one-hot coding image constructions set between loss function value, obtain the best initial weights vector of convolutional neural networks classification based training model；It in test phase, is predicted using the best initial weights vector of convolutional neural networks classification based training model, obtains the corresponding prediction semantic segmentation image of road scene image to semantic segmentation；Advantage is that computation complexity is low, segmentation is high-efficient, segmentation precision is high, and robustness is good.

Description

A road scene segmentation method based on residual network and dilated convolution

technical field

The invention relates to a deep learning semantic segmentation technology, in particular to a road scene segmentation method based on residual network and dilated convolution.

Background technique

Deep learning is a branch of artificial neural network, and artificial neural network with deep network structure is the earliest network model of deep learning. Initially, the applications of deep learning were mainly in the fields of images and speech. Since 2006, deep learning has continued to heat up in academia. Deep learning and neural networks have been widely used in semantic segmentation, computer vision, speech recognition, and tracking. Its high efficiency also makes it suitable for real-time applications, etc. There is huge potential in all aspects.

Convolutional neural networks have achieved success in image classification, localization, and scene understanding. With the proliferation of tasks such as augmented reality and self-driving vehicles, many researchers have turned their attention to scene understanding, and one of the main steps is semantic segmentation, which is to classify each pixel in a given image. Semantic segmentation is of great importance in mobile and robotics related applications.

The problem of semantic segmentation plays a very important role in many application scenarios, such as image understanding, automatic driving, etc. Therefore, in recent years, the problem of semantic segmentation has received extensive attention in academia and industry. Classical semantic segmentation methods include Full Connected Network (FCN) and Convolutional Neural Network SegNet, etc. These methods have good performance in pixel accuracy, average pixel accuracy and average intersection ratio on road scene segmentation databases. However, one of the shortcomings of FCN is that due to the existence of the pooling layer, the size (length and width) of the response tensor is getting smaller and smaller. However, the original design of FCN needs to output the same size as the input, so FCN does Upsampling is used, but upsampling cannot recover all the lost information losslessly; the convolutional neural network SegNet is a network model built on the basis of FCN, but it does not control the problem of information loss well. Therefore, these methods affect the segmentation accuracy due to the loss of information, which leads to a decrease in the robustness of the method.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to provide a road scene segmentation method based on residual network and dilated convolution, which has low computational complexity, high segmentation efficiency, high segmentation accuracy and good robustness.

The technical solution adopted by the present invention to solve the above-mentioned technical problems is: a road scene segmentation method based on residual network and dilated convolution, which is characterized in that it includes two processes: a training phase and a testing phase;

The specific steps of the training phase process are:

Step 1_1: Select Q original road scene images and the real semantic segmentation images corresponding to each original road scene image, and form a training set, and mark the qth original road scene image in the training set as {I ^q (i ,j)}, denote the real semantic segmentation images corresponding to {I ^q (i, j)} in the training set as Then, the one-hot encoding technology is used to process the real semantic segmentation images corresponding to each original road scene image in the training set into 12 one-hot encoded images. The set of 12 processed one-hot encoded images is denoted as Among them, the road scene image is an RGB color image, Q is a positive integer, Q≥100, q is a positive integer, 1≤q≤Q, 1≤i≤W, 1≤j≤H, W represents {I ^q (i, j)}, H represents the height of {I ^q (i, j)}, I ^q (i, j) represents the pixel point at the coordinate position (i, j) in {I ^q (i, j)} Pixel values, express The pixel value of the pixel whose middle coordinate position is (i, j);

Step 1_2: Build a convolutional neural network: The convolutional neural network includes an input layer, a hidden layer and an output layer; the hidden layer consists of 10 Residual blocks set in sequence, among which, each convolutional layer in the first Residual block passes through Set the dilation rate to 1 to form a dilated convolutional layer, each convolutional layer in the second Residual block forms a dilated convolutional layer by setting the dilation rate to 1, and each convolutional layer in the third Residual block is dilated by setting The rate of 2 forms a dilated convolution layer, each convolution layer in the fourth Residual block forms an expanded convolution layer by setting the expansion rate to 2, and each convolution layer in the fifth Residual block is set by setting the expansion rate. 4 forms a dilated convolutional layer, each convolutional layer in the sixth Residualblock forms a dilated convolutional layer by setting a dilation rate of 4, and each convolutional layer in the seventh Residual block forms a dilation by setting a dilation rate of 2. Convolutional layer, each convolutional layer in the 8th Residual block forms a dilated convolutional layer by setting the dilation rate to 2, and each convolutional layer in the 9th Residual block forms a dilated convolutional layer by setting the dilation rate to 1 layer, each convolutional layer in the 10th Residual block forms a dilated convolutional layer by setting the dilation rate to 1;

For the input layer, the input end of the input layer receives the R channel component, G channel component and B channel component of an original input image, and the output end of the input layer outputs the R channel component, G channel component and B channel component of the original input image to Hidden layer; wherein, the width of the original input image received by the input end of the input layer is required to be W and the height is H;

For the first Residual block, the input end of the first Residual block receives the R channel component, G channel component and B channel component of the original input image output by the output end of the input layer, and the output end of the first Residual block outputs 32 images Feature map, the set composed of 32 feature maps is denoted as R ₁ ; wherein, the width of each feature map in R ₁ is W and the height is H;

For the second Residual block, the input of the second Residual block receives all the feature maps in R ₁ , the output of the second Residual block outputs 32 feature maps, and the set of 32 feature maps is recorded as R ₂ ; wherein, the width of each feature map in R ₂ is W and the height is H;

For the third Residual block, the input of the third Residual block receives all the feature maps in R ₂ , the output of the third Residual block outputs 64 feature maps, and the set of 64 feature maps is recorded as R ₃ ; wherein, the width of each feature map in R ₃ is W and the height is H;

For the 4th Residual block, the input of the 4th Residual block receives all the feature maps in _R3 , the output of the _4th Residual block outputs 64 feature maps, and the set of 64 feature maps is recorded as R4 ; wherein, the width of each feature map in R ₄ is W and the height is H;

For the 5th Residual block, the input of the 5th Residual block receives all the feature maps in R ₄ , the output of the 5th Residual block outputs 128 feature maps, and the set of 128 feature maps is recorded as R ₅ ; wherein, the width of each feature map in R ₅ is W and the height is H;

For the 6th Residual block, the input of the 6th Residual block receives all the feature maps in R ₅ , the output of the 6th Residual block outputs 128 feature maps, and the set of 128 feature maps is recorded as R ₆ ; wherein, the width of each feature map in R ₆ is W and the height is H;

For the 7th Residual block, the input of the 7th Residual block receives all the feature maps in R ₆ , the output of the 7th Residual block outputs 64 feature maps, and the set of 64 feature maps is recorded as R ₇ ; wherein, the width of each feature map in R ₇ is W and the height is H;

For the 8th Residual block, the input of the 8th Residual block receives all the feature maps in R ₇ , the output of the 8th Residual block outputs 64 feature maps, and the set of 64 feature maps is recorded as R ₈ ; wherein, the width of each feature map in R ₈ is W and the height is H;

For the ninth Residual block, the input of the ninth Residual block receives all the feature maps in R ₈ , the output of the ninth Residual block outputs 32 feature maps, and the set of 32 feature maps is recorded as R ₉ ; wherein, the width of each feature map in R ₉ is W and the height is H;

For the 10th Residual block, the input of the 10th Residual block receives all the feature maps in R ₉ , the output of the 10th Residual block outputs 32 feature maps, and the set of 32 feature maps is recorded as R ₁₀ ; wherein, the width of each feature map in R ₁₀ is W and the height is H;

For the output layer, it consists of 1 convolutional layer, the input of the output layer receives all the feature maps in R ₁₀ , and the output of the output layer outputs 12 semantic segmentation prediction maps corresponding to the original input image; The width of the semantic segmentation prediction map is W and the height is H;

Step 1_3: Take each original road scene image in the training set as the original input image, input it into the convolutional neural network for training, and obtain 12 semantic segmentation prediction maps corresponding to each original road scene image in the training set. The set of 12 semantic segmentation prediction maps corresponding to {I ^q (i, j)} is denoted as

Step 1_4: Calculate the loss function value between the set of 12 semantic segmentation prediction images corresponding to each original road scene image in the training set and the set of 12 one-hot encoded images processed from the corresponding real semantic segmentation image. ,Will and The loss function value between is denoted as

Step 1_5: Repeat steps 1_3 and 1_4 for a total of V times to obtain a convolutional neural network classification training model, and obtain a total of Q×V loss function values; then find the loss with the smallest value from the Q×V loss function values. function value; then the weight vector and bias term corresponding to the loss function value with the smallest value are corresponding to the optimal weight vector and optimal bias term of the convolutional neural network classification training model, which are correspondingly recorded as W ^best and b ^best ; Wherein, V>1;

The specific steps of the test phase process are:

Step 2_1: Make Indicates the road scene image to be semantically segmented; where 1≤i'≤W', 1≤j'≤H', W' denotes width, H' means the height of, express The pixel value of the pixel whose middle coordinate position is (i, j);

Step 2_2: Put the The R channel component, G channel component and B channel component are input into the convolutional neural network classification training model, and use W ^best and b ^best to predict, get The corresponding predicted semantic segmentation image, denoted as in, express The pixel value of the pixel whose middle coordinate position is (i', j').

In the described steps 1-4, Obtained using categorical cross-entropy.

Compared with the prior art, the advantages of the present invention are:

1) In the process of constructing the convolutional neural network, the method of the present invention introduces the Residual block (residual network block) embedded in the identical shortcut connection in the ResNet (residual network), which is composed of 10 stacked Residual blocks. The hidden layer of the convolutional neural network and the setting of the Residual block increase the ability to extract feature information and fully absorb the structural efficiency of the basic module of the residual network, thereby improving the prediction accuracy of the trained convolutional neural network classification training model. Spend.

2) In the convolutional neural network constructed by the method of the present invention, only 10 Residual blocks are used in the hidden layer, which greatly reduces the cost loss of a series of problems such as redundancy and large amount of data, thereby making its computational complexity low; Each Residual block uses a convolution layer formed by setting the dilation rate. The dilated convolution avoids the loss of information due to the size transformation process. While expanding the receptive field, it ensures the characteristics of The resolution of the graph remains unchanged, and the effective depth information is retained to a large extent, so that the semantic segmentation prediction graph obtained in the training phase and the predicted semantic segmentation image obtained in the testing phase have high resolution, accurate boundaries, and good spatial continuity.

3) The Residual block adopted by the method of the present invention not only greatly improves the extraction strength of feature information, but also prevents the model from overfitting, has strong robustness, and greatly improves the segmentation efficiency.

Description of drawings

Fig. 1 is the overall realization block diagram of the method of the present invention;

Fig. 2 is the composition structure schematic diagram of the convolutional neural network created by the method of the present invention;

Fig. 3a is a selected road scene image to be semantically segmented;

Fig. 3b is the real semantic segmentation image corresponding to the road scene image to be semantically segmented shown in Fig. 3a;

Figure 3c is a predicted semantic segmentation image obtained by using the method of the present invention to predict the road scene image to be semantically segmented as shown in Figure 3a;

Fig. 4a is another selected road scene image to be semantically segmented;

Fig. 4b is the real semantic segmentation image corresponding to the road scene image to be semantically segmented shown in Fig. 4a;

Fig. 4c is a predicted semantic segmentation image obtained by using the method of the present invention to predict the road scene image to be semantically segmented as shown in Fig. 4a.

Detailed ways

The present invention will be further described in detail below with reference to the embodiments of the accompanying drawings.

The overall implementation block diagram of a road scene segmentation method based on residual network and dilated convolution proposed by the present invention is shown in Figure 1, which includes two processes: a training phase and a testing phase.

The specific steps of the training phase process are:

Step 1_1: Select Q original road scene images and the real semantic segmentation images corresponding to each original road scene image, and form a training set, and mark the qth original road scene image in the training set as {I ^q (i ,j)}, denote the real semantic segmentation images corresponding to {I ^q (i, j)} in the training set as Then, the existing one-hot encoding technology (one-hot) is used to process the real semantic segmentation images corresponding to each original road scene image in the training set into 12 one-hot encoded images. The set of 12 processed one-hot encoded images is denoted as Among them, the road scene image is an RGB color image, Q is a positive integer, Q≥100, if Q=100, q is a positive integer, 1≤q≤Q, 1≤i≤W, 1≤j≤H, W means The width of {I ^q (i, j)}, H represents the height of {I ^q (i, j)}, such as W=352, H=480, I ^q (i, j) represents {I ^q (i, j) j)} The pixel value of the pixel whose coordinate position is (i, j), express The pixel value of the pixel point whose middle coordinate position is (i, j); here, the original road scene image directly selects 100 images in the training set of the road scene image database CamVid.

Step 1_2: Build a convolutional neural network: As shown in Figure 2, the convolutional neural network includes an input layer, a hidden layer and an output layer; the hidden layer consists of 10 Residual blocks (residual network blocks) set in sequence, among which, the first Each convolutional layer in one Residual block forms a dilated convolutional layer by setting the dilation rate to 1, and each convolutional layer in the second Residual block forms a dilated convolutional layer by setting the dilation rate to 1, Each convolutional layer in the third Residual block forms a dilated convolutional layer by setting a dilation rate of 2, each convolutional layer in the fourth Residual block forms a dilated convolutional layer by setting a dilation rate of 2, and the fifth Each convolutional layer in the Residual block forms a dilated convolutional layer by setting the dilation rate to 4, and each convolutional layer in the sixth Residual block forms a dilated convolutional layer by setting the dilation rate to 4. The seventh Residual block forms a dilated convolutional layer. Each convolutional layer in the convolutional layer forms a dilated convolutional layer by setting the dilation rate to 2, and each convolutional layer in the 8th Residual block forms a dilated convolutional layer by setting the dilation rate to 2. In the ninth Residual block, the Each convolutional layer forms a dilated convolutional layer by setting the dilation rate to 1, each convolutional layer in the 10th Residualblock forms a dilated convolutional layer by setting the dilation rate to 1, and the dilated convolutional layer in the 10 Residual blocks The size of the convolution kernel remains the same as 3 × 3.

For the input layer, the input end of the input layer receives the R channel component, G channel component and B channel component of an original input image, and the output end of the input layer outputs the R channel component, G channel component and B channel component of the original input image to Hidden layer; wherein, the width of the original input image received by the input end of the input layer is required to be W and the height is H.

For the first Residual block, the input end of the first Residual block receives the R channel component, G channel component and B channel component of the original input image output by the output end of the input layer, and the output end of the first Residual block outputs 32 images For feature maps, the set of 32 feature maps is denoted as R ₁ ; wherein, the width of each feature map in R ₁ is W and the height is H.

For the second Residual block, the input of the second Residual block receives all the feature maps in R ₁ , the output of the second Residual block outputs 32 feature maps, and the set of 32 feature maps is recorded as R ₂ ; where, the width of each feature map in R ₂ is W and the height is H.

For the third Residual block, the input of the third Residual block receives all the feature maps in R ₂ , the output of the third Residual block outputs 64 feature maps, and the set of 64 feature maps is recorded as R ₃ ; where, the width of each feature map in R ₃ is W and the height is H.

For the 4th Residual block, the input of the 4th Residual block receives all the feature maps in _R3 , the output of the _4th Residual block outputs 64 feature maps, and the set of 64 feature maps is recorded as R4 ; where, the width of each feature map in R ₄ is W and the height is H.

For the 5th Residual block, the input of the 5th Residual block receives all the feature maps in R ₄ , the output of the 5th Residual block outputs 128 feature maps, and the set of 128 feature maps is recorded as R ₅ ; where, the width of each feature map in R ₅ is W and the height is H.

For the 6th Residual block, the input of the 6th Residual block receives all the feature maps in R ₅ , the output of the 6th Residual block outputs 128 feature maps, and the set of 128 feature maps is recorded as R ₆ ; where, the width of each feature map in R ₆ is W and the height is H.

For the 7th Residual block, the input of the 7th Residual block receives all the feature maps in R ₆ , the output of the 7th Residual block outputs 64 feature maps, and the set of 64 feature maps is recorded as R ₇ ; wherein, the width of each feature map in R ₇ is W and the height is H.

For the 8th Residual block, the input of the 8th Residual block receives all the feature maps in R ₇ , the output of the 8th Residual block outputs 64 feature maps, and the set of 64 feature maps is recorded as R ₈ ; where, the width of each feature map in R ₈ is W and the height is H.

For the ninth Residual block, the input of the ninth Residual block receives all the feature maps in R ₈ , the output of the ninth Residual block outputs 32 feature maps, and the set of 32 feature maps is recorded as R ₉ ; where, the width of each feature map in R ₉ is W and the height is H.

For the 10th Residual block, the input of the 10th Residual block receives all the feature maps in R ₉ , the output of the 10th Residual block outputs 32 feature maps, and the set of 32 feature maps is recorded as R ₁₀ ; where, the width of each feature map in R ₁₀ is W and the height is H.

For the output layer, it consists of 1 convolutional layer, the input of the output layer receives all the feature maps in R ₁₀ , and the output of the output layer outputs 12 semantic segmentation prediction maps corresponding to the original input image; The width of the semantic segmentation prediction map is W and the height is H.

Step 1_3: Take each original road scene image in the training set as the original input image, input it into the convolutional neural network for training, and obtain 12 semantic segmentation prediction maps corresponding to each original road scene image in the training set. The set of 12 semantic segmentation prediction maps corresponding to {I ^q (i, j)} is denoted as

Step 1_4: Calculate the loss function value between the set of 12 semantic segmentation prediction images corresponding to each original road scene image in the training set and the set of 12 one-hot encoded images processed from the corresponding real semantic segmentation image. ,Will and The loss function value between is denoted as Obtained using categorical crossentropy.

Step 1_5: Repeat steps 1_3 and 1_4 for a total of V times to obtain a convolutional neural network classification training model, and obtain a total of Q×V loss function values; then find the loss with the smallest value from the Q×V loss function values. function value; then the weight vector and bias term corresponding to the loss function value with the smallest value are corresponding to the optimal weight vector and optimal bias term of the convolutional neural network classification training model, which are correspondingly recorded as W ^best and b ^best ; Wherein, V>1, in this embodiment, take V=300.

The specific steps of the test phase process are:

Step 2_1: Make Indicates the road scene image to be semantically segmented; where 1≤i'≤W', 1≤j'≤H', W' denotes width, H' means the height of, express The pixel value of the pixel whose middle coordinate position is (i, j).

Step 2_2: Put the The R channel component, G channel component and B channel component are input into the convolutional neural network classification training model, and use W ^best and b ^best to predict, get The corresponding predicted semantic segmentation image, denoted as in, express The pixel value of the pixel whose middle coordinate position is (i', j').

In order to further verify the feasibility and effectiveness of the method of the present invention, experiments were carried out.

Use the python-based deep learning library Keras2.1.5 to build the architecture of the convolutional neural network. The road scene image database CamVid test set is used to analyze the segmentation effect of the road scene image predicted by the method of the present invention. Here, three common objective parameters for evaluating semantic segmentation methods are used as evaluation indicators, namely Pixel Accuracy (PA), Mean Pixel Accuracy (MPA), and Mean Intersectionover Union (MIoU) to evaluate Predicting segmentation performance for semantically segmented images.

The method of the present invention is used to predict each road scene image in the test set of the road scene image database CamVid, and the predicted semantic segmentation image corresponding to each road scene image is obtained, and the pixel accuracy PA and the average pixel accuracy of the semantic segmentation effect of the method of the present invention are reflected. MPA, mean intersection ratio MIoU are listed in Table 1, the higher the value of pixel precision PA, mean pixel precision MPA, and mean intersection ratio MIoU, the higher the validity and prediction accuracy. From the data listed in Table 1, it can be seen that the segmentation result of the road scene image obtained by the method of the present invention is good, indicating that the method of the present invention is feasible and effective to obtain the predicted semantic segmentation image corresponding to the road scene image.

Table 1. Evaluation results on the test set using the method of the present invention

Fig. 3a shows a selected road scene image to be semantically segmented; Fig. 3b shows the real semantic segmentation image corresponding to the road scene image to be semantically segmented shown in Fig. 3a; Fig. 3c shows the use of the method of the present invention Predict the road scene image to be semantically segmented as shown in Fig. 3a, and obtain the predicted semantic segmentation image; Fig. 4a shows another selected road scene image to be semantically segmented; Fig. 4b shows the image shown in Fig. 4a Figure 4c shows the predicted semantic segmentation image obtained by predicting the road scene image to be semantically segmented as shown in Figure 4a by using the method of the present invention. Comparing Fig. 3b and Fig. 3c, and Fig. 4b and Fig. 4c, it can be seen that the segmentation accuracy of the predicted semantic segmentation image obtained by the method of the present invention is high, which is close to the real semantic segmentation image.

Claims (2)

1. A road scene segmentation method based on residual error network and expansion convolution is characterized by comprising a training stage and a testing stage;

the specific steps of the training phase process are as follows:

step 1_ 1: selecting Q original road scene images and real semantic segmentation images corresponding to each original road scene image, forming a training set, and recording the Q-th original road scene image in the training set as { I }^q(I, j) }, the training set is summed with { I }^q(i, j) } corresponding real semantic segmentation imageIs marked asThen, processing the real semantic segmentation image corresponding to each original road scene image in the training set into 12 single-hot coded images by adopting a single-hot coding technology, and processing the single-hot coded imagesThe processed set of 12 one-hot coded images is denoted asThe road scene image is an RGB color image, Q is a positive integer, Q is more than or equal to 100, Q is a positive integer, Q is more than or equal to 1 and less than or equal to Q, I is more than or equal to 1 and less than or equal to W, j is more than or equal to 1 and less than or equal to H, and W represents { I ≦ I^q(I, j) }, H denotes { I }^qHeight of (I, j) }, I^q(I, j) represents { I^qThe pixel value of the pixel point with the coordinate position (i, j) in (i, j),to representThe middle coordinate position is the pixel value of the pixel point of (i, j);

step 1_ 2: constructing a convolutional neural network: the convolutional neural network comprises an input layer, a hidden layer and an output layer; the hidden layers are composed of 10 sequentially arranged Residual blocks, wherein each convolution layer in the 1 st Residual block forms an expanded convolution layer by setting the expansion ratio to 1, each convolution layer in the 2 nd Residual block forms an expanded convolution layer by setting the expansion ratio to 1, each convolution layer in the 3 rd Residual block forms an expanded convolution layer by setting the expansion ratio to 2, each convolution layer in the 4 th Residual block forms an expanded convolution layer by setting the expansion ratio to 2, each convolution layer in the 5 th Residual block forms an expanded convolution layer by setting the expansion ratio to 4, each convolution layer in the 6 th Residual block forms an expanded convolution layer by setting the expansion ratio to 4, each convolution layer in the 7 th Residual block forms an expanded convolution layer by setting the expansion ratio to 2, each convolution layer in the 8 th Residual block forms an expanded convolution layer by setting the expansion ratio to 2, each convolution layer in the 9 th Residual block forms an expanded convolution layer by setting the expansion rate to be 1, and each convolution layer in the 10 th Residual block forms an expanded convolution layer by setting the expansion rate to be 1;

for an input layer, the input end of the input layer receives an R channel component, a G channel component and a B channel component of an original input image, and the output end of the input layer outputs the R channel component, the G channel component and the B channel component of the original input image to a hidden layer; wherein, the width of the original input image received by the input end of the input layer is required to be W, and the height of the original input image is required to be H;

for the 1 st Residual block, the input end of the 1 st Residual block receives the R channel component, the G channel component and the B channel component of the original input image output by the output end of the input layer, the output end of the 1 st Residual block outputs 32 characteristic graphs, and the set formed by the 32 characteristic graphs is recorded as R₁(ii) a Wherein R is₁Each feature map in (1) has a width W and a height H;

for the 2 nd Residual block, the input terminal of the 2 nd Residual block receives R₁The output end of the 2 nd Residual block outputs 32 characteristic graphs, and the set formed by the 32 characteristic graphs is marked as R₂(ii) a Wherein R is₂Each feature map in (1) has a width W and a height H;

for the 3 rd Residual block, the input terminal of the 3 rd Residual block receives R₂The output end of the 3 rd Residual block outputs 64 characteristic graphs, and the set of the 64 characteristic graphs is marked as R₃(ii) a Wherein R is₃Each feature map in (1) has a width W and a height H;

for the 4 th Residual block, the input terminal of the 4 th Residual block receives R₃The output end of the 4 th Residual block outputs 64 characteristic graphs, and the set of the 64 characteristic graphs is marked as R₄(ii) a Wherein R is₄Each feature map in (1) has a width W and a height H;

for the 5 th Residual block, the input terminal of the 5 th Residual block receives R₄The output end of the 5 th Residual block outputs 128 characteristic graphs, and the set of the 128 characteristic graphs is marked as R₅(ii) a Wherein R is₅Each feature map in (1) has a width W and a height H;

for the 6 th Residual block, the input terminal of the 6 th Residual block receives R₅The output end of the 6 th Residual block outputs 128 characteristic graphs, and the set of the 128 characteristic graphs is marked as R₆(ii) a Wherein R is₆Each feature map in (1) has a width W and a height H;

for the 7 th Residual block, the input terminal of the 7 th Residual block receives R₆The output end of the 7 th Residual block outputs 64 characteristic graphs, and the set of the 64 characteristic graphs is marked as R₇(ii) a Wherein R is₇Each feature map in (1) has a width W and a height H;

for the 8 th Residual block, the input terminal of the 8 th Residual block receives R₇The output end of the 8 th Residual block outputs 64 characteristic graphs, and the set of the 64 characteristic graphs is marked as R₈(ii) a Wherein R is₈Each feature map in (1) has a width W and a height H;

for the 9 th Residual block, the input terminal of the 9 th Residual block receives R₈The output end of the 9 th Residual block outputs 32 characteristic graphs, and the set formed by the 32 characteristic graphs is marked as R₉(ii) a Wherein R is₉Each feature map in (1) has a width W and a height H;

for the 10 th Residual block, the input terminal of the 10 th Residual block receives R₉The 10 th Residual block outputs 32 characteristic graphs, and the set of the 32 characteristic graphs is marked as R₁₀(ii) a Wherein R is₁₀Each feature map in (1) has a width W and a height H;

for the output layer, which consists of 1 convolutional layer, the input of the output layer receives R₁₀All characteristic diagrams in (1)The output end of the output layer outputs 12 semantic segmentation prediction graphs corresponding to the original input image; wherein the width of each semantic segmentation prediction graph is W, and the height of each semantic segmentation prediction graph is H;

step 1_ 3: taking each original road scene image in the training set as an original input image, inputting the original input image into a convolutional neural network for training to obtain 12 semantic segmentation prediction graphs corresponding to each original road scene image in the training set, and performing semantic segmentation on the { I } graph^q(i, j) } the set of 12 semantic segmentation prediction graphs is recorded as

Step 1_ 4: calculating loss function values between a set formed by 12 semantic segmentation prediction images corresponding to each original road scene image in the training set and a set formed by 12 single-hot coded images processed by corresponding real semantic segmentation images, and converting the loss function values into the loss function valuesAndthe value of the loss function in between is recorded as

Step 1_ 5: repeatedly executing the step 1_3 and the step 1_4 for V times to obtain a convolutional neural network classification training model, and obtaining Q multiplied by V loss function values; then finding out the loss function value with the minimum value from the Q multiplied by V loss function values; and then, correspondingly taking the weight vector and the bias item corresponding to the loss function value with the minimum value as the optimal weight vector and the optimal bias item of the convolutional neural network classification training model, and correspondingly marking as W^bestAnd b^best(ii) a Wherein V is greater than 1;

the test stage process comprises the following specific steps:

step 2_ 1: order toRepresenting a road scene image to be semantically segmented; wherein, i ' is more than or equal to 1 and less than or equal to W ', j ' is more than or equal to 1 and less than or equal to H ', and W ' representsWidth of (A), H' representsThe height of (a) of (b),to representThe middle coordinate position is the pixel value of the pixel point of (i, j);

step 2_ 2: will be provided withThe R channel component, the G channel component and the B channel component are input into a convolutional neural network classification training model and are subjected to W-based classification^bestAnd b^bestMaking a prediction to obtainCorresponding predictive semantic segmentation image, denotedWherein,to representAnd the pixel value of the pixel point with the middle coordinate position of (i ', j').

2. The method of claim 1, wherein the road scene segmentation method is based on residual error network and dilation convolutionCharacterized in that in the step 1-4,and obtaining by adopting classification cross entropy.