CN111242955A - Road surface crack image segmentation method based on full convolution neural network - Google Patents
Road surface crack image segmentation method based on full convolution neural network Download PDFInfo
- Publication number
- CN111242955A CN111242955A CN202010070397.7A CN202010070397A CN111242955A CN 111242955 A CN111242955 A CN 111242955A CN 202010070397 A CN202010070397 A CN 202010070397A CN 111242955 A CN111242955 A CN 111242955A
- Authority
- CN
- China
- Prior art keywords
- neural network
- convolution neural
- full convolution
- image segmentation
- steps
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a road surface crack image segmentation method based on a full convolution neural network, which is characterized by comprising the following steps of: the method comprises the following steps: the method comprises the following steps that a camera shoots a pavement crack, a picture data set is established, and images in the data set are divided into two types, namely cracked images and non-cracked images; utilizing Lableme software to label the segmented data of the image with the crack in the data set; dividing the constructed data set into a training set and a test set; constructing a full convolution neural network for image segmentation; training the full convolution neural network by using the divided training set, and optimizing related parameters until a global optimal solution is obtained; and submitting the test concentrated crack image to a full convolution neural network, and outputting an image segmentation result.
Description
Technical Field
The invention belongs to the field of road quality and safety monitoring, and discloses a road surface crack image segmentation method based on a full convolution neural network.
Background
The detection of the pavement cracks is an important traffic maintenance work for ensuring the driving safety, is a crucial step for road management, and aims to obtain pavement maintenance state information. Cracks are the most common type of damage to pavements. Because the automatic crack detection system has the advantages of high safety, low cost, high efficiency, objectivity and the like, the research of the automatic crack detection system is widely concerned.
Image-based crack detection algorithms have been widely discussed and applied over the past few decades. In earlier studies, methods were more based on traditional digital combination or improved image processing techniques such as thresholding, mathematical morphology, and edge detection. These methods are generally based on photometric and geometric assumptions about the properties of the fracture image. However, these methods are very sensitive to noise because they are performed on a single pixel. This remains a challenging task due to crack non-uniformity and background complexity, such as low contrast to the surrounding pavement, and the potential for near-intensity shadows.
In recent years, deep Convolutional Neural Networks (CNNs) have found widespread use in computer image processing.
Deep neural networks have impressive performance in many computer vision tasks, demonstrating the effectiveness of learned deep features. A depth Full Convolution Network (FCN) model is constructed, semantic segmentation is carried out on a pavement crack image, automatic detection of pavement cracks is achieved, and the method has important significance for monitoring pavement conditions.
Disclosure of Invention
The purpose of the invention is as follows: aiming at the defects of the prior art, the invention provides a road surface crack image segmentation method based on a full convolution neural network, so as to realize crack detection, further improve crack detection efficiency and reduce detection cost.
The technical scheme is as follows: the invention provides a road surface crack image segmentation method based on a full convolution neural network, which comprises the following steps:
the method comprises the following steps that an S1 camera photographs a pavement crack, a picture data set is established, and images in the data set are divided into two types, namely crack-carrying images and crack-free images;
s2, utilizing Lableme software to label the segmentation data of the image with the crack in the data set;
s3 dividing the constructed data set into a training set (3/5 of the data set) and a testing set (2/5 of the data set);
s4, constructing a full convolution neural network for image segmentation;
s5, training the full convolution neural network by using the divided training set, and optimizing related parameters until a global optimal solution is obtained;
and S6, submitting the test concentrated crack image to a full convolution neural network, and outputting an image segmentation result.
Step S2 labels the crack image, and the label is a pixel-level label.
The constructed full convolution neural network specifically comprises the following steps:
the full convolution neural network comprises 1 convolution layer, 16 expansion convolution layers, 1 pyramid sampling layer, 2 decoder layers and 1 semantic output layer.
Step S4 is a full convolution neural network model structure and parameters specifically:
the structure of the full convolution neural network expansion module is specifically as follows:
a. span is 1: the input picture is overlapped with the input through the results of point convolution 1x1 (the activation function is Relu6), depth convolution 3x3 (the activation function is Relu6) and point convolution 1x1 (the activation function is linear) to obtain the output.
b. Span is 2 hours: the input picture is directly output after being subjected to point convolution 1x1 (the activation function is Relu6), depth convolution 3x3 (the activation function is Relu6) and point convolution 1x1 (the activation function is linear).
The full convolution neural network pyramid sampling module has 4 ways, namely sampling the average value with the side length of 10 and averaging the whole characteristic diagram; convolving three cavities with the cavity ratios of 1, 2 and 4 respectively, and extracting information from the characteristic diagram by using different receptive fields; the output results of the 4 paths are scaled to the same size and then combined in a stacked manner.
In the full convolution neural network, a decoder 0 is connected with an extended convolution 3#, 16 characteristic graphs are input, and 32 characteristic graphs are output; the decoder 1 is connected to the convolutional layer, and inputs 8 feature maps and outputs 32 feature maps.
The full convolution neural network parameter optimization uses a Moment optimizer, the momentum value of the optimizer is 0.9, the initial learning rate is 0.0001, a descending learning rate updating strategy is adopted, and the attenuation is 0.9 times of the learning rate of the previous step after every 2000 steps; during training, two pictures are taken to calculate the gradient of the error pair network parameters, and the network parameters are further updated.
The target function of the full convolution neural network for image segmentation is the average of the cross entropy of a given label and the prediction result prediction of the neural network at each pixel:
has the advantages that: compared with the prior art, the road surface crack image segmentation method based on the full convolution neural network has the following beneficial effects:
1. the recognition efficiency is high: the method comprises the steps of establishing a picture data set, and dividing images in the data set into two types, namely a cracked type and a non-cracked type; utilizing Lableme software to label the segmented data of the image with the crack in the data set; dividing the constructed data set into a training set and a test set; constructing a full convolution neural network for image segmentation; the full convolution neural network is trained by utilizing the divided training set, relevant parameters are optimized until a global optimal solution is obtained, and the picture recognition efficiency is effectively improved.
2. The crack detection model is suitable for the detection task of the road surface image of the same material after training is completed, and has wide applicability.
3. According to the crack identification method, the detection vehicle with the camera is used for driving on a road to obtain an original image, and then the crack condition is evaluated through neural network processing, so that the requirements on manpower, material resources and financial resources are reduced.
Drawings
FIG. 1 is an overall flow chart designed based on the method of the present invention;
FIG. 2 is a diagram of a full convolutional neural network architecture in accordance with the present invention;
FIG. 3 shows the segmentation result of the road surface crack image by the method of the present invention.
Detailed Description
The present invention is further illustrated by the following figures and specific examples, which are to be understood as illustrative only and not as limiting the scope of the invention, which is to be given the full breadth of the appended claims and any and all equivalent modifications thereof which may occur to those skilled in the art upon reading the present specification.
In the present embodiment, the road surface crack image is subjected to image segmentation to detect cracks. Firstly, preparing a data set, photographing a campus road by using a camera, acquiring an original crack image, and establishing the data set. And then, cutting the image acquired by the camera on a computer, wherein the size of the cut image is 320X320, and after the data set is enhanced, the data set comprises 6000 pictures. Wherein, the number of the images with cracks and without cracks is 3000, and the number of the images for model training and testing is 1800 images and 1200 images respectively. And then, carrying out segmentation data annotation on the image with the crack in the data set by using Lableme software. The construction and training work of the model is completed under a Tensorflow framework, the batch size is all set to be 2, a Moment optimizer is used for optimizing a cost function, the initial learning rate is set to be 0.0001, a descending learning rate updating strategy is adopted, and the attenuation is 0.9 times of the learning rate of the previous step after every 2000 steps. During training, two pictures are taken to calculate the gradient of the error to the network parameters, so that the network parameters are updated. The neural network input image size is unified into a picture of 320 × 320 size. The fracture image segmentation results are shown in fig. 3.
The experimental platform environment is configured as follows:
intel (R) core (TM) i7CPU @3.20GHz, NVIDIA GTX1080Ti video card, 16G DDR4 memory (main frequency 2400MHz), 11G video memory (frequency 1.4 GHz). The whole training process is completed on the GPU in an accelerated mode.
Referring to fig. 1, the method for segmenting the road surface crack image comprises the following specific steps:
s1, photographing the pavement cracks by using a camera, establishing a data set, and dividing images in the data set into two types, namely cracks and no cracks;
s2, utilizing Lableme software to label the segmented data of the image with the crack in the data set;
s3, dividing the constructed data set into a training set (3/5 of the data set) and a testing set (2/5 of the data set);
s4, constructing a full convolution neural network for image segmentation;
the full convolution neural network consists of 1 convolution layer, 16 expansion convolution layers, 1 pyramid sampling layer, 2 decoder layers and 1 semantic output layer.
The structure and parameters of the full product neural network model in step S4 are as follows:
s5, training the full convolution neural network by using the divided training set, and optimizing related parameters until a global optimal solution is obtained;
s6, submitting the test concentrated crack image to a full convolution neural network, and outputting an image segmentation result;
the structure of the full convolution neural network expansion module is specifically as follows:
a. span is 1: the input picture is overlapped with the input through the results of point convolution 1x1 (the activation function is Relu6), depth convolution 3x3 (the activation function is Relu6) and point convolution 1x1 (the activation function is linear) to obtain the output.
b. Span is 2 hours: the input picture is directly output after being subjected to point convolution 1x1 (the activation function is Relu6), depth convolution 3x3 (the activation function is Relu6) and point convolution 1x1 (the activation function is linear).
The full convolution neural network pyramid sampling module has 4 ways, namely sampling the average value with the side length of 10 and averaging the whole characteristic diagram; three hole convolutions with hole ratios 1, 2, 4, respectively, are used to extract information from the feature map using different receptive fields. The output results of the 4 paths are scaled to the same size and then combined in a stacked manner.
In the full convolution neural network, a decoder 0 is connected with an extended convolution 3#, 16 characteristic graphs are input, and 32 characteristic graphs are output; the decoder 1 is connected to the convolutional layer, and inputs 8 feature maps and outputs 32 feature maps.
The full convolution neural network parameter optimization uses a Moment optimizer, the momentum value of the optimizer is 0.9, the initial learning rate is 0.0001, a descending learning rate updating strategy is adopted, and the attenuation is 0.9 times of the learning rate of the previous step after every 2000 steps. During training, two pictures are taken to calculate the gradient of the error to the network parameters, so that the network parameters are updated.
The target function of the full convolution neural network for image segmentation is the average of the cross entropy of a given label and the prediction result prediction of the neural network at each pixel:
it should be noted that, for those skilled in the art, without departing from the principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be construed as the protection scope of the present invention. All the components not specified in the present embodiment can be realized by the prior art.
Claims (9)
1. A road surface crack image segmentation method based on a full convolution neural network is characterized by comprising the following steps: the method comprises the following steps:
the method comprises the following steps that an S1 camera photographs a pavement crack, a picture data set is established, and images in the data set are divided into two types, namely crack-carrying images and crack-free images;
s2, utilizing Lableme software to label the segmentation data of the image with the crack in the data set;
s3 dividing the constructed data set into a training set and a testing set;
s4, constructing a full convolution neural network for image segmentation;
s5, training the full convolution neural network by using the divided training set, and optimizing related parameters until a global optimal solution is obtained;
and S6, submitting the test concentrated crack image to a full convolution neural network, and outputting an image segmentation result.
2. The road surface crack image segmentation method based on the full convolution neural network as claimed in claim 1, wherein the image segmentation method comprises the following steps: step S2 labels the crack image, and the label is a pixel-level label.
3. The road surface crack image segmentation method based on the full convolution neural network as claimed in claim 1, characterized in that: the constructed full convolution neural network specifically comprises the following steps:
the full convolution neural network comprises 1 convolution layer, 16 expansion convolution layers, 1 pyramid sampling layer, 2 decoder layers and 1 semantic output layer.
4. The road surface crack image segmentation method based on the full convolution neural network as claimed in claim 1 or 3, wherein the image segmentation method comprises the following steps:
step S4 is a full convolution neural network model structure and parameters specifically:
5. the road surface crack image segmentation method based on the full convolution neural network as claimed in claim 4, wherein the method comprises the following steps: the structure of the full convolution neural network expansion module is specifically as follows:
a. span is 1: the input picture is overlapped with the input through the results of point convolution 1x1 (the activation function is Relu6), depth convolution 3x3 (the activation function is Relu6) and point convolution 1x1 (the activation function is linear) to obtain the output.
b. Span is 2 hours: the input picture is directly output after being subjected to point convolution 1x1 (the activation function is Relu6), depth convolution 3x3 (the activation function is Relu6) and point convolution 1x1 (the activation function is linear).
6. The road surface crack image segmentation method based on the full convolution neural network as claimed in claim 1, wherein the method comprises the following steps: the full convolution neural network pyramid sampling module has 4 ways, namely sampling the average value with the side length of 10 and averaging the whole characteristic diagram; convolving three cavities with the cavity ratios of 1, 2 and 4 respectively, and extracting information from the characteristic diagram by using different receptive fields; the output results of the 4 paths are scaled to the same size and then combined in a stacked manner.
7. The road surface crack image segmentation method based on the full convolution neural network as claimed in claim 1, wherein the method comprises the following steps: in the full convolution neural network, a decoder 0 is connected with an extended convolution 3#, 16 characteristic graphs are input, and 32 characteristic graphs are output; the decoder 1 is connected to the convolutional layer, and inputs 8 feature maps and outputs 32 feature maps.
8. The road surface crack image segmentation method based on the full convolution neural network as claimed in claim 1, wherein the method comprises the following steps: the full convolution neural network parameter optimization uses a Moment optimizer, the momentum value of the optimizer is 0.9, the initial learning rate is 0.0001, a descending learning rate updating strategy is adopted, and the attenuation is 0.9 times of the learning rate of the previous step after every 2000 steps; during training, two pictures are taken to calculate the gradient of the error pair network parameters, and the network parameters are further updated.
9. The road surface crack image segmentation method based on the full convolution neural network as claimed in claim 1, wherein the method comprises the following steps: the target function of the full convolution neural network for image segmentation is the average of the cross entropy of a given label and the prediction result prediction of the neural network at each pixel:
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010070397.7A CN111242955B (en) | 2020-01-21 | 2020-01-21 | Road surface crack image segmentation method based on full convolution neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010070397.7A CN111242955B (en) | 2020-01-21 | 2020-01-21 | Road surface crack image segmentation method based on full convolution neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111242955A true CN111242955A (en) | 2020-06-05 |
| CN111242955B CN111242955B (en) | 2023-04-21 |
Family
ID=70872858
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010070397.7A Active CN111242955B (en) | 2020-01-21 | 2020-01-21 | Road surface crack image segmentation method based on full convolution neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111242955B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111539494A (en) * | 2020-07-08 | 2020-08-14 | 浙江浙能天然气运行有限公司 | Hydraulic protection damage detection method based on U-Net and SVM |
| CN112233105A (en) * | 2020-10-27 | 2021-01-15 | 江苏科博空间信息科技有限公司 | Road crack detection method based on improved FCN |
| CN113486721A (en) * | 2021-06-09 | 2021-10-08 | 北京科技大学 | Pavement crack detection method based on rolling optimization crack classification recognition model |
| CN114331869A (en) * | 2021-12-01 | 2022-04-12 | 西南科技大学 | Dam face crack semantic segmentation method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107766794A (en) * | 2017-09-22 | 2018-03-06 | 天津大学 | The image, semantic dividing method that a kind of Fusion Features coefficient can learn |
| CN109360206A (en) * | 2018-09-08 | 2019-02-19 | 华中农业大学 | Crop field spike of rice dividing method based on deep learning |
| US20190080456A1 (en) * | 2017-09-12 | 2019-03-14 | Shenzhen Keya Medical Technology Corporation | Method and system for performing segmentation of image having a sparsely distributed object |
| CN110502972A (en) * | 2019-07-05 | 2019-11-26 | 广东工业大学 | A kind of pavement crack segmentation and recognition methods based on deep learning |
| CN110533629A (en) * | 2019-07-10 | 2019-12-03 | 湖南交工智能技术有限公司 | A kind of detection method and detection device of Bridge Crack |
-
2020
- 2020-01-21 CN CN202010070397.7A patent/CN111242955B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190080456A1 (en) * | 2017-09-12 | 2019-03-14 | Shenzhen Keya Medical Technology Corporation | Method and system for performing segmentation of image having a sparsely distributed object |
| CN107766794A (en) * | 2017-09-22 | 2018-03-06 | 天津大学 | The image, semantic dividing method that a kind of Fusion Features coefficient can learn |
| CN109360206A (en) * | 2018-09-08 | 2019-02-19 | 华中农业大学 | Crop field spike of rice dividing method based on deep learning |
| CN110502972A (en) * | 2019-07-05 | 2019-11-26 | 广东工业大学 | A kind of pavement crack segmentation and recognition methods based on deep learning |
| CN110533629A (en) * | 2019-07-10 | 2019-12-03 | 湖南交工智能技术有限公司 | A kind of detection method and detection device of Bridge Crack |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111539494A (en) * | 2020-07-08 | 2020-08-14 | 浙江浙能天然气运行有限公司 | Hydraulic protection damage detection method based on U-Net and SVM |
| CN112233105A (en) * | 2020-10-27 | 2021-01-15 | 江苏科博空间信息科技有限公司 | Road crack detection method based on improved FCN |
| CN113486721A (en) * | 2021-06-09 | 2021-10-08 | 北京科技大学 | Pavement crack detection method based on rolling optimization crack classification recognition model |
| CN113486721B (en) * | 2021-06-09 | 2023-08-29 | 北京科技大学 | Pavement crack detection method based on rolling optimization crack classification recognition model |
| CN114331869A (en) * | 2021-12-01 | 2022-04-12 | 西南科技大学 | Dam face crack semantic segmentation method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111242955B (en) | 2023-04-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111242955B (en) | Road surface crack image segmentation method based on full convolution neural network | |
| CN111062892B (en) | Single image rain removing method based on composite residual error network and deep supervision | |
| Duan et al. | Combining Hough transform and contour algorithm for detecting vehicles' license-plates | |
| CN111767882A (en) | Multi-mode pedestrian detection method based on improved YOLO model | |
| CN113052210B (en) | Rapid low-light target detection method based on convolutional neural network | |
| CN109345547B (en) | Traffic lane line detection method and device based on deep learning multitask network | |
| CN109583345B (en) | Road recognition method, device, computer device and computer readable storage medium | |
| CN112884064A (en) | Target detection and identification method based on neural network | |
| CN104048969A (en) | Tunnel defect recognition method | |
| WO2021238420A1 (en) | Image defogging method, terminal, and computer storage medium | |
| CN112232351B (en) | License plate recognition system based on deep neural network | |
| CN108985343A (en) | Automobile damage detecting method and system based on deep neural network | |
| CN104599256A (en) | Single-image based image rain streak eliminating method and system | |
| CN112529005B (en) | Target detection method based on semantic feature consistency supervision pyramid network | |
| CN113723377A (en) | Traffic sign detection method based on LD-SSD network | |
| CN115272204A (en) | Bearing surface scratch detection method based on machine vision | |
| CN115239644A (en) | Concrete defect identification method and device, computer equipment and storage medium | |
| CN116681636A (en) | Light infrared and visible light image fusion method based on convolutional neural network | |
| CN104462550A (en) | Pedestrian re-recognition method based on similarity and dissimilarity fusion ranking optimization | |
| CN116189162A (en) | Ship plate detection and identification method and device, electronic equipment and storage medium | |
| CN116597270A (en) | Road damage target detection method based on attention mechanism integrated learning network | |
| CN112330618B (en) | Image offset detection method, device and storage medium | |
| CN111079585B (en) | Pedestrian re-identification method combining image enhancement with pseudo-twin convolutional neural network | |
| CN111160282B (en) | Traffic light detection method based on binary Yolov3 network | |
| CN112686247A (en) | Identification card number detection method and device, readable storage medium and terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |