CN114708190A - Road crack detection and evaluation algorithm based on deep learning - Google Patents

Road crack detection and evaluation algorithm based on deep learning Download PDF

Info

Publication number
CN114708190A
CN114708190A CN202210201900.7A CN202210201900A CN114708190A CN 114708190 A CN114708190 A CN 114708190A CN 202210201900 A CN202210201900 A CN 202210201900A CN 114708190 A CN114708190 A CN 114708190A
Authority
CN
China
Prior art keywords
crack
funet
algorithm
model
road
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210201900.7A
Other languages
Chinese (zh)
Inventor
刘宇翔
佘维
谭帅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhengzhou University
Original Assignee
Zhengzhou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhengzhou University filed Critical Zhengzhou University
Priority to CN202210201900.7A priority Critical patent/CN114708190A/en
Publication of CN114708190A publication Critical patent/CN114708190A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • G06T7/0008Industrial image inspection checking presence/absence
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/084Backpropagation, e.g. using gradient descent
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10004Still image; Photographic image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30108Industrial image inspection
    • G06T2207/30132Masonry; Concrete

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a road crack detection and evaluation algorithm based on deep learning, which comprises the following steps: s1, training a neural network model FUnet; s2, road crack detection and evaluation are carried out by using the algorithm flow; compared with the prior art, the invention has the advantages that: the road crack is extracted and analyzed by utilizing the convolutional neural network model FUnet and a subsequent algorithm, a standard flow is provided, the problem of different standards caused by the difference of judgment between people in manual detection is avoided, the road crack is extracted and analyzed through an image, the equipment is simple to use, the cost is reduced, the algorithm flow can finish the extraction and analysis of the crack end to end, the classification task of a pixel level can be finished, meanwhile, the object level division and analysis can be carried out, the internal structure does not need to be manually concerned, and the requirement on operators is low.

Description

Road crack detection and evaluation algorithm based on deep learning
Technical Field
The invention relates to the technical field of crack identification, in particular to a road crack detection and evaluation algorithm based on deep learning.
Background
China has one of the largest road networks in the world, and a road is maintained after being laid, so that the service life of the road is greatly shortened and potential safety hazards are caused if the road is damaged and is not maintained in time. However, the roads have the characteristics of large number, long mileage, wide distribution and the like, and it is difficult to quickly and accurately count and evaluate the road damage conditions of different road sections. The traditional manual detection method is low in speed and high in labor cost, and can accurately detect and count the data by depending on related professional measuring equipment. The professional road surface detection vehicle has the problems of high manufacturing cost, large volume and need of a special person for operation due to the fact that the professional road surface detection vehicle is provided with a plurality of professional measuring devices.
Disclosure of Invention
The invention aims to solve the technical problem of providing a road crack detection statistical algorithm flow based on deep learning, which can extract cracks of a road surface picture and acquire related information so as to provide reference for road surface damage condition evaluation work.
In order to solve the technical problems, the technical scheme provided by the invention is as follows: a road crack detection and evaluation algorithm based on deep learning comprises the following steps:
s1, training a neural network model FUnet;
s11, acquiring a CrackForest data set of the public road crack data set;
s12, inputting mini-batch data into the FUnet model each time to obtain the output of the FUnet model, and calculating a loss value by using a loss function consisting of cross entropy and F1 fraction in combination with the labeling of a data set;
s13, updating the gradient of the FUnet model by using an error back propagation algorithm, performing gradient descent on the FUnet according to the calculated gradient by using a gradient descent algorithm, and updating the parameter of the FUnet model;
s14, repeatedly executing the step S12 and the step S13 until the final loss value is not reduced basically;
s15, storing the parameters of the trained FUnet model for subsequent use;
S2, using the algorithm flow to detect and evaluate the road crack;
s21, constructing a neural network model FUnet, importing the trained data, and adjusting the model into an evaluation mode without changing the values of parameters in the network;
s22, acquiring pictures from the camera or the video, and preprocessing the pictures, wherein the algorithm only needs to perform standardized processing on the pictures;
s23, sending the preprocessed pictures into a FUnet model, and obtaining output of the FUnet model, wherein an output Mask of the FUnet model is the classification of pixel points;
s24, inputting the output Mask into a dfs-based pavement crack target detection algorithm, and dividing examples of cracks;
s25, calculating total and individual data of cracks in subsequent algorithms according to crack example data obtained by a dfs-based pavement crack target detection algorithm, wherein the total and individual data comprise data such as picture crack density, crack individual distribution density and the like, and filtering crack examples with certain indexes not meeting the minimum requirement on the basis, so that noise interference on pictures can be effectively avoided, and only cracks exceeding normal indexes are concerned.
Compared with the prior art, the invention has the advantages that: the road crack is extracted and analyzed by utilizing the convolutional neural network model FUnet and a subsequent algorithm, a standard flow is provided, the problem of different standards caused by the difference of judgment between people in manual detection is avoided, the road crack is extracted and analyzed through an image, the equipment is simple to use, the cost is reduced, the algorithm flow can finish the extraction and analysis of the crack end to end, the classification task of a pixel level can be finished, meanwhile, the object level division and analysis can be carried out, the internal structure does not need to be manually concerned, and the requirement on operators is low.
Preferably, the acquiring of the data set in step S11 is performed by using an image processing method to perform augmentation of the data set, and these operations are performed on the original image and the annotation in the data set at the same time, and the specific operations include: turning over the image, cutting and splicing, locally distorting, adding Gaussian noise and randomly biasing the brightness.
Preferably, the calculation formula of step S12 is as follows:
Loss=CrossEntropyLoss+(1-F1) (1)
wherein
Figure BDA0003529681290000021
Figure BDA0003529681290000022
Figure BDA0003529681290000023
Figure BDA0003529681290000024
Preferably, step S24 may control whether to merge the discontinuous cracks into one crack instance by adjusting the search range in dfs, and may also adjust the merging distance, i.e. draw two discontinuous cracks into the same crack instance in this distance range. And obtaining a crack example after being processed by a dfs-based pavement crack target detection algorithm, wherein the crack example comprises the distribution range [ xmin, ymin, xmax, ymax ] of the crack example in a graph and the number of crack pixel points of the crack example.
Drawings
FIG. 1 is a flow chart of a training and detection process of a road crack detection evaluation algorithm based on deep learning;
FIG. 2 is a general structure diagram of a neural network model FUnet in a deep learning-based road crack detection and evaluation algorithm;
FIG. 3 is a detailed structure diagram of a neural network model FUnet in a deep learning-based road crack detection and evaluation algorithm;
FIG. 4 is a sample example picture of a road crack detection and evaluation algorithm based on deep learning;
fig. 5 is a pixel-level classified picture obtained by an algorithm, black being a background and white being a crack;
FIG. 6 is a crack example obtained by a dfs pavement crack target detection algorithm when the spacing distance is 0, wherein each frame corresponds to one crack example;
fig. 7 is a crack example obtained by a road surface crack target detection algorithm of dfs when the spacing distance is 7, and each frame corresponds to one crack example.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings.
When the invention is specifically implemented, the road crack detection and evaluation algorithm based on deep learning comprises the following steps:
s1, training a neural network model FUnet;
s11, acquiring a CrackForest data set of the public road crack data set;
s12, inputting mini-batch data into the FUnet model each time to obtain the output of the FUnet model, and calculating a loss value by using a loss function composed of cross entropy and F1 score in combination with the labeling of a data set;
s13, updating the gradient of the FUnet model by using an error back propagation algorithm, performing gradient descent on the FUnet according to the calculated gradient by using a gradient descent algorithm, and updating the parameter of the FUnet model;
S14, repeatedly executing the step S12 and the step S13 until the last loss value is basically not reduced;
s15, storing the parameters of the trained FUnet model for subsequent use;
s2, using the algorithm flow to detect and evaluate the road crack;
s21, constructing a neural network model FUnet, importing the trained data, and adjusting the model into an evaluation mode without changing the values of parameters in the network;
s22, acquiring pictures from the camera or the video, and preprocessing the pictures, wherein the algorithm only needs to perform standardized processing on the pictures;
s23, sending the preprocessed pictures into a FUnet model, and obtaining output of the FUnet model, wherein an output Mask of the FUnet model is the classification of pixel points;
s24, inputting the output Mask into a dfs-based pavement crack target detection algorithm, and dividing the crack into examples;
s25, calculating total and individual data of cracks in subsequent algorithms according to crack example data obtained by a dfs-based pavement crack target detection algorithm, wherein the total and individual data comprise data such as picture crack density, crack individual distribution density and the like, and filtering crack examples with certain indexes not meeting the minimum requirement on the basis, so that noise interference on pictures can be effectively avoided, and only cracks exceeding normal indexes are concerned.
Preferably, the acquiring the data set in step S11 uses an image processing method to perform data set augmentation, and these operations are performed simultaneously on the original image and the annotation in the data set, and the specific operations include: turning over the image, cutting and splicing, locally distorting, adding Gaussian noise and randomly biasing the brightness.
Preferably, the calculation formula of step S12 is as follows:
Loss=CrossEntropyLoss+(1-F1) (1)
wherein
Figure BDA0003529681290000041
Figure BDA0003529681290000042
Figure BDA0003529681290000043
Figure BDA0003529681290000044
Preferably, step S24 may control whether to merge the discontinuous cracks into one crack instance by adjusting the search range in dfs, and may also adjust the merging distance, i.e. draw two discontinuous cracks into the same crack instance in this distance range. And obtaining a crack example after being processed by a dfs-based pavement crack target detection algorithm, wherein the crack example comprises the distribution range [ xmin, ymin, xmax, ymax ] of the crack example in a graph and the number of crack pixel points of the crack example.
The working principle of the invention is as follows: the parameter file after the training of the FUnet model is only 1.5MB, the parameter quantity is small, and the training can be quickly finished in a short time only by a small amount of data. The video memory is only required to be operated about 1GB when the video memory is detected singly, and the test can reach about 50 frames per second on the RTX1070 video card. The time complexity of a subsequent pavement crack target detection algorithm of dfs is O (mn), and m and n are the width and height of the picture respectively. Therefore, the algorithm has low performance requirement on the carrying platform and high speed, reduces the cost consumption of deploying the algorithm on one hand, and can be deployed on a plurality of vehicle-mounted platforms on the other hand due to low requirement on the platform.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or to implicitly indicate the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the invention, "plurality" means two or more unless explicitly defined otherwise.
In the present invention, unless otherwise explicitly stated or limited, the terms "mounted," "connected," "fixed," and the like are to be construed broadly, e.g., as being fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood according to specific situations by those of ordinary skill in the art.
In the present invention, unless otherwise expressly stated or limited, "above" or "below" a first feature means that the first and second features are in direct contact, or that the first and second features are not in direct contact but are in contact with each other via another feature therebetween. Also, the first feature "on," "above" and "over" the second feature may include the first feature being directly above and obliquely above the second feature, or simply indicating that the first feature is at a higher level than the second feature. "beneath," "under" and "beneath" a first feature includes the first feature being directly beneath and obliquely beneath the second feature, or simply indicating that the first feature is at a lesser elevation than the second feature.
In the description herein, reference to the terms "one embodiment," "some embodiments," "an example," "a specific example," or "some examples" or the like means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made in the above embodiments by those of ordinary skill in the art without departing from the principle and spirit of the present invention.

Claims (4)

1. A road crack detection and evaluation algorithm based on deep learning is characterized by comprising the following steps:
s1, training a neural network model FUnet;
s11, acquiring a CrackForest data set of the public road crack data set;
S12, inputting mini-batch data into the FUnet model each time to obtain the output of the FUnet model, and calculating a loss value by using a loss function consisting of cross entropy and F1 fraction in combination with the labeling of a data set;
s13, updating the gradient of the FUnet model by using an error back propagation algorithm, performing gradient descent on the FUnet according to the calculated gradient by using a gradient descent algorithm, and updating the parameter of the FUnet model;
s14, repeatedly executing the step S12 and the step S13 until the final loss value is not reduced basically;
s15, storing the parameters of the trained FUnet model for subsequent use;
s2, road crack detection and evaluation are carried out by using the algorithm flow;
s21, constructing a neural network model FUnet, importing the trained data, and adjusting the model into an evaluation mode without changing the value of parameters in the network;
s22, acquiring pictures from the camera or the video, and preprocessing the pictures, wherein the algorithm only needs to perform standardized processing on the pictures;
s23, sending the preprocessed pictures into a FUnet model, and obtaining output of the FUnet model, wherein an output Mask of the FUnet model is the classification of pixel points;
s24, inputting the output Mask into a dfs-based pavement crack target detection algorithm, and dividing the crack into examples;
S25, calculating total and individual data of cracks in subsequent algorithms according to crack example data obtained by a dfs-based pavement crack target detection algorithm, wherein the total and individual data comprise data such as picture crack density, crack individual distribution density and the like, and filtering crack examples with certain indexes not meeting the minimum requirement on the basis, so that noise interference on pictures can be effectively avoided, and only cracks exceeding normal indexes are concerned.
2. The deep learning-based road crack detection and evaluation algorithm as claimed in claim 1, wherein: the acquiring data set of step S11 is augmented by using an image processing method, and these operations are performed on the original image and the label in the data set at the same time, and the specific operations include: turning over the image, cutting and splicing, locally distorting, adding Gaussian noise and randomly biasing the brightness.
3. The deep learning-based road crack detection and evaluation algorithm of claim 1, wherein: the calculation formula of step S12 is as follows:
Loss=CrossEntropyLoss+(1-F1) (1)
wherein
Figure FDA0003529681280000011
Figure FDA0003529681280000012
Figure FDA0003529681280000021
Figure FDA0003529681280000022
4. The deep learning-based road crack detection and evaluation algorithm of claim 1, wherein: step S24 may control whether to merge the discontinuous cracks into one crack instance by adjusting the search range in dfs, and may also adjust the merge distance, that is, two discontinuous cracks are drawn to the same crack instance in this distance range. And obtaining a crack example after being processed by a dfs-based pavement crack target detection algorithm, wherein the crack example comprises the distribution range [ xmin, ymin, xmax, ymax ] of the crack example in a graph and the number of crack pixel points of the crack example.
CN202210201900.7A 2022-03-03 2022-03-03 Road crack detection and evaluation algorithm based on deep learning Pending CN114708190A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210201900.7A CN114708190A (en) 2022-03-03 2022-03-03 Road crack detection and evaluation algorithm based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210201900.7A CN114708190A (en) 2022-03-03 2022-03-03 Road crack detection and evaluation algorithm based on deep learning

Publications (1)

Publication Number Publication Date
CN114708190A true CN114708190A (en) 2022-07-05

Family

ID=82166894

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210201900.7A Pending CN114708190A (en) 2022-03-03 2022-03-03 Road crack detection and evaluation algorithm based on deep learning

Country Status (1)

Country Link
CN (1) CN114708190A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115830007A (en) * 2023-02-06 2023-03-21 山东省滨州公路工程有限公司 Pavement void detection method and system based on image recognition

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109801292A (en) * 2018-12-11 2019-05-24 西南交通大学 A kind of bituminous highway crack image partition method based on generation confrontation network
US20210174492A1 (en) * 2019-12-09 2021-06-10 University Of Central Florida Research Foundation, Inc. Methods of artificial intelligence-assisted infrastructure assessment using mixed reality systems
CN112949783A (en) * 2021-04-29 2021-06-11 南京信息工程大学滨江学院 Road crack detection method based on improved U-Net neural network

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109801292A (en) * 2018-12-11 2019-05-24 西南交通大学 A kind of bituminous highway crack image partition method based on generation confrontation network
US20210174492A1 (en) * 2019-12-09 2021-06-10 University Of Central Florida Research Foundation, Inc. Methods of artificial intelligence-assisted infrastructure assessment using mixed reality systems
CN112949783A (en) * 2021-04-29 2021-06-11 南京信息工程大学滨江学院 Road crack detection method based on improved U-Net neural network

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
BEKHZOD OLIMOV等: "FU-Net: fast biomedical image segmentation model based on bottleneck convolution layers", MULTIMEDIA SYSTEMS, 4 January 2021 (2021-01-04) *
MINA JAFARI等: "FU-Net: Multi-class Image Segmentation Using Feedback Weighted U-Net", INTERNATIONAL CONFERENCE ON IMAGE AND GRAPHICS, 28 November 2019 (2019-11-28) *
ZIPING GAO等: "Generative Adversarial Networks for Road Crack Image Segmentation", 2019 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 30 September 2019 (2019-09-30) *
陈泽斌;罗文婷;李林;: "基于改进U-net模型的路面裂缝智能识别", 数据采集与处理, no. 02, 15 March 2020 (2020-03-15) *
高子平: "基于深度学习的公路路面裂缝图像识别技术研究", 中国优秀硕士论文电子期刊网, 15 July 2021 (2021-07-15) *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115830007A (en) * 2023-02-06 2023-03-21 山东省滨州公路工程有限公司 Pavement void detection method and system based on image recognition

Similar Documents

Publication Publication Date Title
KR102121958B1 (en) Method, system and computer program for providing defect analysis service of concrete structure
US20210319561A1 (en) Image segmentation method and system for pavement disease based on deep learning
TWI431559B (en) A vehicle registration-plate detecting method and system thereof
CN111160301B (en) Tunnel disease target intelligent identification and extraction method based on machine vision
CN111292321B (en) Transmission line insulator defect image identification method
CN108009591A (en) A kind of contact network key component identification method based on deep learning
CN104048969A (en) Tunnel defect recognition method
CN104008550A (en) Wafer surface defect feature analysis method and system and wafer surface detect feature classification method and system
CN109215009B (en) Continuous casting billet surface image defect detection method based on deep convolution neural network
CN110544231A (en) lithium battery electrode surface defect detection method based on background standardization and centralized compensation algorithm
CN110909657A (en) Method for identifying apparent tunnel disease image
CN113240623B (en) Pavement disease detection method and device
KR101874968B1 (en) Visibility measuring system base on image information and method for using the same
CN114708190A (en) Road crack detection and evaluation algorithm based on deep learning
CN111046838A (en) Method and device for identifying wetland remote sensing information
CN117788950A (en) Pavement disease detection method based on improved YOLOv8 model
CN117455917B (en) Establishment of false alarm library of etched lead frame and false alarm on-line judging and screening method
CN114708532A (en) Monitoring video quality evaluation method, system and storage medium
CN117351472A (en) Tobacco leaf information detection method and device and electronic equipment
CN112585655A (en) Unmanned electronic traffic police duty system based on 5G
CN116778357A (en) Power line unmanned aerial vehicle inspection method and system utilizing visible light defect identification
CN116519710A (en) Method and system for detecting surface pollution state of composite insulator
CN116109616A (en) Pavement crack detection and small-surface element fitting detection method based on YOLOv5
CN110070520B (en) Pavement crack detection model construction and detection method based on deep neural network
CN110163081A (en) Regional invasion real-time detection method, system and storage medium based on SSD

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination