CN116167986A - Pavement crack detection method and system - Google Patents

Pavement crack detection method and system Download PDF

Info

Publication number
CN116167986A
CN116167986A CN202310066302.8A CN202310066302A CN116167986A CN 116167986 A CN116167986 A CN 116167986A CN 202310066302 A CN202310066302 A CN 202310066302A CN 116167986 A CN116167986 A CN 116167986A
Authority
CN
China
Prior art keywords
image
pavement
matrix
model
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310066302.8A
Other languages
Chinese (zh)
Inventor
徐国胜
徐国爱
马明昌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing University of Posts and Telecommunications
Original Assignee
Beijing University of Posts and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing University of Posts and Telecommunications filed Critical Beijing University of Posts and Telecommunications
Priority to CN202310066302.8A priority Critical patent/CN116167986A/en
Publication of CN116167986A publication Critical patent/CN116167986A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G06V10/242—Aligning, centring, orientation detection or correction of the image by image rotation, e.g. by 90 degrees
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/28—Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Medical Informatics (AREA)
  • Quality & Reliability (AREA)
  • Databases & Information Systems (AREA)
  • Image Analysis (AREA)

Abstract

The road surface crack detection method and system pre-processes road surface image data, adopts a reinforcement learning automatic pruning model to simultaneously prune a neural network of a trained supervised learning model, and uses the cut supervised learning model to preliminarily predict the pre-processed data to obtain an initial result matrix; the obtained preprocessing data and the initial result matrix form environmental parameters, the environmental parameters are input into the reinforcement learning auxiliary model, and the current situation is judged and a turnover area matrix is output; changing the initial result matrix according to the turnover area matrix to obtain a result matrix after the first processing, and forming new environment parameters by the preprocessed data and the result matrix after the first processing; and ending the prediction to output a final prediction result when the reinforcement learning auxiliary model is executed to reach the set longest execution step number or the turnover area matrix output by the reinforcement learning auxiliary model is empty. The method and the device improve the recognition efficiency and recognition accuracy and achieve the overall light weight of the system.

Description

Pavement crack detection method and system
Technical Field
The application belongs to the technical field of image processing, and particularly relates to a pavement crack detection method and system.
Background
With the increase of road mileage and the extension of service time, various damages to the road surface are gradually generated due to the action of driving load and natural factors, so that the maintenance task of the road becomes more and more heavy. The damage to the road surface adversely affects the road carrying capacity, durability, running speed of the vehicle, fuel consumption, mechanical wear, running comfort, traffic safety, environmental protection, and the like.
Cracks are one of the most common and important manifestations of disease in asphalt pavement. The rapid pavement crack detection and the efficient identification are the precondition of timely maintenance of the road, and are effective ways for improving the service quality of the road, prolonging the service life of the road and reducing the maintenance cost of the road. Therefore, the research of the automatic recognition technology of the asphalt pavement cracks is of great significance to the development of traffic industry.
Disclosure of Invention
In view of the above, the present application is directed to a method and a system for detecting a pavement crack, which are used for solving or partially solving the above-mentioned technical problems.
Based on the above object, a first aspect of the present application provides a pavement crack detection method, including:
preprocessing pavement image data, wherein the preprocessing comprises image filling/cutting and image compression, and normalizing pavement images subjected to the image filling/cutting and the image compression to obtain preprocessed data D;
simultaneously pruning n layers of neural networks of the trained supervised learning model by adopting an reinforcement learning automatic pruning model, and preliminarily predicting the preprocessing data D by using the pruned supervised learning model to obtain an initial result matrix M 0 ；
The obtained preprocessing data D and an initial result matrix M 0 Composition environment<D,M 0 >Inputting the current situation into an Agent of the reinforcement learning auxiliary model, and judging the current situation through the reinforcement learning auxiliary model to output a turnover area matrix M s ；
According to the inversion region matrix M s Altering the initial result matrix M 0 Obtaining a result matrix M after the first treatment 1 Preprocessing data D and a result matrix M after first processing 1 Composition of new environment<D,M1>；
When the Agent executing the reinforcement learning auxiliary model reaches the set longest execution step number N, or the turnover area matrix M output by the reinforcement learning auxiliary model Agent s When the prediction is empty, the final prediction result M is output after the prediction is ended n 。
As a preferable mode of the pavement crack detection method, the image filling/cutting step includes:
reading a pavement image gray level map, and when the size of the target pavement image gray level map is larger than a size preset value, cutting the target pavement image according to the size preset value from the upper left corner;
when the size of the target pavement image gray level map is smaller than the size preset value, 255 pixel values are used for filling in the right side and the lower side of the target pavement image gray level map, and the pavement image gray level map with uniform size is obtained.
As a preferable scheme of the pavement crack detection method, the step of normalizing includes:
and obtaining a pixel value p of the pixel point of the pavement image, calculating a pixel average value mu of the pixel point of the pavement image according to the pixel value p, subtracting the pixel average value mu from the pixel value p of each pixel point, and dividing the pixel average value mu by 255 to obtain normalized preprocessing data D.
As a preferable scheme of the pavement crack detection method, downsampling operation of the pavement image data for 1/2 of the designated times is carried out in the forward transmission process of the supervised learning model, a result matrix M is output, and each value in the result matrix M represents a probability value of disease existing in the position of a designated pixel area in the original pavement image area;
performing supervision training on the supervision learning model by using a Dice Loss function;
Figure BDA0004073589380000021
wherein M is a result matrix, and T is a target matrix.
As a preferable scheme of the pavement crack detection method, the reinforcement learning automatic pruning model Agent sets output actions as selecting the cutting proportion parameters for each layer of neural network at the same time, and the cutting proportion parameters range from 0% to 50%.
A second aspect of the present application provides a pavement crack detection system, comprising:
the data preprocessing module is used for preprocessing road surface image data, wherein the preprocessing comprises image filling/cutting and image compression, and the normalization is carried out on the road surface image after the image filling/cutting and the image compression to obtain preprocessed data D;
the pruning processing module is used for simultaneously pruning the n layers of neural networks of the supervised learning model after training by adopting the reinforcement learning automatic pruning model;
the preliminary prediction module is used for performing preliminary prediction on the preprocessing data D by using the cut supervised learning model to obtain an initial result matrix M 0 ；
The overturn processing module is used for preprocessing the obtained data D and the initial result matrix M 0 Composition environment<D,M 0 >Inputting the current situation into an Agent of the reinforcement learning auxiliary model, and judging the current situation through the reinforcement learning auxiliary model to output a turnover area matrix M s ；
An update processing module for updating the area matrix M according to the inversion s Altering the initial result matrix M 0 Obtaining a result matrix M after the first treatment 1 Will bePreprocessing data D and result matrix M after first processing 1 Composition of new environment<D,M1>；
The result prediction module is used for predicting the maximum execution step number N when the Agent executing the reinforcement learning auxiliary model reaches the set maximum execution step number N or outputting a turnover area matrix M by the reinforcement learning auxiliary model Agent s When the prediction is empty, the final prediction result M is output after the prediction is ended n 。
As a preferable scheme of the pavement crack detection system, the data preprocessing module comprises an image filling/cutting sub-module:
the image filling/cutting submodule is used for reading the gray level image of the road surface image, and when the size of the gray level image of the target road surface image is larger than a size preset value, cutting the target road surface image according to the size preset value from the upper left corner;
the image filling/cutting sub-module is further used for filling the right side and the lower side of the target pavement image gray level map by using 255 pixel values when the size of the target pavement image gray level map is smaller than a size preset value, so that the uniform pavement image gray level map is obtained.
As a preferable scheme of the pavement crack detection system, the data preprocessing module further comprises a normalization processing sub-module:
the normalization processing sub-module is used for obtaining a pixel value p of a pavement image pixel point, calculating a pixel average value mu of the pavement image pixel point according to the pixel value p, subtracting the pixel average value mu from the pixel value p of each pixel point, and dividing 255 to obtain normalized preprocessing data D.
As a preferable scheme of the pavement crack detection system, the pavement crack detection system further comprises a supervised learning model training module:
in the supervised learning model training module, downsampling operation is carried out on road surface image data for 1/2 of the designated times in the forward transmission process of the supervised learning model, a result matrix M is output, and each value in the result matrix M represents a probability value of disease existing at the position of a designated pixel area in an original road surface image area;
the supervised learning model training module performs supervised training on the supervised learning model by using a Dice Loss function;
Figure BDA0004073589380000041
wherein M is a result matrix, and T is a target matrix.
As a preferable scheme of the pavement crack detection system, in the pruning processing module, the reinforcement learning automatic pruning model Agent sets output action as selecting a cutting proportion parameter for each layer of neural network at the same time, and the cutting proportion parameter range is 0-50%.
A third aspect of the present application proposes an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, said processor implementing the pavement crack detection method of the first aspect or any possible implementation thereof when executing said program.
A fourth aspect of the present application proposes a non-transitory computer readable storage medium storing computer instructions for causing a computer to execute a pavement crack detection method implementing the first aspect or any possible implementation thereof.
From the above, it can be seen that, according to the technical scheme provided by the application, the preprocessing is performed on the pavement image data, the preprocessing comprises image filling/cutting and image compression, and the pavement image after the image filling/cutting and the image compression is normalized to obtain the preprocessed data; simultaneously pruning the neural network of the trained supervised learning model by adopting an reinforcement learning automatic pruning model, and preliminarily predicting the preprocessing data by using the pruned supervised learning model to obtain an initial result matrix; the obtained preprocessing data and the initial result matrix form environmental parameters to be input into an enhanced learning auxiliary model Agent, and the current situation is judged through the enhanced learning auxiliary model to output a turnover area matrix; changing the initial result matrix according to the turnover area matrix to obtain a result matrix after the first processing, and forming new environment parameters by the preprocessed data and the result matrix after the first processing; and ending the prediction to output a final prediction result when the reinforcement learning auxiliary model Agent reaches the set longest execution step number or the turnover area matrix output by the reinforcement learning auxiliary model Agent is empty. The method and the device improve the overall recognition efficiency and recognition accuracy and make the overall system lightweight; the method can be used on most mobile equipment in a migration way, the application range of the system is enlarged, the application efficiency of the system is improved, and the physical configuration requirement of the system is reduced to the maximum extent.
Drawings
In order to more clearly illustrate the technical solutions of the present application or related art, the drawings that are required to be used in the description of the embodiments or related art will be briefly described below, and it is apparent that the drawings in the following description are only embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort to those of ordinary skill in the art.
Fig. 1 is a schematic flow chart of a pavement crack detection method according to an embodiment of the present application;
fig. 2 is a schematic diagram of image preprocessing in the pavement crack detection method according to the embodiment of the present application;
fig. 3 is a schematic diagram of supervised learning model training in the pavement crack detection method according to the embodiment of the present application;
fig. 4 is a schematic diagram of a pruning flow in the pavement crack detection method according to the embodiment of the present application;
fig. 5 is a schematic diagram of auxiliary reinforcement learning in the pavement crack detection method according to the embodiment of the present application;
fig. 6 is a schematic diagram of a pavement crack detection system according to an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail below with reference to the accompanying drawings.
It should be noted that unless otherwise defined, technical or scientific terms used in the embodiments of the present application should be given the ordinary meaning as understood by one of ordinary skill in the art to which the present application belongs. The use of the terms "comprising" or "including" and the like in the embodiments of the present application is intended to cover an element or article appearing before the term and equivalents thereof, which are recited after the term, without excluding other elements or articles.
Cracks are one of the most common and important manifestations of disease in asphalt pavement. The cracks indicate that the roadbed or the base layer is damaged, for example, the wheel track belt is cracked, the strength of the roadbed or the base layer is weakened, the roadbed or the base layer can be induced to be damaged, and if rainwater enters the base layer along the cracks, the base layer or the roadbed is unstable, so that the pavement damage is further aggravated. Meanwhile, cracks are not treated in time, so that the service life of the highway is shortened, and the comfort and safety of driving are affected. Pavement crack detection is the basis of highway maintenance management, and is the most important component of pavement damage. The rapid pavement crack detection and the efficient identification are the precondition of timely maintenance of the road, and are effective ways for improving the service quality of the road, prolonging the service life of the road and reducing the maintenance cost of the road. Therefore, the research of the automatic recognition technology of the asphalt pavement cracks is of great significance to the development of highway traffic industry.
For road surface data acquisition, people record road surface conditions by adopting a walking human eye observation method at first, but the efficiency is low, the labor intensity is high and the safety is poor. In the face of the increasing demand for highway maintenance, since the 90 th century, people have begun to develop pavement informatization detection systems, and pavement detection devices based on ultrasonic technology, ground penetrating radar, camera shooting measurement and other technologies have been developed. The image capturing measurement technology can collect damage information widely, and is suitable for various road surfaces, so that the image capturing measurement technology is widely applied to Japanese Komatsu systems, american PCES systems, switzerland CREHOS systems and China CiCs systems. The system gradually realizes a certain automatic detection function, improves the efficiency and the investigation safety, but most of the systems acquire two-dimensional images of the road surface based on the surface or line scanning technology. In recent years, a team can acquire a road surface three-dimensional image with the accuracy of 1 mm/pixel during high-speed running based on a 3D laser scanning imaging technology, so that the road surface form can be more truly embodied, the quality of the road surface image is obviously improved, but the method is not applied to actual production at present, and a two-dimensional image is still used as a data base of analysis.
In the related manual observation technology, most pavement maintenance companies select to use a purely manual naked eye observation mode to detect a disease area when taking a hand of pavement image data, so that the efficiency is low and the cost is high.
In the related automatic detection technology based on the traditional image processing, methods such as a filter, gray correction and the like are adopted to eliminate noise and improve image quality, and meanwhile, the traditional threshold segmentation algorithm and the edge recognition algorithm are improved and researched, so that various crack image automatic recognition algorithms such as a seed recognition algorithm, a machine learning algorithm, a texture analysis algorithm, a global optimization algorithm and the like are provided. Among other features, liu Fanfan et al use features including the black pixel count ratio, the maximum connected black block pixel count ratio, etc., and use a single layer back propagation network to classify after feature vectors are constructed to identify cracks. Similarly, varadharajan et al construct 138-dimensional feature vectors from each picture as input to the classification model. Still other researchers use the fitting degree of the linear regression of the pixel points of the largest connected black block to mark whether the shape of the black block is long and narrow (the slit is generally long and narrow), but the effect cannot reach the accuracy required by practical application.
In the related automatic detection technology based on machine learning, the deep learning technology is applied to road surface image disease area detection engineering, and classical deep supervision network models such as resnet, densenet and the like are used for carrying out end-to-end black box training on road surface image disease area automatic identification.
However, for all the recognition schemes in the current stage, the current algorithm cannot well cope with interference of factors such as illumination, stains and the like. For example, the cic system is still a semi-automatic detection technology based on manual or computer-aided manual evaluation, the main process is that the existing threshold segmentation algorithm, edge recognition algorithm and the like based on the prior art are firstly processed, then the pavement images stored on a computer hard disk are checked by staff to carry out manual evaluation, the pictures are marked with cracks manually when the production requirements cannot be met, and finally the computer is used for statistics.
In view of this, in order to solve the problem of light weight of the model, particularly under the background that mobile devices are popular nowadays, the model is made lighter and more convenient; breaks through the learning bottleneck of the traditional deep learning, and improves the capability of automatically detecting pavement diseases by artificial intelligence; the embodiment of the application provides a pavement crack detection method and system, and the following is specific content of an embodiment of the invention.
Referring to fig. 1, 2, 3, 4 and 5, an embodiment of the present application provides a pavement crack detection method, including the following steps:
s1, preprocessing pavement image data, wherein the preprocessing comprises image filling/cutting and image compression, and normalizing pavement images subjected to the image filling/cutting and the image compression to obtain preprocessed data D;
s2, pruning operation is carried out on the n layers of neural networks of the trained supervised learning model simultaneously by adopting the reinforcement learning automatic pruning model, and preliminary prediction is carried out on the preprocessing data D by using the cut supervised learning model to obtain an initial result matrix M 0 ；
S3, preprocessing the obtained data D and an initial result matrix M 0 Composition environment<D,M 0 >Inputting the current situation into an Agent of the reinforcement learning auxiliary model, and judging the current situation through the reinforcement learning auxiliary model to output a turnover area matrix M s ；
S4, according to the overturning area matrix M s Altering the initial result matrix M 0 Obtaining a result matrix M after the first treatment 1 Preprocessing data D and a result matrix M after first processing 1 Composition of new environment<D,M1>；
S5, when the Agent executing the reinforcement learning auxiliary model reaches the set longest execution step number N, or the turnover area matrix M output by the reinforcement learning auxiliary model Agent s In the case of the air-conditioner,ending the prediction to output a final prediction result M n 。
The current mainstream method for automatically identifying the pavement image disease area is to construct an identification model based on traditional supervised learning to identify the pavement image disease area end to end, but because of different model structures, different training parameters and training modes and the like, different models have different performances on the automatic detection task of the pavement image disease area under different parameter and training mode settings. In order to eliminate the influence of other interference factors, the embodiment of the application performs unified processing on data processing, training parameters, training mode setting and the like, and achieves the learning bottleneck which can be achieved in the road surface image disease area automatic identification task under the unified parameter background of different structure neural networks.
In the embodiment, the auxiliary fig. 2 is that the pavement image data is preprocessed, the pavement image is filled and compressed, and the size of the original pavement image is set as follows: and when the size of the target image gray level is smaller than 2200 x 3400, 255 pixel values are used for filling in the right side and the lower side of the image gray level, and finally, the image size is unified into 2200 x 3400 gray level.
Then compressing the image to 704 x 1088, finally performing normalization processing to obtain a pixel value p of a pavement image pixel point, calculating a pixel average value mu of the pavement image pixel point according to the pixel value p, subtracting the pixel average value mu from the pixel value p of each pixel point, dividing the pixel average value mu by 255 to obtain normalized preprocessing data D, and ending the data preprocessing: d= (p- μ)/255.
In the embodiment, the training is performed by using a modified supervised learning model, the downsampling operation is performed 5 times 1/2 of the road surface image data in the forward transmission process of the supervised learning model, and the dimension of the final output result matrix M is 704/2 5 *1088/2 5 I.e. 22 x 34, each value in the result matrix represents the probability value of disease in 100 x 100 pixel area at a certain position in the original 2200 x 3400 area, and the label is used for supervision trainingTarget matrix T, also labeled 22 x 34, was used to supervise the model using the Dice Loss function:
Figure BDA0004073589380000081
until the supervised learning model no longer converges.
In this embodiment, in order to improve overall recognition efficiency, efficiency optimization is performed on a traditional supervised learning model, so that overall recognition efficiency is improved.
Specifically, the reinforcement learning automatic pruning model is selected to prune the pre-trained supervised learning model, the trained supervised learning model consists of a layer of parameter weights weight and bias, and in forward transmission, not all parameters are acted, only a small part of parameter operation can activate neurons at the lower layer, but calculation of all neurons can additionally increase calculation burden, so that the overall operation efficiency of a corresponding system is improved by selecting a mode of inactivating part of neurons (the value of which is set to 0).
FIG. 4 is assisted by a supervised learning model neural network having n layers, each of which is defined by L from the first layer 1 ，L 2 ，...，L n Representing n layers of the neural network in sequential progression. Simultaneously pruning operation is carried out on each layer of neural network by using an reinforcement learning automatic pruning model, wherein the reinforcement learning automatic pruning model Agent sets output action as a corresponding parameter cutting proportion for each layer, the range is 0-50%, the environmental parameter is set as currently input image data, a comparison group, namely a model which does not prune is placed, meanwhile, the change of the running efficiency FLPs (floating point of operations, floating point operation times) and the change of the price Loss are recorded, when the ratio of the New price Loss of the reinforcement learning automatic pruning model to the price Loss calculated by the non-pruning model is not lower than 5%, the running efficiency New FLPs of the reinforcement learning automatic pruning model is increased compared with the FLPs calculated by the non-pruning model, positive rewards +1 are given to the reinforcement learning Agent, and otherwise negative rewards-1 are given to the Agent:
namely:
Figure BDA0004073589380000091
thus, every m steps are performed, the reinforcement learning automatic pruning model Agent is subjected to total reward and forward iterative updating until the reinforcement learning automatic pruning model is not converged.
In the embodiment, in order to break the learning bottleneck of the supervised learning model in the automatic recognition of the pavement image disease area, the recognition result is further optimized based on the recognition result of the supervised learning model, so that the overall recognition effect is improved.
Specifically, the reinforcement learning auxiliary model is adopted to train on the basis of the supervision learning model and the reinforcement learning automatic pruning model, and the result output in the earlier stage is combined with the original image input to judge the mark area for increasing/decreasing the cracks. The Agent output obtaining action of the reinforcement learning auxiliary model is to judge whether the judgment result in the initial result matrix of the dimension 22 x 34 needs to be subjected to the overturning operation, namely, simultaneously carrying out the value overturning operation on partial areas in the initial result matrix, wherein the value range of each point in the initial result matrix is 0-1.0, representing the possibility of diseases in the 100 x 100 pixel area of the original image, and setting M s Is the region of selected values in the initial result matrix: m is M n ＝1-M s I.e. the roll-over prediction value M n The + original predicted value is equal to 1.
Wherein the environment of the reinforcement learning auxiliary model is set as the data D which is currently preprocessed and the result matrix M which is obtained by the last processing n (the representative result matrix is processed n times by the reinforcement learning auxiliary model)<D,M n >Setting an upper limit N of the number of steps in the training process of the reinforcement learning auxiliary model, ending the prediction when the reinforcement learning auxiliary model is deployed N times or the area of the value selected by the current prediction is empty, and obtaining a result matrix M n As a final prediction result matrix, taking a current result matrix M n And last processing result matrix M n-1 The difference percentage of the Dice Loss is accumulated as the report, and the Agent of the reinforcement learning auxiliary model is updated according to the report after the n times of operation are finished: reward=reward+ (NewDiceLoss-DiceLoss)/DiceLoss。
In summary, the application preprocesses the pavement image data, wherein the preprocessing comprises image filling/cutting and image compression, and normalizes the pavement image after the image filling/cutting and the image compression to obtain the preprocessed data; simultaneously pruning the neural network of the trained supervised learning model by adopting an reinforcement learning automatic pruning model, and preliminarily predicting the preprocessing data by using the pruned supervised learning model to obtain an initial result matrix; the obtained preprocessing data and the initial result matrix form environmental parameters to be input into an enhanced learning auxiliary model Agent, and the current situation is judged through the enhanced learning auxiliary model to output a turnover area matrix; changing the initial result matrix according to the turnover area matrix to obtain a result matrix after the first processing, and forming new environment parameters by the preprocessed data and the result matrix after the first processing; and ending the prediction to output a final prediction result when the reinforcement learning auxiliary model Agent reaches the set longest execution step number or the turnover area matrix output by the reinforcement learning auxiliary model Agent is empty. The method and the device improve the overall recognition efficiency and recognition accuracy and make the overall system lightweight; the method can be used on most mobile equipment in a migration way, the application range of the system is enlarged, the application efficiency of the system is improved, and the physical configuration requirements of the system are reduced to the maximum extent.
It should be noted that, the method of the embodiments of the present application may be performed by a single device, for example, a computer or a server. The method of the embodiment can also be applied to a distributed scene, and is completed by mutually matching a plurality of devices. In the case of such a distributed scenario, one of the devices may perform only one or more steps of the methods of embodiments of the present application, and the devices may interact with each other to complete the methods.
It should be noted that some embodiments of the present application are described above. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments described above and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
Referring to fig. 6, based on the same inventive concept, corresponding to the method of any embodiment described above, an embodiment of the present application further provides a pavement crack detection system, including:
the data preprocessing module 1 is used for preprocessing pavement image data, wherein the preprocessing comprises image filling/cutting and image compression, and the pavement image after the image filling/cutting and the image compression is normalized to obtain preprocessed data D;
the pruning processing module 2 is used for simultaneously pruning the n layers of neural networks of the trained supervised learning model by adopting the reinforcement learning automatic pruning model;
a preliminary prediction module 3 for performing preliminary prediction on the preprocessed data D by using the cut supervised learning model to obtain an initial result matrix M 0 ；
A flipping processing module 4 for generating the obtained preprocessed data D and the initial result matrix M 0 Composition environment<D,M 0 >Inputting the current situation into an Agent of the reinforcement learning auxiliary model, and judging the current situation through the reinforcement learning auxiliary model to output a turnover area matrix M s ；
An update processing module 5 for updating the area matrix M according to the inversion s Altering the initial result matrix M 0 Obtaining a result matrix M after the first treatment 1 Preprocessing data D and a result matrix M after first processing 1 Composition of new environment<D,M1>；
A result prediction module 6 for performing a reinforcement learning auxiliary model Agent to reach a set longest execution step number N or a turnover area matrix M outputted by the reinforcement learning auxiliary model Agent s When the prediction is empty, the final prediction result M is output after the prediction is ended n 。
In this embodiment, the data preprocessing module 1 includes an image filling/cutting sub-module 11:
the image filling/cutting sub-module 11 is configured to read a grayscale image of a road surface, and when the size of the grayscale image of the target road surface is greater than a preset size value, cut the image of the target road surface according to the preset size value from the upper left corner;
the image filling/cutting sub-module 11 is further configured to, when the size of the target pavement image gray scale is smaller than the size preset value, fill up the right side and the lower side of the target pavement image gray scale with 255 pixels, and obtain the pavement image gray scale with uniform size.
In this embodiment, the data preprocessing module 1 further includes a normalization processing sub-module 12:
the normalization processing sub-module 12 is configured to obtain a pixel value p of a pixel point of the pavement image, calculate a pixel average value μ of the pixel point of the pavement image according to the pixel value p, and divide the pixel value p of each pixel point by 255 to obtain normalized preprocessed data D.
In this embodiment, the method further includes a supervised learning model training module 7:
in the supervised learning model training module 7, downsampling operation is carried out on road surface image data for 1/2 of the designated times in the forward transmission process of the supervised learning model, a result matrix M is output, and each value in the result matrix M represents a probability value of disease existing at the position of a designated pixel area in the original road surface image area;
the supervised learning model training module 7 performs supervised training on the supervised learning model by using a Dice Loss function;
Figure BDA0004073589380000111
wherein M is a result matrix, and T is a target matrix.
In this embodiment, in the pruning processing module 2, the reinforcement learning automatic pruning model Agent sets the output action as selecting the clipping ratio parameter for each layer of neural network at the same time, and the clipping ratio parameter ranges from 0% to 50%.
A third aspect of the embodiments of the present application proposes an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the pavement crack detection method of the first aspect or any possible implementation thereof when executing the program.
A fourth aspect of the present application proposes a non-transitory computer readable storage medium storing computer instructions for causing a computer to execute a pavement crack detection method implementing the first aspect or any possible implementation thereof
The device of the foregoing embodiment is used to implement the corresponding pavement crack detection method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiment, which is not described herein.
Based on the same inventive concept, the application also provides an electronic device corresponding to the method of any embodiment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the pavement crack detection method of any embodiment when executing the program.
Fig. 7 is a schematic diagram of a hardware structure of an electronic device according to the embodiment, where the device may include: a processor 1010, a memory 1020, an input/output interface 1030, a communication interface 1040, and a bus 1050. Wherein processor 1010, memory 1020, input/output interface 1030, and communication interface 1040 implement communication connections therebetween within the device via a bus 1050.
The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit ), microprocessor, application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc. for executing relevant programs to implement the technical solutions provided in the embodiments of the present disclosure.
The Memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory ), static storage device, dynamic storage device, or the like. Memory 1020 may store an operating system and other application programs, and when the embodiments of the present specification are implemented in software or firmware, the associated program code is stored in memory 1020 and executed by processor 1010.
The input/output interface 1030 is used to connect with an input/output module for inputting and outputting information. The input/output module may be configured as a component in a device (not shown) or may be external to the device to provide corresponding functionality. Wherein the input devices may include a keyboard, mouse, touch screen, microphone, various types of sensors, etc., and the output devices may include a display, speaker, vibrator, indicator lights, etc.
Communication interface 1040 is used to connect communication modules (not shown) to enable communication interactions of the present device with other devices. The communication module may implement communication through a wired manner (such as USB, network cable, etc.), or may implement communication through a wireless manner (such as mobile network, WIFI, bluetooth, etc.).
Bus 1050 includes a path for transferring information between components of the device (e.g., processor 1010, memory 1020, input/output interface 1030, and communication interface 1040).
It should be noted that although the above-described device only shows processor 1010, memory 1020, input/output interface 1030, communication interface 1040, and bus 1050, in an implementation, the device may include other components necessary to achieve proper operation. Furthermore, it will be understood by those skilled in the art that the above-described apparatus may include only the components necessary to implement the embodiments of the present description, and not all the components shown in the drawings.
The electronic device of the foregoing embodiment is configured to implement the corresponding pavement crack detection method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiment, which is not described herein.
Based on the same inventive concept, corresponding to any of the above embodiments of the method, the present application further provides a non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the pavement crack detection method according to any of the above embodiments.
The computer readable media of the present embodiments, including both permanent and non-permanent, removable and non-removable media, may be used to implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device.
The storage medium of the foregoing embodiments stores computer instructions for causing the computer to execute the pavement crack detection method according to any one of the foregoing embodiments, and has the advantages of the corresponding method embodiments, which are not described herein.
Those of ordinary skill in the art will appreciate that: the discussion of any of the embodiments above is merely exemplary and is not intended to suggest that the scope of the application (including the claims) is limited to these examples; the technical features of the above embodiments or in the different embodiments may also be combined within the idea of the present application, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of brevity.
Additionally, well-known power/ground connections to Integrated Circuit (IC) chips and other components may or may not be shown within the provided figures, in order to simplify the illustration and discussion, and so as not to obscure the embodiments of the present application. Furthermore, the devices may be shown in block diagram form in order to avoid obscuring the embodiments of the present application, and this also takes into account the fact that specifics with respect to implementation of such block diagram devices are highly dependent upon the platform on which the embodiments of the present application are to be implemented (i.e., such specifics should be well within purview of one skilled in the art). Where specific details (e.g., circuits) are set forth in order to describe example embodiments of the application, it should be apparent to one skilled in the art that embodiments of the application can be practiced without, or with variation of, these specific details. Accordingly, the description is to be regarded as illustrative in nature and not as restrictive.
While the present application has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of those embodiments will be apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.
The present embodiments are intended to embrace all such alternatives, modifications and variances which fall within the broad scope of the appended claims. Accordingly, any omissions, modifications, equivalents, improvements and/or the like which are within the spirit and principles of the embodiments are intended to be included within the scope of the present application.

Claims (10)

1. A pavement crack detection method comprising:
preprocessing pavement image data, wherein the preprocessing comprises image filling/cutting and image compression, and normalizing pavement images subjected to the image filling/cutting and the image compression to obtain preprocessed data D;
simultaneously pruning n layers of neural networks of the trained supervised learning model by adopting an reinforcement learning automatic pruning model, and preliminarily predicting the preprocessing data D by using the pruned supervised learning model to obtain an initial result matrix M 0 ；
The obtained preprocessing data D and an initial result matrix M 0 Composition environment<D,M 0 >Inputting the current data into an Agent of the reinforcement learning auxiliary model, and carrying out current analysis on the current data through the reinforcement learning auxiliary modelJudging and outputting a turnover area matrix M under the condition s ；
According to the inversion region matrix M s Altering the initial result matrix M 0 Obtaining a result matrix M after the first treatment 1 Preprocessing data D and a result matrix M after first processing 1 Composition of new environment<D,M1>；
When the Agent executing the reinforcement learning auxiliary model reaches the set longest execution step number N, or the turnover area matrix M output by the reinforcement learning auxiliary model Agent s When the prediction is empty, the final prediction result M is output after the prediction is ended n 。
2. The pavement crack detection method as set forth in claim 1, wherein the image filling/cutting step includes:
reading a pavement image gray level map, and when the size of the target pavement image gray level map is larger than a size preset value, cutting the target pavement image according to the size preset value from the upper left corner;
when the size of the target pavement image gray level map is smaller than the size preset value, 255 pixel values are used for filling in the right side and the lower side of the target pavement image gray level map, and the pavement image gray level map with uniform size is obtained.
3. The pavement crack detection method as set forth in claim 2, wherein the step of normalizing includes:
and obtaining a pixel value p of the pixel point of the pavement image, calculating a pixel average value mu of the pixel point of the pavement image according to the pixel value p, subtracting the pixel average value mu from the pixel value p of each pixel point, and dividing the pixel average value mu by 255 to obtain normalized preprocessing data D.
4. The pavement crack detection method as set forth in claim 1, wherein downsampling operation is performed on pavement image data for a specified number of times of 1/2 in a forward transfer process of the supervised learning model, a result matrix M is output, and each value in the result matrix M represents a probability value that a disease exists at a specified pixel area position in an original pavement image area;
performing supervision training on the supervision learning model by using a Dice Loss function;
Figure FDA0004073589370000021
wherein M is a result matrix, and T is a target matrix.
5. The pavement crack detection method as set forth in claim 4, wherein the reinforcement learning automatic pruning model Agent sets the output action as selecting the clipping ratio parameter for each layer of neural network at the same time, and the clipping ratio parameter ranges from 0% to 50%.
6. A pavement crack detection system, comprising:
the data preprocessing module is used for preprocessing road surface image data, wherein the preprocessing comprises image filling/cutting and image compression, and the normalization is carried out on the road surface image after the image filling/cutting and the image compression to obtain preprocessed data D;
the pruning processing module is used for simultaneously pruning the n layers of neural networks of the supervised learning model after training by adopting the reinforcement learning automatic pruning model;
the preliminary prediction module is used for performing preliminary prediction on the preprocessing data D by using the cut supervised learning model to obtain an initial result matrix M 0 ；
The overturn processing module is used for preprocessing the obtained data D and the initial result matrix M 0 Composition environment<D,M 0 >Inputting the current situation into an Agent of the reinforcement learning auxiliary model, and judging the current situation through the reinforcement learning auxiliary model to output a turnover area matrix M s ；
An update processing module for updating the area matrix M according to the inversion s Altering the initial result matrix M 0 Obtaining a result matrix M after the first treatment 1 Preprocessing data D and a result matrix M after first processing 1 Composition of new environment<D,M1>；
Result prediction module forWhen the Agent executing the reinforcement learning auxiliary model reaches the set longest execution step number N, or the turnover area matrix M output by the reinforcement learning auxiliary model Agent s When the prediction is empty, the final prediction result M is output after the prediction is ended n 。
7. The pavement crack detection system of claim 6, wherein the data preprocessing module includes an image filling/cutting sub-module:
the image filling/cutting submodule is used for reading the gray level image of the road surface image, and when the size of the gray level image of the target road surface image is larger than a size preset value, cutting the target road surface image according to the size preset value from the upper left corner;
the image filling/cutting sub-module is further used for filling the right side and the lower side of the target pavement image gray level map by using 255 pixel values when the size of the target pavement image gray level map is smaller than a size preset value, so that the uniform pavement image gray level map is obtained.
8. The pavement crack detection system of claim 7, wherein the data preprocessing module further comprises a normalization processing sub-module:
the normalization processing sub-module is used for obtaining a pixel value p of a pavement image pixel point, calculating a pixel average value mu of the pavement image pixel point according to the pixel value p, subtracting the pixel average value mu from the pixel value p of each pixel point, and dividing 255 to obtain normalized preprocessing data D.
9. The pavement crack detection system of claim 8, further comprising a supervised learning model training module:
in the supervised learning model training module, downsampling operation is carried out on road surface image data for 1/2 of the designated times in the forward transmission process of the supervised learning model, a result matrix M is output, and each value in the result matrix M represents a probability value of disease existing at the position of a designated pixel area in an original road surface image area;
the supervised learning model training module performs supervised training on the supervised learning model by using a Dice Loss function;
Figure FDA0004073589370000031
wherein M is a result matrix, and T is a target matrix.
10. The pavement crack detection system of claim 9, wherein the reinforcement learning automatic pruning model Agent in the pruning processing module sets the output action as selecting the clipping proportion parameter for each layer of neural network at the same time, and the clipping proportion parameter ranges from 0% to 50%.
CN202310066302.8A 2023-01-12 2023-01-12 Pavement crack detection method and system Pending CN116167986A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310066302.8A CN116167986A (en) 2023-01-12 2023-01-12 Pavement crack detection method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310066302.8A CN116167986A (en) 2023-01-12 2023-01-12 Pavement crack detection method and system

Publications (1)

Publication Number Publication Date
CN116167986A true CN116167986A (en) 2023-05-26

Family

ID=86417767

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310066302.8A Pending CN116167986A (en) 2023-01-12 2023-01-12 Pavement crack detection method and system

Country Status (1)

Country Link
CN (1) CN116167986A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116558578A (en) * 2023-07-12 2023-08-08 中国公路工程咨询集团有限公司 Road surface condition detection method, device and storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116558578A (en) * 2023-07-12 2023-08-08 中国公路工程咨询集团有限公司 Road surface condition detection method, device and storage medium
CN116558578B (en) * 2023-07-12 2023-10-17 中国公路工程咨询集团有限公司 Road surface condition detection method, device and storage medium

Similar Documents

Publication Publication Date Title
CN111179229B (en) Industrial CT defect detection method based on deep learning
Zhang et al. Pavement distress detection using convolutional neural network (CNN): A case study in Montreal, Canada
KR102166458B1 (en) Defect inspection method and apparatus using image segmentation based on artificial neural network
Nguyen et al. Pavement crack detection using convolutional neural network
Zhou et al. Deep learning-based crack segmentation for civil infrastructure: Data types, architectures, and benchmarked performance
Wan et al. Ceramic tile surface defect detection based on deep learning
CN109840483B (en) Landslide crack detection and identification method and device
Ni et al. A Generative adversarial learning strategy for enhanced lightweight crack delineation networks
Kanaeva et al. Road pavement crack detection using deep learning with synthetic data
CN116167986A (en) Pavement crack detection method and system
CN111144425B (en) Method and device for detecting shot screen picture, electronic equipment and storage medium
Camilleri et al. Detecting road potholes using computer vision techniques
CN115601682A (en) Method and device for detecting foreign matters of underground belt conveyor
CN114418937B (en) Pavement crack detection method and related equipment
CN115272224A (en) Unsupervised pavement damage detection method for smart city construction
CN114283383A (en) Smart city highway maintenance method, computer equipment and medium
Lu Advances in deep learning methods for pavement surface crack detection and identification with visible light visual images
CN117079145B (en) Comprehensive road condition evaluation method, electronic equipment and storage medium
CN117422699A (en) Highway detection method, highway detection device, computer equipment and storage medium
CN111242898A (en) Train pantograph abrasion detection method and system based on deep neural network
Zhao et al. MED-YOLOv8s: a new real-time road crack, pothole, and patch detection model
CN117058069A (en) Automatic detection method for apparent diseases of pavement in panoramic image
Ji et al. Bucket teeth detection based on faster region convolutional neural network
Li et al. An unsupervised concrete crack detection method based on nnU-Net
Dhakal et al. Surface identification of top-down, bottom-up, and cement-treated reflective cracks using convolutional neural network and artificial neural networks

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination